IAM Service Account Key vs Google Credentials File

Question

I'm writing code to generate and download a private key for a Google Cloud service account.

Using the IAM API, I was able to create a service account, and my call to generate a key seems to be working. I get back a Service Account Key as described on the IAM API create key page, like

{ 
  "privateKeyType": "TYPE_GOOGLE_CREDENTIALS_FILE",
  "privateKeyData": "random-key-stringkajdkjakjfke", ...
}

I downloaded this file as a JSON response and am trying to authenticate with it:

gcloud auth activate-service-account --key-file=service-account-key-file.json

Unfortunately, I get an error stating The .json key file is not in a valid format.

When I go though the Google Cloud Console flow (IAM & Admin -> Service accounts -> ... -> Create Key -> Create) I get a downloaded JSON file that looks like

{
  "type": "service_account",
  "private_key": "----BEGIN-PRIVATE-KEY-----",
  "auth_uri": "https://gaiastaging.corp.google.com/o/oauth2/auth",
}

This file looks completely different than the response from the IAM API. Explains my error! Unfortunately, this format doesn't seem to be described anywhere. It's mentioned briefly in some docs. Is it a Google Credentials File?

I'd like to take the IAM response file/JSON and convert it to the second credentials file. I've tried writing some code to convert it, but there are some fields like "auth_provider_x509_cert_url" that I don't understand.

Perhaps converting the file is the wrong approach as well? More generally:

How can I generate a file and then use it to authenticate with gcloud?

How should I describe/distinguish between both of the above files? Why is each type of file useful?

Answer 1

About the two files:

A Google Credentials file and a Service Account Credentials file are the same thing - they're both the second type of file that I downloaded off the Google Cloud Console page. No great official docs pages on them, but they're referenced a lot. Probably also Application Default Credentials.

The JSON response from the IAM create call - this is just a response to an API call. It's not useful outside of parsing it with your application code.

To generate a Google Credentials file:

In the JSON response to the IAM create, there's a field privateKeyData. This field actually contains the entire Google Credentials file. It's just encoded as a base64 string. I just downloaded the file from HTML as

<a href="data:attachment/json;base64;charset=utf-8,THAT-LONG-privateKeyData-base64-string-here" download="service-account-key.json">
  Download key
</a>

Or if you just want to confirm that it contains all the information quickly, copy paste the base64 privateKeyData field into a file google-credentials and decode it (on Linux) with:

base64 -d google-credentials

I was then able to run

gcloud auth activate-service-account --key-file=google-credentials.json

and got

Activated service account credentials for: [[e-mail]]