Authenticating to Hashicorp Vault using GCE Signed Metadata

Applications often require access to small pieces of sensitive data at build or run time, referred to as secrets. Secrets are generally more sensitive than other environment variables or parts of your repository as they may grant access to additional data, such as user information.

HashiCorp Vault is a popular open source tool for secret management, which allows a developer to store, manage and control access to tokens, passwords, certificates, API keys and other secrets. Vault has authentication backends, which allow developers to use many kinds of identities to access Vault, including tokens, or usernames and passwords.

Today, we’re announcing a new Google Compute Engine signed metadata authentication backend for Vault. This allows a developer to use an existing running instance to authenticate directly to Vault, using a JWT of this instance’s signed metadata as its identity. Learn more about the latest release of Vault.

The following example shows how a user can enable and configure the GCP auth backend and then authenticate an instance with Vault. See the docs to learn more.

In the following example, an admin mounts the backend at auth/gcp and adds:

• Credentials that the backend can use to verify the instance (requiring some read-only permissions)
• A Vault-specific role (roles determine what Vault secrets the authenticating GCE VM instance can access, and can restrict login to a set of instances by zone or region, instance group, or GCP labels)

# Mount the backend at ‘auth/gcp’.
$ vault auth-enable ‘gcp’
...

# Add configuration. This can include credentials Vault will 
# use to make calls to GCP APIs to verify authenticating instances. 
$ vault write auth/gcp/config credentials=@path/to/creds.json
...


# Add configuration. This can include credentials Vault will 
# use to make calls to GCP APIs to verify authenticating instances. 
$ vault write auth/gcp/role/my-gce-role \
type='gce' \
policies=’dev,prod’ \
project_id=”project-123456” \
zone="us-central1-c"
instance_group=”my-instance-group” \
labels=”foo:bar,prod:true,...” \
...

Then, a Compute Engine instance can authenticate to the Vault server using the following script:

#! /bin/bash

# [START PARAMS]
# Subtitute real vault address or env variable.
VAULT_ADDR="https://vault.rocks"

# Substitute another service account for the VM instance or use the
# built-in default.
SERVICE_ACCOUNT="default"

# The instance will attempt login under this role.
ROLE="my-gce-role"
# [END PARAMS]


# Generate a metadata identity token JWT with the expected audience.
# The GCP backend expect a JWT 'aud' claim ending in “vault/$ROLE”.
AUD="$VAULT_ADDR/vault/$ROLE"
TOKEN="$(curl -H "Metadata-Flavor: Google" \
http://metadata/computeMetadata/v1/instance/service-accounts/default/identity?audience=$AUD&format=full)"


# Attempt login against VAULT. 
# Using API calls:

cat > payload.json <<-END 
{ 
  "role": "$ROLE", 
  "jwt": "$TOKEN" 
}
END

$ RESP="$(curl \
    --request POST \
    --data @payload.json \
    $VAULT_ADDR/v1/auth/gcp/login)"

# OR

# Use CL vault tool if downloaded
vault write auth/gcp/login role=”$ROLE" jwt="$TOKEN" > response

# Get auth token from JSON
# response (response) and get
# your secrets from Vault!
# ...

A few weeks ago, we also announced a Google Cloud Platform IAM authentication backend for HashiCorp Vault, allowing authentication using an existing IAM service account identity. For further guidance on using HashiCorp Vault with Google Cloud, take a look at today’s Google Cloud blog post.

By Emily Ye, Software Engineer

Authenticating to HashiCorp Vault using Google Cloud IAM

Applications often require access to small pieces of sensitive data at build or run time, referred to as secrets. Secrets are generally more sensitive than other environment variables or parts of your repository as they may grant access to additional data, such as user data.

HashiCorp Vault is a popular open source tool for secret management, which allows a developer to store, manage and control access to tokens, passwords, certificates, API keys and other secrets. Vault has many options for authentication, called authentication backends. These allow developers to use many kinds of identities to access Vault, including tokens, or usernames and passwords. As the number of developers on a team grows, these kinds of authentication options become impractical; and in enterprise scenarios, managing and auditing these identities becomes burdensome.

Today, we are pleased to announce a Google Cloud Platform IAM authentication backend for Vault. This allows a developer to use an existing IAM identity to authenticate to Vault. Using a service account, you can sign a JWT to show it came from a particular account, and use that to authenticate to Vault. Learn more in the documentation.

The following example in Go shows how a user can authenticate with Vault using this backend. This example assumes the Vault server has already been mounted at auth/gcp and configured.

package main

import (
 ...
 vaultapi "github.com/hashicorp/vault/api"
 "golang.org/x/oauth2"
 "golang.org/x/oauth2/google"
 "google.golang.org/api/iam/v1"
 ...
)

func main() {
 // Start [PARAMS]
 project := "project-123456"
 serviceAccount := "[email protected]"
 credsPath := "path/to/creds.json"

 os.Setenv("VAULT_ADDR", "https://vault.mycompany.com")
 defer os.Setenv("VAULT_ADDR", "")
 // End [PARAMS]

 // Start [GCP IAM Setup]
 jsonBytes, err := ioutil.ReadFile(credsPath)
 if err != nil {
  log.Fatal(err)
 }
 config, err := google.JWTConfigFromJSON(jsonBytes, iam.CloudPlatformScope)
 if err != nil {
  log.Fatal(err)
 }

 httpClient := config.Client(oauth2.NoContext)
 iamClient, err := iam.New(httpClient)
 if err != nil {
  log.Fatal(err)
 }
 // End [GCP IAM Setup]
 
 // 1. Generate signed JWT using IAM.
 resourceName := fmt.Sprintf("projects/%s/serviceAccounts/%s", project, serviceAccount)
 jwtPayload := map[string]interface{}{
  "aud": "auth/gcp/login",
  "sub": serviceAccount,
  "exp": time.Now().Add(time.Minute * 10).Unix(),
 }

 payloadBytes, err := json.Marshal(jwtPayload)
 if err != nil {
  log.Fatal(err)
 }
 signJwtReq := &iam.SignJwtRequest{
  Payload: string(payloadBytes),
 }

 resp, err := iamClient.Projects.ServiceAccounts.SignJwt(
resourceName, signJwtReq).Do()
 if err != nil {
  log.Fatal(err)
 }

 // 2. Send signed JWT in login request to Vault.
 vaultClient, err := vaultapi.NewClient(vaultapi.DefaultConfig())
 if err != nil {
  log.Fatal(err)
 }

 vaultResp, err := vaultClient.Logical().Write(
"auth/gcp/login", 
map[string]interface{}{
   "role": "test",
   "jwt":  resp.SignedJwt,
  })

 if err != nil {
  log.Fatal(err)
 }

 // 3. Use auth token from response.
 log.Println("Access token %s", vaultResp.Auth.ClientToken)
 vaultClient.SetToken(vaultResp.Auth.ClientToken)
 // ...
}

Vault is just one way of managing secrets in development. For further reading on choosing a solution that’s right for you, see Google Cloud Platform’s documentation on Secret Management.

By Emily Ye, Software Engineer