AWS: Sourcing AWS CLI Credentials using a Custom AWS CLI Credential Provider and AWS Vault

by No Comments

Introduction

AWS Vault is actually a great tool to securely store and access your AWS Credentials when using the AWS CLI.

One of the major benefits of AWS Vault is that it stores your IAM Credential Sets within your Operating Systems’ own Secure Keystore (e.g. KeyChain, Windows Credential Manager, Kwallet etc.), and generates temporary credentials from those for your shell and applications to use.

After adding an IAM Credential Set (profile) in AWS Vault you can directly run programs such as the AWS CLI using these credentials sets. See below for an example.

#--- Install AWS Vault
$ choco install aws-vault

#--- Store AWS credentials for the "jonsmith" profile
$ aws-vault add default
Enter Access Key Id: ABDCDEFDASDASF
Enter Secret Key: ***************************
Added credentials to profile "default" in vault

#--- Execute a command (using temporary credentials)
$ aws-vault exec default -- aws s3 ls
bucket_1
bucket_2

 

The Problem is in the Prefix!

Noticed how we needed to run AWS CLI commands with the prefix “aws-vault exec [profile]”?

Having to constantly use the prefix might be a blocking factor sometimes. For example when using the AWS CLI from CI/CD Pipelines, VS Code Extensions etc.

Reading through the AWS Documentation I found out that it is possible to use an External Process to source the Credentials to the AWS CLI. You can find the article here.

[profile developer]
credential_process = /opt/bin/awscreds-custom --username johnsmith

Many of the bigger Corporate or Enterprise Companies use this feature to build their own Custom AWS CLI Credential Provider, and of course to avoid having developers store their cleartext IAM Access Secrets in files on their computers.

First I thought about writing my own Credential Source Provider for the AWS CLI. But when thinking about the design, I quickly discovered I would actually be re-inventing the wheel as I would probably end up writing an AWS Vault knockoff.

 

So, combining it all together?

  • Looking at the AWS Documentation it seems the AWS CLI is expecting an STDOUT Stream in Json Output that matches the following syntax:
{
  "Version": 1,
  "AccessKeyId": "an AWS access key",
  "SecretAccessKey": "your AWS secret access key",
  "SessionToken": "the AWS session token for temporary credentials", 
  "Expiration": "ISO8601 timestamp when the credentials expire"
}
  • When running AWS Vault with the Debug option (–debug) we can see that before running the actual command (exec), AWS Vault sets the Environment Variables for AWS CLI. (AWS_ACCESS_KEY_ID, AWS_SESSION_TOKEN, AWS_SESSION_EXPIRATION)
#--- Running AWS Vault with the Debug option
$ aws-vault --debug exec default aws s3 ls
2020/05/24 13:41:34 aws-vault v5.4.4
2020/05/24 13:41:34 [keyring] Considering backends: [wincred pass file]
2020/05/24 13:41:34 Loading config file C:\Users\sandbox\.aws\config
2020/05/24 13:41:34 Parsing config file C:\Users\sandbox\.aws\config
2020/05/24 13:41:34 profile default: using stored credentials
2020/05/24 13:41:34 profile default: using GetSessionToken
2020/05/24 13:41:34 Looking for sessions for default
2020/05/24 13:41:34 Looking up all keys in keyring
2020/05/24 13:41:34 Session "session,ZGVmYLVsdA,,1590334079" expires in 59m44.6242639s
2020/05/24 13:41:34 Re-using cached credentials ****************32EC generated from GetSessionToken, expires in 59m44.6232311s
2020/05/24 13:41:34 Setting subprocess env: AWS_DEFAULT_REGION=us-east-1, AWS_REGION=us-east-1
2020/05/24 13:41:34 Setting subprocess env: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
2020/05/24 13:41:34 Setting subprocess env: AWS_SESSION_TOKEN, AWS_SECURITY_TOKEN
2020/05/24 13:41:34 Setting subprocess env: AWS_SESSION_EXPIRATION
2020-05-24 12:23:19 bucket_1
2020-05-24 12:23:19 bucket_2

So wrapping it all up, we will be building a Custom AWS CLI Credential Provider using an intermediate process between AWS CLI and AWS Vault. This process will use the AWS ENV Variable set by AWS Vault and push them in a Json String using the STDOUT Stream to AWS CLI.

 

Building our Custom AWS CLI Credential Provider

  1. Register your AWS IAM Credentials with AWS Vault, using e.g. the Default Profile:
 #--- Add Default IAM Profile to AWS Vault
$ aws-vault add default
Enter Access Key Id: ABDCDEFDASDASF
Enter Secret Key: ***************************
Added credentials to profile "default" in vault

 

  1. Within AWS Vault’s installation directory we create our Custom AWS CLI Credential Provider:
  • Create File ‘%PROGRAMDATA%\chocolatey\bin\aws-vault-credprovider.cmd‘
@ECHO OFF
ECHO { "Version": 1, "AccessKeyId": "%AWS_ACCESS_KEY_ID%", "SecretAccessKey": "%AWS_SECRET_ACCESS_KEY%", "SessionToken": "%AWS_SESSION_TOKEN%" }

 

  1. In our AWS Configuration File we register our Custom AWS CLI Credential Provider for the concerning IAM Profiles, in this case we will be using the default IAM Profile:
  • Modify File ‘%USERPROFILE%\.aws\config’
[default]
credential_process = "C:\ProgramData\chocolatey\bin\aws-vault.exe" exec default "C:\ProgramData\chocolatey\bin\aws-vault-credprovider.cmd"
region = us-east-1

 

  1. Don’t forget to clean-up your previously entered cleartext credentials from the AWS CLI Credentials File. Because we have now securely stored them within our Operating System’s Secure Vault using AWS Vault.
  • Remove from File ‘%USERPROFILE%\.aws\credentials’
[default]
aws_access_key_id = ABDCDEFDASDASF
aws_secret_access_key = ***************************

 

Run it!

If all went well you should be able now to just run the AWS CLI without prefixing it with the AWS Vault.

#--- Run AWS CLI without the AWS Vault Prefix but with the Credential Provider in the Background
$ aws s3 ls
bucket_1
bucket_2

 

Of couse we’ve kept the demo as clean as possible, so it might seem we’ve left out some crucial security considerations. But it’s upto you guys to implement those in case you would need them!

SECOTRON, Thomas Geens
Tags