Managing Secrets in CircleCI Pipelines

📘

Deeper Dive

For more in-depth information, check out our detailed documentation on the following topics:

CircleCI Plugin

Oauth2.0/JWT Auth Method

👍

Need any help?

If something in this tutorial isn't working as expected, feel free to contact our support team via Slack.

Below is a text-only guide for users based on the above video

The Akeyless plugin for CircleCI enables a secure, easy, and integrative way to fetch Secrets into CircleCI pipelines, either integrating the native CircleCI short-lived OIDC authentication tokens, or using any other Authentication Methods with Akeyless native RBAC. In this demo, we will walk through how to do this with Akeyless to update a MySQL database table.

Prerequisites

1. A GitHub, GitLab, or Bitbucket project set up in CircleCI.
2. Permissions to create a CircleCI context that will be used to secure and share environment variables across projects.

Set Up Authentication Method

The first step in this process is to ensure we have a CircleCI Auth Method set up in our console (video on basic Auth Method creation here).

OpenID Connect tokens

In CircleCI jobs that use at least one context, the OpenID Connect ID token is available in the environment variable $CIRCLE_OIDC_TOKEN. The OpenID Provider is unique to your organization. The URL is https://oidc.circleci.com/org/ORGANIZATION_ID, where ORGANIZATION_ID is the organization ID (a universally unique identifier) representing your organization.

Go to your CircleCI web application. In the left-side menu, click "Organization Settings".

You will then be brought to the Overview section.

Create an Auth Method via Akeyless Console

Go back to the console and create your CircleCI Auth Method by clicking "Users & Auth Methods" > "New" > "OAuth 2.0 / JWT".

Then add the following information:

Name: Give the Auth Method a name.

JWKs URL: The URL to the JWKS that contains the public keys that should be used for JWT verification. In our example, we will use https://gitlab.com/-/jwks.

Unique Identifier: A unique claim name that contains details uniquely identifying the request. In the following example, we will use the GitLab user_login claim.

JWT TTL: Choose the length of time the token will be available for use (in minutes).

Require Sub Claim on role association: Tick this box to ensure a sub-claim is required.

Click 'Save' to save the Auth Method.

Create Auth Method via Akeyless CLI

You can also create the Auth Method using the CLI as follows:

akeyless create-auth-method-oauth2 --name CircleCIAuth \ 
--jwks-uri https://oidc.circleci.com/org/<ORGANIZATION ID>/.well-known/jwks-pub.json \
--unique-identifier iss \
--force-sub-claims

Create an Access Role

Create Access Role via Akeyless console

Next, create a dedicated Access Role (video on Access Role creation here). Please note that you will assign it the necessary permissions at a later stage.

In your console, click "Access Roles" > "New" and give it a Name.

Next, Associate the CircleCIAuth Auth Method created earlier and add the Sub-Claim iss=https://oidc.circleci.com/org/<ORGANIZATION_ID>.

Then, grant read and list permissions for "Secrets & Keys".

Create Access Role via Akeyless CLI

Create a role.

akeyless create-role --name CircleCIRole

Associate your new Role with the created Authentication Method and add the Sub-Claim:

akeyless assoc-role-am --role-name \CircleCIRole \
--am-name \CircleCIAuth \
--sub-claims iss=https://oidc.circleci.com/org/<ORGANIZATION_ID>

🚧

Sub Claims:

It is mandatory to add an appropriate Sub Claim based on the claims available in the CircleCI documentation to prevent access of unauthorized users. This can also be used to limit access to specific workflows as described on the same CircleCI page under Additional Claims.

Grant read and list permissions for Secrets & Keys.

akeyless set-role-rule --role-name \CircleCIRole \
--path /'*' \
--capability read --capability list

CircleCI Global Configuration

Instead of checking your Auth Method access-id, or your Gateway URL into version control, it's more secure to store them in CircleCI environment variables.

In your CircleCI Project, go to Project Settings > Environment variables > Add Environment Variable.

Create an environment variable in CircleCI called ACCESS_ID and store your Auth Method's access-id in it.

In jobs using a context, CircleCI provides OpenID Connect ID (OIDC) tokens in environment variables. A job can use these tokens to access Akeyless without storing long-lived credentials in CircleCI.

Go to Organization Settings > Contexts > Add a context andnName it akeyless. We will add this context to a job later by adding the context key to the workflows section of our circleci/config.yml file.

Example Usage

In this example, we connected CircleCI to our GitLab repo and we're running a set of commands in order to update a MySQL database.

Open your CircleCI project and create/update your .circleci/config.yml file for CircleCI.

# Use the latest 2.1 version of CircleCI pipeline process engine.
# See: https://circleci.com/docs/2.0/configuration-reference
version: 2.1

# Define a job to be invoked later in a workflow.
# See: https://circleci.com/docs/2.0/configuration-reference/#jobs
jobs:
  mysql:
    docker:
      - image: 'akeyless/ci_base:latest-alpine'
    # Add steps to the job
    # See: https://circleci.com/docs/2.0/configuration-reference/#steps
    steps:
      - checkout
      - run:
          name: "Authenticate To Akeyless"
          command: export TOKEN=$(akeyless auth --access-id $ACCESS_ID --access-type jwt --jwt $CIRCLE_OIDC_TOKEN) >> $BASH_ENV
      - run:
          name: "Fetch Akeyless secrets and update MySQL DB"
          command: |
            export DATA=$(akeyless get-dynamic-secret-value -n mysqlDS)
            export RSA=$(akeyless get-secret-value -n jeremy-demo)
            export REMOTE_HOST=$(akeyless get-secret-value -n remote-host)
            export LC_user=$(echo $DATA | jq -r .id)
            export LC_pass=$(echo $DATA | jq -r .password)
            echo $RSA | base64 -d >> jeremy-demo.pem
            chmod 600 jeremy-demo.pem
            apk add --update openssh-client
            scp -o StrictHostKeyChecking=no -i "jeremy-demo.pem" employee.sh ubuntu@$REMOTE_HOST:~/.
            ssh -o "SendEnv LC_*" -o StrictHostKeyChecking=no -i "jeremy-demo.pem" ubuntu@$REMOTE_HOST -t "bash employee.sh"
            echo "Database Employees updated!"

# Invoke jobs via workflows
# See: https://circleci.com/docs/2.0/configuration-reference/#workflows
workflows:
  mysql-workflow:
    jobs:
      - mysql:
          context:
            - akeyless

Here is our very simplistic employee.sh file, found the main folder of our project, which adds a new employee into the table using our temporary credentials from the MySQL Dynamic Secret (LC_user, LC_pass):

#!/bin/bash
mysql -u$LC_user -p$LC_pass -e 'USE Employees; INSERT INTO employees (First_Name,Last_Name) VALUES ("Steve","Rogers");'

Here is what's happening in this example:

We then Commit our changes in GitLab:

From there, CircleCI starts a new Workflow to run what we added in the config file.

Once it is successful, you will see all green status indications:

And our secrets are also not shown anywhere in the job:

We then can see that our database was also updated: