Managing Secrets in CircleCI Pipelines

๐Ÿ“˜

Deeper Dive

For more in-depth information, check out our detailed documentation on the following topics:

CircleCI Plugin

Oauth2.0/JWT Auth Method

๐Ÿ‘

Need any help?

If something in this tutorial isn't working as expected, feel free to contact our support team via Slack.

Below is a text-only guide for users based on the above video

The Akeyless plugin for CircleCI enables a secure, easy, and integrative way to fetch Secrets into CircleCI pipelines, either integrating the native CircleCI short-lived OIDC authentication tokens, or using any other Authentication Methods with Akeyless native RBAC. In this demo, we will walk through how to do this with Akeyless to update a MySQL database table.

Prerequisites

  1. A GitHub, GitLab, or Bitbucket project set up in CircleCI.
  2. Permissions to create a CircleCI context that will be used to secure and share environment variables across projects.

Set Up Authentication Method

The first step in this process is to ensure we have a CircleCI Auth Method set up in our console (video on basic Auth Method creation here).

OpenID Connect tokens

In CircleCI jobs that use at least one context, the OpenID Connect ID token is available in the environment variable $CIRCLE_OIDC_TOKEN. The OpenID Provider is unique to your organization. The URL is https://oidc.circleci.com/org/ORGANIZATION_ID, where ORGANIZATION_ID is the organization ID (a universally unique identifier) representing your organization.

Go to your CircleCI web application. In the left-side menu, click "Organization Settings".

You will then be brought to the Overview section.

Create an Auth Method via Akeyless Console

Go back to the console and create your CircleCI Auth Method by clicking "Users & Auth Methods" > "New" > "OAuth 2.0 / JWT".

Then add the following information:

Name: Give the Auth Method a name.

JWKs URL: The URL to the JWKS that contains the public keys that should be used for JWT verification. In our example, we will use https://gitlab.com/-/jwks.

Unique Identifier: A unique claim name that contains details uniquely identifying the request. In the following example, we will use the GitLab user_login claim.

JWT TTL: Choose the length of time the token will be available for use (in minutes).

Require Sub Claim on role association: Tick this box to ensure a sub-claim is required.

Click 'Save' to save the Auth Method.

Create Auth Method via Akeyless CLI

You can also create the Auth Method using the CLI as follows:

akeyless create-auth-method-oauth2 --name CircleCIAuth \ 
--jwks-uri https://oidc.circleci.com/org/<ORGANIZATION ID>/.well-known/jwks-pub.json \
--unique-identifier iss \
--force-sub-claims

Create an Access Role

Create Access Role via Akeyless console

Next, create a dedicated Access Role (video on Access Role creation here). Please note that you will assign it the necessary permissions at a later stage.

In your console, click "Access Roles" > "New" and give it a Name.

Next, Associate the CircleCIAuth Auth Method created earlier and add the Sub-Claim iss=https://oidc.circleci.com/org/<ORGANIZATION_ID>.

Then, grant read and list permissions for "Secrets & Keys".

Create Access Role via Akeyless CLI

Create a role.

akeyless create-role --name CircleCIRole

Associate your new Role with the created Authentication Method and add the Sub-Claim:

akeyless assoc-role-am --role-name \CircleCIRole \
--am-name \CircleCIAuth \
--sub-claims iss=https://oidc.circleci.com/org/<ORGANIZATION_ID>

๐Ÿšง

Sub Claims:

It is mandatory to add an appropriate Sub Claim based on the claims available in the CircleCI documentation to prevent access of unauthorized users. This can also be used to limit access to specific workflows as described on the same CircleCI page under Additional Claims.

Grant read and list permissions for Secrets & Keys.

akeyless set-role-rule --role-name \CircleCIRole \
--path /'*' \
--capability read --capability list

CircleCI Global Configuration

Instead of checking your Auth Method access-id, or your Gateway URL into version control, it's more secure to store them in CircleCI environment variables.

In your CircleCI Project, go to Project Settings > Environment variables > Add Environment Variable.

Create an environment variable in CircleCI called ACCESS_ID and store your Auth Method's access-id in it.

In jobs using a context, CircleCI provides OpenID Connect ID (OIDC) tokens in environment variables. A job can use these tokens to access Akeyless without storing long-lived credentials in CircleCI.

Go to Organization Settings > Contexts > Add a context andnName it akeyless. We will add this context to a job later by adding the context key to the workflows section of our circleci/config.yml file.

Example Usage

In this example, we connected CircleCI to our GitLab repo and we're running a set of commands in order to update a MySQL database.

Open your CircleCI project and create/update your .circleci/config.yml file for CircleCI.

# Use the latest 2.1 version of CircleCI pipeline process engine.
# See: https://circleci.com/docs/2.0/configuration-reference
version: 2.1

# Define a job to be invoked later in a workflow.
# See: https://circleci.com/docs/2.0/configuration-reference/#jobs
jobs:
  mysql:
    docker:
      - image: 'akeyless/ci_base:latest-alpine'
    # Add steps to the job
    # See: https://circleci.com/docs/2.0/configuration-reference/#steps
    steps:
      - checkout
      - run:
          name: "Authenticate To Akeyless"
          command: export TOKEN=$(akeyless auth --access-id $ACCESS_ID --access-type jwt --jwt $CIRCLE_OIDC_TOKEN) >> $BASH_ENV
      - run:
          name: "Fetch Akeyless secrets and update MySQL DB"
          command: |
            export DATA=$(akeyless get-dynamic-secret-value -n mysqlDS)
            export RSA=$(akeyless get-secret-value -n jeremy-demo)
            export REMOTE_HOST=$(akeyless get-secret-value -n remote-host)
            export LC_user=$(echo $DATA | jq -r .id)
            export LC_pass=$(echo $DATA | jq -r .password)
            echo $RSA | base64 -d >> jeremy-demo.pem
            chmod 600 jeremy-demo.pem
            apk add --update openssh-client
            scp -o StrictHostKeyChecking=no -i "jeremy-demo.pem" employee.sh ubuntu@$REMOTE_HOST:~/.
            ssh -o "SendEnv LC_*" -o StrictHostKeyChecking=no -i "jeremy-demo.pem" ubuntu@$REMOTE_HOST -t "bash employee.sh"
            echo "Database Employees updated!"

# Invoke jobs via workflows
# See: https://circleci.com/docs/2.0/configuration-reference/#workflows
workflows:
  mysql-workflow:
    jobs:
      - mysql:
          context:
            - akeyless

Here is our very simplistic employee.sh file, found the main folder of our project, which adds a new employee into the table using our temporary credentials from the MySQL Dynamic Secret (LC_user, LC_pass):

#!/bin/bash
mysql -u$LC_user -p$LC_pass -e 'USE Employees; INSERT INTO employees (First_Name,Last_Name) VALUES ("Steve","Rogers");'

Here is what's happening in this example:

  1. We pull the Akeyless _ci_base_ Docker image.
  2. Authenticate to Akeyless by running `akeyless auth` with the Access ID we stored as an environment variable as well as the CircleCI OIDC token.
  3. Fetch the secrets and update the MySQL database by running a list of commands where we:
    1. Export our Dynamic Secret, which we grab from our MySQL Dynamic Secret Provider, (see video on creating and using dynamic secrets as an Env Variable.
    2. Export our private key, by running `akeyless get secret value`, where the private key is sitting within our Akeyless account, as an Env Variable.
    3. Export the remote host by running `akeyless get secret value` grabbing the remote host secret from Akeyless that I saved, which is the host where our MySQL database is located.
    4. Then, we grab just the temp username from this Dynamic secret using `jq` because this Dynamic Secret Value gives us both an ID as well as a password. So we're going to split those pieces of information.
    5. Next, we grab the temp password
    6. We then need to decode our base64 encoded private key and put it into a pem file which we then give the proper permissions.
    7. Next, we add and update the OpenSSH client within the machine.
    8. Then, we run our SSH copy command to copy the employee.sh file which updates the database.
    9. Next, SSH into the remote host sending the values of the username and the password in order to log in and then run the `employee.sh` file which will have those environment variables for the username and password to log into the MySQL database.
    10. And we just echo that the database called `employees` was updated.
    11. The last part of the config is to invoke jobs via workflows where we call the `akeyless` context that we created earlier.

We then Commit our changes in GitLab:

From there, CircleCI starts a new Workflow to run what we added in the config file.

Once it is successful, you will see all green status indications:

And our secrets are also not shown anywhere in the job:

We then can see that our database was also updated: