Injecting Secrets into a Kubernetes Cluster
Deeper Dive
For more in-depth information, check out our detailed documentation on the following topics:
Need any help?
If something in this tutorial isn't working as expected, feel free to contact our support team via Slack.
Below is a text-only guide for users based on the above video
Why use a Secrets Management Platform to Manage Kubernetes Secrets?
There are a few reasons to use an external secrets management platform for managing Kubernetes secrets:
- Kubernetes stores secrets unencrypted. Therefore, anyone with access to the cluster can decode the values.
- With secrets in so many scripts, repos, and config files it can cause bottlenecks and operational problems for an organization.
- It's better to keep secrets externally for modern app development where the app has no state locally and everything is kept in environment variables or external files to enable a more microservices-based architecture.
How does Akeyless Inject Secrets into Kubernetes?
Akeyless has a Kubernetes plugin with a webhook which listens for events and injects an executable into containers inside a pod which request secrets from Akeyless through annotations in the file. You can inject secrets using an init container or sidecar container.
Init Container
With an init container, the secret is pre-populated into the pod before the app starts as part of the pod lifecycle. The webhook looks for annotations that correspond to specific schema and adds an init container that authenticates. The app then reads secrets from Akeyless through environment variables.
Sidecar Container
With a sidecar container, another container runs alongside the init container which tracks change to secrets and you can configure the interval that it checks for any changes to secrets and inject secrets into the filesystem of the pod. This is good for when the secret changes or the application is long-lasting and needs to re-authenticate regularly to grab the secret.
Set Up a Secret in Akeyless
Prerequisites
- Ensure that you have a K8s cluster and Helm installed before you begin. Again, we are using Amazon EKS for this demo.
- A K8s authentication method (see here if you aren't sure).
- K8s v1.16 and later (to use admissionregistration.k8s.io/v1).
- MutatingAdmissionWebhook admission controllers enabled (should be by default). This topic provides a recommended set of admission controllers to enable in general.
Create a static secret or dynamic secret in Akeyless.
We will use the static secret we already have called MyFirstSecret:
Create a K8s authentication method.
We already have one set up called my-k8s-auth-method that we will use (if you don't have one, see this link):
Create a Role and associate the authentication method.
We already have it associates with the 'admin' role. Best practice is to associate with a role with tighter permissions.
Configure the Akeyless K8s Plugin
Install Helm. The Akeyless Helm chart is available here.
Run the following commands to install the Helm chart:
helm repo add akeyless https://akeylesslabs.github.io/helm-charts
helm repo update
Fetch the values.yaml file locally, and modify the access credential values in the file using the following command:
helm show values akeyless/akeyless-secrets-injection > values.yaml
Edit the file values.yaml on your desktop or using an editor such as 'vi'. For example:
vi values.yaml
Once inside the file, find the following values and edit them as follows:
Important
Make sure to use port 8080 in your Gateway URL.
AKEYLESS_URL: "https://vault.akeyless.io"
AKEYLESS_ACCESS_ID: "<your-k8s-auth-method-access-id>"
AKEYLESS_ACCESS_TYPE: "k8s"
AKEYLESS_K8S_AUTH_CONF_NAME: "<your-k8s-conf-name-in-the-gateway>"
AKEYLESS_API_GW_URL: "https://gateway.url:8080"
On your K8s cluster, create a new namespace (we called it Akeyless):
kubectl create namespace akeyless
kubectl label namespace akeyless name=akeyless
Deploy the Helm chart to the namespace:
helm install aks akeyless/akeyless-secrets-injection --namespace akeyless -f values.yaml
Validate the deployment state:
kubectl get all -n akeyless
Output should look something like this:
kubectl get all -n akeyless
NAME READY STATUS RESTARTS AGE
pod/aks-akeyless-secrets-injection-77c857d496-r5xth 1/1 Running 1 (73s ago) 1d
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/aks-akeyless-secrets-injection ClusterIP 10.97.228.133 <none> 443/TCP 1d
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/aks-akeyless-secrets-injection 1/1 1 1 1d
NAME DESIRED CURRENT READY AGE
replicaset.apps/aks-akeyless-secrets-injection-77c857d496 1 1 1 1d
Create the init Container and Inject a Secret at Runtime
Note
In this demo, we are using the basic environment variable secret injection option. You can also inject multiple files or even a folder. For more info, check out the docs.
Create a file called env.yaml with the following command which will enable the Akeyless webhook and env annotations to inject 'MyFirstSecret' into the pod's filesystem:
cat << EOF > env.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: test
spec:
replicas: 1
selector:
matchLabels:
app: hello-secrets
template:
metadata:
labels:
app: hello-secrets
annotations:
akeyless/enabled: "true"
spec:
containers:
- name: alpine
image: alpine
command:
- "sh"
- "-c"
- "echo $MY_SECRET && echo going to sleep... && sleep 10000"
env:
- name: MY_SECRET
value: akeyless:/MyFirstSecret
EOF
Then run the following command to run it:
kubectl apply -f env.yaml
This will create a new pod called 'test-...' which you can find by running:
kubectl get pods
Copy the full pod name and run:
kubectl logs test-...
You should see output similar to the following showing the Secret Value of the secret inside your pod:
Defaulted container "alpine" out of: alpine, init (init)
<your secret value>
going to sleep...
Sidecar Mode
Note
Sidecar mode will only work with file injection mode.
Create a file called sidecar.yaml with the following command which will enable the Akeyless webhook, sidecar, and injectfile annotations to inject 'MyFirstSecret' into the pod's filesystem as a file:
cat << EOF > sidecar.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: test-file
spec:
replicas: 1
selector:
matchLabels:
app: file-secrets
template:
metadata:
labels:
app: file-secrets
annotations:
akeyless/enabled: "true"
akeyless/inject_file: "/MyFirstSecret|location=/secrets/secretsVersion.json"
akeyless/side_car_enabled: "true"
akeyless/side_car_refresh_interval: "30s"
akeyless/side_car_versions_to_retrieve: "2"
spec:
containers:
- name: alpine
image: alpine
command:
- "sh"
- "-c"
- "cat /secrets/secretsVersion.json && echo going to sleep... && sleep 10000"
EOF
Then run the following command to run it:
kubectl apply -f sidecar.yaml
This will create a new pod called 'test-file-...' which you can find by running:
kubectl get pods
Copy the full pod name and run:
kubectl logs test-file-... -c alpine
Because of the annotation akeyless/side_car_versions_to_retrieve: "2", you should see output similar to the following showing the two latest versions of the Secret Value for the secret inside your pod:
[
{
"version": 2,
"secret_value": "your-latest-secret-value",
"creation_date": 1234567890
},
{
"version": 1,
"secret_value": "your-previous-secret-value",
"creation_date": 1234567890
}
]going to sleep...
For an in-depth list of annotations and some troubleshooting info, see the docs.
Updated 7 months ago