Sync Secrets to K8s with External Secrets Operator

👍

Need any help?

If something in this tutorial isn't working as expected, feel free to contact our support team via Slack.

Below is a text-only guide for users based on the above video

Introduction & Architecture Overview

Integrating the Akeyless Platform with the External Secrets Operator (ESO) allows organizations to keep familiar Kubernetes workflows while strengthening security posture with centralized lifecycle management, robust encryption, and fine-grained access control. Akeyless acts as the central source of truth, managing key rotation, dynamic credential generation, audit logging, and securing secrets at rest with Zero-Knowledge DFC encryption.

Through custom resources such as ExternalSecret, secrets can be fetched from Akeyless directly into Kubernetes. Complementing this, PushSecret allows native Kubernetes secrets to be pushed back into Akeyless, enabling full bidirectional sync for all secret types—including static, dynamic, and rotated secrets, as well as certificates.

While ESO synchronizes these values into native Kubernetes Secrets in etcd, full end-to-end security requires enabling Kubernetes Encryption at Rest on the control plane, ensuring data remains encrypted both within Akeyless and on disk within the cluster.

Installing the External Secrets Operator

Install the External Secrets Operator into your Kubernetes cluster via Helm.

helm repo add external-secrets https://charts.external-secrets.io
helm repo update
helm install external-secrets external-secrets/external-secrets \
  --namespace external-secrets \
  --create-namespace

Verify that the operator pods are running:

kubectl get pods -n external-secrets

Namespace & Workload Identity Setup

Create an isolated application namespace and a dedicated ServiceAccount. The ServiceAccount token acts as the identity that Akeyless validates.

kubectl create namespace akeyless-eso-demo

Create akeyless-sa.yaml:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: akeyless-sa
  namespace: akeyless-eso-demo

Apply the ServiceAccount:

kubectl apply -f akeyless-sa.yaml

Akeyless Authentication & Access Role Configuration

Configure authentication and permissions inside the Akeyless Console:

1. Create Kubernetes Auth Method:

• Navigate to Auth Methods > New > Kubernetes.
• Note the generated Access ID (p-123456).
• Specify your cluster's Kubernetes Auth configuration name (your-k8s-conf-name).
• Enforce Least Privilege: Restrict the authentication method by adding a sub-claim rule targeting the application namespace (namespace=akeyless-eso-demo).

2. Configure Role & Role Rules:

• Create an Access Role named k8s-role and associate it with the Kubernetes Auth Method.
• Under Role Rules, grant Read and List permissions on paths required for pulling secrets (e.g., /test-secret, /devops/*, and /devopsadmin/*).
• Grant Create and Update permissions on /devops/* to support secret pushing.

Defining the SecretStore

Create a SecretStore Custom Resource in Kubernetes. The SecretStore serves as the bridge between Kubernetes and Akeyless, defining the Gateway endpoint, the Access ID, and the workload ServiceAccount used for JWT authentication.

Create secret-store.yaml:

apiVersion: external-secrets.io/v1
kind: SecretStore
metadata:
  name: akeyless-secret-store
  namespace: akeyless-eso-demo
spec:
  provider:
    akeyless:
      akeylessGWApiURL: "https://<YOUR_GW_URL>/api/v2"
      authSecretRef:
        kubernetesAuth:
          accessID: "p-123456"
          k8sConfName: "your-k8s-conf-name"
          serviceAccountRef:
            name: "akeyless-sa"

Apply the SecretStore:

kubectl apply -f secret-store.yaml

Verify the SecretStore status:

kubectl get secretstore akeyless-secret-store -n akeyless-eso-demo

Pulling Secrets: ExternalSecret Workflow Example

Synchronize static, dynamic, and rotated secrets from Akeyless into Kubernetes native Secret objects using an ExternalSecret manifest.

Create external-secrets.yaml:

---
# 1. Static Secret Sync
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: static-secret-sync
  namespace: akeyless-eso-demo
spec:
  refreshInterval: "1h"
  secretStoreRef:
    name: akeyless-secret-store
    kind: SecretStore
  target:
    name: static-access-id
    creationPolicy: Owner
  data:
    - secretKey: access_id
      remoteRef:
        key: "/test-secret"
---
# 2. Dynamic Secret Sync
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: dynamic-secret-sync
  namespace: akeyless-eso-demo
spec:
  refreshInterval: "15m"
  secretStoreRef:
    name: akeyless-secret-store
    kind: SecretStore
  target:
    name: mysql-dynamic-creds
    creationPolicy: Owner
  data:
    - secretKey: username
      remoteRef:
        key: "/devops/mysql-dynamic-secret"
        property: user
    - secretKey: password
      remoteRef:
        key: "/devops/mysql-dynamic-secret"
        property: password
---
# 3. Rotated Secret Sync
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: rotated-secret-sync
  namespace: akeyless-eso-demo
spec:
  refreshInterval: "10m"
  secretStoreRef:
    name: akeyless-secret-store
    kind: SecretStore
  target:
    name: mysql-privileged-creds
    creationPolicy: Owner
  data:
    - secretKey: password
      remoteRef:
        key: "/devopsadmin/mysql-rotated-secret"

Apply the ExternalSecret configurations:

kubectl apply -f external-secrets.yaml

Verify that all three secret types are synchronized:

kubectl get externalsecrets -n akeyless-eso-demo

List the created native Kubernetes secrets:

kubectl get secrets -n akeyless-eso-demo

Retrieve and decode the payload of each native Kubernetes secret using kubectl and jq:

# 1. Inspect Static Secret payload
kubectl get secret static-access-id -n akeyless-eso-demo -o jsonpath='{.data}' | jq 'map_values(@base64d)'

# 2. Inspect Dynamic Secret credentials
kubectl get secret mysql-dynamic-creds -n akeyless-eso-demo -o jsonpath='{.data}' | jq 'map_values(@base64d)'

# 3. Inspect Rotated Secret payload
kubectl get secret mysql-privileged-creds -n akeyless-eso-demo -o jsonpath='{.data}' | jq 'map_values(@base64d)'

Validating Sub-Claim Enforcement (Unauthorized Access)

Demonstrate least-privilege security by attempting to fetch a secret from the default namespace, which does not match the configured sub-claim restriction (namespace=akeyless-eso-demo).

Step 1: Create Credentials and SecretStore in the Default Namespace

Apply the auth credentials secret directly into the default namespace:

kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: akeyless-k8s-creds
  namespace: default
type: Opaque
stringData:
  accessId: "p-123456"
  accessType: "k8s"
  accessTypeParam: "your-k8s-conf-name"
EOF

Apply the SecretStore into the default namespace referencing the default ServiceAccount:

kubectl apply -f - <<EOF
apiVersion: external-secrets.io/v1
kind: SecretStore
metadata:
  name: akeyless-secret-store
  namespace: default
spec:
  provider:
    akeyless:
      akeylessGWApiURL: "https://<YOUR_GW_URL>/api/v2"
      authSecretRef:
        kubernetesAuth:
          accessID: "p-123456"
          k8sConfName: "your-k8s-conf-name"
          serviceAccountRef:
            name: "default"
EOF

Step 2: Attempt to Sync a Secret via ExternalSecret

Create and apply the static-secret-sync-unauthorized resource in the default namespace:

kubectl apply -f - <<EOF
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: static-secret-sync-unauthorized
  namespace: default
spec:
  refreshInterval: "1h"
  secretStoreRef:
    name: akeyless-secret-store
    kind: SecretStore
  target:
    name: static-access-id-failed
    creationPolicy: Owner
  data:
    - secretKey: access_id
      remoteRef:
        key: "/test-secret"
EOF

Step 3: Verify the Rejection and Inspect Authentication Errors

Check the status of the unauthorized ExternalSecret:

kubectl get externalsecret static-secret-sync-unauthorized -n default

Expected Output: Status shows STATUS: SecretSyncedError and READY: False.

Inspect the controller event details to confirm sub-claim enforcement rejected the request:

kubectl describe externalsecret static-secret-sync-unauthorized -n default

Expected Error in Events:

Warning  UpdateFailed  ...  external-secrets  error processing spec.data[0] (key: /test-secret) failed to obtain item description: Desc: Failed to get item. Status 401 Unauthorized, Error: UnauthorizedAccess. unauthorized access for access id p-123456.

Pushing Secrets: PushSecret Workflow Example

Push a local Kubernetes Secret back into the Akeyless Platform using the PushSecret custom resource.

Prerequisites

Before pushing secrets, ensure that the Akeyless Access Role associated with your Kubernetes authentication method has the required write permissions. Navigate to Roles in the Akeyless Console, select your role, and verify that Create and Update permissions are enabled on the target remote path (for example, /devops/*).

Step 1: Create the Local Kubernetes Secret

kubectl create secret generic k8s-source-secret \
  --namespace akeyless-eso-demo \
  --from-literal=db_password='MySuperSecureLocalPassword123!'

Step 2: Define and Apply the PushSecret

Create push-secret.yaml:

apiVersion: external-secrets.io/v1alpha1
kind: PushSecret
metadata:
  name: pushsecret-to-akeyless
  namespace: akeyless-eso-demo
spec:
  refreshInterval: "10m"
  updatePolicy: Replace
  deletionPolicy: None
  secretStoreRefs:
    - name: akeyless-secret-store
      kind: SecretStore
  selector:
    secret:
      name: k8s-source-secret
  data:
    - match:
        secretKey: db_password
        remoteRef:
          remoteKey: "/devops/pushed-k8s-secret"

Apply the manifest:

kubectl apply -f push-secret.yaml

Step 3: Verify the Push Synchronization

Confirm that the PushSecret resource status is Synced:

kubectl get pushsecret pushsecret-to-akeyless -n akeyless-eso-demo

Verify that the secret exists in Akeyless:

• Via the Akeyless CLI:

akeyless get-secret-value --name "/devops/pushed-k8s-secret"

• Via the Akeyless Console UI:
  Navigate to Items in the left navigation sidebar, locate and open the devops folder, and select pushed-k8s-secret. Click the eye icon next to Secret Value to view the pushed plaintext payload (MySuperSecureLocalPassword123!).