hub: audit destination credentials inline only (0.175.0) (#227)

Destinations take credentials only as inline values: s3.accessKey /
s3.secretKey, bearerToken and secretHeaders.<name>.value. credentialsSecret,
bearerTokenSecret and secretHeaders secretName/secretKey are removed from the
values and README and fail the render with the inline field to use.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

charts/hub/Chart.yaml

@@ -16,7 +16,7 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.174.0
+version: 0.175.0
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to

charts/hub/README.md

@@ -902,7 +902,7 @@ kerberospipeline:
 | `kerberoshub.audit.dispatchInterval` | How often the mounted destination configuration is reloaded and eligible destinations are polled. | `"5s"` |
 | `kerberoshub.audit.leaseDuration` | Per-destination lease duration. It must exceed every destination timeout by at least 30 seconds. | `"2m"` |
 | `kerberoshub.audit.terminationGracePeriodSeconds` | Pod termination grace period. Keep this longer than `leaseDuration` so an in-flight cycle can finish. | `150` |
-| `kerberoshub.audit.destinations` | Audit destinations: webhooks that receive batches as JSON, or S3 buckets that archive them. Every entry is validated when the chart renders. Inline credentials go into Secret `hub-audit-<id>-credentials`; changes to it take effect within a few minutes without a restart. Delete a Secret with that name that the chart did not create before setting inline credentials. Delivery semantics and per-provider S3 settings: https://learn.uug.ai/docs/hub/audit-events/#export-audit-events. | `[]` |
+| `kerberoshub.audit.destinations` | Audit destinations: webhooks that receive batches as JSON, or S3 buckets that archive them. Every entry is validated when the chart renders. Credentials (`bearerToken`, `secretHeaders.<name>.value`, `s3.accessKey` / `s3.secretKey`) are set inline; the chart stores them in Secret `hub-audit-<id>-credentials`, and changes take effect within a few minutes without a restart. Delivery semantics and per-provider S3 settings: https://learn.uug.ai/docs/hub/audit-events/#export-audit-events. | `[]` |
 | `kerberoshub.audit.destinations[].id` | Required. Unique name, `^[a-z][a-z0-9-]{0,62}$`. The delivery checkpoint is stored under it, so renaming starts a new destination. | |
 | `kerberoshub.audit.destinations[].type` | Optional. `webhook` or `s3`. | `webhook` |
 | `kerberoshub.audit.destinations[].enabled` | Optional. `false` pauses delivery and keeps the checkpoint. | `true` |
@@ -913,17 +913,15 @@ kerberospipeline:
 | `kerberoshub.audit.destinations[].filters` | Optional. Lists `organisationIds`, `projectIds` (ObjectIds), `actions`, `categories` and `outcomes` (`success`, `failure`). An event must match every list that is set. | all events |
 | `kerberoshub.audit.destinations[].url` | Required for webhooks. `http` or `https` endpoint that receives each batch. | |
 | `kerberoshub.audit.destinations[].headers` | Optional, webhooks. Public request headers. `Authorization` and `Idempotency-Key` are reserved. | `{}` |
-| `kerberoshub.audit.destinations[].bearerToken` | Optional, webhooks. Sent as `Authorization: Bearer <token>`. Stored in Secret `hub-audit-<id>-credentials`. Use this or `bearerTokenSecret`. | |
-| `kerberoshub.audit.destinations[].bearerTokenSecret` | Optional, webhooks. Existing Secret with the bearer token: `name` (required) and `key` (default `token`). | |
-| `kerberoshub.audit.destinations[].secretHeaders.<name>` | Optional, webhooks. Header with a secret value: `value` (stored in Secret `hub-audit-<id>-credentials`), or `secretName` and `secretKey` of an existing Secret. | |
+| `kerberoshub.audit.destinations[].bearerToken` | Optional, webhooks. Sent as `Authorization: Bearer <token>`. | |
+| `kerberoshub.audit.destinations[].secretHeaders.<name>.value` | Optional, webhooks. Header with a secret value, such as an API key. | |
 | `kerberoshub.audit.destinations[].s3.bucket` | Required for S3. Bucket with versioning, Object Lock and a `COMPLIANCE` default retention. | |
 | `kerberoshub.audit.destinations[].s3.region` | Required for S3, also for S3-compatible stores. | |
 | `kerberoshub.audit.destinations[].s3.prefix` | Optional. Key prefix; objects are written as `<prefix>/<id>/yyyy/mm/dd/<firstId>-<lastId>.ndjson.gz`. | |
 | `kerberoshub.audit.destinations[].s3.endpoint` | Optional. Base URL of an S3-compatible store, without the bucket. Omit for AWS S3. | |
 | `kerberoshub.audit.destinations[].s3.forcePathStyle` | Optional. Path-style URLs, usually needed for Ceph RGW, MinIO and Garage. | `false` |
 | `kerberoshub.audit.destinations[].s3.allowUnlockedBucket` | Optional. Skips the Object Lock check for stores without Object Lock, such as Garage; archived batches are then deletable. | `false` |
-| `kerberoshub.audit.destinations[].s3.accessKey` / `kerberoshub.audit.destinations[].s3.secretKey` | Optional, both or neither, not empty. Stored in Secret `hub-audit-<id>-credentials`. Without credentials the pod's AWS identity is used (AWS S3 only). Use these or `credentialsSecret`. | |
-| `kerberoshub.audit.destinations[].credentialsSecret` | Optional, S3. Existing Secret with the S3 keys: `name` (required), `accessKeyIdKey` (default `access-key-id`) and `secretAccessKeyKey` (default `secret-access-key`). The key fields name keys inside the Secret. | |
+| `kerberoshub.audit.destinations[].s3.accessKey` / `kerberoshub.audit.destinations[].s3.secretKey` | Optional, both or neither, not empty. Without them the pod's AWS identity is used (AWS S3 only). | |
 | `kerberoshub.audit.serviceAccount.create` | Create a ServiceAccount used only by hub-audit, for example for S3 access through IRSA. | `false` |
 | `kerberoshub.audit.serviceAccount.name` | Name of the hub-audit ServiceAccount; with `create: false`, an existing ServiceAccount to use. Defaults to `hub-audit` when created; when empty and not created, `kerberoshub.serviceAccount` applies. | `""` |
 | `kerberoshub.audit.serviceAccount.annotations` | Annotations for the hub-audit ServiceAccount, such as `eks.amazonaws.com/role-arn`. | `{}` |

charts/hub/templates/_helpers.tpl

@@ -110,25 +110,29 @@ Usage: {{- include "hub.opentelemetry.samplerEnv" $ | trim | nindent 8 }}
 {{/*
 hub.audit.destinations resolves the audit destinations into JSON. Inline
 credentials (s3.accessKey / s3.secretKey on S3 destinations, bearerToken and
-secretHeaders.<name>.value on webhooks) are turned into a chart-created Secret named
-hub-audit-<id>-credentials, so every template below reads one shape:
-credentialsSecret / bearerTokenSecret / secretHeaders with create: true and the
-values to store.
+secretHeaders.<name>.value on webhooks) are the only way to pass credentials.
+They are stored in a chart-created Secret named hub-audit-<id>-credentials;
+the internal credentialsSecret / bearerTokenSecret / secretHeaders entries
+below describe that Secret for the other templates.
 Use: fromJsonArray (include "hub.audit.destinations" .)
 */}}
 {{- define "hub.audit.destinations" -}}
 {{- $resolved := list -}}
 {{- range $index, $original := .Values.kerberoshub.audit.destinations -}}
   {{- $destination := deepCopy $original -}}
+  {{- $label := default (printf "%d" $index) $destination.id -}}
+  {{- if hasKey $destination "credentialsSecret" -}}
+    {{- fail (printf "audit destination %s: credentialsSecret is not supported; set s3.accessKey and s3.secretKey" $label) -}}
+  {{- end -}}
+  {{- if hasKey $destination "bearerTokenSecret" -}}
+    {{- fail (printf "audit destination %s: bearerTokenSecret is not supported; set bearerToken" $label) -}}
+  {{- end -}}
   {{- $s3 := default dict $destination.s3 -}}
   {{- if or (hasKey $s3 "accessKey") (hasKey $s3 "secretKey") -}}
     {{- $id := required (printf "id is required for audit destination %d" $index) $destination.id -}}
     {{- if not (and $s3.accessKey $s3.secretKey) -}}
       {{- fail (printf "audit destination %s must set both s3.accessKey and s3.secretKey, and neither may be empty" $id) -}}
     {{- end -}}
-    {{- if $destination.credentialsSecret -}}
-      {{- fail (printf "audit destination %s sets s3.accessKey/secretKey and credentialsSecret; use one of them" $id) -}}
-    {{- end -}}
     {{- $_ := set $destination "credentialsSecret" (dict
         "name" (printf "hub-audit-%s-credentials" $id)
         "create" true
@@ -142,9 +146,6 @@ Use: fromJsonArray (include "hub.audit.destinations" .)
     {{- if not $destination.bearerToken -}}
       {{- fail (printf "audit destination %s sets an empty bearerToken" $id) -}}
     {{- end -}}
-    {{- if $destination.bearerTokenSecret -}}
-      {{- fail (printf "audit destination %s sets bearerToken and bearerTokenSecret; use one of them" $id) -}}
-    {{- end -}}
     {{- $_ := set $destination "bearerTokenSecret" (dict
         "name" (printf "hub-audit-%s-credentials" $id)
         "key" "bearer-token"
@@ -153,18 +154,16 @@ Use: fromJsonArray (include "hub.audit.destinations" .)
     {{- $_ := unset $destination "bearerToken" -}}
   {{- end -}}
   {{- range $header, $secret := (default dict $destination.secretHeaders) -}}
-    {{- if and (kindIs "map" $secret) (hasKey $secret "value") -}}
-      {{- $id := required (printf "id is required for audit destination %d" $index) $destination.id -}}
-      {{- if not $secret.value -}}
-        {{- fail (printf "audit destination %s header %s sets an empty value" $id $header) -}}
-      {{- end -}}
-      {{- if or $secret.secretName $secret.secretKey -}}
-        {{- fail (printf "audit destination %s header %s sets value and secretName/secretKey; use one of them" $id $header) -}}
-      {{- end -}}
-      {{- $_ := set $secret "secretName" (printf "hub-audit-%s-credentials" $id) -}}
-      {{- $_ := set $secret "secretKey" (printf "header-%s" (sha256sum $header | trunc 12)) -}}
-      {{- $_ := set $secret "create" true -}}
+    {{- if or (not (kindIs "map" $secret)) (not (hasKey $secret "value")) (hasKey $secret "secretName") (hasKey $secret "secretKey") -}}
+      {{- fail (printf "audit destination %s header %s: set secretHeaders.%s.value" $label $header $header) -}}
     {{- end -}}
+    {{- $id := required (printf "id is required for audit destination %d" $index) $destination.id -}}
+    {{- if not $secret.value -}}
+      {{- fail (printf "audit destination %s header %s sets an empty value" $id $header) -}}
+    {{- end -}}
+    {{- $_ := set $secret "secretName" (printf "hub-audit-%s-credentials" $id) -}}
+    {{- $_ := set $secret "secretKey" (printf "header-%s" (sha256sum $header | trunc 12)) -}}
+    {{- $_ := set $secret "create" true -}}
   {{- end -}}
   {{- $resolved = append $resolved $destination -}}
 {{- end -}}

charts/hub/templates/configMaps/configmap-hub-audit.yaml

@@ -58,9 +58,6 @@
     {{- $credentials := default dict $destination.credentialsSecret -}}
     {{- if gt (len $credentials) 0 -}}
       {{- $secretName := required (printf "credentialsSecret.name is required for destination %s" $id) $credentials.name -}}
-      {{- if and (not $credentials.create) (or $credentials.accessKeyId $credentials.secretAccessKey) -}}
-        {{- fail (printf "audit destination %s sets credentialsSecret.accessKeyId/secretAccessKey without create: true; set create: true to let the chart create Secret %s from them, or remove them and create the Secret yourself" $id $secretName) -}}
-      {{- end -}}
       {{- $_ := set $settings "accessKeyIdFile" (printf "%s/access-key-id-%s" $secretRoot (sha256sum (printf "%s:%s" $secretName (default "access-key-id" $credentials.accessKeyIdKey)) | trunc 12)) -}}
       {{- $_ := set $settings "secretAccessKeyFile" (printf "%s/secret-access-key-%s" $secretRoot (sha256sum (printf "%s:%s" $secretName (default "secret-access-key" $credentials.secretAccessKeyKey)) | trunc 12)) -}}
     {{- end -}}

charts/hub/templates/kerberos-hub/hub-audit-credentials-secret.yaml

@@ -1,8 +1,7 @@
 {{- if .Values.audit.enabled }}
 {{- /*
-Secrets the chart creates for audit destinations: inline credentials
-(s3.accessKey / s3.secretKey, bearerToken, secretHeaders.<name>.value) and credentialsSecret.create: true.
-Destinations that reference an existing Secret are not rendered here.
+Secret the chart creates per audit destination from its inline credentials
+(s3.accessKey / s3.secretKey, bearerToken, secretHeaders.<name>.value).
 */}}
 {{- $created := dict }}
 {{- range $index, $destination := fromJsonArray (include "hub.audit.destinations" .) }}
@@ -32,7 +31,7 @@ Destinations that reference an existing Secret are not rendered here.
 {{- end }}
 {{- if $name }}
 {{- if hasKey $created $name }}
-{{- fail (printf "audit destinations %s and %s both create Secret %q; give each its own credentialsSecret.name, or set create: false on one and reuse the Secret" (get $created $name) $destination.id $name) }}
+{{- fail (printf "audit destinations %s and %s both create Secret %q" (get $created $name) $destination.id $name) }}
 {{- end }}
 {{- $_ := set $created $name $destination.id }}
 ---

charts/hub/values.yaml

@@ -861,7 +861,7 @@ kerberoshub:
         cpu: 10m
   # Dispatches canonical audit_events to one or more destinations: webhook
   # sinks, or S3 Object Lock buckets for immutable storage.
-  # Credentials are always read from existing Secrets and never stored here.
+  # Destination credentials are set inline on each destination.
   audit:
     repository: ghcr.io/uug-ai/hub-audit
     pullPolicy: IfNotPresent
@@ -910,14 +910,6 @@ kerberoshub:
     #   secretHeaders:
     #     X-Api-Key:
     #       value: "xxx"
-    #   # Or reference an existing Secret instead of the inline values:
-    #   # bearerTokenSecret:
-    #   #   name: audit-primary-siem # required
-    #   #   key: token # optional, default token
-    #   # secretHeaders:
-    #   #   X-Api-Key:
-    #   #     secretName: audit-primary-siem # required
-    #   #     secretKey: api-key # required
     #   filters: # optional, default all events; an event must match every list set
     #     organisationIds: []
     #     projectIds: []
@@ -949,14 +941,8 @@ kerberoshub:
     #     # in Secret hub-audit-<id>-credentials and mounts them.
     #     accessKey: "xxx"
     #     secretKey: "xxx"
-    #   # Or reference an existing Secret instead of s3.accessKey/secretKey.
-    #   # accessKeyIdKey/secretAccessKeyKey name keys inside that Secret.
-    #   # credentialsSecret:
-    #   #   name: audit-immutable-archive # required
-    #   #   accessKeyIdKey: access-key-id # optional, default access-key-id
-    #   #   secretAccessKeyKey: secret-access-key # optional, default secret-access-key
-    #   # Without either, hub-audit uses the pod's AWS identity (IRSA via
-    #   # kerberoshub.audit.serviceAccount, or an instance profile): AWS S3 only.
+    #     # Without them, hub-audit uses the pod's AWS identity (IRSA via
+    #     # kerberoshub.audit.serviceAccount, or an instance profile): AWS S3 only.
   # hub-workflows is the standalone, queue-driven workflow engine. It consumes
   # pipeline events and dispatches the stages declared in its workflow
   # definitions, tracking each run in its own `workflow_runs` collection. It shares events