deployment/base/vault

Vault base manifests

This directory holds the raw Kubernetes manifests for Vault. Vault receives recordings from Agents, stores recording metadata in MongoDB, and sends recording bytes to storage providers configured through the Vault UI.

Scope of this README. At the top level of this repository Vault is normally installed through Kustomize (see overlays/ and README.kustomize.md). This README documents the alternative: applying the manifests in this folder directly with kubectl. Use it when you want to install only Vault, inspect each object, or integrate these manifests into your own deployment tooling.

What gets deployed

File	Kind	Purpose
mongodb-configmap.yaml	ConfigMap	Supplies Vault's MongoDB connection, backend flavor, retry-write, and TLS settings.
kerberos-vault-deployment.yaml	Deployment	ConfigMap-backed Vault example selected by the base Kustomization.
kerberos-vault-deployment-with-secrets.yaml	Deployment	Alternative Secret-backed Vault example.
kerberos-vault-service.yaml	Service	Exposes Vault on NodePort 30080. A commented LoadBalancer variant is included.
data-filtering-deployment.yaml	Deployment	Optional YOLO data-filtering worker. The default manifest requests one NVIDIA GPU.
vault-runtime-configuration-configmap.yaml	ConfigMap	Plaintext local defaults used by the base Kustomization; managed-resource lists are empty.
vault-runtime-configuration-secret.yaml	Secret	Optional provider, integration, and account example for production secret handling.

The matching kerberos-vault namespace is defined in ../namespaces/kerberos-vault.yaml.

Hardened Vault image

Both deployment variants use the hardened Vault image ghcr.io/uug-ai/vault:v1.7.19. The Vault container runs as non-root UID/GID 65532, drops Linux capabilities, disables privilege escalation, uses the RuntimeDefault seccomp profile, and has a read-only root filesystem.

The image listens on container port 8080. The Service keeps its existing port 80 and NodePort 30080, forwarding to the named http container port. Startup, readiness, and liveness probes use /api/health.

The image uses /run/vault for temporary upload/finalization files. Each Deployment mounts a writable, disk-backed emptyDir there, limited to 1Gi; the pod fsGroup grants the non-root Vault process access. Size this scratch limit for expected concurrent uploads and available node ephemeral storage.

Size the writable scratch volume

There is no one-size-fits-all volume size: estimate how many full recordings Vault may be writing to scratch at the same time, and the maximum size of each.

• A regular HTTP upload is spooled to a temporary file. The default VAULT_MAX_PUBLISH_FILE_BYTES limit is 1 GiB per upload. If you change this limit, use the new value in your sizing estimate.
• Resumable tus chunks are stored in the configured object-storage provider, not in this volume. When an upload is finalized, Vault downloads the completed recording to a temporary file, so count concurrent tus finalizations and their maximum recording sizes.
• Set TUS_MAX_SIZE_BYTES to a positive byte limit to cap tus upload size. It is unlimited when unset. For example, 1073741824 limits each tus upload to 1 GiB. Keep this cap within the scratch capacity you have planned.

Use this conservative estimate, then round up to a convenient GiB value:

scratch size >= 1.5 x (
  concurrent HTTP uploads x maximum HTTP upload size
  + concurrent tus finalizations x maximum tus upload size
)

The 1.5 multiplier allows roughly 50% headroom. For example, allowing two simultaneous 1 GiB HTTP uploads and two simultaneous 1 GiB tus finalizations uses up to 4 GiB for those temporary files; choose a scratch limit of at least 6 GiB. If you only expect two full-file operations at once in total, a 3 GiB limit is a reasonable starting point. Increase it for larger files or more concurrent operations.

To apply the 6 GiB example, set the tus cap and scratch limit in the Vault container and pod template of the Deployment you use. Make the same changes to the Secret-backed Deployment if you use that variant:

spec:
  template:
    spec:
      containers:
        - name: vault
          env:
            - name: TUS_MAX_SIZE_BYTES
              value: "1073741824"
          volumeMounts:
            - name: vault-tmp
              mountPath: /run/vault
      volumes:
        - name: vault-tmp
          emptyDir:
            sizeLimit: 6Gi

Keep the existing GIN_MODE and other environment variables and the vault-tmp mount; merge these entries rather than replacing them. The disk-backed emptyDir is charged to node ephemeral storage: its sizeLimit is a ceiling, not a reservation. Make sure the node and your pod's ephemeral-storage requests/limits can accommodate scratch use as well as container logs and other temporary files. If the limit is exhausted, uploads or finalizations can fail. Apply the updated Deployment and check that the Vault pod becomes Ready before relying on the new limits.

Prerequisites

• A running Kubernetes cluster and kubectl configured to reach it.
• A MongoDB-compatible database reachable from the Vault pod.
• A storage provider such as MinIO, Amazon S3, Google Cloud Storage, or Azure Blob Storage. Configure it in the Vault UI or through the optional runtime configuration ConfigMap or Secret after deployment.
• An NVIDIA-capable node and device plugin only when deploying the optional data-filtering worker with its default resource settings.

MongoDB configuration

Vault imports every key from the mongodb ConfigMap through envFrom. The base manifest connects to the in-cluster MongoDB service at mongodb.mongodb.

Variable	Default here	Meaning
MONGODB_DATABASE_STORAGE	KerberosStorage	Database where Vault stores recording metadata and configuration.
MONGODB_URI	unset	Complete MongoDB connection URI. When set, it takes precedence over component settings.
MONGODB_HOST	mongodb.mongodb	MongoDB host and optional port for component-based configuration.
MONGODB_DATABASE_CREDENTIALS	admin	Authentication database or auth source.
MONGODB_USERNAME / MONGODB_PASSWORD	root / yourpassword	Database credentials. Replace these demo values before deployment.
MONGODB_FLAVOR	mongodb	Backend compatibility mode: mongodb or documentdb.
MONGODB_RETRY_WRITES	true	Enables retryable writes for MongoDB. Vault always disables them for DocumentDB.
MONGODB_TLS	false	Enables TLS for the database connection.
MONGODB_TLS_CA_FILE	empty	Path to a mounted PEM CA bundle. A non-empty value also enables TLS.
MONGODB_TLS_INSECURE_SKIP_VERIFY	false	Disables certificate and hostname verification. Use only for isolated local testing.

The ConfigMap also contains legacy Factory and Hub database names, but Vault uses MONGODB_DATABASE_STORAGE for its own data.

Warning

ConfigMaps are not appropriate for production credentials. Move the MongoDB username, password, or credential-bearing URI to a Kubernetes Secret in a production deployment and expose those keys to the Vault container.

MongoDB Atlas or another URI connection

Set MONGODB_URI in mongodb-configmap.yaml. The component settings are ignored when the URI is non-empty.

The Deployment's wait-for-mongodb-before-starup init container currently probes the base service name mongodb.mongodb:27017. When using Atlas or another external database, change that command to probe the external host and port, or replace it with a readiness mechanism suitable for your environment. Otherwise Vault will remain in Init even when its configured database is reachable.

DocumentDB or MongoDB with a custom CA

For AWS DocumentDB, use a connection URI, set the flavor to documentdb, disable retryable writes, and enable TLS:

MONGODB_URI: "mongodb://<username>:<password>@<endpoint>:27017/?replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false"
MONGODB_FLAVOR: "documentdb"
MONGODB_RETRY_WRITES: "false"
MONGODB_TLS: "true"
MONGODB_TLS_CA_FILE: "/certs/global-bundle.pem"
MONGODB_TLS_INSECURE_SKIP_VERIFY: "false"

Create a Secret from the trusted CA bundle:

kubectl create namespace kerberos-vault --dry-run=client -o yaml | kubectl apply -f -
kubectl create secret generic mongodb-ca \
  --from-file=global-bundle.pem \
  -n kerberos-vault

Then add the Secret volume and mount to kerberos-vault-deployment.yaml:

spec:
  template:
    spec:
      containers:
        - name: vault
          volumeMounts:
            - name: mongodb-ca
              mountPath: /certs
              readOnly: true
      volumes:
        - name: mongodb-ca
          secret:
            secretName: mongodb-ca

Also update or remove the MongoDB wait init-container as described above. For Amazon DocumentDB, the shared Vault database client defaults component-based authentication to SCRAM-SHA-1 and does not enable MongoDB Stable API.

Vault configuration

The selected runtime ConfigMap or Secret supplies these environment variables through envFrom. Only GIN_MODE remains directly on either Deployment:

Variable	Default here	Meaning
KERBEROS_LOGIN_USERNAME / KERBEROS_LOGIN_PASSWORD	root / kerberos	Vault UI login. Change these demo credentials.
MQTT_URI	tcp://mqtt.kerberos.io:1883	MQTT broker used for on-demand forwarding.
MQTT_USERNAME / MQTT_PASSWORD	empty	Optional MQTT credentials.
VAULT_STORAGE_PROVIDERS	unset	JSON array of runtime-managed storage providers.
VAULT_INTEGRATIONS	unset	JSON array of runtime-managed integrations.
VAULT_ACCOUNTS	unset	JSON array of runtime-managed Vault accounts.

Vault accounts, storage providers, and integrations can be supplied as JSON arrays through the three environment variables. Two alternative Deployment manifests show how to import them from a vault-runtime-configuration ConfigMap or Secret. Environment definitions are grayed out and read-only in Vault, override database definitions with the same name, and are loaded when the pod starts. Invalid definitions prevent startup. An enabled Vault integration automatically starts the forwarding worker. See README.configure.md.

The base kustomization.yaml includes the plaintext ConfigMap and standard Deployment so the demo login remains usable. The credential-bearing Secret and Secret-backed Deployment remain standalone:

• vault-runtime-configuration-configmap.yaml contains the demo Vault login and MQTT defaults. Its provider, integration, and account arrays are [], so those resources remain managed through the UI and database. ConfigMap values are plaintext.
• vault-runtime-configuration-secret.yaml stores the same eight variables with stringData, so values can be entered without manual base64 encoding. Use an external secret manager or encrypted GitOps workflow in production.

For direct kubectl installation, apply one matching pair in the Vault namespace. Replace every Secret example credential first. Both Deployment manifests manage Deployment/vault; choose one and do not apply both.

# Plaintext local-development variant:
kubectl apply -n kerberos-vault \
  -f ./vault-runtime-configuration-configmap.yaml
kubectl apply -n kerberos-vault \
  -f ./kerberos-vault-deployment.yaml

# Recommended Secret-backed variant:
kubectl apply -n kerberos-vault \
  -f ./vault-runtime-configuration-secret.yaml
kubectl apply -n kerberos-vault \
  -f ./kerberos-vault-deployment-with-secrets.yaml

Restart the Deployment after changing any value. Leave a key absent or set it to [] to use only database definitions for that resource type. Switching variants is also an apply: the selected manifest replaces the existing Deployment/vault definition.

Deploy with kubectl without Kustomize

The manifest files do not hard-code a namespace because Kustomize normally injects it. When applying them directly, target the namespace explicitly.

# 1. Create the namespace.
kubectl apply -f ../namespaces/kerberos-vault.yaml

# 2. Review the database settings, then create the ConfigMap.
kubectl apply -n kerberos-vault -f ./mongodb-configmap.yaml

# 3. Optionally apply the plaintext runtime configuration, then deploy Vault.
# For the Secret-backed alternative, use the matching pair documented above.
# kubectl apply -n kerberos-vault -f ./vault-runtime-configuration-configmap.yaml
kubectl apply -n kerberos-vault -f ./kerberos-vault-deployment.yaml
kubectl apply -n kerberos-vault -f ./kerberos-vault-service.yaml

Verify the rollout:

kubectl get pods,svc -n kerberos-vault
kubectl rollout status deployment/vault -n kerberos-vault
kubectl logs -n kerberos-vault deploy/vault

If the pod remains in Init, inspect the database wait container:

kubectl logs -n kerberos-vault deploy/vault \
  -c wait-for-mongodb-before-starup

Access the UI

With the NodePort service, Vault is reachable on port 30080 of any node:

kubectl get nodes -o wide
# Browse http://<node-ip>:30080

# Or use a local port-forward without exposing a node port.
kubectl port-forward -n kerberos-vault svc/vault-nodeport 8080:80
# Browse http://localhost:8080

Log in with the configured KERBEROS_LOGIN_USERNAME and KERBEROS_LOGIN_PASSWORD values. The base defaults are root and kerberos.

To use a cloud LoadBalancer, uncomment the vault-lb service at the bottom of kerberos-vault-service.yaml and remove or comment out the NodePort service.

Optional data filtering

The data-filtering worker is independent of the Vault Deployment. Before applying it, configure its queue, Vault credentials, model settings, and GPU resources in data-filtering-deployment.yaml.

kubectl apply -n kerberos-vault -f ./data-filtering-deployment.yaml
kubectl rollout status deployment/data-filtering -n kerberos-vault

Clusters without an NVIDIA device plugin must remove the nvidia.com/gpu requests and limits or leave this optional Deployment unapplied. See README.extensions.md for the integration flow.

Uninstall

kubectl delete -n kerberos-vault -f ./data-filtering-deployment.yaml --ignore-not-found
kubectl delete -n kerberos-vault -f ./kerberos-vault-service.yaml
kubectl delete -n kerberos-vault -f ./kerberos-vault-deployment.yaml
kubectl delete -n kerberos-vault -f ./mongodb-configmap.yaml
kubectl delete secret mongodb-ca -n kerberos-vault --ignore-not-found
kubectl delete -f ../namespaces/kerberos-vault.yaml