Align promoted frontend values with hardened Hub chart

Remove the invalid Docker Hub prefix on GHCR images, use frontend port 8080 and named NodePort routing, and add a pre-infrastructure runtime preflight for chart 0.170.0+.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

.devcontainer/devcontainer.json

@@ -3,6 +3,7 @@
 	"dockerFile": "Dockerfile",
     "runArgs": ["--privileged"],
 	"features": {
+		"ghcr.io/devcontainers/features/azure-cli:1": {},
         "ghcr.io/devcontainers/features/docker-outside-of-docker:1": {}
     },
 	"customizations": {

.github/workflows/kind.yaml

@@ -27,6 +27,10 @@ jobs:
           kubectl get sc
       - name: Checkout repository
         uses: actions/checkout@v2
+      - name: Check promoted Hub runtime
+        run: |
+          helm repo add kerberos https://charts.kerberos.io
+          bash scripts/check-hub-runtime.sh
       - name: Install OpenEBS
         run: |
           kubectl apply -f https://openebs.github.io/charts/openebs-operator.yaml

.gitignore

@@ -1,2 +1,17 @@
 **/charts
-modules/amazon-eks-documentdb/tfplan
+
+# Terraform generated and local files
+.terraform/
+.terraform.lock.hcl
+*.tfstate
+*.tfstate.*
+*.tfplan
+crash.log
+crash.*.log
+override.tf
+override.tf.json
+*_override.tf
+*_override.tf.json
+terraform.tfvars
+*.auto.tfvars
+*.auto.tfvars.json

README.md

@@ -17,6 +17,21 @@ The Kerberos.io stack offers flexible installation options, supporting deploymen
 
 This repository continuously tests the Kerberos.io deployment using GitHub Actions across various platforms. Refer to the badges for an overview of the operating systems and Kubernetes distributions we are deploying to. We welcome suggestions for additional automations to consider.
 
+### Hardened Hub frontend
+
+The promoted frontend image requires Hub chart **0.170.0 or newer**, which
+provides port `8080`, non-root/read-only runtime settings and a writable `/tmp`
+mount. Both primary and demo frontends use these settings; Admin remains on its
+legacy image and port `80`. Keep `global.imageRegistry` empty for the supplied
+fully qualified `ghcr.io` image names. Service ports remain `80` and the frontend
+NodePort remains `32080`, targeting the container port named `http`.
+
+Run `bash scripts/check-hub-runtime.sh` to check the published chart with these
+values before installation, or pass a local chart path as the first argument.
+When adding custom branding mounts, preserve the chart's `frontend-tmp` volume
+and `/tmp` mount. Existing deployments may require the explicit
+[HTTP port migration](https://github.com/kerberos-io/helm-charts/blob/main/charts/hub/README.md#recovering-a-duplicate-http-port-during-upgrades).
+
 ## Architecture
 
 As previously indicated, several integral components, including Object Storage, Database, and Message Brokers, are deployed and configured to ensure the proper functionality of Kerberos.io. The architecture diagrams below aims to provide a visual representation of these components and their interactions with one another depending the deployment; self-hosted or managed cloud deployment.
@@ -68,3 +83,4 @@ Based on your technology experience and preferences, you can choose from the fol
 Or provision the infrastructure yourself with infrastructure as code:
 
 - [[AWS] Terraform: EKS cluster + Amazon DocumentDB](./modules/amazon-eks-documentdb/README.md)
+- [[Azure] Terraform: Event Hub with 10 partitions](./modules/azure-event-hub/README.md)

base/hub/kerberos-hub-import-database-job.yaml

@@ -43,14 +43,71 @@ data:
         conn = db.getMongo();
         db = conn.getDB('Kerberos');
 
+        const ownerUsername = 'example-user';
+        const applicationUsername = 'example-application';
+        const organisationName = 'example-orgranisation';
+        const zeroObjectId = ObjectId('000000000000000000000000');
+        const ownerCreatedAt = ISODate('2016-09-20T09:27:58.811Z');
+        const ownerUpdatedAt = ISODate('2020-06-14T05:01:35.000Z');
+
+        function resolveUserId(username) {
+            const users = db.users.find(
+                { username: username },
+                { _id: 1 }
+            ).limit(2).toArray();
+            if (users.length > 1) {
+                throw new Error(`Cannot import duplicate username: ${username}`);
+            }
+            if (users.length === 1) {
+                if (!users[0]._id || typeof users[0]._id.toHexString !== 'function') {
+                    throw new Error(`Existing user ${username} does not have an ObjectId _id`);
+                }
+                return users[0]._id;
+            }
+            return ObjectId();
+        }
+
+        const ownerId = resolveUserId(ownerUsername);
+        const applicationId = resolveUserId(applicationUsername);
+
+        function assertAcknowledged(result, operation) {
+            if (!result.acknowledged) {
+                throw new Error(`${operation} was not acknowledged`);
+            }
+        }
+
+        function insertIfMissing(collection, id, values) {
+            const result = db.getCollection(collection).updateOne(
+                { _id: id },
+                { $setOnInsert: values },
+                { upsert: true }
+            );
+            assertAcknowledged(result, `Insert for ${collection}/${id}`);
+        }
+
+        function setObjectIdIfMissing(collection, id, field, value) {
+            const result = db.getCollection(collection).updateOne(
+                {
+                    _id: id,
+                    $or: [
+                        { [field]: { $exists: false } },
+                        { [field]: null },
+                        { [field]: zeroObjectId }
+                    ]
+                },
+                { $set: { [field]: value } }
+            );
+            assertAcknowledged(result, `Initialisation of ${collection}.${field}`);
+        }
+
         // This will create a user with the following credentials.
         // username: example-user
         // password: example-password
         db.users.updateOne(
-            { '_id': ObjectId('57e1011e3178aa6c5cc774d1') },
+            { '_id': ownerId },
             {
                 $set: {
-                'username': 'example-user',
+                'username': ownerUsername,
                 'email': 'example-user@email.com',
                 'password': '$2a$10$jwLcD/.UT/1WLK7ct1XuHewI3GQXwW3zerPhCCs7QDrReEuIHbVYi',
                 'role': 'owner',
@@ -58,8 +115,8 @@ data:
                 'timezone': 'Europe/Brussels',
                 'isActive': NumberLong(1),
                 'registerToken': '',
-                'updated_at': ISODate('2020-06-14T05:01:35.000Z'),
-                'created_at': ISODate('2016-09-20T09:27:58.811Z'),
+                'updated_at': ownerUpdatedAt,
+                'created_at': ownerCreatedAt,
                 'amazon_secret_access_key': 'K6rRLBI1xxxCk3C1H',
                 'amazon_access_key_id': 'AKIAxxxxxxG5Q',
                 'card_brand': 'Visa',
@@ -71,16 +128,85 @@ data:
             { upsert: true }
         );
 
+        // The owner gets a random ObjectId on first import and is resolved by
+        // username on reruns. Its organisation and default project share it.
+        insertIfMissing('organisation', ownerId, {
+            name: organisationName,
+            ownerId: ownerId,
+            isActive: true,
+            settings: {
+                timezone: 'Europe/Brussels'
+            },
+            audit: {
+                createdBy: ownerId.toHexString(),
+                createdAt: ownerCreatedAt,
+                updatedBy: ownerId.toHexString(),
+                updatedAt: ownerUpdatedAt,
+                lastAction: 'organisation.created'
+            }
+        });
+
+        const membershipResult = db.organisation_users.updateOne(
+            {
+                userId: ownerId,
+                organisationId: ownerId
+            },
+            {
+                $setOnInsert: {
+                    userId: ownerId,
+                    organisationId: ownerId,
+                    status: 'active',
+                    invitedBy: ownerId,
+                    joinedAt: ownerCreatedAt,
+                    audit: {
+                        createdBy: ownerId.toHexString(),
+                        createdAt: ownerCreatedAt,
+                        updatedBy: ownerId.toHexString(),
+                        updatedAt: ownerUpdatedAt,
+                        lastAction: 'organisation.membership.created'
+                    }
+                }
+            },
+            { upsert: true }
+        );
+        assertAcknowledged(membershipResult, 'Owner organisation membership');
+
+        insertIfMissing('project', ownerId, {
+            organisationId: ownerId,
+            name: 'Project 1',
+            slug: 'project-1',
+            isActive: true,
+            audit: {
+                createdBy: ownerId.toHexString(),
+                createdAt: ownerCreatedAt,
+                updatedBy: ownerId.toHexString(),
+                updatedAt: ownerUpdatedAt,
+                lastAction: 'project.default.created'
+            }
+        });
+
+        // Initialise only missing selections. Re-running the import must not
+        // reset a user who has since selected another organisation or project.
+        setObjectIdIfMissing('users', ownerId, 'organisationId', ownerId);
+        const importedOwner = db.users.findOne(
+            { _id: ownerId, organisationId: { $type: 'objectId' } },
+            { organisationId: 1 }
+        );
+        if (!importedOwner) {
+            throw new Error('Imported owner has an invalid organisationId');
+        }
+        setObjectIdIfMissing('users', ownerId, 'projectId', importedOwner.organisationId);
+
 
         // This will create an application with the following credentials, which
         // can be used to authenticate with the admin panel.
         // username: example-application
         // password: example-password
         db.users.updateOne(
-            { '_id': ObjectId('57e1011e3178aa6c5cc774d2') },
+            { '_id': applicationId },
             {
                 $set: {
-                'username': 'example-application',
+                'username': applicationUsername,
                 'email': 'example-application@email.com',
                 'password': '$2a$10$jwLcD/.UT/1WLK7ct1XuHewI3GQXwW3zerPhCCs7QDrReEuIHbVYi',
                 'role': 'application',
@@ -99,7 +225,7 @@ data:
         // Subscriptions allow you to modify specific permissions and features for a user.
         // As you'll see below you can specify specific threshold for each subscription.
         db.subscriptions.updateOne(
-            { '_id': ObjectId('57e1011e3178aa6c5cc774d1') },
+            { '_id': ownerId },
             {
                 $set: {
                 'name': 'default',
@@ -108,7 +234,8 @@ data:
                 'quantity': 1,
                 'trial_ends_at': null,
                 'ends_at': null,
-                'user_id': '57e1011e3178aa6c5cc774d1',
+                'user_id': ownerId.toHexString(),
+                'status': 'active',
                 'updated_at': ISODate('2021-04-27T09:45:30.169Z'),
                 'created_at': ISODate('2016-09-20T09:35:03.448Z'),
                 'stripe_status': 'active'
@@ -116,6 +243,7 @@ data:
             },
             { upsert: true }
         );
+        setObjectIdIfMissing('subscriptions', ownerId, 'organisation_id', ownerId);
 
         // This will create a settings object for the subscription.
         // and will allow you to modify the settings for the subscription.
@@ -240,4 +368,87 @@ data:
                 }
             },
             { upsert: true }
-        );
+        );
+
+        const importedUsers = db.users.countDocuments({
+            _id: {
+                $in: [ownerId, applicationId]
+            }
+        });
+        const importedOwnerUsernames = db.users.countDocuments({ username: ownerUsername });
+        const importedApplicationUsernames = db.users.countDocuments({ username: applicationUsername });
+        const importedOwnerSelection = db.users.findOne(
+            {
+                _id: ownerId,
+                organisationId: { $type: 'objectId' },
+                projectId: { $type: 'objectId' }
+            },
+            { organisationId: 1, projectId: 1 }
+        );
+        const importedOrganisations = db.organisation.countDocuments({
+            _id: ownerId,
+            ownerId: ownerId,
+            isActive: true
+        });
+        const importedMemberships = db.organisation_users.countDocuments({
+            userId: ownerId,
+            organisationId: ownerId,
+            status: 'active'
+        });
+        const importedProjects = db.project.countDocuments({
+            _id: ownerId,
+            organisationId: ownerId,
+            slug: 'project-1',
+            isActive: true
+        });
+        const importedSubscriptions = db.subscriptions.countDocuments({
+            _id: ownerId,
+            organisation_id: ownerId
+        });
+        const importedSettings = db.settings.countDocuments({
+            _id: {
+                $in: [
+                    ObjectId('5a72c509e17699d18ada9154'),
+                    ObjectId('63f346ec64011a574161cf99'),
+                    ObjectId('5a43fa12d885eb7da57046b3'),
+                    ObjectId('5a4d3a6bd885eb7da5e6b297'),
+                    ObjectId('5a53d0a0d885eb7da53ed5a6')
+                ]
+            }
+        });
+
+        if (!importedOwnerSelection) {
+            throw new Error('Imported owner is missing ObjectId organisation/project selections');
+        }
+        const selectedOrganisationExists = db.organisation.countDocuments({
+            _id: importedOwnerSelection.organisationId
+        });
+        const selectedProjectExists = db.project.countDocuments({
+            _id: importedOwnerSelection.projectId,
+            organisationId: importedOwnerSelection.organisationId
+        });
+        const selectedProjectIsLogicalDefault = importedOwnerSelection.projectId.equals(
+            importedOwnerSelection.organisationId
+        );
+
+        if (
+            importedUsers !== 2 ||
+            importedOwnerUsernames !== 1 ||
+            importedApplicationUsernames !== 1 ||
+            importedOrganisations !== 1 ||
+            importedMemberships !== 1 ||
+            importedProjects !== 1 ||
+            importedSubscriptions !== 1 ||
+            importedSettings !== 5 ||
+            selectedOrganisationExists !== 1 ||
+            (!selectedProjectIsLogicalDefault && selectedProjectExists !== 1)
+        ) {
+            throw new Error(
+                `Import verification failed: users=${importedUsers}, ` +
+                `organisations=${importedOrganisations}, memberships=${importedMemberships}, ` +
+                `projects=${importedProjects}, subscriptions=${importedSubscriptions}, ` +
+                `settings=${importedSettings}`
+            );
+        }
+
+        print('Imported 2 users, 1 organisation, 1 membership, 1 project, 1 subscription, and 5 settings records into Kerberos.');

base/hub/kerberos-hub-nodeports.yaml

@@ -9,7 +9,7 @@ spec:
   type: NodePort
   ports:
     - port: 80
-      targetPort: 80
+      targetPort: http
       nodePort: 32080 # You can specify a port in the range 30000-32767 or let Kubernetes assign one automatically
       name: frontend
       protocol: TCP

base/hub/kerberos-hub-values.yaml

@@ -19,7 +19,7 @@ licenseServer:
 
 # When using a private registry, you can specify the registry here.
 global:
-  imageRegistry: "docker.io/"
+  imageRegistry: ""
 # Environment: set to 'production', 'develop', 'demo', 'staging' or 'acceptance'.
 # Set to 'true' if this is a private deployment.
 environment: "production"
@@ -157,7 +157,7 @@ kerberoshub:
   api:
     repository: ghcr.io/uug-ai/hub-api
     pullPolicy: IfNotPresent
-    tag: "v1.9.48"
+    tag: "v1.11.31"
     replicas: 1
     jwtSecret: "I1JcwzW3A0t_THIS-IS_NOT_A_SECRET_WJK9jnPkipbnVTpf0efMy" # change to a random value, this is for generating JWT tokens.
     schema: "http"
@@ -240,7 +240,9 @@ kerberoshub:
   frontend:
     repository: ghcr.io/uug-ai/hub-frontend
     pullPolicy: IfNotPresent
-    tag: "v1.13.0"
+    tag: "v1.19.28"
+    # Requires Hub chart 0.170.0+ for the matching hardened runtime defaults.
+    containerPort: 8080
     replicas: 1
     schema: "http"
     url: "yourdomain.com"
@@ -355,7 +357,7 @@ kerberoshub:
   cleanup:
     repository: ghcr.io/uug-ai/hub-cleanup
     pullPolicy: IfNotPresent
-    tag: "v1.4.16"
+    tag: "v1.4.26"
     resources:
       requests:
         memory: 50Mi
@@ -363,7 +365,7 @@ kerberoshub:
   monitordevice:
     repository: ghcr.io/uug-ai/hub-monitor-device
     pullPolicy: IfNotPresent
-    tag: "v1.4.1"
+    tag: "v1.4.10"
     resources:
       requests:
         memory: 50Mi
@@ -415,7 +417,7 @@ kerberospipeline:
   event:
     repository: ghcr.io/uug-ai/hub-pipeline-event
     pullPolicy: IfNotPresent
-    tag: "v1.3.1"
+    tag: "v1.3.9"
     replicas: 1
     resources:
       requests:
@@ -427,7 +429,7 @@ kerberospipeline:
   monitor:
     repository: ghcr.io/uug-ai/hub-pipeline-monitor
     pullPolicy: IfNotPresent
-    tag: "v1.3.11"
+    tag: "v1.3.29"
     replicas: 1
     resources:
       requests:
@@ -439,7 +441,7 @@ kerberospipeline:
   sequence:
     repository: ghcr.io/uug-ai/hub-pipeline-sequence
     pullPolicy: IfNotPresent
-    tag: "v1.6.23"
+    tag: "v1.7.14"
     replicas: 1
     resources:
       requests:
@@ -449,9 +451,9 @@ kerberospipeline:
         memory: 50Mi
         cpu: 50m
   throttler:
-    repository: uugai/hub-pipeline-throttler
+    repository: ghcr.io/uug-ai/hub-pipeline-throttler
     pullPolicy: IfNotPresent
-    tag: "v1.2.0"
+    tag: "v1.2.10"
     replicas: 1
     resources:
       requests:
@@ -463,7 +465,7 @@ kerberospipeline:
   notify:
     repository: ghcr.io/uug-ai/hub-pipeline-notification
     pullPolicy: IfNotPresent
-    tag: "v1.3.17"
+    tag: "v1.3.46"
     replicas: 1
     resources:
       requests:
@@ -481,9 +483,9 @@ kerberospipeline:
         #    persistentVolumeClaim:
         #      claimName: custom-layout-claim
   notifyTest:
-    repository: uugai/hub-pipeline-notification-test
+    repository: ghcr.io/uug-ai/hub-pipeline-notification-test
     pullPolicy: IfNotPresent
-    tag: "v1.2.1"
+    tag: "v1.2.6"
     replicas: 1
     resources:
       requests:
@@ -503,7 +505,7 @@ kerberospipeline:
   analysis:
     repository: ghcr.io/uug-ai/hub-pipeline-analysis
     pullPolicy: IfNotPresent
-    tag: "v1.8.4"
+    tag: "v1.8.29"
     replicas: 1
     resources:
       requests:
@@ -515,7 +517,7 @@ kerberospipeline:
   dominantColor:
     repository: ghcr.io/uug-ai/hub-pipeline-dominantcolors
     pullPolicy: IfNotPresent
-    tag: "v2.0.3"
+    tag: "v2.0.9"
     replicas: 1
     resources:
       requests:
@@ -527,7 +529,7 @@ kerberospipeline:
   thumbnail:
     repository: ghcr.io/uug-ai/hub-pipeline-thumbnail
     pullPolicy: IfNotPresent
-    tag: "v1.3.10"
+    tag: "v1.3.22"
     replicas: 1
     quality: "1" # 1 (best) - 31 (worst)
     width: "600"
@@ -542,9 +544,9 @@ kerberospipeline:
         memory: 50Mi
         cpu: 50m
   counting:
-    repository: uugai/hub-pipeline-counting
+    repository: ghcr.io/uug-ai/hub-pipeline-counting
     pullPolicy: IfNotPresent
-    tag: "v1.6.3"
+    tag: "v2.0.6"
     replicas: 1
     resources:
       requests:
@@ -554,7 +556,7 @@ kerberospipeline:
     enabled: true # Enable or disable the sprite generation 'true' or 'false
     repository: ghcr.io/uug-ai/hub-pipeline-sprite
     pullPolicy: IfNotPresent
-    tag: "v1.1.16"
+    tag: "v1.1.27"
     replicas: 1
     interval: "1" # Number of secondes between each thumbnail in the sprite
     width: "240" # Should not be changed for the moment (hard coded in UI)
@@ -569,7 +571,7 @@ kerberospipeline:
   export:
     repository: ghcr.io/uug-ai/hub-pipeline-export
     pullPolicy: IfNotPresent
-    tag: "v1.2.9"
+    tag: "v1.2.20"
     replicas: 1
     resources:
       requests:

base/vault/README.md

@@ -18,9 +18,12 @@ recording bytes to storage providers configured through the Vault UI.
 | File | Kind | Purpose |
 | ---- | ---- | ------- |
 | [`mongodb-configmap.yaml`](./mongodb-configmap.yaml) | `ConfigMap` | Supplies Vault's MongoDB connection, backend flavor, retry-write, and TLS settings. |
-| [`kerberos-vault-deployment.yaml`](./kerberos-vault-deployment.yaml) | `Deployment` | Runs the Vault API and UI on container port `80`. |
+| [`kerberos-vault-deployment.yaml`](./kerberos-vault-deployment.yaml) | `Deployment` | ConfigMap-backed Vault example selected by the base Kustomization. |
+| [`kerberos-vault-deployment-with-secrets.yaml`](./kerberos-vault-deployment-with-secrets.yaml) | `Deployment` | Alternative Secret-backed Vault example. |
 | [`kerberos-vault-service.yaml`](./kerberos-vault-service.yaml) | `Service` | Exposes Vault on `NodePort` **30080**. A commented `LoadBalancer` variant is included. |
 | [`data-filtering-deployment.yaml`](./data-filtering-deployment.yaml) | `Deployment` | Optional YOLO data-filtering worker. The default manifest requests one NVIDIA GPU. |
+| [`vault-runtime-configuration-configmap.yaml`](./vault-runtime-configuration-configmap.yaml) | `ConfigMap` | Plaintext local defaults used by the base Kustomization; managed-resource lists are empty. |
+| [`vault-runtime-configuration-secret.yaml`](./vault-runtime-configuration-secret.yaml) | `Secret` | Optional provider, integration, and account example for production secret handling. |
 
 The matching `kerberos-vault` namespace is defined in
 [`../namespaces/kerberos-vault.yaml`](../namespaces/kerberos-vault.yaml).
@@ -30,7 +33,8 @@ The matching `kerberos-vault` namespace is defined in
 - A running Kubernetes cluster and `kubectl` configured to reach it.
 - A MongoDB-compatible database reachable from the Vault pod.
 - A storage provider such as MinIO, Amazon S3, Google Cloud Storage, or Azure
-  Blob Storage. Configure it in the Vault UI after deployment.
+  Blob Storage. Configure it in the Vault UI or through the optional runtime
+  configuration ConfigMap or Secret after deployment.
 - An NVIDIA-capable node and device plugin only when deploying the optional
   data-filtering worker with its default resource settings.
 
@@ -119,17 +123,62 @@ authentication to `SCRAM-SHA-1` and does not enable MongoDB Stable API.
 
 ## Vault configuration
 
-Relevant environment variables are defined directly on the Vault Deployment:
+The selected runtime ConfigMap or Secret supplies these environment variables
+through `envFrom`. Only `GIN_MODE` remains directly on either Deployment:
 
 | Variable | Default here | Meaning |
 | -------- | ------------ | ------- |
 | `KERBEROS_LOGIN_USERNAME` / `KERBEROS_LOGIN_PASSWORD` | `root` / `kerberos` | Vault UI login. Change these demo credentials. |
-| `MQTTURI` | `tcp://mqtt.kerberos.io:1883` | MQTT broker used for on-demand forwarding. |
+| `MQTT_URI` | `tcp://mqtt.kerberos.io:1883` | MQTT broker used for on-demand forwarding. |
 | `MQTT_USERNAME` / `MQTT_PASSWORD` | empty | Optional MQTT credentials. |
-| `CONTINUOUS_FORWARDING` | `false` | Enables forwarding for a chained Vault setup. |
+| `VAULT_STORAGE_PROVIDERS` | unset | JSON array of runtime-managed storage providers. |
+| `VAULT_INTEGRATIONS` | unset | JSON array of runtime-managed integrations. |
+| `VAULT_ACCOUNTS` | unset | JSON array of runtime-managed Vault accounts. |
 
-Storage providers, integrations, and Vault accounts are configured through the
-UI after the pod starts. See [`README.configure.md`](../../README.configure.md).
+Vault accounts, storage providers, and integrations can be supplied as JSON
+arrays through the three environment variables. Two alternative Deployment
+manifests show how to import them from a `vault-runtime-configuration` ConfigMap
+or Secret.
+Environment definitions are grayed out and read-only in Vault, override database
+definitions with the same name, and are loaded when the pod starts. Invalid
+definitions prevent startup. An enabled Vault integration automatically starts
+the forwarding worker. See [`README.configure.md`](../../README.configure.md).
+
+The base [`kustomization.yaml`](./kustomization.yaml) includes the plaintext
+ConfigMap and standard Deployment so the demo login remains usable. The
+credential-bearing Secret and Secret-backed Deployment remain standalone:
+
+- [`vault-runtime-configuration-configmap.yaml`](./vault-runtime-configuration-configmap.yaml)
+  contains the demo Vault login and MQTT defaults. Its provider, integration,
+  and account arrays are `[]`, so those resources remain managed through the UI
+  and database. ConfigMap values are plaintext.
+- [`vault-runtime-configuration-secret.yaml`](./vault-runtime-configuration-secret.yaml)
+  stores the same eight variables with `stringData`, so values can be entered
+  without manual base64 encoding. Use an external secret manager or encrypted
+  GitOps workflow in production.
+
+For direct `kubectl` installation, apply one matching pair in the Vault
+namespace. Replace every Secret example credential first. Both Deployment
+manifests manage `Deployment/vault`; choose one and do not apply both.
+
+```bash
+# Plaintext local-development variant:
+kubectl apply -n kerberos-vault \
+  -f ./vault-runtime-configuration-configmap.yaml
+kubectl apply -n kerberos-vault \
+  -f ./kerberos-vault-deployment.yaml
+
+# Recommended Secret-backed variant:
+kubectl apply -n kerberos-vault \
+  -f ./vault-runtime-configuration-secret.yaml
+kubectl apply -n kerberos-vault \
+  -f ./kerberos-vault-deployment-with-secrets.yaml
+```
+
+Restart the Deployment after changing any value. Leave a key absent or set it
+to `[]` to use only database definitions for that resource type. Switching
+variants is also an `apply`: the selected manifest replaces the existing
+`Deployment/vault` definition.
 
 ## Deploy with kubectl without Kustomize
 
@@ -143,7 +192,9 @@ kubectl apply -f ../namespaces/kerberos-vault.yaml
 # 2. Review the database settings, then create the ConfigMap.
 kubectl apply -n kerberos-vault -f ./mongodb-configmap.yaml
 
-# 3. Deploy Vault and expose it through NodePort.
+# 3. Optionally apply the plaintext runtime configuration, then deploy Vault.
+# For the Secret-backed alternative, use the matching pair documented above.
+# kubectl apply -n kerberos-vault -f ./vault-runtime-configuration-configmap.yaml
 kubectl apply -n kerberos-vault -f ./kerberos-vault-deployment.yaml
 kubectl apply -n kerberos-vault -f ./kerberos-vault-service.yaml
 ```

base/vault/kerberos-vault-deployment-with-secrets.yaml

@@ -0,0 +1,47 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: vault
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: vault
+  minReadySeconds: 10
+  strategy:
+    type: RollingUpdate
+    rollingUpdate:
+      maxUnavailable: 1
+      maxSurge: 1
+  template:
+    metadata:
+      labels:
+        app: vault
+    spec:
+      initContainers:
+        - name: wait-for-mongodb-before-starup
+          image: busybox
+          command: ["sh", "-c", "until nc -z mongodb.mongodb 27017 > /dev/null; do echo Waiting for master.; sleep 2; done;"]
+      containers:
+        - name: vault
+          image: ghcr.io/uug-ai/vault:v1.4.9
+          resources:
+            requests:
+              memory: 128Mi
+              cpu: 100m
+            limits:
+              memory: 128Mi
+              cpu: 100m
+          ports:
+            - containerPort: 80
+          envFrom:
+            - configMapRef:
+                name: mongodb
+            # Optional runtime configuration containing provider, integration,
+            # and account credentials.
+            - secretRef:
+                name: vault-runtime-configuration
+                optional: true
+          env:
+            - name: GIN_MODE
+              value: release

base/vault/kerberos-vault-deployment.yaml

@@ -24,7 +24,7 @@ spec:
           command: ["sh", "-c", "until nc -z mongodb.mongodb 27017 > /dev/null; do echo Waiting for master.; sleep 2; done;"]
       containers:
         - name: vault
-          image: ghcr.io/uug-ai/vault:v1.4.9
+          image: ghcr.io/uug-ai/vault:v1.7.19
           resources:
             requests:
               memory: 128Mi
@@ -37,36 +37,10 @@ spec:
           envFrom:
             - configMapRef:
                 name: mongodb
+            # Optional plaintext runtime configuration for local development.
+            - configMapRef:
+                name: vault-runtime-configuration
+                optional: true
           env:
             - name: GIN_MODE
               value: release
-            - name: KERBEROS_LOGIN_USERNAME
-              value: "root"
-            - name: KERBEROS_LOGIN_PASSWORD
-              value: "kerberos"
-            # MQTT broker to be used for on-demand forwarding.
-            - name: MQTTURI
-              value: "tcp://mqtt.kerberos.io:1883"
-            - name: MQTT_USERNAME
-              value: ""
-            - name: MQTT_PASSWORD
-              value: ""
-            # If you have a chained vault setup, enable this.
-            - name: CONTINUOUS_FORWARDING
-              value: "false"
-# MongoDB URI (for example for a SaaS service like MongoDB Atlas)
-# If uri is set, the below properties are not used (host, adminDatabase, username, password)
-#MONGODB_URI: "mongodb+srv://xx:xx@kerberos-hub.xxx.mongodb.net/?retryWrites=true&w=majority&appName=xxx"
-# Mongodb configuration (if you are not using the configmap)
-# This is the mongodb database where data will be stored, you might use a different name if you want.
-#- name: MONGODB_DATABASE_STORAGE
-#  value : "KerberosStorage"
-# If you do not wish to use the URI, you can specify the individual values.
-#- name: MONGODB_HOST
-#  value: "mongodb.mongodb"
-#- name: MONGODB_DATABASE_CREDENTIALS
-#  value: "admin"
-#- name: MONGODB_USERNAME
-#  value: "root"
-#- name: MONGODB_PASSWORD
-#  value: "yourmongodbpassword"

base/vault/kustomization.yaml

@@ -1,6 +1,7 @@
 namespace: kerberos-vault
 resources:
   - mongodb-configmap.yaml
+  - vault-runtime-configuration-configmap.yaml
   - kerberos-vault-deployment.yaml
   - kerberos-vault-service.yaml
   - data-filtering-deployment.yaml

base/vault/vault-runtime-configuration-configmap.yaml

@@ -0,0 +1,16 @@
+# Plaintext defaults for local development. Provider, integration, and account
+# lists stay empty so credentials can be managed through the Vault UI/database.
+# Use vault-runtime-configuration-secret.yaml for environment-managed resources.
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: vault-runtime-configuration
+data:
+  KERBEROS_LOGIN_USERNAME: "root"
+  KERBEROS_LOGIN_PASSWORD: "kerberos"
+  MQTT_URI: "tcp://mqtt.kerberos.io:1883"
+  MQTT_USERNAME: ""
+  MQTT_PASSWORD: ""
+  VAULT_STORAGE_PROVIDERS: "[]"
+  VAULT_INTEGRATIONS: "[]"
+  VAULT_ACCOUNTS: "[]"

base/vault/vault-runtime-configuration-secret.yaml

@@ -0,0 +1,19 @@
+# Replace all example credentials before applying this manifest.
+# Keep the populated file outside source control in production.
+apiVersion: v1
+kind: Secret
+metadata:
+  name: vault-runtime-configuration
+type: Opaque
+stringData:
+  KERBEROS_LOGIN_USERNAME: "root"
+  KERBEROS_LOGIN_PASSWORD: "kerberos"
+  MQTT_URI: "tcp://mqtt.kerberos.io:1883"
+  MQTT_USERNAME: ""
+  MQTT_PASSWORD: ""
+  VAULT_STORAGE_PROVIDERS: >-
+    [{"name":"archive","provider":"minio","host":"minio.minio.svc:9000","region":"us-east-1","bucket":"recordings","access_key":"example-key","secret":"example-secret","use_ssl":"false"}]
+  VAULT_INTEGRATIONS: >-
+    [{"name":"events","queue":"rabbitmq","broker":"amqp://rabbitmq.rabbitmq.svc:5672","topic":"recordings","username":"example-user","password":"example-password"}]
+  VAULT_ACCOUNTS: >-
+    [{"account":"camera-fleet","directory":"fleet","provider":"archive","access_key":"example-account-key","secret_access_key":"example-account-secret","limit":"30","queues":["events"]}]

modules/amazon-eks-documentdb/README.md

@@ -166,9 +166,15 @@ helm install hub kerberos/hub \
   --version 0.127.0 \
   -n kerberos-hub \
   -f your-hub-values.yaml \
-  -f hub-documentdb-values.yaml
+  -f hub-documentdb-values.yaml \
+  --set-string kerberoshub.api.tag=v1.9.49
 ```
 
+Hub API `v1.9.49` is the first release that disables the MongoDB Stable API in
+DocumentDB mode. Older images can fail the SCRAM-SHA-1 handshake with
+`sasl conversation error: ... Query syntax error`, even when the username,
+password and authentication mechanism are correct.
+
 The `hub_values_snippet` output contains credentials, so treat the generated
 file as a secret and do not commit it.
 
@@ -177,8 +183,15 @@ file as a secret and do not commit it.
 ```bash
 kubectl logs -n kerberos-hub deploy/hub-api | head -50
 kubectl exec -n kerberos-hub deploy/hub-api -- ls -l /certs
+kubectl get deployment hub-api -n kerberos-hub \
+  -o jsonpath='{.spec.template.spec.containers[0].image}{"\n"}'
+kubectl get configmap mongodb-config -n kerberos-hub \
+  -o jsonpath='{.data.MONGODB_FLAVOR}{"\n"}'
 ```
 
+The last two commands must report Hub API `v1.9.49` or newer and
+`documentdb`, respectively.
+
 A one-off connectivity check from inside the cluster:
 
 ```bash
@@ -274,6 +287,7 @@ helm upgrade hub kerberos/hub \
   --file your-hub-values.yaml \
   --file hub-documentdb-values.yaml \
   --file ingress/hub-public-values.yaml \
+  --set-string kerberoshub.api.tag=v1.9.49 \
   --atomic \
   --wait
 ```

modules/amazon-eks-documentdb/database-import/hub-import.js

@@ -1,8 +1,15 @@
 const database = db.getSiblingDB('Kerberos');
+const ownerUsername = 'example-user';
+const applicationUsername = 'example-application';
+const organisationName = 'example-orgranisation';
+const zeroObjectId = ObjectId('000000000000000000000000');
+const ownerCreatedAt = ISODate('2016-09-20T09:27:58.811Z');
+const ownerUpdatedAt = ISODate('2020-06-14T05:01:35.000Z');
 
 function upsert(collection, id, values) {
+  const objectId = typeof id === 'string' ? ObjectId(id) : id;
   const result = database.getCollection(collection).updateOne(
-    { _id: ObjectId(id) },
+    { _id: objectId },
     { $set: values },
     { upsert: true },
   );
@@ -12,9 +19,59 @@ function upsert(collection, id, values) {
   }
 }
 
+function resolveUserId(username) {
+  const users = database.users.find(
+    { username },
+    { _id: 1 },
+  ).limit(2).toArray();
+  if (users.length > 1) {
+    throw new Error(`Cannot import duplicate username: ${username}`);
+  }
+  if (users.length === 1) {
+    if (!users[0]._id || typeof users[0]._id.toHexString !== 'function') {
+      throw new Error(`Existing user ${username} does not have an ObjectId _id`);
+    }
+    return users[0]._id;
+  }
+  return ObjectId();
+}
+
+function insertIfMissing(collection, id, values) {
+  const result = database.getCollection(collection).updateOne(
+    { _id: id },
+    { $setOnInsert: values },
+    { upsert: true },
+  );
+
+  if (!result.acknowledged) {
+    throw new Error(`Insert was not acknowledged for ${collection}/${id}`);
+  }
+}
+
+function setObjectIdIfMissing(collection, id, field, value) {
+  const result = database.getCollection(collection).updateOne(
+    {
+      _id: id,
+      $or: [
+        { [field]: { $exists: false } },
+        { [field]: null },
+        { [field]: zeroObjectId },
+      ],
+    },
+    { $set: { [field]: value } },
+  );
+
+  if (!result.acknowledged) {
+    throw new Error(`Initialisation was not acknowledged for ${collection}.${field}`);
+  }
+}
+
+const ownerId = resolveUserId(ownerUsername);
+const applicationId = resolveUserId(applicationUsername);
+
 // Login: example-user / example-password
-upsert('users', '57e1011e3178aa6c5cc774d1', {
-  username: 'example-user',
+upsert('users', ownerId, {
+  username: ownerUsername,
   email: 'example-user@email.com',
   password: '$2a$10$jwLcD/.UT/1WLK7ct1XuHewI3GQXwW3zerPhCCs7QDrReEuIHbVYi',
   role: 'owner',
@@ -32,9 +89,76 @@ upsert('users', '57e1011e3178aa6c5cc774d1', {
   card_status_message: null,
 });
 
+insertIfMissing('organisation', ownerId, {
+  name: organisationName,
+  ownerId,
+  isActive: true,
+  settings: {
+    timezone: 'Europe/Brussels',
+  },
+  audit: {
+    createdBy: ownerId.toHexString(),
+    createdAt: ownerCreatedAt,
+    updatedBy: ownerId.toHexString(),
+    updatedAt: ownerUpdatedAt,
+    lastAction: 'organisation.created',
+  },
+});
+
+const membershipResult = database.organisation_users.updateOne(
+  {
+    userId: ownerId,
+    organisationId: ownerId,
+  },
+  {
+    $setOnInsert: {
+      userId: ownerId,
+      organisationId: ownerId,
+      status: 'active',
+      invitedBy: ownerId,
+      joinedAt: ownerCreatedAt,
+      audit: {
+        createdBy: ownerId.toHexString(),
+        createdAt: ownerCreatedAt,
+        updatedBy: ownerId.toHexString(),
+        updatedAt: ownerUpdatedAt,
+        lastAction: 'organisation.membership.created',
+      },
+    },
+  },
+  { upsert: true },
+);
+if (!membershipResult.acknowledged) {
+  throw new Error('Owner organisation membership was not acknowledged');
+}
+
+insertIfMissing('project', ownerId, {
+  organisationId: ownerId,
+  name: 'Project 1',
+  slug: 'project-1',
+  isActive: true,
+  audit: {
+    createdBy: ownerId.toHexString(),
+    createdAt: ownerCreatedAt,
+    updatedBy: ownerId.toHexString(),
+    updatedAt: ownerUpdatedAt,
+    lastAction: 'project.default.created',
+  },
+});
+
+setObjectIdIfMissing('users', ownerId, 'organisationId', ownerId);
+const importedOwner = database.users.findOne(
+  { _id: ownerId, organisationId: { $type: 'objectId' } },
+  { organisationId: 1 },
+);
+if (!importedOwner) {
+  throw new Error('Imported owner has an invalid organisationId');
+}
+setObjectIdIfMissing('users', ownerId, 'projectId', importedOwner.organisationId);
+
 // Admin login: example-application / example-password
-upsert('users', '57e1011e3178aa6c5cc774d2', {
-  username: 'example-application',
+upsert('users', applicationId, {
+  username: applicationUsername,
   email: 'example-application@email.com',
   password: '$2a$10$jwLcD/.UT/1WLK7ct1XuHewI3GQXwW3zerPhCCs7QDrReEuIHbVYi',
   role: 'application',
@@ -46,18 +170,20 @@ upsert('users', '57e1011e3178aa6c5cc774d2', {
   created_at: ISODate('2016-09-20T09:27:58.811Z'),
 });
 
-upsert('subscriptions', '57e1011e3178aa6c5cc774d1', {
+upsert('subscriptions', ownerId, {
   name: 'default',
   stripe_id: 'sub_9ECyjjMz3R7etK',
   stripe_plan: 'enterprise',
   quantity: 1,
   trial_ends_at: null,
   ends_at: null,
-  user_id: '57e1011e3178aa6c5cc774d1',
+  user_id: ownerId.toHexString(),
+  status: 'active',
   updated_at: ISODate('2021-04-27T09:45:30.169Z'),
   created_at: ISODate('2016-09-20T09:35:03.448Z'),
   stripe_status: 'active',
 });
+setObjectIdIfMissing('subscriptions', ownerId, 'organisation_id', ownerId);
 
 upsert('settings', '5a72c509e17699d18ada9154', {
   key: 'plan',
@@ -132,14 +258,38 @@ upsert('settings', '5a53d0a0d885eb7da53ed5a6', {
 
 const importedUsers = database.users.countDocuments({
   _id: {
-    $in: [
-      ObjectId('57e1011e3178aa6c5cc774d1'),
-      ObjectId('57e1011e3178aa6c5cc774d2'),
-    ],
+    $in: [ownerId, applicationId],
   },
 });
+const importedOwnerUsernames = database.users.countDocuments({ username: ownerUsername });
+const importedApplicationUsernames = database.users.countDocuments({ username: applicationUsername });
+const importedOwnerSelection = database.users.findOne(
+  {
+    _id: ownerId,
+    organisationId: { $type: 'objectId' },
+    projectId: { $type: 'objectId' },
+  },
+  { organisationId: 1, projectId: 1 },
+);
+const importedOrganisations = database.organisation.countDocuments({
+  _id: ownerId,
+  ownerId,
+  isActive: true,
+});
+const importedMemberships = database.organisation_users.countDocuments({
+  userId: ownerId,
+  organisationId: ownerId,
+  status: 'active',
+});
+const importedProjects = database.project.countDocuments({
+  _id: ownerId,
+  organisationId: ownerId,
+  slug: 'project-1',
+  isActive: true,
+});
 const importedSubscriptions = database.subscriptions.countDocuments({
-  _id: ObjectId('57e1011e3178aa6c5cc774d1'),
+  _id: ownerId,
+  organisation_id: ownerId,
 });
 const importedSettings = database.settings.countDocuments({
   _id: {
@@ -153,11 +303,38 @@ const importedSettings = database.settings.countDocuments({
   },
 });
 
-if (importedUsers !== 2 || importedSubscriptions !== 1 || importedSettings !== 5) {
+if (!importedOwnerSelection) {
+  throw new Error('Imported owner is missing ObjectId organisation/project selections');
+}
+const selectedOrganisationExists = database.organisation.countDocuments({
+  _id: importedOwnerSelection.organisationId,
+});
+const selectedProjectExists = database.project.countDocuments({
+  _id: importedOwnerSelection.projectId,
+  organisationId: importedOwnerSelection.organisationId,
+});
+const selectedProjectIsLogicalDefault = importedOwnerSelection.projectId.equals(
+  importedOwnerSelection.organisationId,
+);
+
+if (
+  importedUsers !== 2
+  || importedOwnerUsernames !== 1
+  || importedApplicationUsernames !== 1
+  || importedOrganisations !== 1
+  || importedMemberships !== 1
+  || importedProjects !== 1
+  || importedSubscriptions !== 1
+  || importedSettings !== 5
+  || selectedOrganisationExists !== 1
+  || (!selectedProjectIsLogicalDefault && selectedProjectExists !== 1)
+) {
   throw new Error(
     `Import verification failed: users=${importedUsers}, `
-      + `subscriptions=${importedSubscriptions}, settings=${importedSettings}`,
+      + `organisations=${importedOrganisations}, memberships=${importedMemberships}, `
+      + `projects=${importedProjects}, subscriptions=${importedSubscriptions}, `
+      + `settings=${importedSettings}`,
   );
 }
 
-print('Imported 2 users, 1 subscription, and 5 settings records into Kerberos.');
+print('Imported 2 users, 1 organisation, 1 membership, 1 project, 1 subscription, and 5 settings records into Kerberos.');

scripts/check-hub-runtime.sh

@@ -0,0 +1,44 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+CHART="${1:-kerberos/hub}"
+
+assert_contains() {
+  if ! grep -Eq -- "$2" <<< "$1"; then
+    printf 'Missing expected deployment runtime setting: %s\n' "$2" >&2
+    exit 1
+  fi
+}
+
+nodeport="$(awk '
+  /^---/ { selected=0 }
+  /name: hub-frontend-nodeport$/ { selected=1 }
+  selected { print }
+' "$ROOT/base/hub/kerberos-hub-nodeports.yaml")"
+assert_contains "$nodeport" 'targetPort: http$'
+assert_contains "$nodeport" 'nodePort: 32080'
+
+rendered="$(helm template hub "$CHART" --values "$ROOT/base/hub/kerberos-hub-values.yaml" \
+  --set mode=ui --set kerberoshub.frontend.demoEnabled=true)"
+if grep -q 'image: "docker.io/ghcr.io/' <<< "$rendered"; then
+  echo "Fully qualified GHCR image was prefixed with docker.io" >&2
+  exit 1
+fi
+for name in hub-frontend hub-frontend-demo; do
+  deployment="$(awk -v name="$name" '
+    function emit() { if (deployment && matched) printf "%s", block }
+    /^---$/ { emit(); block=""; deployment=0; matched=0 }
+    { block=block $0 "\n" }
+    /^kind: Deployment$/ { deployment=1 }
+    $0 == "  name: " name { matched=1 }
+    END { emit() }
+  ' <<< "$rendered")"
+  for setting in 'containerPort: 8080' 'runAsNonRoot: true' 'runAsUser: 65532' \
+    'readOnlyRootFilesystem: true' 'mountPath: /tmp' 'name: frontend-tmp'; do
+    assert_contains "$deployment" "^[[:space:]]*(- )?$setting$"
+  done
+  assert_contains "$deployment" 'image: "ghcr.io/uug-ai/hub-frontend:v1.19.28"$'
+  echo "OK: $name uses the promoted image and matching hardened runtime"
+done
+echo "Deployment Hub runtime preflight passed."