kerberos-io/deployment
1
0
Fork 0
mirror of https://github.com/kerberos-io/deployment.git synced 2026-08-23 15:18:32 +00:00
Code Issues Packages Projects Releases Wiki Activity

adjust deployment microk8s

Browse Source
This commit is contained in:
Cedric Verstraeten
2024-08-13 14:19:31 +02:00
parent bf303f83cf
commit 85390e82c5
3 changed files with 285 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files

235
README.k8s.md
View File

@@ -2,8 +2,6 @@
Within this tutorial we will install the Kerberos.io edge stack (Kerberos Agent, Kerberos Vault and the Data filtering service). This will allow us to store recordings from multiple cameras at the edge
## Kerberos Vault
### OpenEBS
Some of the services we'll leverage such as MongoDB or Minio require storage, to persist their data safely. In a managed Kubernetes cluster, the relevant cloud provider will allocate storage automatically for you, as you might expect this is not the case for a self-hosted cluster.
@@ -31,27 +29,101 @@ Once you are ok with the `BasePath` go ahead and apply the operator.
Once done it should start installing several resources in the `openebs` namespace. If all resources are created successfully we can launch the `helm install` for MongoDB.
### MongoDB
### Object storage: MinIO
MinIO is a high-performance, distributed object storage system that is compatible with Amazon S3 cloud storage service. It is designed to handle large-scale data storage and retrieval, making it an ideal choice for modern cloud-native applications.
In the context of the Kerberos.io stack, MinIO will be used to store recordings from the Kerberos Agents. These recordings are crucial for surveillance and monitoring purposes, and having a reliable storage solution like MinIO ensures that the data is stored securely and can be accessed efficiently.
```bash
kubectl create namespace minio-tenant
```
```bash
kubectl apply -k github.com/minio/operator\?ref=v6.0.1
```
Next we'll create a tenant
```bash
sed -i 's/openebs-hostpath/microk8s-hostpath/g' ./minio-tenant-base.yaml
kubectl apply -f minio-tenant-base.yaml
```
We create a bucket in the minio tenant
```bash
kubectl port-forward svc/myminio-hl 9000 -n minio-tenant
```
You might need to install the minio client if not yet available.
```bash
curl https://dl.min.io/client/mc/release/linux-amd64/mc \
--create-dirs \
-o $HOME/minio-binaries/mc
chmod +x $HOME/minio-binaries/mc
export PATH=$PATH:$HOME/minio-binaries/
```
```bash
mc alias set myminio http://localhost:9000 minio minio123 --insecure
mc mb myminio/mybucket --insecure
```
or if not possible we will access the minio console using a reverse tunnel.
```bash
kubectl port-forward svc/myminio-console -n minio-tenant 8080:9090
ssh -L 8080:localhost:8080 youruser@x.x.x.x
```
To access the application, open your browser and navigate to `localhost:8080`. Use the credentials specified in the `minio-tenant-base.yaml` configuration file to log in. Once logged in, you can create a new bucket, such as `mybucket`, or choose a name of your preference.
### Database: MongoDB
When using Kerberos Vault, it will persist references to the recordings stored in your storage provider in a MongoDB database. As used before, we are using `helm` to install MongoDB in our Kubernetes cluster. Within the Kerberos Vault project we are using the latest official mongodb driver, so we support all major MongoDB versions (4.x, 5.x, 6.x, 7.x).
Have a look into the `./mongodb/values.yaml` file, you will find plenty of configurations for the MongoDB helm chart. To change the username and password of the MongoDB instance, go ahead and [find the attribute where](https://github.com/kerberos-io/vault/blob/master/kubernetes/mongodb/values.yaml#L148) you can change the root password. Please note that we are using the official [Bitnami Mongodb helm chart](https://github.com/bitnami/charts/tree/main/bitnami/mongodb), so please use their repository for more indepth configuration.
Have a look into the `./mongodb-values.yaml` file, you will find plenty of configurations for the MongoDB helm chart. To change the username and password of the MongoDB instance, go ahead and [find the attribute where](https://github.com/kerberos-io/vault/blob/master/kubernetes/mongodb/values.yaml#L148) you can change the root password. Please note that we are using the official [Bitnami Mongodb helm chart](https://github.com/bitnami/charts/tree/main/bitnami/mongodb), so please use their repository for more indepth configuration.
Next to that you might also consider a SaaS MongoDB deployment using MongoDB Atlas or using a managed cloud like AWS, GCP, Azure or Alibaba cloud. A managed service takes away a lot of management and maintenance from your side (backups, security, sharing, etc). If you do want to install MongoDB in your own cluster then please continue with this tutorial.
helm repo add bitnami https://charts.bitnami.com/bitnami
kubectl create namespace mongodb
Note: If you are installing a self-hosted Kubernetes cluster, we recommend using `openebs`. Therefore make sure to uncomment the `global`.`storageClass` attribute, and make sure it's using `openebs-hostpath` instead.
Note: If you are installing a self-hosted Kubernetes cluster, we recommend using `openebs`. Therefore make sure to uncomment the `global`.`storageClass` attribute, and make sure it's using `microk8s-hostpath` instead.
sed -i 's/openebs-hostpath/microk8s-hostpath/g' ./mongodb-values.yaml
helm install mongodb -n mongodb bitnami/mongodb --values ./mongodb-values.yaml
Once installed successfully, we should verify if the password has been set correctly. Print out the password using `echo $MONGODB_ROOT_PASSWORD` and confirm the password is what you've specified in the `values.yaml` file.
Or after updating the `./mongodb-values.yaml` file again
export MONGODB_ROOT_PASSWORD=$(kubectl get secret -n mongodb mongodb -o jsonpath="{.data.mongodb-root-password}" | base64 --decode)
echo $MONGODB_ROOT_PASSWORD
helm upgrade mongodb -n mongodb bitnami/mongodb --values ./mongodb-values.yaml
### Config Map
### Message broker: RabbitMQ
```bash
kubectl create namespace rabbitmq
```
```bash
sed -i 's/openebs-hostpath/microk8s-hostpath/g' ./rabbitmq-values.yaml
helm install rabbitmq bitnami/rabbitmq -n rabbitmq -f rabbitmq-values.yaml
kubectl get po -A -w
```
```bash
helm upgrade rabbitmq bitnami/rabbitmq -n rabbitmq -f rabbitmq-values.yaml
```
```bash
helm del rabbitmq -n rabbitmq
```
### Kerberos Vault
#### Config Map
Kerberos Vault requires a configuration to connect to the MongoDB instance. To handle this `configmap` map is created in the `./mongodb/mongodb.config.yaml` file. However you might also use the environment variables within the `./kerberos-vault/deployment.yaml` file to configure the mongodb connection.
@@ -59,61 +131,144 @@ Modify the MongoDB credentials in the `./mongodb/mongodb.config.yaml`, and make
As mentioned above a managed MongoDB is easier to setup and manage, for example for MongoDB Atlas, you will get a MongoDB URI in the form of `"mongodb+srv://xx:xx@kerberos-hub.xxx.mongodb.net/?retryWrites=true&w=majority&appName=xxx"`. By applying this value into the `MONGODB_URI` field, you will have setup your MongoDB connection successfully.
- name: MONGODB_URI
value: "mongodb+srv://xx:xx@kerberos-hub.xxx.mongodb.net/?retryWrites=true&w=majority&appName=xxx"
```yaml
- name: MONGODB_URI
value: "mongodb+srv://xx:xx@kerberos-hub.xxx.mongodb.net/?retryWrites=true&w=majority&appName=xxx"
```
Once you applied this value, the other values like `MONGODB_USERNAME`, `MONGODB_PASSWORD` and others will be ignored. If you don't like the `MONGODB_URI` format you can still use the old way of defining the MongoDB connection by providing the different values.
- name: MONGODB_USERNAME
value: "root"
- name: MONGODB_PASSWORD
--> value: "yourmongodbpassword"
```yaml
- name: MONGODB_USERNAME
value: "root"
- name: MONGODB_PASSWORD
-> value: "yourmongodbpassword"
```
Create the config map in the `kerberos-vault` namespace.
kubectl create namespace kerberos-vault
kubectl apply -f ./mongodb-config.yaml -n kerberos-vault
```bash
kubectl create namespace kerberos-vault
```
### Deployment
Apply the mongodb configuration file, so the Kerberos Vault application knows how to connect to the MongoDB.
```bash
kubectl apply -f ./mongodb-config.yaml -n kerberos-vault
```
#### Deployment
To install the Kerberos Vault web app inside your cluster, simply execute below `kubectl` command. This will create the deployment for us with the necessary configurations, and exposed it on internal/external IP address, thanks to our `LoadBalancer` MetalLB or cloud provider.
kubectl apply -f ./kerberos-vault-deployment.yaml -n kerberos-vault
```bash
kubectl apply -f ./kerberos-vault-deployment.yaml -n kerberos-vault
```
### Access the Kerberos Vault
Verify if the pod is running
If you have chosen to use the `NodePort` configuration you should be able to reach the Kerberos Vault using the `http://localhost:30080` endpoint in your browser. However if you have a server installation without a GUI, you might choose to do a reverse proxy so you can open the browser on your local machine.
```bash
kubectl get po -w -A
```
ssh -L 8080:localhost:30080 user@server-ip -p 22
#### Access the UI
### MinIO storage provider
If you have opted for the `NodePort` configuration, you can access the Kerberos Vault via the `http://localhost:30080` endpoint in your browser. For server installations without a GUI, consider setting up a reverse proxy to enable browser access from your local machine. Alternatively, you may utilize a `LoadBalancer` if one is available or if you are deploying on a managed Kubernetes service.
kubectl create namespace minio-tenant
```bash
ssh -L 8080:localhost:30080 user@server-ip -p 22
```
kubectl apply -f minio-storageclass.yaml
#### Configure the Kerberos Vault
# This will create the operator
kubectl kustomize github.com/minio/operator?ref=v6.0.1 | kubectl apply -f -
With the Kerberos Vault installed, we can proceed to configure the various components. Currently, this must be done through the Kerberos Vault UI, but we plan to make it configurable via environment variables, eliminating the need for manual UI configurations.
# Install the tenants
kubectl apply -f tenant-base.yaml
- Navigate to the `Storage Providers` menu and select the (+ Add Storage Provider) button. A modal will appear where you can input the required details. After entering the information, click the "Verify" button to ensure the configuration is valid. Once you receive a "Configuration is valid and working" message, click the "Add Storage Provider" button to complete the process.
# Watch the tenant creation
watch kubectl get all -n minio-tenant
- Minio
- Enabled: true
- Provider name: minio
- Bucket name: mybucket
- Region: na
- Hostname: myminio-hl.minio-tenant:9000
- Access key: minio
- Secret key: minio123
### Create the integration RabbitMQ
- Navigate to the `Integrations` menu and select the (+ Add Integration) button. A modal will appear where you can input the required details. After entering the information, click the "Verify" button to ensure the configuration is valid. Once you receive a "Configuration is valid and working" message, click the "Add Integration" button to complete the process.
kubectl create namespace rabbitmq
- RabbitMQ
- Enabled: true
- Integration name: rabbitmq
- Broker: rabbitmq.rabbitmq:5672
- Exchange:
- Queue: data-filtering
- Username: yourusername
- Password: yourpassword
helm install rabbitmq bitnami/rabbitmq -n rabbitmq -f rabbitmq-values.yaml
- Navigate to the `Accounts` menu and click the (+ Add Account) button. A modal will appear where you can input the required details. After entering the information, click the "Add Account" button to complete the process.
### Configure the Kerberos Vault
- Enabled: true
- Account name: myaccount
- Main provider: minio
- Day limit: 30
- Integration: rabbitmq
- Directory: \*
- Access key: XJoi2@bgSOvOYBy#
- Secret key: OGGqat4lXRpL@9XBYc8FUaId@5
..... (should be done through env files so we do not need to get in the UI)
create the minio provider, add integration
### Create a Kerberos Agent
### Create an agent
After deploying the Kerberos Vault and configuring the necessary services for storage, database, and integration, you can proceed to deploy the Kerberos Agent with the appropriate configuration. Review the `kerberos-agent-deployment.yaml` file and adjust the relevant settings, such as the RTSP URL, to ensure proper functionality. Please note that you can allow opt for the [Kerberos Factory](https://github.com/kerberos-io/factory/tree/master/kubernetes) which gives you a UI to manage the creation of Kerberos Agents.
### Create the data filtering
```bash
kubectl apply -f kerberos-agent-deployment.yaml
```
### add forwarding integration
Review the creation of the Kerberos Agent and review the logs of the container to validate the Kerberos Agent is able to connect to the IP camera, and if a recording is being created and transferred to the Kerberos Vault
```bash
kubectl get po -w -A
kubectl logs -f kerberos-agent...
```
To validate the Kerberos Vault and review any stored recordings, access the user interface at `http://localhost:30080` (after establishing the reverse tunnel).
### Optimized Data Filtering for Enhanced Bandwidth Efficiency and Relevance
Once your Kerberos Agents are properly connected and all recordings are stored in the Kerberos Vault, you may encounter additional challenges such as bandwidth limitations, storage constraints, and the need to efficiently locate relevant data. To accomplish this, we can configure an integration to filter the recordings, ensuring that only the relevant ones are retained.
Assuming all configurations are correctly set and all Kubernetes deployments are operational, you can apply the `data-filtering-deployment.yaml` deployment. This deployment will schedule a pod that listens to the configured integration in Kerberos Vault and runs a YOLOv8 model to evaluate the recordings and match them against specified conditions.
```bash
kubectl apply -f data-filtering-deployment.yaml
```
Each time a recording is stored in the Kerberos Vault, the `data-filtering` pod will receive a notification and execute the specified model (YOLOv8 by default). Based on the defined conditions, the `data-filtering` pod may forward the recording to a remote Kerberos Vault, trigger alerts, or send notifications.
Ensure that the `data-filtering` workload is actively running, receiving messages from the integration, and performing the necessary processing tasks.
```bash
kubectl get po -w -A
kubectl logs -f data...
```
### Add forwarding integration
We'll need to access the UI again to add the integration
```bash
ssh -L 8080:localhost:30080 user@server-ip -p 22
```
Go to the Kerberos Vault application in your browser and open the integration section, add a new integration.
- Add an integration
- Kerberos Vault
- Enabled: true
- Integration name: rabbitmq
- Broker: rabbitmq.rabbitmq:5672
- Exchange:
- Queue: data-filtering
- Username: yourusername
- Password: yourpassword

109
README.microk8s.md
View File

@@ -71,7 +71,7 @@ One of the key advantages of MicroK8s is its out-of-the-box addons, which can be
```bash
microk8s enable dns
microk8s enable dashboard
microk8s enable gpu
microk8s enable nvidia
microk8s enable hostpath-storage
```
@@ -87,6 +87,15 @@ Or view the pod status with:
kubectl get po -w -A
```
### Clone repository
Next, we will clone this repository to our local environment. This will allow us to execute the necessary configuration files for installing the Minio operator, MongoDB Helm chart, and other required components.
```bash
git clone https://github.com/kerberos-io/deployment
cd deployment
```
### Object storage: MinIO
MinIO is a high-performance, distributed object storage system that is compatible with Amazon S3 cloud storage service. It is designed to handle large-scale data storage and retrieval, making it an ideal choice for modern cloud-native applications.
@@ -94,11 +103,13 @@ MinIO is a high-performance, distributed object storage system that is compatibl
In the context of the Kerberos.io stack, MinIO will be used to store recordings from the Kerberos Agents. These recordings are crucial for surveillance and monitoring purposes, and having a reliable storage solution like MinIO ensures that the data is stored securely and can be accessed efficiently.
```bash
kubectl create namespace minio-tenant
git clone --depth 1 --branch v6.0.1 https://github.com/minio/operator.git && kubectl apply -k operator/
```
View the minio operator status with:
```bash
kubectl apply -k github.com/minio/operator\?ref=v6.0.1
kubectl get po -w -A
```
Next we'll create a tenant
@@ -108,12 +119,20 @@ sed -i 's/openebs-hostpath/microk8s-hostpath/g' ./minio-tenant-base.yaml
kubectl apply -f minio-tenant-base.yaml
```
We create a bucket in the minio tenant
View the minio tenant status with:
```bash
kubectl port-forward svc/myminio-hl 9000 -n minio-tenant
kubectl get po -w -A
```
You should see the `myminio` tenant being created
```bash
minio-tenant myminio-pool-0-0 2/2 Running 0 60s
```
We create a bucket in the minio tenant
You might need to install the minio client if not yet available.
```bash
@@ -125,11 +144,27 @@ chmod +x $HOME/minio-binaries/mc
export PATH=$PATH:$HOME/minio-binaries/
```
Expose the minio service so we can reach it from our local station.
```bash
kubectl port-forward svc/myminio-hl 9000 -n minio-tenant &
```
Create the `mybucket` bucket in the `myminio` tenant.
```bash
mc alias set myminio http://localhost:9000 minio minio123 --insecure
mc mb myminio/mybucket --insecure
```
The expected output should resemble the following:
```bash
root@microk8s:~/deployment# mc mb myminio/mybucket --insecure
Handling connection for 9000
Bucket created successfully `myminio/mybucket`.
```
or if not possible we will access the minio console using a reverse tunnel.
```bash
@@ -147,36 +182,47 @@ Have a look into the `./mongodb-values.yaml` file, you will find plenty of confi
Next to that you might also consider a SaaS MongoDB deployment using MongoDB Atlas or using a managed cloud like AWS, GCP, Azure or Alibaba cloud. A managed service takes away a lot of management and maintenance from your side (backups, security, sharing, etc). If you do want to install MongoDB in your own cluster then please continue with this tutorial.
helm repo add bitnami https://charts.bitnami.com/bitnami
kubectl create namespace mongodb
```bash
helm repo add bitnami https://charts.bitnami.com/bitnami
kubectl create namespace mongodb
```
Note: If you are installing a self-hosted Kubernetes cluster, we recommend using `openebs`. Therefore make sure to uncomment the `global`.`storageClass` attribute, and make sure it's using `microk8s-hostpath` instead.
sed -i 's/openebs-hostpath/microk8s-hostpath/g' ./mongodb-values.yaml
helm install mongodb -n mongodb bitnami/mongodb --values ./mongodb-values.yaml
```bash
sed -i 's/openebs-hostpath/microk8s-hostpath/g' ./mongodb-values.yaml
helm install mongodb -n mongodb bitnami/mongodb --values ./mongodb-values.yaml
```
Or after updating the `./mongodb-values.yaml` file again
helm upgrade mongodb -n mongodb bitnami/mongodb --values ./mongodb-values.yaml
```bash
helm upgrade mongodb -n mongodb bitnami/mongodb --values ./mongodb-values.yaml
```
View the MongoDB status and wait until it's properly running
```bash
kubectl get po -w -A
```
### Message broker: RabbitMQ
Now we can store recordings in `MinIO` and metadata in `MongoDB`. The remaining task is to store events in a message broker such as `RabbitMQ`. This setup enables an asynchronous event-driven approach, allowing you to receive real-time event each time a recording is uploaded. By doing so, you can develop custom logic and abstract the camera network from your machine learning models or computer vision algorithms. The primary focus is on the recordings, not the complex camera infrastructure.
```bash
kubectl create namespace rabbitmq
kubectl create namespace rabbitmq
```
```bash
sed -i 's/openebs-hostpath/microk8s-hostpath/g' ./rabbitmq-values.yaml
helm install rabbitmq bitnami/rabbitmq -n rabbitmq -f rabbitmq-values.yaml
kubectl get po -A -w
```
```bash
helm upgrade rabbitmq bitnami/rabbitmq -n rabbitmq -f rabbitmq-values.yaml
```
View the RabbitMQ status and wait until it's properly running
```bash
helm del rabbitmq -n rabbitmq
kubectl get po -w -A
```
### Kerberos Vault
@@ -258,7 +304,7 @@ With the Kerberos Vault installed, we can proceed to configure the various compo
- Enabled: true
- Integration name: rabbitmq
- Broker: rabbitmq.rabbitmq:5672
- Exchange:
- Exchange: <empty>
- Queue: data-filtering
- Username: yourusername
- Password: yourpassword
@@ -271,12 +317,12 @@ With the Kerberos Vault installed, we can proceed to configure the various compo
- Day limit: 30
- Integration: rabbitmq
- Directory: \*
- Access key: XJoi2@bgSOvOYBy#
- Secret key: OGGqat4lXRpL@9XBYc8FUaId@5
- Access key: XJoi2@bgSOvOYBy# (or generate new keys, but don't forget to update them in the next steps)
- Secret key: OGGqat4lXRpL@9XBYc8FUaId@5 (or generate new keys, but don't forget to update them in the next steps)
### Create a Kerberos Agent
After deploying the Kerberos Vault and configuring the necessary services for storage, database, and integration, you can proceed to deploy the Kerberos Agent with the appropriate configuration. Review the `kerberos-agent-deployment.yaml` file and adjust the relevant settings, such as the RTSP URL, to ensure proper functionality. Please note that you can allow opt for the [Kerberos Factory](https://github.com/kerberos-io/factory/tree/master/kubernetes) which gives you a UI to manage the creation of Kerberos Agents.
After deploying the Kerberos Vault and configuring the necessary services for storage, database, and integration, you can proceed to deploy the Kerberos Agent with the appropriate configuration. Review the `kerberos-agent-deployment.yaml` file and adjust the relevant settings, such as the RTSP URL, to ensure proper functionality. Please note that you can allow opt for the [Kerberos Factory](https://github.com/kerberos-io/factory/tree/master/kubernetes) which gives you a UI to manage the creation of Kerberos Agents. Also please note if you generated new the keys in the previous Kerberos Vault account creation, you need to update those in the Kerberos Agent deployment.
```bash
kubectl apply -f kerberos-agent-deployment.yaml
@@ -295,7 +341,7 @@ To validate the Kerberos Vault and review any stored recordings, access the user
Once your Kerberos Agents are properly connected and all recordings are stored in the Kerberos Vault, you may encounter additional challenges such as bandwidth limitations, storage constraints, and the need to efficiently locate relevant data. To accomplish this, we can configure an integration to filter the recordings, ensuring that only the relevant ones are retained.
Assuming all configurations are correctly set and all Kubernetes deployments are operational, you can apply the `data-filtering-deployment.yaml` deployment. This deployment will schedule a pod that listens to the configured integration in Kerberos Vault and runs a YOLOv8 model to evaluate the recordings and match them against specified conditions.
Assuming all configurations are correctly set and all Kubernetes deployments are operational, you can apply the `data-filtering-deployment.yaml` deployment. This deployment will schedule a pod that listens to the configured integration in Kerberos Vault and runs a YOLOv8 model to evaluate the recordings and match them against specified conditions. Please note that if you do not have a GPU on the device, you will need to disable the resource limit of the nvidia/gpu. Once done the filtering will run on the CPU.
```bash
kubectl apply -f data-filtering-deployment.yaml
@@ -330,3 +376,24 @@ Go to the Kerberos Vault application in your browser and open the integration se
- Queue: data-filtering
- Username: yourusername
- Password: yourpassword
## Cleanup
If you consider to remove the Kerberos.io stack you might just disable the microk8s installation
```bash
microk8s reset
sudo snap remove microk8s
```
or if you want to keep the microk8s installation you can also delete the individual deployments.
```bash
kubectl delete -f data-filtering-deployment.yaml
kubectl delete -f kerberos-agent-deployment.yaml
kubectl delete -f ./kerberos-vault-deployment.yaml -n kerberos-vault
kubectl delete -f ./mongodb-config.yaml -n kerberos-vault
helm del rabbitmq -n rabbitmq
helm del mongodb -n mongodb
git clone --depth 1 --branch v6.0.1 https://github.com/minio/operator.git && kubectl delete -k operator/
```

4
data-filtering-deployment.yaml
View File

@@ -39,9 +39,9 @@ spec:
- name: STORAGE_URI
value: "http://vault-lb.kerberos-vault/api"
- name: STORAGE_ACCESS_KEY
value: "52gyELgxutOXUWhF"
value: "XJoi2@bgSOvOYBy#"
- name: STORAGE_SECRET_KEY
value: "k8DrcB@hQ5XfxDENzDKcnkxBHx"
value: "OGGqat4lXRpL@9XBYc8FUaId@5"
- name: LOGGING
value: "True"