mirror of
https://github.com/kerberos-io/agent.git
synced 2026-10-21 17:58:24 +00:00
Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Kilian Boute
|
07edec5d55 |
@@ -18,7 +18,7 @@
|
||||
},
|
||||
"forwardPorts": [
|
||||
3000,
|
||||
8082
|
||||
8080
|
||||
],
|
||||
"postCreateCommand": "cd ui && yarn install && yarn build && cd ../machinery && go mod download && bash ./verify-moq-devcontainer.sh"
|
||||
}
|
||||
@@ -20,7 +20,9 @@ jobs:
|
||||
go-version: "1.25.x"
|
||||
check-latest: true
|
||||
cache: true
|
||||
cache-dependency-path: machinery/go.sum
|
||||
cache-dependency-path: |
|
||||
machinery/go.sum
|
||||
examples/frame-processor/go.sum
|
||||
- name: Install native dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
@@ -39,3 +41,5 @@ jobs:
|
||||
run: cd machinery && go vet -v ./...
|
||||
- name: Test
|
||||
run: cd machinery && go test -v ./...
|
||||
- name: Test frame processor example
|
||||
run: cd examples/frame-processor && GOWORK=off go test -v ./...
|
||||
|
||||
@@ -10,7 +10,7 @@ jobs:
|
||||
build:
|
||||
runs-on: ${{ matrix.runner }}
|
||||
permissions:
|
||||
contents: read
|
||||
contents: write
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
@@ -21,6 +21,11 @@ jobs:
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
- name: Login to DockerHub
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
username: ${{ secrets.DOCKER_USERNAME }}
|
||||
password: ${{ secrets.DOCKER_PASSWORD }}
|
||||
- name: Run Build
|
||||
run: |
|
||||
docker build --build-arg VERSION=$(echo "${{ github.event.pull_request.head.sha }}" | cut -c1-7) -t ${{ matrix.architecture }} .
|
||||
|
||||
Vendored
+1
-1
@@ -14,7 +14,7 @@
|
||||
"-action",
|
||||
"run",
|
||||
"-port",
|
||||
"8082"
|
||||
"8080"
|
||||
],
|
||||
"envFile": "${workspaceFolder}/machinery/.env.local",
|
||||
"buildFlags": "--tags dynamic,moq",
|
||||
|
||||
@@ -76,57 +76,6 @@ There are a myriad of cameras out there (USB, IP and other cameras), and it migh
|
||||
1. [What is new?](#what-is-new)
|
||||
1. [Contributors](#contributors)
|
||||
|
||||
### Direct S3 recording layout
|
||||
|
||||
Direct S3 uploads store recordings as
|
||||
`<s3.username>/yyyy/mm/dd/<timestamp>-<deviceKey>.mp4`. The timestamp is the
|
||||
recording's Unix timestamp in milliseconds, and the date is its UTC date, not the upload
|
||||
date. Timestamp and device key come from the recording's `.metadata` upload
|
||||
checkpoint, which stores the timestamp in milliseconds. The uploader also
|
||||
preserves checkpoint duration, FPS (when available), and encryption status as S3
|
||||
object metadata. The device key must be non-empty and contain no path separators.
|
||||
Retried uploads use the same object key, even after midnight. Millisecond
|
||||
precision keeps recordings starting within the same second distinct.
|
||||
|
||||
This replaces the previous `<s3.username>/<filename>.mp4` layout for new uploads.
|
||||
Existing objects are not moved; consumers of direct S3 uploads must support both
|
||||
layouts when reading historical recordings. Updated Kerberos Vault deployments
|
||||
use the same filename and dated layout under the authenticated account directory
|
||||
for both direct Vault uploads and Hub-proxied uploads.
|
||||
|
||||
Only historical recordings with missing, empty, or FPS-only queue markers fall
|
||||
back to the legacy filename format and configured Agent key. A structured
|
||||
checkpoint takes precedence over both the filename and current configured key;
|
||||
an invalid or unreadable checkpoint is reported as an upload error, not silently
|
||||
replaced with filename-derived values.
|
||||
|
||||
The shorter filename cannot be decoded by legacy filename-only consumers.
|
||||
Integrations must preserve structured recording metadata, including the device
|
||||
key (`productid`) and recording timestamp (`event-timestamp`), as well as the full
|
||||
object key.
|
||||
|
||||
### Local recording layout
|
||||
|
||||
New local recordings are stored under
|
||||
`data/recordings/yyyy/mm/dd/<timestamp>-<deviceKey>.mp4`, using the recording's UTC
|
||||
start date and Unix-millisecond timestamp. For example:
|
||||
`data/recordings/2026/01/02/1767355200123-device-key.mp4`.
|
||||
The filename no longer embeds the camera name, region, change count, or duration,
|
||||
and it does not change when the recording is finalized.
|
||||
Those attributes, together with FPS and encryption status, are captured in the
|
||||
`.metadata` upload checkpoint and sent as structured HTTP/tus metadata.
|
||||
|
||||
Existing flat recordings and dated recordings with older long filenames remain
|
||||
supported without a migration. Recording lists, day filters, playback, uploaders,
|
||||
and automatic cleanup handle both filename formats and layouts. Upload queue
|
||||
checkpoints remain flat in `data/cloud/`. Successful-upload removal also resolves
|
||||
both local layouts. Deploy the updated Vault alongside Agent for canonical
|
||||
`<account-directory>/yyyy/mm/dd/<timestamp>-<deviceKey>.mp4` storage keys and
|
||||
metadata-driven Hub integration events.
|
||||
Directory decryption also processes dated subdirectories and writes each output
|
||||
to a sibling `decrypted/` directory; those outputs are excluded from recording
|
||||
lists and automatic cleanup.
|
||||
|
||||
## Quickstart - Docker
|
||||
|
||||
The easiest way to get your Kerberos Agent up and running is to use our public image on [Docker hub](https://hub.docker.com/r/kerberos/agent). Once you have selected a specific tag, run `docker` command below, which will open the web interface of your Kerberos agent on port `80`, and off you go. For a more configurable and persistent deployment have a look at [Running and automating a Kerberos Agent](#running-and-automating-a-kerberos-agent).
|
||||
@@ -416,13 +365,11 @@ See [RTSPS and TLS certificates](README-RTSPS-TLS.md) for the complete Bosch UI,
|
||||
| `AGENT_KERBEROSVAULT_SECRET_KEY` | The secret key of a Kerberos Vault account. | "" |
|
||||
| `AGENT_KERBEROSVAULT_PROVIDER` | A Kerberos Vault provider you have created (optional). | "" |
|
||||
| `AGENT_KERBEROSVAULT_DIRECTORY` | The directory, in the Kerberos vault, where the recordings will be stored. | "" |
|
||||
| `AGENT_KERBEROSVAULT_CUSTOM_HEADERS` | JSON object of custom upload headers that Vault includes as integration metadata. | `{"site_id":"site-1"}` |
|
||||
| `AGENT_KERBEROSVAULT_SECONDARY_URI` | The Kerberos Vault API url. | "https://vault.domain.com/api" |
|
||||
| `AGENT_KERBEROSVAULT_SECONDARY_ACCESS_KEY` | The access key of a secondary Kerberos Vault account. | "" |
|
||||
| `AGENT_KERBEROSVAULT_SECONDARY_SECRET_KEY` | The secret key of a secondary Kerberos Vault account. | "" |
|
||||
| `AGENT_KERBEROSVAULT_SECONDARY_PROVIDER` | A secondary Kerberos Vault provider you have created (optional). | "" |
|
||||
| `AGENT_KERBEROSVAULT_SECONDARY_DIRECTORY` | The directory, in the secondary Kerberos vault, where the recordings will be stored. | "" |
|
||||
| `AGENT_KERBEROSVAULT_SECONDARY_CUSTOM_HEADERS` | JSON object of custom upload headers for the secondary Vault. | `{"site_id":"site-1"}` |
|
||||
| `AGENT_DROPBOX_ACCESS_TOKEN` | The Access Token from your Dropbox app, that is used to leverage the Dropbox SDK. | "" |
|
||||
| `AGENT_DROPBOX_DIRECTORY` | The directory, in Dropbox, where the recordings will be stored. | "" |
|
||||
| `AGENT_ENCRYPTION` | Enable 'true' or disable 'false' end-to-end encryption for MQTT messages. | "false" |
|
||||
@@ -461,94 +408,6 @@ Smaller chunks provide more frequent resumable checkpoints but create more HTTP
|
||||
requests. Larger chunks reduce request overhead but require more data to be
|
||||
retransmitted when a request fails.
|
||||
|
||||
### Custom metadata headers for Kerberos Vault
|
||||
|
||||
Use `AGENT_KERBEROSVAULT_CUSTOM_HEADERS` to attach your own metadata (for
|
||||
example a site, line, or equipment identifier) to every recording uploaded to
|
||||
Kerberos Vault. The value is a JSON object whose keys are header names and whose
|
||||
values are strings. The Agent sends these as HTTP headers on every upload, and
|
||||
Vault adds them to the `metadata` object of its integration messages (Kafka,
|
||||
MQTT, SQS, RabbitMQ, etc.):
|
||||
|
||||
```json
|
||||
{
|
||||
"timestamp": "2026-09-24T15:06:45Z",
|
||||
"key": "agent1/1790262405_3-425_agent1_0-0-0-0_-1_10000.mp4",
|
||||
"device": "agent1",
|
||||
"metadata": {
|
||||
"site_id": "site-1",
|
||||
"line_id": "line-2",
|
||||
"equipment_id": "equipment-3"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The value must be valid JSON on a single line, so wrap it in single quotes in a
|
||||
shell:
|
||||
|
||||
```bash
|
||||
docker run -p 80:80 --name mycamera \
|
||||
-e AGENT_CLOUD=kstorage \
|
||||
-e AGENT_KERBEROSVAULT_URI=https://vault.domain.com/api \
|
||||
-e AGENT_KERBEROSVAULT_ACCESS_KEY=xxx \
|
||||
-e AGENT_KERBEROSVAULT_SECRET_KEY=xxx \
|
||||
-e AGENT_KERBEROSVAULT_DIRECTORY=agent1 \
|
||||
-e AGENT_KERBEROSVAULT_CUSTOM_HEADERS='{"site_id":"site-1","line_id":"line-2","equipment_id":"equipment-3"}' \
|
||||
-d --restart=always kerberos/agent:latest
|
||||
```
|
||||
|
||||
In `docker compose` or an `.env` file:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
agent:
|
||||
image: kerberos/agent:latest
|
||||
environment:
|
||||
AGENT_CLOUD: kstorage
|
||||
AGENT_KERBEROSVAULT_CUSTOM_HEADERS: '{"site_id":"site-1","line_id":"line-2","equipment_id":"equipment-3"}'
|
||||
```
|
||||
|
||||
```dotenv
|
||||
AGENT_KERBEROSVAULT_CUSTOM_HEADERS={"site_id":"site-1","line_id":"line-2","equipment_id":"equipment-3"}
|
||||
```
|
||||
|
||||
In Kubernetes, quote the value so YAML keeps it as a string:
|
||||
|
||||
```yaml
|
||||
env:
|
||||
- name: AGENT_KERBEROSVAULT_CUSTOM_HEADERS
|
||||
value: '{"site_id":"site-1","line_id":"line-2","equipment_id":"equipment-3"}'
|
||||
```
|
||||
|
||||
Use `AGENT_KERBEROSVAULT_SECONDARY_CUSTOM_HEADERS` for the secondary Vault. In
|
||||
ConfigMap deployments you can set shared defaults with the `GLOBAL_` prefix
|
||||
(for example `GLOBAL_AGENT_KERBEROSVAULT_CUSTOM_HEADERS`) and override them per
|
||||
agent with `AGENT_KERBEROSVAULT_CUSTOM_HEADERS`. You can also set the same JSON
|
||||
under **Settings → Persistence → Custom upload headers** in the Agent UI, which
|
||||
stores it as `kstorage.custom_headers` in `config.json`.
|
||||
|
||||
The rules for this value are:
|
||||
|
||||
- It must be a JSON object with string values. Numbers, `null`, arrays, and
|
||||
nested objects are rejected.
|
||||
- It can contain up to 32 headers, with names up to 128 characters and values
|
||||
up to 4096 bytes.
|
||||
- Header names must be valid HTTP tokens (letters, digits, and ``!#$%&'*+-.^_`|~``).
|
||||
Keys are matched case-insensitively, so `site_id` and `SITE_ID` cannot both
|
||||
be used.
|
||||
- Reserved names are rejected: anything starting with `X-Kerberos-`,
|
||||
`Authorization`, `Cookie`, `Host`, `Content-Type`, `Content-Length`,
|
||||
`Trace-Id`, tus protocol headers (`Tus-Resumable`, `Upload-*`), and
|
||||
hop-by-hop headers.
|
||||
- Values cannot contain line breaks, tabs, or other control characters.
|
||||
|
||||
If the value is invalid, the Agent logs an error and does not upload to that
|
||||
Vault until the configuration is fixed. The recording stays queued, so it will
|
||||
upload once the value is corrected. Header names reach Vault unchanged, but
|
||||
proxies may normalise their casing. If Vault redirects an upload to another
|
||||
host, the Agent removes the custom headers and Vault credentials before
|
||||
following the redirect.
|
||||
|
||||
|
||||
## Encryption
|
||||
|
||||
@@ -653,10 +512,7 @@ The `web` is a **React** project which is the main entry point for an end user t
|
||||
cd ui
|
||||
yarn start
|
||||
|
||||
This starts the web app on port `3000`. In development, the UI connects to the
|
||||
Agent API on port `8082`. Override that port when needed:
|
||||
|
||||
REACT_APP_AGENT_API_PORT=9090 yarn start
|
||||
This will start a webserver and launches the web app on port `3000`.
|
||||
|
||||
|
||||
|
||||
@@ -668,18 +524,18 @@ Once signed in you'll see the dashboard page. After successfull configuration of
|
||||
|
||||
The `machinery` is a **Golang** project which delivers two functions: it acts as the Kerberos Agent which is doing all the heavy lifting with camera processing and other kinds of logic and on the other hand it acts as a webserver (Rest API) that allows communication from the web (React) or any other custom application. The API is documented using `swagger`.
|
||||
|
||||
An executable reference for the Agent frame-processing HTTP and MQTT contracts is
|
||||
available in [`examples/frame-processor`](examples/frame-processor). It provides
|
||||
deterministic processing profiles for integration testing without requiring a
|
||||
machine-learning runtime.
|
||||
|
||||
You can simply run the `machinery` using following commands.
|
||||
|
||||
git clone https://github.com/kerberos-io/agent
|
||||
cd machinery
|
||||
go run main.go -action run -port 8082
|
||||
go run main.go -action run -port 80
|
||||
|
||||
This will launch the Kerberos Agent and run a webserver on port `8082`, matching
|
||||
the UI development default. You can change the port together with
|
||||
`REACT_APP_AGENT_API_PORT`. We strongly support the usage of
|
||||
[Goland](https://www.jetbrains.com/go/) or
|
||||
[Visual Studio Code](https://code.visualstudio.com/), as it comes with all the
|
||||
debugging and linting features built in.
|
||||
This will launch the Kerberos Agent and run a webserver on port `80`. You can change the port by your own preference. We strongly support the usage of [Goland](https://www.jetbrains.com/go/) or [Visual Studio Code](https://code.visualstudio.com/), as it comes with all the debugging and linting features built in.
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
FROM golang:1.24-bookworm AS build
|
||||
|
||||
WORKDIR /src
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
COPY . .
|
||||
RUN CGO_ENABLED=0 go build -trimpath -o /frame-processor .
|
||||
|
||||
FROM gcr.io/distroless/static-debian12:nonroot
|
||||
COPY --from=build /frame-processor /frame-processor
|
||||
EXPOSE 8080
|
||||
ENTRYPOINT ["/frame-processor"]
|
||||
@@ -0,0 +1,88 @@
|
||||
# MQTT control contract
|
||||
|
||||
The Frame Processor publishes control messages to
|
||||
`kerberos/agent/<hubKey>`. Each message targets one Agent through the envelope's
|
||||
`device_id`. It subscribes to `kerberos/hub/<hubKey>` for correlated status
|
||||
events.
|
||||
|
||||
MQTT transports control data only. JPEG frames use `POST /v1/frames` and full
|
||||
recordings use the Agent's existing Vault upload path.
|
||||
|
||||
## Envelope
|
||||
|
||||
```json
|
||||
{
|
||||
"mid": "0c556fc7-637b-4b2a-8a90-2d1cf8450956",
|
||||
"device_id": "camera-1",
|
||||
"timestamp": 1789380000,
|
||||
"encrypted": false,
|
||||
"hidden": false,
|
||||
"public_key": "",
|
||||
"fingerprint": "",
|
||||
"payload": {
|
||||
"version": "1.0",
|
||||
"action": "capture-frame",
|
||||
"device_id": "camera-1",
|
||||
"signature": "",
|
||||
"encrypted_value": "",
|
||||
"hidden_value": "",
|
||||
"value": {}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The reference service publishes plaintext envelopes for local contract testing.
|
||||
Production deployment must use a trusted broker and should adopt the Agent's
|
||||
encrypted-message packaging before commands cross an untrusted broker.
|
||||
|
||||
## `capture-frame`
|
||||
|
||||
```json
|
||||
{
|
||||
"schemaVersion": "1.0",
|
||||
"requestId": "request-1",
|
||||
"processingProfile": "always-trigger",
|
||||
"expiresAt": 1789380030000,
|
||||
"traceId": "optional-trace-id"
|
||||
}
|
||||
```
|
||||
|
||||
## `request-recording-window`
|
||||
|
||||
```json
|
||||
{
|
||||
"schemaVersion": "1.0",
|
||||
"requestId": "request-1",
|
||||
"frameId": "frame-1",
|
||||
"capturedAt": 1789380000123,
|
||||
"preRollSeconds": 10,
|
||||
"eventClipSeconds": 30,
|
||||
"expiresAt": 1789380030000,
|
||||
"processingProfile": "always-trigger",
|
||||
"traceId": "optional-trace-id"
|
||||
}
|
||||
```
|
||||
|
||||
`capturedAt` is generated by the Agent and must be echoed unchanged. The Agent
|
||||
uses `requestId` for command idempotency and selects the local recording that
|
||||
contains `capturedAt`.
|
||||
|
||||
## `frame-processing-status`
|
||||
|
||||
```json
|
||||
{
|
||||
"schemaVersion": "1.0",
|
||||
"requestId": "request-1",
|
||||
"frameId": "frame-1",
|
||||
"deviceId": "camera-1",
|
||||
"status": "queued",
|
||||
"occurredAt": 1789380001000,
|
||||
"retryable": false,
|
||||
"message": "",
|
||||
"traceId": "optional-trace-id"
|
||||
}
|
||||
```
|
||||
|
||||
Expected statuses are `accepted`, `captured`, `submitted`, `no-event`, `event`,
|
||||
`pending-finalisation`, `queued`, `uploaded`, `expired`, `not-found`, `rejected`,
|
||||
and `failed`.
|
||||
@@ -0,0 +1,74 @@
|
||||
# Example Frame Processor
|
||||
|
||||
This reference service defines and exercises the Kerberos Agent frame-processing
|
||||
contract. It accepts Agent JPEGs over HTTP, makes a deterministic decision, and
|
||||
publishes Agent control commands over MQTT. It has no RabbitMQ or machine-learning
|
||||
runtime dependency.
|
||||
|
||||
## Endpoints
|
||||
|
||||
- `GET /health`
|
||||
- `POST /v1/frames` with multipart `metadata` JSON and `frame` JPEG parts
|
||||
- `POST /v1/frame-requests` with JSON to request capture from one or more Agents
|
||||
|
||||
See [openapi.yaml](openapi.yaml) and [MQTT.md](MQTT.md) for the versioned wire
|
||||
contracts.
|
||||
|
||||
## Run
|
||||
|
||||
```bash
|
||||
export FRAME_PROCESSOR_API_TOKEN=development-token
|
||||
export FRAME_PROCESSOR_MQTT_URI=tcp://localhost:1883
|
||||
export FRAME_PROCESSOR_HUB_KEY=development-hub
|
||||
export FRAME_PROCESSOR_PROFILE=never-trigger
|
||||
go run .
|
||||
```
|
||||
|
||||
The broker credentials are optional when the local broker permits anonymous
|
||||
connections:
|
||||
|
||||
```bash
|
||||
export FRAME_PROCESSOR_MQTT_USERNAME=...
|
||||
export FRAME_PROCESSOR_MQTT_PASSWORD=...
|
||||
```
|
||||
|
||||
Request a frame from an Agent:
|
||||
|
||||
```bash
|
||||
curl --fail-with-body \
|
||||
-H 'Authorization: Bearer development-token' \
|
||||
-H 'Content-Type: application/json' \
|
||||
--data @testdata/frame-request.json \
|
||||
http://localhost:8080/v1/frame-requests
|
||||
```
|
||||
|
||||
## Profiles
|
||||
|
||||
- `never-trigger`
|
||||
- `always-trigger`
|
||||
- `every-nth-frame`
|
||||
- `brightness-threshold`
|
||||
|
||||
Use `FRAME_PROCESSOR_EVERY_N` and `FRAME_PROCESSOR_BRIGHTNESS_THRESHOLD` to tune
|
||||
the last two profiles. `FRAME_PROCESSOR_DELAY_MILLISECONDS` and
|
||||
`FRAME_PROCESSOR_FORCE_ERROR` provide deterministic latency and failure
|
||||
simulation.
|
||||
|
||||
Recording commands default to a 30-second event clip with 10 seconds of pre-roll.
|
||||
Configure them with `FRAME_PROCESSOR_EVENT_CLIP_SECONDS` and
|
||||
`FRAME_PROCESSOR_PRE_ROLL_SECONDS`.
|
||||
|
||||
The reference MQTT publisher emits plaintext Agent envelopes for local contract
|
||||
testing. Use a trusted broker. Production support for untrusted brokers requires
|
||||
the same encrypted-message packaging used by Hub and Agent.
|
||||
|
||||
Frame idempotency is guaranteed until the submitted frame's `expiresAt`. The
|
||||
service rejects frame TTLs longer than `FRAME_PROCESSOR_MAX_FRAME_TTL_SECONDS`
|
||||
(five minutes by default), then evicts the cached result at expiry.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
GOWORK=off go test ./...
|
||||
GOWORK=off go vet ./...
|
||||
```
|
||||
@@ -0,0 +1,162 @@
|
||||
package contract
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
)
|
||||
|
||||
const (
|
||||
SchemaVersion = "1.0"
|
||||
MaxImageDimension = 8192
|
||||
|
||||
ActionCaptureFrame = "capture-frame"
|
||||
ActionRequestRecordingWindow = "request-recording-window"
|
||||
ActionFrameStatus = "frame-processing-status"
|
||||
)
|
||||
|
||||
type FrameMetadata struct {
|
||||
SchemaVersion string `json:"schemaVersion"`
|
||||
RequestID string `json:"requestId"`
|
||||
FrameID string `json:"frameId"`
|
||||
DeviceID string `json:"deviceId"`
|
||||
CapturedAt int64 `json:"capturedAt"`
|
||||
ExpiresAt int64 `json:"expiresAt"`
|
||||
ProcessingProfile string `json:"processingProfile"`
|
||||
SourceStream string `json:"sourceStream"`
|
||||
Width int `json:"width"`
|
||||
Height int `json:"height"`
|
||||
TraceID string `json:"traceId,omitempty"`
|
||||
}
|
||||
|
||||
func (m FrameMetadata) Validate(nowMillis int64) error {
|
||||
if m.SchemaVersion != SchemaVersion {
|
||||
return fmt.Errorf("unsupported schemaVersion %q", m.SchemaVersion)
|
||||
}
|
||||
if m.RequestID == "" || m.FrameID == "" || m.DeviceID == "" {
|
||||
return errors.New("requestId, frameId, and deviceId are required")
|
||||
}
|
||||
if m.CapturedAt <= 0 {
|
||||
return errors.New("capturedAt must be a positive Unix millisecond timestamp")
|
||||
}
|
||||
if m.ExpiresAt <= m.CapturedAt {
|
||||
return errors.New("expiresAt must be later than capturedAt")
|
||||
}
|
||||
if nowMillis > 0 && m.ExpiresAt <= nowMillis {
|
||||
return errors.New("frame has expired")
|
||||
}
|
||||
if m.ProcessingProfile == "" {
|
||||
return errors.New("processingProfile is required")
|
||||
}
|
||||
if m.SourceStream != "main" && m.SourceStream != "sub" {
|
||||
return errors.New("sourceStream must be main or sub")
|
||||
}
|
||||
if m.Width <= 0 || m.Height <= 0 || m.Width > MaxImageDimension || m.Height > MaxImageDimension {
|
||||
return fmt.Errorf("width and height must be between 1 and %d", MaxImageDimension)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type FrameRequest struct {
|
||||
SchemaVersion string `json:"schemaVersion"`
|
||||
RequestID string `json:"requestId,omitempty"`
|
||||
DeviceIDs []string `json:"deviceIds"`
|
||||
ProcessingProfile string `json:"processingProfile"`
|
||||
ExpiresAt int64 `json:"expiresAt"`
|
||||
TraceID string `json:"traceId,omitempty"`
|
||||
}
|
||||
|
||||
func (r FrameRequest) Validate(nowMillis int64) error {
|
||||
if r.SchemaVersion != SchemaVersion {
|
||||
return fmt.Errorf("unsupported schemaVersion %q", r.SchemaVersion)
|
||||
}
|
||||
if len(r.DeviceIDs) == 0 {
|
||||
return errors.New("at least one deviceId is required")
|
||||
}
|
||||
for _, deviceID := range r.DeviceIDs {
|
||||
if deviceID == "" {
|
||||
return errors.New("deviceIds cannot contain empty values")
|
||||
}
|
||||
}
|
||||
if r.ProcessingProfile == "" {
|
||||
return errors.New("processingProfile is required")
|
||||
}
|
||||
if r.ExpiresAt <= nowMillis {
|
||||
return errors.New("expiresAt must be in the future")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type CaptureFrameCommand struct {
|
||||
SchemaVersion string `json:"schemaVersion"`
|
||||
RequestID string `json:"requestId"`
|
||||
ProcessingProfile string `json:"processingProfile"`
|
||||
ExpiresAt int64 `json:"expiresAt"`
|
||||
TraceID string `json:"traceId,omitempty"`
|
||||
}
|
||||
|
||||
type RecordingWindowCommand struct {
|
||||
SchemaVersion string `json:"schemaVersion"`
|
||||
RequestID string `json:"requestId"`
|
||||
FrameID string `json:"frameId"`
|
||||
CapturedAt int64 `json:"capturedAt"`
|
||||
PreRollSeconds int64 `json:"preRollSeconds"`
|
||||
EventClipSeconds int64 `json:"eventClipSeconds"`
|
||||
ExpiresAt int64 `json:"expiresAt"`
|
||||
ProcessingProfile string `json:"processingProfile"`
|
||||
TraceID string `json:"traceId,omitempty"`
|
||||
}
|
||||
|
||||
func (c RecordingWindowCommand) Validate(nowMillis int64) error {
|
||||
if c.SchemaVersion != SchemaVersion {
|
||||
return fmt.Errorf("unsupported schemaVersion %q", c.SchemaVersion)
|
||||
}
|
||||
if c.RequestID == "" || c.FrameID == "" {
|
||||
return errors.New("requestId and frameId are required")
|
||||
}
|
||||
if c.CapturedAt <= 0 {
|
||||
return errors.New("capturedAt must be a positive Unix millisecond timestamp")
|
||||
}
|
||||
if c.EventClipSeconds <= 0 {
|
||||
return errors.New("eventClipSeconds must be positive")
|
||||
}
|
||||
if c.PreRollSeconds < 0 || c.PreRollSeconds > c.EventClipSeconds {
|
||||
return errors.New("preRollSeconds must be between zero and eventClipSeconds")
|
||||
}
|
||||
if c.ExpiresAt <= nowMillis {
|
||||
return errors.New("expiresAt must be in the future")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type StatusEvent struct {
|
||||
SchemaVersion string `json:"schemaVersion"`
|
||||
RequestID string `json:"requestId"`
|
||||
FrameID string `json:"frameId,omitempty"`
|
||||
DeviceID string `json:"deviceId"`
|
||||
Status string `json:"status"`
|
||||
OccurredAt int64 `json:"occurredAt"`
|
||||
Retryable bool `json:"retryable,omitempty"`
|
||||
Message string `json:"message,omitempty"`
|
||||
TraceID string `json:"traceId,omitempty"`
|
||||
}
|
||||
|
||||
type MQTTMessage struct {
|
||||
MID string `json:"mid"`
|
||||
DeviceID string `json:"device_id"`
|
||||
Timestamp int64 `json:"timestamp"`
|
||||
Encrypted bool `json:"encrypted"`
|
||||
Hidden bool `json:"hidden"`
|
||||
PublicKey string `json:"public_key"`
|
||||
Fingerprint string `json:"fingerprint"`
|
||||
Payload MQTTPayload `json:"payload"`
|
||||
}
|
||||
|
||||
type MQTTPayload struct {
|
||||
Version string `json:"version"`
|
||||
Action string `json:"action"`
|
||||
DeviceID string `json:"device_id"`
|
||||
Signature string `json:"signature"`
|
||||
EncryptedValue string `json:"encrypted_value"`
|
||||
HiddenValue string `json:"hidden_value"`
|
||||
Value any `json:"value"`
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
package contract
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestFrameMetadataValidate(t *testing.T) {
|
||||
now := int64(1_000)
|
||||
metadata := FrameMetadata{
|
||||
SchemaVersion: SchemaVersion,
|
||||
RequestID: "request-1",
|
||||
FrameID: "frame-1",
|
||||
DeviceID: "device-1",
|
||||
CapturedAt: 900,
|
||||
ExpiresAt: 1_100,
|
||||
ProcessingProfile: "always-trigger",
|
||||
SourceStream: "sub",
|
||||
Width: 640,
|
||||
Height: 480,
|
||||
}
|
||||
|
||||
if err := metadata.Validate(now); err != nil {
|
||||
t.Fatalf("Validate() error = %v", err)
|
||||
}
|
||||
|
||||
metadata.ExpiresAt = now
|
||||
if err := metadata.Validate(now); err == nil {
|
||||
t.Fatal("Validate() accepted an expired frame")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRecordingWindowCommandValidate(t *testing.T) {
|
||||
command := RecordingWindowCommand{
|
||||
SchemaVersion: SchemaVersion,
|
||||
RequestID: "request-1",
|
||||
FrameID: "frame-1",
|
||||
CapturedAt: 900,
|
||||
PreRollSeconds: 10,
|
||||
EventClipSeconds: 30,
|
||||
ExpiresAt: 2_000,
|
||||
}
|
||||
|
||||
if err := command.Validate(1_000); err != nil {
|
||||
t.Fatalf("Validate() error = %v", err)
|
||||
}
|
||||
|
||||
command.PreRollSeconds = 31
|
||||
if err := command.Validate(1_000); err == nil {
|
||||
t.Fatal("Validate() accepted pre-roll longer than the event clip")
|
||||
}
|
||||
}
|
||||
|
||||
func TestContractFixturesDecode(t *testing.T) {
|
||||
tests := []struct {
|
||||
path string
|
||||
target any
|
||||
}{
|
||||
{"../testdata/frame-request.json", &FrameRequest{}},
|
||||
{"../testdata/capture-frame.json", &CaptureFrameCommand{}},
|
||||
{"../testdata/request-recording-window.json", &RecordingWindowCommand{}},
|
||||
}
|
||||
for _, test := range tests {
|
||||
value, err := os.ReadFile(test.path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := json.Unmarshal(value, test.target); err != nil {
|
||||
t.Fatalf("decode %s: %v", test.path, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
module github.com/kerberos-io/agent/examples/frame-processor
|
||||
|
||||
go 1.24.2
|
||||
|
||||
require github.com/eclipse/paho.mqtt.golang v1.5.0
|
||||
|
||||
require (
|
||||
github.com/gorilla/websocket v1.5.3 // indirect
|
||||
golang.org/x/net v0.27.0 // indirect
|
||||
golang.org/x/sync v0.7.0 // indirect
|
||||
)
|
||||
@@ -0,0 +1,8 @@
|
||||
github.com/eclipse/paho.mqtt.golang v1.5.0 h1:EH+bUVJNgttidWFkLLVKaQPGmkTUfQQqjOsyvMGvD6o=
|
||||
github.com/eclipse/paho.mqtt.golang v1.5.0/go.mod h1:du/2qNQVqJf/Sqs4MEL77kR8QTqANF7XU7Fk0aOTAgk=
|
||||
github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
|
||||
github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
|
||||
golang.org/x/net v0.27.0 h1:5K3Njcw06/l2y9vpGCSdcxWOYHOUk3dVNGDXN+FvAys=
|
||||
golang.org/x/net v0.27.0/go.mod h1:dDi0PyhWNoiUOrAS8uXv/vnScO4wnHQO4mj9fn/RytE=
|
||||
golang.org/x/sync v0.7.0 h1:YsImfSBoP9QPYL0xyKJPq0gcaJdG3rInoqxTWbfQu9M=
|
||||
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
|
||||
@@ -0,0 +1,184 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strconv"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/kerberos-io/agent/examples/frame-processor/contract"
|
||||
"github.com/kerberos-io/agent/examples/frame-processor/mqttpublisher"
|
||||
"github.com/kerberos-io/agent/examples/frame-processor/processor"
|
||||
"github.com/kerberos-io/agent/examples/frame-processor/service"
|
||||
)
|
||||
|
||||
func main() {
|
||||
config, err := loadConfig()
|
||||
if err != nil {
|
||||
slog.Error("invalid configuration", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
publisher, err := mqttpublisher.New(config.mqtt, func(status contract.StatusEvent) {
|
||||
slog.Info("Agent frame-processing status",
|
||||
"deviceId", status.DeviceID,
|
||||
"requestId", status.RequestID,
|
||||
"frameId", status.FrameID,
|
||||
"status", status.Status,
|
||||
)
|
||||
})
|
||||
if err != nil {
|
||||
slog.Error("failed to initialize MQTT", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
defer publisher.Close()
|
||||
|
||||
engine := processor.New(config.processor)
|
||||
application := service.New(config.service, engine, publisher, nil)
|
||||
server := &http.Server{
|
||||
Addr: config.address,
|
||||
Handler: application.Handler(),
|
||||
ReadHeaderTimeout: 5 * time.Second,
|
||||
IdleTimeout: 60 * time.Second,
|
||||
}
|
||||
|
||||
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
|
||||
defer stop()
|
||||
go func() {
|
||||
<-ctx.Done()
|
||||
shutdownContext, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
_ = server.Shutdown(shutdownContext)
|
||||
}()
|
||||
|
||||
slog.Info("Frame Processor listening", "address", config.address, "profile", config.processor.DefaultProfile)
|
||||
if err := server.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
|
||||
slog.Error("Frame Processor stopped", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
type applicationConfig struct {
|
||||
address string
|
||||
mqtt mqttpublisher.Config
|
||||
processor processor.Config
|
||||
service service.Config
|
||||
}
|
||||
|
||||
func loadConfig() (applicationConfig, error) {
|
||||
brightnessThreshold := envInt("FRAME_PROCESSOR_BRIGHTNESS_THRESHOLD", 200)
|
||||
config := applicationConfig{
|
||||
address: envString("FRAME_PROCESSOR_ADDRESS", ":8080"),
|
||||
mqtt: mqttpublisher.Config{
|
||||
BrokerURI: os.Getenv("FRAME_PROCESSOR_MQTT_URI"),
|
||||
Username: os.Getenv("FRAME_PROCESSOR_MQTT_USERNAME"),
|
||||
Password: os.Getenv("FRAME_PROCESSOR_MQTT_PASSWORD"),
|
||||
HubKey: os.Getenv("FRAME_PROCESSOR_HUB_KEY"),
|
||||
ClientID: os.Getenv("FRAME_PROCESSOR_MQTT_CLIENT_ID"),
|
||||
Timeout: envDurationSeconds("FRAME_PROCESSOR_MQTT_TIMEOUT_SECONDS", 10),
|
||||
},
|
||||
processor: processor.Config{
|
||||
DefaultProfile: envString("FRAME_PROCESSOR_PROFILE", processor.ProfileNeverTrigger),
|
||||
EveryN: envInt("FRAME_PROCESSOR_EVERY_N", 2),
|
||||
BrightnessThreshold: uint8(brightnessThreshold),
|
||||
Delay: envDurationMillis("FRAME_PROCESSOR_DELAY_MILLISECONDS", 0),
|
||||
ForceError: envBool("FRAME_PROCESSOR_FORCE_ERROR", false),
|
||||
},
|
||||
service: service.Config{
|
||||
APIToken: os.Getenv("FRAME_PROCESSOR_API_TOKEN"),
|
||||
MaxFrameBytes: int64(envInt("FRAME_PROCESSOR_MAX_FRAME_BYTES", 4<<20)),
|
||||
MaxFrameTTL: envDurationSeconds("FRAME_PROCESSOR_MAX_FRAME_TTL_SECONDS", 300),
|
||||
CommandTTL: envDurationSeconds("FRAME_PROCESSOR_COMMAND_TTL_SECONDS", 30),
|
||||
PreRollSeconds: int64(envInt("FRAME_PROCESSOR_PRE_ROLL_SECONDS", 10)),
|
||||
EventClipSeconds: int64(envInt("FRAME_PROCESSOR_EVENT_CLIP_SECONDS", 30)),
|
||||
},
|
||||
}
|
||||
if config.service.APIToken == "" {
|
||||
return applicationConfig{}, errors.New("FRAME_PROCESSOR_API_TOKEN is required")
|
||||
}
|
||||
if config.mqtt.BrokerURI == "" || config.mqtt.HubKey == "" {
|
||||
return applicationConfig{}, errors.New("FRAME_PROCESSOR_MQTT_URI and FRAME_PROCESSOR_HUB_KEY are required")
|
||||
}
|
||||
if config.processor.EveryN <= 0 {
|
||||
return applicationConfig{}, errors.New("FRAME_PROCESSOR_EVERY_N must be positive")
|
||||
}
|
||||
if !processor.IsProfileSupported(config.processor.DefaultProfile) {
|
||||
return applicationConfig{}, fmt.Errorf("unsupported FRAME_PROCESSOR_PROFILE %q", config.processor.DefaultProfile)
|
||||
}
|
||||
if brightnessThreshold < 0 || brightnessThreshold > 255 {
|
||||
return applicationConfig{}, errors.New("FRAME_PROCESSOR_BRIGHTNESS_THRESHOLD must be between 0 and 255")
|
||||
}
|
||||
if config.processor.Delay < 0 || config.processor.Delay > time.Minute {
|
||||
return applicationConfig{}, errors.New("FRAME_PROCESSOR_DELAY_MILLISECONDS must be between 0 and 60000")
|
||||
}
|
||||
if config.service.MaxFrameBytes <= 0 || config.service.MaxFrameBytes > 100<<20 {
|
||||
return applicationConfig{}, errors.New("FRAME_PROCESSOR_MAX_FRAME_BYTES must be between 1 and 104857600")
|
||||
}
|
||||
if config.service.MaxFrameTTL <= 0 || config.service.MaxFrameTTL > time.Hour {
|
||||
return applicationConfig{}, errors.New("FRAME_PROCESSOR_MAX_FRAME_TTL_SECONDS must be between 1 and 3600")
|
||||
}
|
||||
if config.service.PreRollSeconds < 0 || config.service.PreRollSeconds > config.service.EventClipSeconds {
|
||||
return applicationConfig{}, errors.New("FRAME_PROCESSOR_PRE_ROLL_SECONDS must be between zero and FRAME_PROCESSOR_EVENT_CLIP_SECONDS")
|
||||
}
|
||||
if config.service.CommandTTL <= 0 || config.service.CommandTTL > time.Hour {
|
||||
return applicationConfig{}, errors.New("FRAME_PROCESSOR_COMMAND_TTL_SECONDS must be between 1 and 3600")
|
||||
}
|
||||
if config.service.EventClipSeconds <= 0 || config.service.EventClipSeconds > 24*60*60 {
|
||||
return applicationConfig{}, errors.New("FRAME_PROCESSOR_EVENT_CLIP_SECONDS must be between 1 and 86400")
|
||||
}
|
||||
if config.mqtt.Timeout <= 0 || config.mqtt.Timeout > time.Minute {
|
||||
return applicationConfig{}, errors.New("FRAME_PROCESSOR_MQTT_TIMEOUT_SECONDS must be between 1 and 60")
|
||||
}
|
||||
return config, nil
|
||||
}
|
||||
|
||||
func envString(name, fallback string) string {
|
||||
if value := os.Getenv(name); value != "" {
|
||||
return value
|
||||
}
|
||||
return fallback
|
||||
}
|
||||
|
||||
func envInt(name string, fallback int) int {
|
||||
value := os.Getenv(name)
|
||||
if value == "" {
|
||||
return fallback
|
||||
}
|
||||
parsed, err := strconv.Atoi(value)
|
||||
if err != nil {
|
||||
slog.Warn("invalid integer environment value, using default", "name", name)
|
||||
return fallback
|
||||
}
|
||||
return parsed
|
||||
}
|
||||
|
||||
func envBool(name string, fallback bool) bool {
|
||||
value := os.Getenv(name)
|
||||
if value == "" {
|
||||
return fallback
|
||||
}
|
||||
parsed, err := strconv.ParseBool(value)
|
||||
if err != nil {
|
||||
slog.Warn("invalid boolean environment value, using default", "name", name)
|
||||
return fallback
|
||||
}
|
||||
return parsed
|
||||
}
|
||||
|
||||
func envDurationSeconds(name string, fallback int) time.Duration {
|
||||
return time.Duration(envInt(name, fallback)) * time.Second
|
||||
}
|
||||
|
||||
func envDurationMillis(name string, fallback int) time.Duration {
|
||||
return time.Duration(envInt(name, fallback)) * time.Millisecond
|
||||
}
|
||||
|
||||
func (c applicationConfig) String() string {
|
||||
return fmt.Sprintf("address=%s profile=%s", c.address, c.processor.DefaultProfile)
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
package main
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestLoadConfigRejectsInvalidBounds(t *testing.T) {
|
||||
t.Setenv("FRAME_PROCESSOR_API_TOKEN", "secret")
|
||||
t.Setenv("FRAME_PROCESSOR_MQTT_URI", "tcp://localhost:1883")
|
||||
t.Setenv("FRAME_PROCESSOR_HUB_KEY", "hub")
|
||||
t.Setenv("FRAME_PROCESSOR_BRIGHTNESS_THRESHOLD", "256")
|
||||
|
||||
if _, err := loadConfig(); err == nil {
|
||||
t.Fatal("loadConfig() accepted an invalid brightness threshold")
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadConfigRejectsUnknownProfile(t *testing.T) {
|
||||
t.Setenv("FRAME_PROCESSOR_API_TOKEN", "secret")
|
||||
t.Setenv("FRAME_PROCESSOR_MQTT_URI", "tcp://localhost:1883")
|
||||
t.Setenv("FRAME_PROCESSOR_HUB_KEY", "hub")
|
||||
t.Setenv("FRAME_PROCESSOR_PROFILE", "unknown")
|
||||
|
||||
if _, err := loadConfig(); err == nil {
|
||||
t.Fatal("loadConfig() accepted an unknown profile")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,171 @@
|
||||
package mqttpublisher
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
mqtt "github.com/eclipse/paho.mqtt.golang"
|
||||
"github.com/kerberos-io/agent/examples/frame-processor/contract"
|
||||
)
|
||||
|
||||
const (
|
||||
commandQoS = byte(1)
|
||||
statusQoS = byte(1)
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
BrokerURI string
|
||||
Username string
|
||||
Password string
|
||||
HubKey string
|
||||
ClientID string
|
||||
Timeout time.Duration
|
||||
}
|
||||
|
||||
type StatusHandler func(contract.StatusEvent)
|
||||
|
||||
type Publisher struct {
|
||||
client mqtt.Client
|
||||
commandTopic string
|
||||
timeout time.Duration
|
||||
}
|
||||
|
||||
func New(config Config, handler StatusHandler) (*Publisher, error) {
|
||||
if config.BrokerURI == "" || config.HubKey == "" {
|
||||
return nil, errors.New("MQTT broker URI and Hub key are required")
|
||||
}
|
||||
if config.Timeout <= 0 {
|
||||
config.Timeout = 10 * time.Second
|
||||
}
|
||||
if config.ClientID == "" {
|
||||
config.ClientID = "frame-processor-" + randomID()
|
||||
}
|
||||
|
||||
statusTopic := "kerberos/hub/" + config.HubKey
|
||||
options := mqtt.NewClientOptions().
|
||||
AddBroker(config.BrokerURI).
|
||||
SetClientID(config.ClientID).
|
||||
SetUsername(config.Username).
|
||||
SetPassword(config.Password).
|
||||
SetCleanSession(false).
|
||||
SetResumeSubs(true).
|
||||
SetAutoReconnect(true).
|
||||
SetConnectRetry(true).
|
||||
SetConnectRetryInterval(5 * time.Second).
|
||||
SetMaxReconnectInterval(time.Minute).
|
||||
SetKeepAlive(30 * time.Second).
|
||||
SetPingTimeout(10 * time.Second)
|
||||
if handler != nil {
|
||||
options.SetOnConnectHandler(func(client mqtt.Client) {
|
||||
token := client.Subscribe(statusTopic, statusQoS, statusMessageHandler(handler))
|
||||
if !token.WaitTimeout(config.Timeout) || token.Error() != nil {
|
||||
slog.Error("failed to subscribe to Agent status events", "topic", statusTopic, "error", token.Error())
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
client := mqtt.NewClient(options)
|
||||
token := client.Connect()
|
||||
if !token.WaitTimeout(config.Timeout) {
|
||||
return nil, errors.New("MQTT connection timed out")
|
||||
}
|
||||
if err := token.Error(); err != nil {
|
||||
return nil, fmt.Errorf("connect MQTT: %w", err)
|
||||
}
|
||||
return &Publisher{
|
||||
client: client, commandTopic: "kerberos/agent/" + config.HubKey,
|
||||
timeout: config.Timeout,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (p *Publisher) Close() {
|
||||
if p != nil && p.client != nil && p.client.IsConnected() {
|
||||
p.client.Disconnect(250)
|
||||
}
|
||||
}
|
||||
|
||||
func (p *Publisher) PublishCaptureFrame(ctx context.Context, deviceID string, command contract.CaptureFrameCommand) error {
|
||||
return p.publish(ctx, deviceID, contract.ActionCaptureFrame, command)
|
||||
}
|
||||
|
||||
func (p *Publisher) PublishRecordingWindow(ctx context.Context, deviceID string, command contract.RecordingWindowCommand) error {
|
||||
return p.publish(ctx, deviceID, contract.ActionRequestRecordingWindow, command)
|
||||
}
|
||||
|
||||
func (p *Publisher) publish(ctx context.Context, deviceID, action string, value any) error {
|
||||
message := newMessage(deviceID, action, value, time.Now())
|
||||
payload, err := json.Marshal(message)
|
||||
if err != nil {
|
||||
return fmt.Errorf("marshal MQTT command: %w", err)
|
||||
}
|
||||
token := p.client.Publish(p.commandTopic, commandQoS, false, payload)
|
||||
timer := time.NewTimer(p.timeout)
|
||||
defer timer.Stop()
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case <-timer.C:
|
||||
return errors.New("MQTT publish timed out")
|
||||
case <-token.Done():
|
||||
if err := token.Error(); err != nil {
|
||||
return fmt.Errorf("publish MQTT command: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
func newMessage(deviceID, action string, value any, now time.Time) contract.MQTTMessage {
|
||||
return contract.MQTTMessage{
|
||||
MID: randomID(),
|
||||
DeviceID: deviceID,
|
||||
Timestamp: now.Unix(),
|
||||
Payload: contract.MQTTPayload{
|
||||
Version: contract.SchemaVersion,
|
||||
Action: action,
|
||||
DeviceID: deviceID,
|
||||
Value: value,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func randomID() string {
|
||||
value := make([]byte, 16)
|
||||
if _, err := rand.Read(value); err != nil {
|
||||
return fmt.Sprintf("%d", time.Now().UnixNano())
|
||||
}
|
||||
value[6] = (value[6] & 0x0f) | 0x40
|
||||
value[8] = (value[8] & 0x3f) | 0x80
|
||||
encoded := hex.EncodeToString(value)
|
||||
return strings.Join([]string{encoded[0:8], encoded[8:12], encoded[12:16], encoded[16:20], encoded[20:32]}, "-")
|
||||
}
|
||||
|
||||
func statusMessageHandler(handler StatusHandler) mqtt.MessageHandler {
|
||||
return func(_ mqtt.Client, message mqtt.Message) {
|
||||
var envelope contract.MQTTMessage
|
||||
if err := json.Unmarshal(message.Payload(), &envelope); err != nil {
|
||||
slog.Warn("discarding malformed Agent status envelope", "error", err)
|
||||
return
|
||||
}
|
||||
if envelope.Payload.Action != contract.ActionFrameStatus {
|
||||
return
|
||||
}
|
||||
value, err := json.Marshal(envelope.Payload.Value)
|
||||
if err != nil {
|
||||
slog.Warn("discarding unencodable Agent status value", "error", err)
|
||||
return
|
||||
}
|
||||
var status contract.StatusEvent
|
||||
if err := json.Unmarshal(value, &status); err != nil {
|
||||
slog.Warn("discarding malformed Agent status value", "error", err)
|
||||
return
|
||||
}
|
||||
handler(status)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
package mqttpublisher
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/kerberos-io/agent/examples/frame-processor/contract"
|
||||
)
|
||||
|
||||
func TestNewMessageMatchesAgentEnvelope(t *testing.T) {
|
||||
command := contract.CaptureFrameCommand{
|
||||
SchemaVersion: contract.SchemaVersion,
|
||||
RequestID: "request-1",
|
||||
ProcessingProfile: "always-trigger",
|
||||
ExpiresAt: 2_000,
|
||||
}
|
||||
message := newMessage("device-1", contract.ActionCaptureFrame, command, time.Unix(1_000, 0))
|
||||
payload, err := json.Marshal(message)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
var decoded map[string]any
|
||||
if err := json.Unmarshal(payload, &decoded); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if decoded["device_id"] != "device-1" || decoded["timestamp"] != float64(1_000) {
|
||||
t.Fatalf("envelope = %s", payload)
|
||||
}
|
||||
inner := decoded["payload"].(map[string]any)
|
||||
if inner["action"] != contract.ActionCaptureFrame || inner["device_id"] != "device-1" {
|
||||
t.Fatalf("payload = %#v", inner)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStatusMessageHandlerIgnoresOtherActions(t *testing.T) {
|
||||
called := false
|
||||
handler := statusMessageHandler(func(contract.StatusEvent) { called = true })
|
||||
handler(nil, fakeMessage(`{"payload":{"action":"motion","value":{}}}`))
|
||||
if called {
|
||||
t.Fatal("handler accepted an unrelated action")
|
||||
}
|
||||
}
|
||||
|
||||
type fakeMessage string
|
||||
|
||||
func (m fakeMessage) Duplicate() bool { return false }
|
||||
func (m fakeMessage) Qos() byte { return 1 }
|
||||
func (m fakeMessage) Retained() bool { return false }
|
||||
func (m fakeMessage) Topic() string { return "test" }
|
||||
func (m fakeMessage) MessageID() uint16 { return 1 }
|
||||
func (m fakeMessage) Payload() []byte { return []byte(m) }
|
||||
func (m fakeMessage) Ack() {}
|
||||
@@ -0,0 +1,216 @@
|
||||
openapi: 3.0.3
|
||||
info:
|
||||
title: Kerberos Frame Processing API
|
||||
version: 1.0.0
|
||||
description: Reference contract for Agent frame submission and externally requested capture.
|
||||
servers:
|
||||
- url: http://localhost:8080
|
||||
security:
|
||||
- bearerAuth: []
|
||||
paths:
|
||||
/health:
|
||||
get:
|
||||
security: []
|
||||
summary: Check service liveness
|
||||
responses:
|
||||
"200":
|
||||
description: Service is healthy
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
required: [status]
|
||||
properties:
|
||||
status:
|
||||
type: string
|
||||
enum: [healthy]
|
||||
/v1/frames:
|
||||
post:
|
||||
summary: Process one Agent frame
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
multipart/form-data:
|
||||
schema:
|
||||
type: object
|
||||
required: [metadata, frame]
|
||||
properties:
|
||||
metadata:
|
||||
$ref: "#/components/schemas/FrameMetadata"
|
||||
frame:
|
||||
type: string
|
||||
format: binary
|
||||
encoding:
|
||||
metadata:
|
||||
contentType: application/json
|
||||
frame:
|
||||
contentType: image/jpeg
|
||||
responses:
|
||||
"200":
|
||||
description: Frame processed
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/FrameResponse"
|
||||
"400":
|
||||
$ref: "#/components/responses/BadRequest"
|
||||
"401":
|
||||
$ref: "#/components/responses/Unauthorized"
|
||||
"413":
|
||||
description: Frame exceeds the configured request limit
|
||||
"415":
|
||||
description: Frame is not a valid JPEG
|
||||
"422":
|
||||
description: Metadata is invalid or expired
|
||||
"502":
|
||||
description: Processing or MQTT command publication failed
|
||||
/v1/frame-requests:
|
||||
post:
|
||||
summary: Request a new frame from one or more Agents
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/FrameRequest"
|
||||
responses:
|
||||
"202":
|
||||
description: Capture commands accepted for publication
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/FrameRequestResponse"
|
||||
"400":
|
||||
$ref: "#/components/responses/BadRequest"
|
||||
"401":
|
||||
$ref: "#/components/responses/Unauthorized"
|
||||
"422":
|
||||
description: Request is invalid or expired
|
||||
"502":
|
||||
description: MQTT command publication failed
|
||||
components:
|
||||
securitySchemes:
|
||||
bearerAuth:
|
||||
type: http
|
||||
scheme: bearer
|
||||
responses:
|
||||
BadRequest:
|
||||
description: Malformed request
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/Error"
|
||||
Unauthorized:
|
||||
description: Missing or invalid bearer token
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/Error"
|
||||
schemas:
|
||||
FrameMetadata:
|
||||
type: object
|
||||
required:
|
||||
- schemaVersion
|
||||
- requestId
|
||||
- frameId
|
||||
- deviceId
|
||||
- capturedAt
|
||||
- expiresAt
|
||||
- processingProfile
|
||||
- sourceStream
|
||||
- width
|
||||
- height
|
||||
properties:
|
||||
schemaVersion:
|
||||
type: string
|
||||
enum: ["1.0"]
|
||||
requestId:
|
||||
type: string
|
||||
frameId:
|
||||
type: string
|
||||
deviceId:
|
||||
type: string
|
||||
capturedAt:
|
||||
type: integer
|
||||
format: int64
|
||||
description: Agent wall-clock capture time in Unix milliseconds.
|
||||
expiresAt:
|
||||
type: integer
|
||||
format: int64
|
||||
description: Unix milliseconds after which the frame must not be processed.
|
||||
processingProfile:
|
||||
type: string
|
||||
sourceStream:
|
||||
type: string
|
||||
enum: [main, sub]
|
||||
width:
|
||||
type: integer
|
||||
minimum: 1
|
||||
height:
|
||||
type: integer
|
||||
minimum: 1
|
||||
traceId:
|
||||
type: string
|
||||
FrameRequest:
|
||||
type: object
|
||||
required: [schemaVersion, deviceIds, processingProfile, expiresAt]
|
||||
properties:
|
||||
schemaVersion:
|
||||
type: string
|
||||
enum: ["1.0"]
|
||||
requestId:
|
||||
type: string
|
||||
description: Generated by the service when omitted.
|
||||
deviceIds:
|
||||
type: array
|
||||
minItems: 1
|
||||
items:
|
||||
type: string
|
||||
processingProfile:
|
||||
type: string
|
||||
expiresAt:
|
||||
type: integer
|
||||
format: int64
|
||||
traceId:
|
||||
type: string
|
||||
FrameResponse:
|
||||
type: object
|
||||
required: [schemaVersion, requestId, frameId, decision, reason]
|
||||
properties:
|
||||
schemaVersion:
|
||||
type: string
|
||||
enum: ["1.0"]
|
||||
requestId:
|
||||
type: string
|
||||
frameId:
|
||||
type: string
|
||||
decision:
|
||||
type: string
|
||||
enum: [no-event, event]
|
||||
reason:
|
||||
type: string
|
||||
FrameRequestResponse:
|
||||
type: object
|
||||
required: [schemaVersion, requestId, deviceIds, status]
|
||||
properties:
|
||||
schemaVersion:
|
||||
type: string
|
||||
enum: ["1.0"]
|
||||
requestId:
|
||||
type: string
|
||||
deviceIds:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
status:
|
||||
type: string
|
||||
enum: [accepted]
|
||||
Error:
|
||||
type: object
|
||||
required: [schemaVersion, error]
|
||||
properties:
|
||||
schemaVersion:
|
||||
type: string
|
||||
enum: ["1.0"]
|
||||
error:
|
||||
type: string
|
||||
@@ -0,0 +1,117 @@
|
||||
package processor
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"image/jpeg"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/kerberos-io/agent/examples/frame-processor/contract"
|
||||
)
|
||||
|
||||
const (
|
||||
ProfileNeverTrigger = "never-trigger"
|
||||
ProfileAlwaysTrigger = "always-trigger"
|
||||
ProfileEveryNthFrame = "every-nth-frame"
|
||||
ProfileBrightnessThreshold = "brightness-threshold"
|
||||
)
|
||||
|
||||
type Decision struct {
|
||||
Triggered bool `json:"triggered"`
|
||||
Reason string `json:"reason"`
|
||||
}
|
||||
|
||||
type Config struct {
|
||||
DefaultProfile string
|
||||
EveryN int
|
||||
BrightnessThreshold uint8
|
||||
Delay time.Duration
|
||||
ForceError bool
|
||||
}
|
||||
|
||||
type Engine struct {
|
||||
config Config
|
||||
mu sync.Mutex
|
||||
counts map[string]int
|
||||
}
|
||||
|
||||
func IsProfileSupported(profile string) bool {
|
||||
switch profile {
|
||||
case ProfileNeverTrigger, ProfileAlwaysTrigger, ProfileEveryNthFrame, ProfileBrightnessThreshold:
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
func New(config Config) *Engine {
|
||||
if config.DefaultProfile == "" {
|
||||
config.DefaultProfile = ProfileNeverTrigger
|
||||
}
|
||||
if config.EveryN <= 0 {
|
||||
config.EveryN = 2
|
||||
}
|
||||
return &Engine{config: config, counts: make(map[string]int)}
|
||||
}
|
||||
|
||||
func (e *Engine) Process(ctx context.Context, metadata contract.FrameMetadata, frame []byte) (Decision, error) {
|
||||
if e.config.Delay > 0 {
|
||||
timer := time.NewTimer(e.config.Delay)
|
||||
defer timer.Stop()
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return Decision{}, ctx.Err()
|
||||
case <-timer.C:
|
||||
}
|
||||
}
|
||||
if e.config.ForceError {
|
||||
return Decision{}, errors.New("configured processing failure")
|
||||
}
|
||||
|
||||
profile := metadata.ProcessingProfile
|
||||
if profile == "" {
|
||||
profile = e.config.DefaultProfile
|
||||
}
|
||||
switch profile {
|
||||
case ProfileNeverTrigger:
|
||||
return Decision{Reason: ProfileNeverTrigger}, nil
|
||||
case ProfileAlwaysTrigger:
|
||||
return Decision{Triggered: true, Reason: ProfileAlwaysTrigger}, nil
|
||||
case ProfileEveryNthFrame:
|
||||
e.mu.Lock()
|
||||
e.counts[metadata.DeviceID]++
|
||||
count := e.counts[metadata.DeviceID]
|
||||
e.mu.Unlock()
|
||||
return Decision{
|
||||
Triggered: count%e.config.EveryN == 0,
|
||||
Reason: fmt.Sprintf("frame %d of every %d", count, e.config.EveryN),
|
||||
}, nil
|
||||
case ProfileBrightnessThreshold:
|
||||
image, err := jpeg.Decode(bytes.NewReader(frame))
|
||||
if err != nil {
|
||||
return Decision{}, fmt.Errorf("decode JPEG: %w", err)
|
||||
}
|
||||
bounds := image.Bounds()
|
||||
var total uint64
|
||||
for y := bounds.Min.Y; y < bounds.Max.Y; y++ {
|
||||
for x := bounds.Min.X; x < bounds.Max.X; x++ {
|
||||
gray, _, _, _ := image.At(x, y).RGBA()
|
||||
total += uint64(gray >> 8)
|
||||
}
|
||||
}
|
||||
pixels := uint64(bounds.Dx() * bounds.Dy())
|
||||
if pixels == 0 {
|
||||
return Decision{}, errors.New("JPEG has no pixels")
|
||||
}
|
||||
average := uint8(total / pixels)
|
||||
return Decision{
|
||||
Triggered: average >= e.config.BrightnessThreshold,
|
||||
Reason: fmt.Sprintf("average brightness %d, threshold %d", average, e.config.BrightnessThreshold),
|
||||
}, nil
|
||||
default:
|
||||
return Decision{}, fmt.Errorf("unknown processing profile %q", profile)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
package processor
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/kerberos-io/agent/examples/frame-processor/contract"
|
||||
)
|
||||
|
||||
func TestEveryNthFrameIsTrackedPerDevice(t *testing.T) {
|
||||
engine := New(Config{EveryN: 2})
|
||||
metadata := contract.FrameMetadata{ProcessingProfile: ProfileEveryNthFrame, DeviceID: "device-1"}
|
||||
|
||||
first, err := engine.Process(context.Background(), metadata, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
second, err := engine.Process(context.Background(), metadata, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
metadata.DeviceID = "device-2"
|
||||
otherDevice, err := engine.Process(context.Background(), metadata, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if first.Triggered || !second.Triggered || otherDevice.Triggered {
|
||||
t.Fatalf("decisions = first:%t second:%t other:%t", first.Triggered, second.Triggered, otherDevice.Triggered)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUnknownProfileFails(t *testing.T) {
|
||||
engine := New(Config{})
|
||||
_, err := engine.Process(context.Background(), contract.FrameMetadata{
|
||||
ProcessingProfile: "missing-profile",
|
||||
}, nil)
|
||||
if err == nil {
|
||||
t.Fatal("Process() accepted an unknown profile")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,358 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/subtle"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"image"
|
||||
_ "image/jpeg"
|
||||
"io"
|
||||
"log/slog"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/kerberos-io/agent/examples/frame-processor/contract"
|
||||
"github.com/kerberos-io/agent/examples/frame-processor/processor"
|
||||
)
|
||||
|
||||
const maxMetadataBytes = 64 << 10
|
||||
|
||||
var errRequestTooLarge = errors.New("request exceeds maximum size")
|
||||
|
||||
type Publisher interface {
|
||||
PublishCaptureFrame(context.Context, string, contract.CaptureFrameCommand) error
|
||||
PublishRecordingWindow(context.Context, string, contract.RecordingWindowCommand) error
|
||||
}
|
||||
|
||||
type Processor interface {
|
||||
Process(context.Context, contract.FrameMetadata, []byte) (processor.Decision, error)
|
||||
}
|
||||
|
||||
type Config struct {
|
||||
APIToken string
|
||||
MaxFrameBytes int64
|
||||
MaxFrameTTL time.Duration
|
||||
CommandTTL time.Duration
|
||||
PreRollSeconds int64
|
||||
EventClipSeconds int64
|
||||
}
|
||||
|
||||
type Service struct {
|
||||
config Config
|
||||
processor Processor
|
||||
publisher Publisher
|
||||
now func() time.Time
|
||||
newID func() string
|
||||
|
||||
resultsMu sync.Mutex
|
||||
results map[string]*frameResult
|
||||
}
|
||||
|
||||
type frameResult struct {
|
||||
done chan struct{}
|
||||
expiresAt int64
|
||||
response FrameResponse
|
||||
err error
|
||||
}
|
||||
|
||||
type FrameResponse struct {
|
||||
SchemaVersion string `json:"schemaVersion"`
|
||||
RequestID string `json:"requestId"`
|
||||
FrameID string `json:"frameId"`
|
||||
Decision string `json:"decision"`
|
||||
Reason string `json:"reason"`
|
||||
}
|
||||
|
||||
type FrameRequestResponse struct {
|
||||
SchemaVersion string `json:"schemaVersion"`
|
||||
RequestID string `json:"requestId"`
|
||||
DeviceIDs []string `json:"deviceIds"`
|
||||
Status string `json:"status"`
|
||||
}
|
||||
|
||||
func New(config Config, frameProcessor Processor, publisher Publisher, newID func() string) *Service {
|
||||
if config.MaxFrameBytes <= 0 {
|
||||
config.MaxFrameBytes = 4 << 20
|
||||
}
|
||||
if config.CommandTTL <= 0 {
|
||||
config.CommandTTL = 30 * time.Second
|
||||
}
|
||||
if config.MaxFrameTTL <= 0 {
|
||||
config.MaxFrameTTL = 5 * time.Minute
|
||||
}
|
||||
if config.EventClipSeconds <= 0 {
|
||||
config.EventClipSeconds = 30
|
||||
}
|
||||
if newID == nil {
|
||||
newID = func() string { return fmt.Sprintf("request-%d", time.Now().UnixNano()) }
|
||||
}
|
||||
return &Service{
|
||||
config: config, processor: frameProcessor, publisher: publisher,
|
||||
now: time.Now, newID: newID, results: make(map[string]*frameResult),
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Service) Handler() http.Handler {
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("GET /health", s.handleHealth)
|
||||
mux.HandleFunc("POST /v1/frames", s.authorize(s.handleFrame))
|
||||
mux.HandleFunc("POST /v1/frame-requests", s.authorize(s.handleFrameRequest))
|
||||
return mux
|
||||
}
|
||||
|
||||
func (s *Service) authorize(next http.HandlerFunc) http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
if s.config.APIToken == "" {
|
||||
writeError(w, http.StatusServiceUnavailable, "service authentication is not configured")
|
||||
return
|
||||
}
|
||||
provided := strings.TrimPrefix(r.Header.Get("Authorization"), "Bearer ")
|
||||
if subtle.ConstantTimeCompare([]byte(provided), []byte(s.config.APIToken)) != 1 {
|
||||
writeError(w, http.StatusUnauthorized, "unauthorized")
|
||||
return
|
||||
}
|
||||
next(w, r)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Service) handleHealth(w http.ResponseWriter, _ *http.Request) {
|
||||
writeJSON(w, http.StatusOK, map[string]string{"status": "healthy"})
|
||||
}
|
||||
|
||||
func (s *Service) handleFrame(w http.ResponseWriter, r *http.Request) {
|
||||
metadata, frame, err := readFrame(w, r, s.config.MaxFrameBytes)
|
||||
if err != nil {
|
||||
if errors.Is(err, errRequestTooLarge) {
|
||||
writeError(w, http.StatusRequestEntityTooLarge, err.Error())
|
||||
return
|
||||
}
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
nowMillis := s.now().UnixMilli()
|
||||
if err := metadata.Validate(nowMillis); err != nil {
|
||||
writeError(w, http.StatusUnprocessableEntity, err.Error())
|
||||
return
|
||||
}
|
||||
if time.Duration(metadata.ExpiresAt-nowMillis)*time.Millisecond > s.config.MaxFrameTTL {
|
||||
writeError(w, http.StatusUnprocessableEntity, "expiresAt exceeds maximum frame TTL")
|
||||
return
|
||||
}
|
||||
imageConfig, _, err := image.DecodeConfig(bytes.NewReader(frame))
|
||||
if err != nil {
|
||||
writeError(w, http.StatusUnsupportedMediaType, "frame must be a valid JPEG")
|
||||
return
|
||||
}
|
||||
if imageConfig.Width != metadata.Width || imageConfig.Height != metadata.Height {
|
||||
writeError(w, http.StatusUnprocessableEntity, "frame dimensions do not match metadata")
|
||||
return
|
||||
}
|
||||
|
||||
result, owner := s.beginFrame(metadata.FrameID, metadata.ExpiresAt, nowMillis)
|
||||
if !owner {
|
||||
select {
|
||||
case <-r.Context().Done():
|
||||
writeError(w, http.StatusRequestTimeout, "request cancelled")
|
||||
return
|
||||
case <-result.done:
|
||||
}
|
||||
if result.err != nil {
|
||||
writeError(w, http.StatusBadGateway, result.err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, result.response)
|
||||
return
|
||||
}
|
||||
|
||||
response, processErr := s.processFrame(r.Context(), metadata, frame)
|
||||
s.finishFrame(result, response, processErr)
|
||||
if processErr != nil {
|
||||
writeError(w, http.StatusBadGateway, processErr.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, response)
|
||||
}
|
||||
|
||||
func (s *Service) processFrame(ctx context.Context, metadata contract.FrameMetadata, frame []byte) (FrameResponse, error) {
|
||||
decision, err := s.processor.Process(ctx, metadata, frame)
|
||||
if err != nil {
|
||||
return FrameResponse{}, fmt.Errorf("process frame: %w", err)
|
||||
}
|
||||
response := FrameResponse{
|
||||
SchemaVersion: contract.SchemaVersion,
|
||||
RequestID: metadata.RequestID,
|
||||
FrameID: metadata.FrameID,
|
||||
Decision: "no-event",
|
||||
Reason: decision.Reason,
|
||||
}
|
||||
if !decision.Triggered {
|
||||
return response, nil
|
||||
}
|
||||
command := contract.RecordingWindowCommand{
|
||||
SchemaVersion: contract.SchemaVersion,
|
||||
RequestID: metadata.RequestID,
|
||||
FrameID: metadata.FrameID,
|
||||
CapturedAt: metadata.CapturedAt,
|
||||
PreRollSeconds: s.config.PreRollSeconds,
|
||||
EventClipSeconds: s.config.EventClipSeconds,
|
||||
ExpiresAt: s.now().Add(s.config.CommandTTL).UnixMilli(),
|
||||
ProcessingProfile: metadata.ProcessingProfile,
|
||||
TraceID: metadata.TraceID,
|
||||
}
|
||||
if err := command.Validate(s.now().UnixMilli()); err != nil {
|
||||
return FrameResponse{}, fmt.Errorf("build recording command: %w", err)
|
||||
}
|
||||
if err := s.publisher.PublishRecordingWindow(ctx, metadata.DeviceID, command); err != nil {
|
||||
return FrameResponse{}, fmt.Errorf("publish recording command: %w", err)
|
||||
}
|
||||
response.Decision = "event"
|
||||
return response, nil
|
||||
}
|
||||
|
||||
func (s *Service) handleFrameRequest(w http.ResponseWriter, r *http.Request) {
|
||||
var request contract.FrameRequest
|
||||
decoder := json.NewDecoder(http.MaxBytesReader(w, r.Body, maxMetadataBytes))
|
||||
decoder.DisallowUnknownFields()
|
||||
if err := decoder.Decode(&request); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid JSON request")
|
||||
return
|
||||
}
|
||||
if request.RequestID == "" {
|
||||
request.RequestID = s.newID()
|
||||
}
|
||||
nowMillis := s.now().UnixMilli()
|
||||
if err := request.Validate(nowMillis); err != nil {
|
||||
writeError(w, http.StatusUnprocessableEntity, err.Error())
|
||||
return
|
||||
}
|
||||
if time.Duration(request.ExpiresAt-nowMillis)*time.Millisecond > s.config.MaxFrameTTL {
|
||||
writeError(w, http.StatusUnprocessableEntity, "expiresAt exceeds maximum frame TTL")
|
||||
return
|
||||
}
|
||||
for _, deviceID := range request.DeviceIDs {
|
||||
command := contract.CaptureFrameCommand{
|
||||
SchemaVersion: contract.SchemaVersion,
|
||||
RequestID: request.RequestID,
|
||||
ProcessingProfile: request.ProcessingProfile,
|
||||
ExpiresAt: request.ExpiresAt,
|
||||
TraceID: request.TraceID,
|
||||
}
|
||||
if err := s.publisher.PublishCaptureFrame(r.Context(), deviceID, command); err != nil {
|
||||
writeError(w, http.StatusBadGateway, "failed to publish capture command")
|
||||
return
|
||||
}
|
||||
}
|
||||
writeJSON(w, http.StatusAccepted, FrameRequestResponse{
|
||||
SchemaVersion: contract.SchemaVersion,
|
||||
RequestID: request.RequestID,
|
||||
DeviceIDs: request.DeviceIDs,
|
||||
Status: "accepted",
|
||||
})
|
||||
}
|
||||
|
||||
func (s *Service) beginFrame(frameID string, expiresAt, nowMillis int64) (*frameResult, bool) {
|
||||
s.resultsMu.Lock()
|
||||
defer s.resultsMu.Unlock()
|
||||
for id, result := range s.results {
|
||||
if result.expiresAt <= nowMillis {
|
||||
delete(s.results, id)
|
||||
}
|
||||
}
|
||||
if result, ok := s.results[frameID]; ok {
|
||||
return result, false
|
||||
}
|
||||
result := &frameResult{done: make(chan struct{}), expiresAt: expiresAt}
|
||||
s.results[frameID] = result
|
||||
delay := time.Duration(expiresAt-nowMillis) * time.Millisecond
|
||||
time.AfterFunc(delay, func() {
|
||||
s.resultsMu.Lock()
|
||||
if s.results[frameID] == result {
|
||||
delete(s.results, frameID)
|
||||
}
|
||||
s.resultsMu.Unlock()
|
||||
})
|
||||
return result, true
|
||||
}
|
||||
|
||||
func (s *Service) finishFrame(result *frameResult, response FrameResponse, err error) {
|
||||
s.resultsMu.Lock()
|
||||
result.response = response
|
||||
result.err = err
|
||||
close(result.done)
|
||||
s.resultsMu.Unlock()
|
||||
}
|
||||
|
||||
func readFrame(w http.ResponseWriter, r *http.Request, maxFrameBytes int64) (contract.FrameMetadata, []byte, error) {
|
||||
r.Body = http.MaxBytesReader(w, r.Body, maxFrameBytes+maxMetadataBytes)
|
||||
reader, err := r.MultipartReader()
|
||||
if err != nil {
|
||||
return contract.FrameMetadata{}, nil, errors.New("content type must be multipart/form-data")
|
||||
}
|
||||
var metadata contract.FrameMetadata
|
||||
var frame []byte
|
||||
for {
|
||||
part, err := reader.NextPart()
|
||||
if errors.Is(err, io.EOF) {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
var tooLarge *http.MaxBytesError
|
||||
if errors.As(err, &tooLarge) {
|
||||
return contract.FrameMetadata{}, nil, errRequestTooLarge
|
||||
}
|
||||
return contract.FrameMetadata{}, nil, errors.New("invalid multipart body")
|
||||
}
|
||||
switch part.FormName() {
|
||||
case "metadata":
|
||||
if err := decodeMetadataPart(part, &metadata); err != nil {
|
||||
return contract.FrameMetadata{}, nil, err
|
||||
}
|
||||
case "frame":
|
||||
if part.Header.Get("Content-Type") != "image/jpeg" {
|
||||
return contract.FrameMetadata{}, nil, errors.New("frame content type must be image/jpeg")
|
||||
}
|
||||
frame, err = io.ReadAll(io.LimitReader(part, maxFrameBytes+1))
|
||||
if err != nil || int64(len(frame)) > maxFrameBytes {
|
||||
return contract.FrameMetadata{}, nil, errRequestTooLarge
|
||||
}
|
||||
}
|
||||
}
|
||||
if metadata.FrameID == "" || len(frame) == 0 {
|
||||
return contract.FrameMetadata{}, nil, errors.New("metadata and frame parts are required")
|
||||
}
|
||||
return metadata, frame, nil
|
||||
}
|
||||
|
||||
func decodeMetadataPart(part *multipart.Part, target *contract.FrameMetadata) error {
|
||||
value, err := io.ReadAll(io.LimitReader(part, maxMetadataBytes+1))
|
||||
if err != nil || len(value) > maxMetadataBytes {
|
||||
return errRequestTooLarge
|
||||
}
|
||||
decoder := json.NewDecoder(bytes.NewReader(value))
|
||||
decoder.DisallowUnknownFields()
|
||||
if err := decoder.Decode(target); err != nil {
|
||||
return errors.New("invalid metadata JSON")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func writeError(w http.ResponseWriter, status int, message string) {
|
||||
writeJSON(w, status, map[string]any{
|
||||
"schemaVersion": contract.SchemaVersion,
|
||||
"error": message,
|
||||
})
|
||||
}
|
||||
|
||||
func writeJSON(w http.ResponseWriter, status int, value any) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(status)
|
||||
if err := json.NewEncoder(w).Encode(value); err != nil {
|
||||
slog.Error("failed to encode HTTP response", "error", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,267 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"image"
|
||||
"image/color"
|
||||
"image/jpeg"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/textproto"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/kerberos-io/agent/examples/frame-processor/contract"
|
||||
"github.com/kerberos-io/agent/examples/frame-processor/processor"
|
||||
)
|
||||
|
||||
type recordingPublish struct {
|
||||
deviceID string
|
||||
command contract.RecordingWindowCommand
|
||||
}
|
||||
|
||||
type fakePublisher struct {
|
||||
mu sync.Mutex
|
||||
captures []contract.CaptureFrameCommand
|
||||
recordings []recordingPublish
|
||||
}
|
||||
|
||||
type blockingProcessor struct {
|
||||
started chan struct{}
|
||||
release chan struct{}
|
||||
mu sync.Mutex
|
||||
calls int
|
||||
}
|
||||
|
||||
func (p *blockingProcessor) Process(ctx context.Context, _ contract.FrameMetadata, _ []byte) (processor.Decision, error) {
|
||||
p.mu.Lock()
|
||||
p.calls++
|
||||
if p.calls == 1 {
|
||||
close(p.started)
|
||||
}
|
||||
p.mu.Unlock()
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return processor.Decision{}, ctx.Err()
|
||||
case <-p.release:
|
||||
return processor.Decision{Triggered: true, Reason: "test"}, nil
|
||||
}
|
||||
}
|
||||
|
||||
func (p *fakePublisher) PublishCaptureFrame(_ context.Context, _ string, command contract.CaptureFrameCommand) error {
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
p.captures = append(p.captures, command)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p *fakePublisher) PublishRecordingWindow(_ context.Context, deviceID string, command contract.RecordingWindowCommand) error {
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
p.recordings = append(p.recordings, recordingPublish{deviceID: deviceID, command: command})
|
||||
return nil
|
||||
}
|
||||
|
||||
func TestFrameAlwaysTriggerPublishesOneIdempotentRecordingCommand(t *testing.T) {
|
||||
publisher := &fakePublisher{}
|
||||
service := New(Config{
|
||||
APIToken: "secret", CommandTTL: time.Minute,
|
||||
PreRollSeconds: 10, EventClipSeconds: 30,
|
||||
}, processor.New(processor.Config{}), publisher, nil)
|
||||
service.now = func() time.Time { return time.UnixMilli(1_000) }
|
||||
server := httptest.NewServer(service.Handler())
|
||||
defer server.Close()
|
||||
|
||||
metadata := validMetadata()
|
||||
for range 2 {
|
||||
response := postFrame(t, server.URL, "secret", metadata, jpegFrame(t, 2, 2, 255))
|
||||
if response.StatusCode != http.StatusOK {
|
||||
t.Fatalf("POST /v1/frames status = %d", response.StatusCode)
|
||||
}
|
||||
response.Body.Close()
|
||||
}
|
||||
|
||||
if got := len(publisher.recordings); got != 1 {
|
||||
t.Fatalf("recording commands = %d, want 1", got)
|
||||
}
|
||||
published := publisher.recordings[0]
|
||||
if published.deviceID != metadata.DeviceID || published.command.CapturedAt != metadata.CapturedAt {
|
||||
t.Fatalf("published command = %#v", published)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFrameRequestPublishesCaptureCommand(t *testing.T) {
|
||||
publisher := &fakePublisher{}
|
||||
service := New(Config{APIToken: "secret"}, processor.New(processor.Config{}), publisher, func() string { return "generated-request" })
|
||||
service.now = func() time.Time { return time.UnixMilli(1_000) }
|
||||
server := httptest.NewServer(service.Handler())
|
||||
defer server.Close()
|
||||
|
||||
body := `{"schemaVersion":"1.0","deviceIds":["device-1"],"processingProfile":"always-trigger","expiresAt":2000}`
|
||||
request, err := http.NewRequest(http.MethodPost, server.URL+"/v1/frame-requests", strings.NewReader(body))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
request.Header.Set("Authorization", "Bearer secret")
|
||||
request.Header.Set("Content-Type", "application/json")
|
||||
response, err := http.DefaultClient.Do(request)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer response.Body.Close()
|
||||
if response.StatusCode != http.StatusAccepted {
|
||||
t.Fatalf("POST /v1/frame-requests status = %d", response.StatusCode)
|
||||
}
|
||||
if got := len(publisher.captures); got != 1 || publisher.captures[0].RequestID != "generated-request" {
|
||||
t.Fatalf("capture commands = %#v", publisher.captures)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFrameRejectsUnauthorizedRequest(t *testing.T) {
|
||||
service := New(Config{APIToken: "secret"}, processor.New(processor.Config{}), &fakePublisher{}, nil)
|
||||
server := httptest.NewServer(service.Handler())
|
||||
defer server.Close()
|
||||
|
||||
response := postFrame(t, server.URL, "wrong", validMetadata(), jpegFrame(t, 2, 2, 255))
|
||||
defer response.Body.Close()
|
||||
if response.StatusCode != http.StatusUnauthorized {
|
||||
t.Fatalf("POST /v1/frames status = %d", response.StatusCode)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFrameFailsClosedWithoutConfiguredToken(t *testing.T) {
|
||||
application := New(Config{}, processor.New(processor.Config{}), &fakePublisher{}, nil)
|
||||
server := httptest.NewServer(application.Handler())
|
||||
defer server.Close()
|
||||
|
||||
response := postFrame(t, server.URL, "", validMetadata(), jpegFrame(t, 2, 2, 255))
|
||||
defer response.Body.Close()
|
||||
if response.StatusCode != http.StatusServiceUnavailable {
|
||||
t.Fatalf("POST /v1/frames status = %d", response.StatusCode)
|
||||
}
|
||||
}
|
||||
|
||||
func TestConcurrentDuplicateFramesPublishOneRecordingCommand(t *testing.T) {
|
||||
publisher := &fakePublisher{}
|
||||
frameProcessor := &blockingProcessor{started: make(chan struct{}), release: make(chan struct{})}
|
||||
application := New(Config{
|
||||
APIToken: "secret", CommandTTL: time.Minute,
|
||||
PreRollSeconds: 10, EventClipSeconds: 30,
|
||||
}, frameProcessor, publisher, nil)
|
||||
application.now = func() time.Time { return time.UnixMilli(1_000) }
|
||||
server := httptest.NewServer(application.Handler())
|
||||
defer server.Close()
|
||||
|
||||
metadata := validMetadata()
|
||||
statuses := make(chan int, 2)
|
||||
go func() {
|
||||
response := postFrame(t, server.URL, "secret", metadata, jpegFrame(t, 2, 2, 255))
|
||||
defer response.Body.Close()
|
||||
statuses <- response.StatusCode
|
||||
}()
|
||||
<-frameProcessor.started
|
||||
go func() {
|
||||
response := postFrame(t, server.URL, "secret", metadata, jpegFrame(t, 2, 2, 255))
|
||||
defer response.Body.Close()
|
||||
statuses <- response.StatusCode
|
||||
}()
|
||||
close(frameProcessor.release)
|
||||
|
||||
for range 2 {
|
||||
if status := <-statuses; status != http.StatusOK {
|
||||
t.Fatalf("POST /v1/frames status = %d", status)
|
||||
}
|
||||
}
|
||||
if got := len(publisher.recordings); got != 1 {
|
||||
t.Fatalf("recording commands = %d, want 1", got)
|
||||
}
|
||||
frameProcessor.mu.Lock()
|
||||
defer frameProcessor.mu.Unlock()
|
||||
if frameProcessor.calls != 1 {
|
||||
t.Fatalf("processor calls = %d, want 1", frameProcessor.calls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFrameRejectsTTLAboveConfiguredMaximum(t *testing.T) {
|
||||
application := New(Config{APIToken: "secret", MaxFrameTTL: time.Second}, processor.New(processor.Config{}), &fakePublisher{}, nil)
|
||||
application.now = func() time.Time { return time.UnixMilli(1_000) }
|
||||
server := httptest.NewServer(application.Handler())
|
||||
defer server.Close()
|
||||
|
||||
metadata := validMetadata()
|
||||
metadata.ExpiresAt = 2_001
|
||||
response := postFrame(t, server.URL, "secret", metadata, jpegFrame(t, 2, 2, 255))
|
||||
defer response.Body.Close()
|
||||
if response.StatusCode != http.StatusUnprocessableEntity {
|
||||
t.Fatalf("POST /v1/frames status = %d", response.StatusCode)
|
||||
}
|
||||
}
|
||||
|
||||
func validMetadata() contract.FrameMetadata {
|
||||
return contract.FrameMetadata{
|
||||
SchemaVersion: contract.SchemaVersion,
|
||||
RequestID: "request-1", FrameID: "frame-1", DeviceID: "device-1",
|
||||
CapturedAt: 900, ExpiresAt: 2_000, ProcessingProfile: processor.ProfileAlwaysTrigger,
|
||||
SourceStream: "sub", Width: 2, Height: 2,
|
||||
}
|
||||
}
|
||||
|
||||
func jpegFrame(t *testing.T, width, height int, brightness uint8) []byte {
|
||||
t.Helper()
|
||||
frame := image.NewGray(image.Rect(0, 0, width, height))
|
||||
for index := range frame.Pix {
|
||||
frame.Pix[index] = brightness
|
||||
}
|
||||
frame.SetGray(0, 0, color.Gray{Y: brightness})
|
||||
var output bytes.Buffer
|
||||
if err := jpeg.Encode(&output, frame, nil); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return output.Bytes()
|
||||
}
|
||||
|
||||
func postFrame(t *testing.T, baseURL, token string, metadata contract.FrameMetadata, frame []byte) *http.Response {
|
||||
t.Helper()
|
||||
var body bytes.Buffer
|
||||
writer := multipart.NewWriter(&body)
|
||||
metadataHeader := make(textproto.MIMEHeader)
|
||||
metadataHeader.Set("Content-Disposition", `form-data; name="metadata"`)
|
||||
metadataHeader.Set("Content-Type", "application/json")
|
||||
part, err := writer.CreatePart(metadataHeader)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := json.NewEncoder(part).Encode(metadata); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
frameHeader := make(textproto.MIMEHeader)
|
||||
frameHeader.Set("Content-Disposition", `form-data; name="frame"; filename="frame.jpg"`)
|
||||
frameHeader.Set("Content-Type", "image/jpeg")
|
||||
part, err = writer.CreatePart(frameHeader)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := part.Write(frame); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := writer.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
request, err := http.NewRequest(http.MethodPost, baseURL+"/v1/frames", &body)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
request.Header.Set("Authorization", "Bearer "+token)
|
||||
request.Header.Set("Content-Type", writer.FormDataContentType())
|
||||
response, err := http.DefaultClient.Do(request)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return response
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"schemaVersion": "1.0",
|
||||
"requestId": "request-example-1",
|
||||
"processingProfile": "always-trigger",
|
||||
"expiresAt": 4102444800000,
|
||||
"traceId": "trace-example-1"
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"schemaVersion": "1.0",
|
||||
"requestId": "request-example-1",
|
||||
"deviceIds": ["camera-1"],
|
||||
"processingProfile": "always-trigger",
|
||||
"expiresAt": 4102444800000,
|
||||
"traceId": "trace-example-1"
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"schemaVersion": "1.0",
|
||||
"requestId": "request-example-1",
|
||||
"frameId": "frame-example-1",
|
||||
"capturedAt": 1789380000123,
|
||||
"preRollSeconds": 10,
|
||||
"eventClipSeconds": 30,
|
||||
"expiresAt": 4102444800000,
|
||||
"processingProfile": "always-trigger",
|
||||
"traceId": "trace-example-1"
|
||||
}
|
||||
@@ -1296,9 +1296,6 @@ const docTemplate = `{
|
||||
"description": "old way, remove this",
|
||||
"type": "string"
|
||||
},
|
||||
"custom_headers": {
|
||||
"type": "string"
|
||||
},
|
||||
"directory": {
|
||||
"type": "string"
|
||||
},
|
||||
|
||||
@@ -1289,9 +1289,6 @@
|
||||
"description": "old way, remove this",
|
||||
"type": "string"
|
||||
},
|
||||
"custom_headers": {
|
||||
"type": "string"
|
||||
},
|
||||
"directory": {
|
||||
"type": "string"
|
||||
},
|
||||
|
||||
+1
-1
@@ -25,7 +25,7 @@ require (
|
||||
github.com/gorilla/websocket v1.5.3
|
||||
github.com/kellydunn/golang-geo v0.7.0
|
||||
github.com/kerberos-io/joy4 v1.0.64
|
||||
github.com/kerberos-io/onvif v1.2.3
|
||||
github.com/kerberos-io/onvif v1.2.2
|
||||
github.com/minio/minio-go/v6 v6.0.57
|
||||
github.com/moq-dev/moq-go v0.5.7
|
||||
github.com/nfnt/resize v0.0.0-20180221191011-83c6a9932646
|
||||
|
||||
+2
-2
@@ -776,8 +776,8 @@ github.com/kellydunn/golang-geo v0.7.0 h1:A5j0/BvNgGwY6Yb6inXQxzYwlPHc6WVZR+Mrar
|
||||
github.com/kellydunn/golang-geo v0.7.0/go.mod h1:YYlQPJ+DPEzrHx8kT3oPHC/NjyvCCXE+IuKGKdrjrcU=
|
||||
github.com/kerberos-io/joy4 v1.0.64 h1:gTUSotHSOhp9mNqEecgq88tQHvpj7TjmrvPUsPm0idg=
|
||||
github.com/kerberos-io/joy4 v1.0.64/go.mod h1:nZp4AjvKvTOXRrmDyAIOw+Da+JA5OcSo/JundGfOlFU=
|
||||
github.com/kerberos-io/onvif v1.2.3 h1:RDnDvLBd92VCgFOrlbGavzCVN9lQA/qs9novdNFIAOU=
|
||||
github.com/kerberos-io/onvif v1.2.3/go.mod h1:XSgEQXmEDjUQTbdXvsaRJt6Az8YPGj7L+j5iXKEGijU=
|
||||
github.com/kerberos-io/onvif v1.2.2 h1:QnxITps7xvAVD2abWRsa3+p9QexjKESQQldUFMx/mYA=
|
||||
github.com/kerberos-io/onvif v1.2.2/go.mod h1:XSgEQXmEDjUQTbdXvsaRJt6Az8YPGj7L+j5iXKEGijU=
|
||||
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
|
||||
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
|
||||
github.com/klauspost/asmfmt v1.3.2/go.mod h1:AG8TuvYojzulgDAMCnYn50l/5QV3Bs/tp6j0HLHbNSE=
|
||||
|
||||
@@ -10,58 +10,11 @@ import (
|
||||
"github.com/kerberos-io/agent/machinery/src/models"
|
||||
)
|
||||
|
||||
func TestPickRecordingToCleanupAcrossDatedDirectories(t *testing.T) {
|
||||
for _, legacyMarker := range []bool{false, true} {
|
||||
t.Run(map[bool]string{false: "metadata", true: "legacy"}[legacyMarker], func(t *testing.T) {
|
||||
recordingsDir, cloudDir := newCleanupDirs(t)
|
||||
pendingName := "2026/01/01/oldest_pending.mp4"
|
||||
safeName := "2026/01/02/uploaded.mp4"
|
||||
writeRecording(t, recordingsDir, pendingName, 30)
|
||||
if legacyMarker {
|
||||
if err := os.WriteFile(filepath.Join(cloudDir, filepath.Base(pendingName)), nil, 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
} else {
|
||||
markPending(t, cloudDir, pendingName)
|
||||
}
|
||||
writeRecording(t, recordingsDir, safeName, 20)
|
||||
writeRecording(t, recordingsDir, "flat_uploaded.mp4", 10)
|
||||
name, pending, err := pickRecordingToCleanup(recordingsDir, cloudDir)
|
||||
if err != nil || pending || name != safeName {
|
||||
t.Fatalf("cleanup = (%q, %v, %v), want (%q, false, nil)", name, pending, err, safeName)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestCleanupDatedPendingRecordingRemovesFlatMarker(t *testing.T) {
|
||||
recordingsDir, cloudDir := newCleanupDirs(t)
|
||||
name := "2026/01/01/pending.mp4"
|
||||
if err := os.MkdirAll(filepath.Dir(filepath.Join(recordingsDir, name)), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
writeSizedRecording(t, recordingsDir, name, 2*1000*1000)
|
||||
markPending(t, cloudDir, name)
|
||||
config := &models.Configuration{Config: models.Config{AutoClean: "true", MaxDirectorySize: 1}}
|
||||
CleanupRecordingDirectory(filepath.Dir(filepath.Dir(recordingsDir)), config)
|
||||
for _, filePath := range []string{
|
||||
filepath.Join(recordingsDir, name),
|
||||
filepath.Join(cloudDir, models.RecordingUploadMetadataFileName(name)),
|
||||
} {
|
||||
if _, err := os.Stat(filePath); !os.IsNotExist(err) {
|
||||
t.Fatalf("expected %s to be removed, got %v", filePath, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// writeRecording creates a file under recordingsDir and sets its modtime so the
|
||||
// tests can control the "oldest" ordering deterministically.
|
||||
func writeRecording(t *testing.T, recordingsDir, name string, ageMinutes int) {
|
||||
t.Helper()
|
||||
full := filepath.Join(recordingsDir, name)
|
||||
if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil {
|
||||
t.Fatalf("mkdir recording directory: %v", err)
|
||||
}
|
||||
if err := os.WriteFile(full, []byte("data"), 0o644); err != nil {
|
||||
t.Fatalf("write recording %s: %v", name, err)
|
||||
}
|
||||
|
||||
@@ -32,7 +32,6 @@ import (
|
||||
"github.com/bluenviron/gortsplib/v5/pkg/format/rtplpcm"
|
||||
"github.com/bluenviron/gortsplib/v5/pkg/format/rtpmpeg4audio"
|
||||
"github.com/bluenviron/gortsplib/v5/pkg/format/rtpsimpleaudio"
|
||||
"github.com/bluenviron/mediacommon/pkg/bits"
|
||||
"github.com/bluenviron/mediacommon/pkg/codecs/h264"
|
||||
"github.com/bluenviron/mediacommon/pkg/codecs/h265"
|
||||
"github.com/bluenviron/mediacommon/pkg/codecs/mpeg4audio"
|
||||
@@ -160,10 +159,13 @@ type Golibrtsp struct {
|
||||
}
|
||||
|
||||
// streamHealth instruments the RTSP read path. gortsplib delivers every RTP
|
||||
// packet on a single read goroutine, so prolonged queue lock contention or
|
||||
// process starvation can delay socket reads. Queue consumers process packets
|
||||
// after releasing the lock; a slow recorder therefore falls behind its cursor
|
||||
// instead of directly blocking WritePacket.
|
||||
// packet on a single read goroutine; queue.WritePacket() is synchronous, so if
|
||||
// a downstream consumer (recording, muxing, WebRTC) is slow or the process is
|
||||
// CPU-starved, WritePacket() blocks, the TCP socket is not drained, and the
|
||||
// camera advances RTP sequence numbers -> "RTP packets lost". This type makes
|
||||
// the two failure modes distinguishable:
|
||||
// - large writeMax / writeAvg => downstream back-pressure (our side).
|
||||
// - large gapMax with fast writes => upstream network / camera stall.
|
||||
type streamHealth struct {
|
||||
mu sync.Mutex
|
||||
windowStart time.Time
|
||||
@@ -176,152 +178,6 @@ type streamHealth struct {
|
||||
decodeErrs int64
|
||||
}
|
||||
|
||||
type videoReferenceState struct {
|
||||
haveSequence bool
|
||||
lastSequence uint16
|
||||
valid bool
|
||||
droppedAccessUnits uint64
|
||||
}
|
||||
|
||||
type h264FrameNumTracker struct {
|
||||
sps *h264.SPS
|
||||
haveFrameNum bool
|
||||
lastFrameNum uint32
|
||||
}
|
||||
|
||||
const maxH264SliceHeaderBytes = 64
|
||||
|
||||
func (t *h264FrameNumTracker) setSPS(sps *h264.SPS) {
|
||||
if sps == nil {
|
||||
return
|
||||
}
|
||||
if t.sps == nil ||
|
||||
t.sps.Log2MaxFrameNumMinus4 != sps.Log2MaxFrameNumMinus4 ||
|
||||
t.sps.SeparateColourPlaneFlag != sps.SeparateColourPlaneFlag ||
|
||||
t.sps.FrameMbsOnlyFlag != sps.FrameMbsOnlyFlag {
|
||||
t.haveFrameNum = false
|
||||
}
|
||||
copy := *sps
|
||||
t.sps = ©
|
||||
}
|
||||
|
||||
func (t *h264FrameNumTracker) observe(au [][]byte) (uint32, uint32, bool, bool) {
|
||||
if t.sps == nil || !t.sps.FrameMbsOnlyFlag {
|
||||
return 0, 0, false, false
|
||||
}
|
||||
|
||||
for _, nalu := range au {
|
||||
if len(nalu) == 0 {
|
||||
continue
|
||||
}
|
||||
naluType := h264.NALUType(nalu[0] & 0x1F)
|
||||
if naluType != h264.NALUTypeIDR && naluType != h264.NALUTypeNonIDR {
|
||||
continue
|
||||
}
|
||||
|
||||
frameNum, err := h264SliceFrameNum(nalu, t.sps)
|
||||
if err != nil {
|
||||
return 0, 0, false, false
|
||||
}
|
||||
if naluType == h264.NALUTypeIDR {
|
||||
t.haveFrameNum = true
|
||||
t.lastFrameNum = frameNum
|
||||
return frameNum, frameNum, false, true
|
||||
}
|
||||
|
||||
// Non-reference pictures do not advance the reference-picture frame_num
|
||||
// sequence and therefore cannot prove that a reference picture is missing.
|
||||
if (nalu[0]>>5)&0x03 == 0 {
|
||||
return frameNum, frameNum, false, true
|
||||
}
|
||||
if !t.haveFrameNum {
|
||||
t.haveFrameNum = true
|
||||
t.lastFrameNum = frameNum
|
||||
return frameNum, frameNum, false, true
|
||||
}
|
||||
|
||||
frameNumBits := t.sps.Log2MaxFrameNumMinus4 + 4
|
||||
if frameNumBits > 16 {
|
||||
return 0, 0, false, false
|
||||
}
|
||||
maxFrameNum := uint32(1) << frameNumBits
|
||||
expected := (t.lastFrameNum + 1) & (maxFrameNum - 1)
|
||||
// When the SPS permits frame_num gaps, the decoder is required to
|
||||
// synthesize non-existing pictures and the discontinuity is not evidence
|
||||
// of transport loss. RTP/depacketizer errors still invalidate references.
|
||||
gap := frameNum != expected && !t.sps.GapsInFrameNumValueAllowedFlag
|
||||
t.lastFrameNum = frameNum
|
||||
return expected, frameNum, gap, true
|
||||
}
|
||||
|
||||
return 0, 0, false, false
|
||||
}
|
||||
|
||||
func h264SliceFrameNum(nalu []byte, sps *h264.SPS) (uint32, error) {
|
||||
if len(nalu) < 2 || sps == nil {
|
||||
return 0, errors.New("H264 slice or SPS is missing")
|
||||
}
|
||||
frameNumBits := sps.Log2MaxFrameNumMinus4 + 4
|
||||
if frameNumBits > 16 {
|
||||
return 0, fmt.Errorf("invalid H264 frame_num width: %d", frameNumBits)
|
||||
}
|
||||
header := nalu[1:]
|
||||
if len(header) > maxH264SliceHeaderBytes {
|
||||
header = header[:maxH264SliceHeaderBytes]
|
||||
}
|
||||
header = h264.EmulationPreventionRemove(header)
|
||||
pos := 0
|
||||
|
||||
for range 3 {
|
||||
if _, err := bits.ReadGolombUnsigned(header, &pos); err != nil {
|
||||
return 0, fmt.Errorf("read H264 slice header: %w", err)
|
||||
}
|
||||
}
|
||||
if sps.SeparateColourPlaneFlag {
|
||||
if _, err := bits.ReadBits(header, &pos, 2); err != nil {
|
||||
return 0, fmt.Errorf("read H264 colour plane: %w", err)
|
||||
}
|
||||
}
|
||||
frameNum, err := bits.ReadBits(header, &pos, int(frameNumBits))
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("read H264 frame_num: %w", err)
|
||||
}
|
||||
return uint32(frameNum), nil
|
||||
}
|
||||
|
||||
func (s *videoReferenceState) observeSequence(sequence uint16) (uint16, bool) {
|
||||
expected := sequence
|
||||
gap := false
|
||||
if s.haveSequence {
|
||||
expected = s.lastSequence + 1
|
||||
gap = sequence != expected
|
||||
if gap {
|
||||
s.valid = false
|
||||
}
|
||||
}
|
||||
s.haveSequence = true
|
||||
s.lastSequence = sequence
|
||||
return expected, gap
|
||||
}
|
||||
|
||||
func (s *videoReferenceState) invalidate() {
|
||||
s.valid = false
|
||||
}
|
||||
|
||||
func (s *videoReferenceState) accept(randomAccess bool) (bool, uint64) {
|
||||
if randomAccess {
|
||||
dropped := s.droppedAccessUnits
|
||||
s.droppedAccessUnits = 0
|
||||
s.valid = true
|
||||
return true, dropped
|
||||
}
|
||||
if !s.valid {
|
||||
s.droppedAccessUnits++
|
||||
return false, 0
|
||||
}
|
||||
return true, 0
|
||||
}
|
||||
|
||||
const (
|
||||
streamHealthWindow = 10 * time.Second
|
||||
streamHealthWriteWarn = 150 * time.Millisecond
|
||||
@@ -334,9 +190,9 @@ func newStreamHealth() *streamHealth {
|
||||
}
|
||||
|
||||
// observePacket records one processed video frame: the wall-clock gap since the
|
||||
// previous frame and how long WritePacket spent acquiring and updating the
|
||||
// queue. It emits an immediate warning when either side stalls and a periodic
|
||||
// summary every streamHealthWindow.
|
||||
// previous frame (arrival cadence) and how long WritePacket() blocked
|
||||
// (back-pressure). It emits an immediate warning when either side stalls and a
|
||||
// periodic summary every streamHealthWindow.
|
||||
func (h *streamHealth) observePacket(streamType string, writeDur time.Duration) {
|
||||
if h == nil {
|
||||
return
|
||||
@@ -408,9 +264,9 @@ func (h *streamHealth) observePacket(streamType string, writeDur time.Duration)
|
||||
}
|
||||
}
|
||||
|
||||
// observeLost is invoked by gortsplib when RTP sequence numbers skip. With the
|
||||
// configured TCP transport, the sender or an upstream RTSP component omitted
|
||||
// those packets; sender-side pressure is one possible cause.
|
||||
// observeLost is invoked by gortsplib when RTP sequence numbers skip. On a TCP
|
||||
// transport this means the sender (camera) dropped packets because we were not
|
||||
// reading fast enough, not loss on the wire.
|
||||
func (h *streamHealth) observeLost(streamType string, lost uint64) {
|
||||
if h == nil {
|
||||
return
|
||||
@@ -1031,14 +887,6 @@ func (g *Golibrtsp) Start(ctx context.Context, streamType string, queue *packets
|
||||
// called when a video RTP packet arrives for H264
|
||||
var filteredAU [][]byte
|
||||
if g.VideoH264Media != nil && g.VideoH264Forma != nil {
|
||||
referenceState := &videoReferenceState{}
|
||||
frameNumTracker := &h264FrameNumTracker{}
|
||||
if len(g.VideoH264Forma.SPS) > 0 {
|
||||
var sps h264.SPS
|
||||
if sps.Unmarshal(g.VideoH264Forma.SPS) == nil {
|
||||
frameNumTracker.setSPS(&sps)
|
||||
}
|
||||
}
|
||||
|
||||
// Extracts DTS from the bitstream to support B-frame H264 streams.
|
||||
// Created once per stream (tracks reorder state across access units).
|
||||
@@ -1054,16 +902,6 @@ func (g *Golibrtsp) Start(ctx context.Context, streamType string, queue *packets
|
||||
default:
|
||||
}
|
||||
|
||||
if expected, gap := referenceState.observeSequence(rtppkt.SequenceNumber); gap {
|
||||
log.WithFields(log.Fields{
|
||||
"component": "capture",
|
||||
"event": "video_rtp_sequence_gap",
|
||||
"expected_sequence_number": expected,
|
||||
"received_sequence_number": rtppkt.SequenceNumber,
|
||||
"stream": streamType,
|
||||
}).Warn("RTP sequence gap invalidated video reference chain")
|
||||
}
|
||||
|
||||
if len(rtppkt.Payload) > 0 {
|
||||
|
||||
// decode timestamps — validate each call separately
|
||||
@@ -1079,9 +917,6 @@ func (g *Golibrtsp) Start(ctx context.Context, streamType string, queue *packets
|
||||
// this is a keyframe.
|
||||
au, errDecode := g.VideoH264Decoder.Decode(rtppkt)
|
||||
if errDecode != nil {
|
||||
if errDecode != rtph264.ErrMorePacketsNeeded {
|
||||
referenceState.invalidate()
|
||||
}
|
||||
if errDecode != rtph264.ErrNonStartingPacketAndNoPrevious && errDecode != rtph264.ErrMorePacketsNeeded {
|
||||
log.Error("capture.golibrtsp.Start(): " + errDecode.Error())
|
||||
}
|
||||
@@ -1126,7 +961,6 @@ func (g *Golibrtsp) Start(ctx context.Context, streamType string, queue *packets
|
||||
if errSPS == nil {
|
||||
// Debug SPS information
|
||||
g.debugSPSInfo(&sps, streamType)
|
||||
frameNumTracker.setSPS(&sps)
|
||||
|
||||
// Get width
|
||||
g.Streams[g.VideoH264Index].Width = sps.Width()
|
||||
@@ -1188,39 +1022,6 @@ func (g *Golibrtsp) Start(ctx context.Context, streamType string, queue *packets
|
||||
return
|
||||
}
|
||||
|
||||
if expected, received, gap, ok := frameNumTracker.observe(au); ok && gap {
|
||||
referenceState.invalidate()
|
||||
log.WithFields(log.Fields{
|
||||
"component": "capture",
|
||||
"event": "h264_frame_num_gap",
|
||||
"expected_frame_num": expected,
|
||||
"received_frame_num": received,
|
||||
"stream": streamType,
|
||||
}).Warn("H264 frame_num gap invalidated video reference chain")
|
||||
}
|
||||
|
||||
accepted, recoveredDropped := referenceState.accept(idrPresent)
|
||||
if !accepted {
|
||||
if referenceState.droppedAccessUnits == 1 {
|
||||
log.WithFields(log.Fields{
|
||||
"codec": "H264",
|
||||
"component": "capture",
|
||||
"event": "dependent_access_units_dropped",
|
||||
"stream": streamType,
|
||||
}).Warn("Dropping dependent video frames until the next IDR")
|
||||
}
|
||||
return
|
||||
}
|
||||
if recoveredDropped > 0 {
|
||||
log.WithFields(log.Fields{
|
||||
"codec": "H264",
|
||||
"component": "capture",
|
||||
"dropped_access_units": recoveredDropped,
|
||||
"event": "video_reference_chain_recovered",
|
||||
"stream": streamType,
|
||||
}).Info("Video reference chain recovered at IDR")
|
||||
}
|
||||
|
||||
if idrPresent {
|
||||
log.WithFields(log.Fields{
|
||||
"component": "capture",
|
||||
@@ -1364,7 +1165,6 @@ func (g *Golibrtsp) Start(ctx context.Context, streamType string, queue *packets
|
||||
|
||||
// called when a video RTP packet arrives for H265
|
||||
if g.VideoH265Media != nil && g.VideoH265Forma != nil {
|
||||
referenceState := &videoReferenceState{}
|
||||
|
||||
// Extracts DTS from the bitstream to support B-frame H265 streams.
|
||||
// Created once per stream (tracks reorder state across access units).
|
||||
@@ -1380,16 +1180,6 @@ func (g *Golibrtsp) Start(ctx context.Context, streamType string, queue *packets
|
||||
default:
|
||||
}
|
||||
|
||||
if expected, gap := referenceState.observeSequence(rtppkt.SequenceNumber); gap {
|
||||
log.WithFields(log.Fields{
|
||||
"component": "capture",
|
||||
"event": "video_rtp_sequence_gap",
|
||||
"expected_sequence_number": expected,
|
||||
"received_sequence_number": rtppkt.SequenceNumber,
|
||||
"stream": streamType,
|
||||
}).Warn("RTP sequence gap invalidated video reference chain")
|
||||
}
|
||||
|
||||
if len(rtppkt.Payload) > 0 {
|
||||
|
||||
// decode timestamps — validate each call separately
|
||||
@@ -1405,9 +1195,6 @@ func (g *Golibrtsp) Start(ctx context.Context, streamType string, queue *packets
|
||||
// this is a keyframe.
|
||||
au, errDecode := g.VideoH265Decoder.Decode(rtppkt)
|
||||
if errDecode != nil {
|
||||
if errDecode != rtph265.ErrMorePacketsNeeded {
|
||||
referenceState.invalidate()
|
||||
}
|
||||
if errDecode != rtph265.ErrNonStartingPacketAndNoPrevious && errDecode != rtph265.ErrMorePacketsNeeded {
|
||||
log.Error("capture.golibrtsp.Start(): " + errDecode.Error())
|
||||
}
|
||||
@@ -1457,28 +1244,6 @@ func (g *Golibrtsp) Start(ctx context.Context, streamType string, queue *packets
|
||||
return
|
||||
}
|
||||
|
||||
accepted, recoveredDropped := referenceState.accept(isRandomAccess)
|
||||
if !accepted {
|
||||
if referenceState.droppedAccessUnits == 1 {
|
||||
log.WithFields(log.Fields{
|
||||
"codec": "H265",
|
||||
"component": "capture",
|
||||
"event": "dependent_access_units_dropped",
|
||||
"stream": streamType,
|
||||
}).Warn("Dropping dependent video frames until the next random-access frame")
|
||||
}
|
||||
return
|
||||
}
|
||||
if recoveredDropped > 0 {
|
||||
log.WithFields(log.Fields{
|
||||
"codec": "H265",
|
||||
"component": "capture",
|
||||
"dropped_access_units": recoveredDropped,
|
||||
"event": "video_reference_chain_recovered",
|
||||
"stream": streamType,
|
||||
}).Info("Video reference chain recovered at random-access frame")
|
||||
}
|
||||
|
||||
// add VPS, SPS and PPS before random access access unit
|
||||
if isRandomAccess {
|
||||
au = append([][]byte{
|
||||
|
||||
@@ -11,8 +11,6 @@ import (
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/bluenviron/mediacommon/pkg/codecs/h264"
|
||||
)
|
||||
|
||||
func TestPreRecordingGOPCount(t *testing.T) {
|
||||
@@ -136,114 +134,3 @@ func TestRTSPSTLSConfig(t *testing.T) {
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestVideoReferenceStateDropsDependentFramesUntilRandomAccess(t *testing.T) {
|
||||
state := &videoReferenceState{}
|
||||
|
||||
accepted, recovered := state.accept(false)
|
||||
if accepted || recovered != 0 || state.droppedAccessUnits != 1 {
|
||||
t.Fatalf("initial dependent frame = (accepted=%t, recovered=%d, dropped=%d), want (false, 0, 1)",
|
||||
accepted, recovered, state.droppedAccessUnits)
|
||||
}
|
||||
|
||||
accepted, recovered = state.accept(true)
|
||||
if !accepted || recovered != 1 || !state.valid {
|
||||
t.Fatalf("random-access recovery = (accepted=%t, recovered=%d, valid=%t), want (true, 1, true)",
|
||||
accepted, recovered, state.valid)
|
||||
}
|
||||
|
||||
if expected, gap := state.observeSequence(100); gap || expected != 100 {
|
||||
t.Fatalf("first sequence observation = (expected=%d, gap=%t), want (100, false)", expected, gap)
|
||||
}
|
||||
if expected, gap := state.observeSequence(102); !gap || expected != 101 || state.valid {
|
||||
t.Fatalf("sequence gap = (expected=%d, gap=%t, valid=%t), want (101, true, false)",
|
||||
expected, gap, state.valid)
|
||||
}
|
||||
|
||||
accepted, recovered = state.accept(false)
|
||||
if accepted || recovered != 0 || state.droppedAccessUnits != 1 {
|
||||
t.Fatalf("post-loss dependent frame = (accepted=%t, recovered=%d, dropped=%d), want (false, 0, 1)",
|
||||
accepted, recovered, state.droppedAccessUnits)
|
||||
}
|
||||
|
||||
accepted, recovered = state.accept(true)
|
||||
if !accepted || recovered != 1 || !state.valid {
|
||||
t.Fatalf("post-loss recovery = (accepted=%t, recovered=%d, valid=%t), want (true, 1, true)",
|
||||
accepted, recovered, state.valid)
|
||||
}
|
||||
}
|
||||
|
||||
func TestVideoReferenceStateAcceptsSequenceWraparound(t *testing.T) {
|
||||
state := &videoReferenceState{valid: true}
|
||||
|
||||
if _, gap := state.observeSequence(^uint16(0)); gap {
|
||||
t.Fatal("first sequence observation reported a gap")
|
||||
}
|
||||
if expected, gap := state.observeSequence(0); gap || expected != 0 || !state.valid {
|
||||
t.Fatalf("sequence wraparound = (expected=%d, gap=%t, valid=%t), want (0, false, true)",
|
||||
expected, gap, state.valid)
|
||||
}
|
||||
}
|
||||
|
||||
func TestH264FrameNumTrackerDetectsMissingIDR(t *testing.T) {
|
||||
tracker := &h264FrameNumTracker{}
|
||||
tracker.setSPS(&h264.SPS{
|
||||
Log2MaxFrameNumMinus4: 1,
|
||||
FrameMbsOnlyFlag: true,
|
||||
})
|
||||
|
||||
if _, _, gap, ok := tracker.observe([][]byte{testH264Slice(h264.NALUTypeIDR, 0)}); !ok || gap {
|
||||
t.Fatalf("initial IDR observation = (ok=%t, gap=%t), want (true, false)", ok, gap)
|
||||
}
|
||||
for frameNum := uint32(1); frameNum <= 24; frameNum++ {
|
||||
if expected, received, gap, ok := tracker.observe([][]byte{testH264Slice(h264.NALUTypeNonIDR, frameNum)}); !ok || gap {
|
||||
t.Fatalf("frame_num %d observation = (expected=%d, received=%d, ok=%t, gap=%t), want no gap",
|
||||
frameNum, expected, received, ok, gap)
|
||||
}
|
||||
}
|
||||
|
||||
expected, received, gap, ok := tracker.observe([][]byte{testH264Slice(h264.NALUTypeNonIDR, 1)})
|
||||
if !ok || !gap || expected != 25 || received != 1 {
|
||||
t.Fatalf("missing IDR transition = (expected=%d, received=%d, ok=%t, gap=%t), want (25, 1, true, true)",
|
||||
expected, received, ok, gap)
|
||||
}
|
||||
}
|
||||
|
||||
func TestH264FrameNumTrackerResetsAtIDR(t *testing.T) {
|
||||
tracker := &h264FrameNumTracker{}
|
||||
tracker.setSPS(&h264.SPS{
|
||||
Log2MaxFrameNumMinus4: 1,
|
||||
FrameMbsOnlyFlag: true,
|
||||
})
|
||||
|
||||
tracker.observe([][]byte{testH264Slice(h264.NALUTypeIDR, 0)})
|
||||
tracker.observe([][]byte{testH264Slice(h264.NALUTypeNonIDR, 1)})
|
||||
if expected, received, gap, ok := tracker.observe([][]byte{testH264Slice(h264.NALUTypeIDR, 0)}); !ok || gap {
|
||||
t.Fatalf("IDR reset = (expected=%d, received=%d, ok=%t, gap=%t), want no gap",
|
||||
expected, received, ok, gap)
|
||||
}
|
||||
if expected, received, gap, ok := tracker.observe([][]byte{testH264Slice(h264.NALUTypeNonIDR, 1)}); !ok || gap || expected != 1 || received != 1 {
|
||||
t.Fatalf("post-IDR frame = (expected=%d, received=%d, ok=%t, gap=%t), want (1, 1, true, false)",
|
||||
expected, received, ok, gap)
|
||||
}
|
||||
}
|
||||
|
||||
func TestH264FrameNumTrackerHonorsPermittedGaps(t *testing.T) {
|
||||
tracker := &h264FrameNumTracker{}
|
||||
tracker.setSPS(&h264.SPS{
|
||||
Log2MaxFrameNumMinus4: 1,
|
||||
FrameMbsOnlyFlag: true,
|
||||
GapsInFrameNumValueAllowedFlag: true,
|
||||
})
|
||||
|
||||
tracker.observe([][]byte{testH264Slice(h264.NALUTypeIDR, 0)})
|
||||
expected, received, gap, ok := tracker.observe([][]byte{testH264Slice(h264.NALUTypeNonIDR, 2)})
|
||||
if !ok || gap || expected != 1 || received != 2 {
|
||||
t.Fatalf("permitted frame_num gap = (expected=%d, received=%d, ok=%t, gap=%t), want (1, 2, true, false)",
|
||||
expected, received, ok, gap)
|
||||
}
|
||||
}
|
||||
|
||||
func testH264Slice(naluType h264.NALUType, frameNum uint32) []byte {
|
||||
return []byte{0x60 | byte(naluType), 0xE0 | byte(frameNum&0x1F)}
|
||||
}
|
||||
|
||||
+161
-43
@@ -53,16 +53,13 @@ func publishRecordingState(mqttClient mqtt.Client, hubKey string, configuration
|
||||
}
|
||||
}
|
||||
|
||||
func recordingUploadMetadata(name string, config models.Config, timestamp int64, mp4Video *video.MP4, encrypted bool, coordinates string, changes int) models.RecordingUploadMetadata {
|
||||
func recordingUploadMetadata(name, deviceKey string, timestamp int64, mp4Video *video.MP4, encrypted bool) models.RecordingUploadMetadata {
|
||||
metadata := models.RecordingUploadMetadata{
|
||||
FileName: filepath.Base(name),
|
||||
DeviceKey: config.Key,
|
||||
Timestamp: timestamp,
|
||||
Duration: mp4Video.VideoTotalDuration,
|
||||
Encrypted: encrypted,
|
||||
DeviceName: config.Name,
|
||||
RegionCoordinates: coordinates,
|
||||
NumberOfChanges: strconv.Itoa(changes),
|
||||
FileName: filepath.Base(name),
|
||||
DeviceKey: deviceKey,
|
||||
Timestamp: timestamp,
|
||||
Duration: mp4Video.VideoTotalDuration,
|
||||
Encrypted: encrypted,
|
||||
}
|
||||
value := mp4Video.AverageFPS()
|
||||
if value > 0 && value <= 240 && !math.IsInf(value, 0) && !math.IsNaN(value) {
|
||||
@@ -193,7 +190,7 @@ func CleanupRecordingDirectory(configDirectory string, configuration *models.Con
|
||||
return
|
||||
}
|
||||
|
||||
if err := utils.RemoveRecording(recordingsDirectory, name); err != nil {
|
||||
if err := os.Remove(recordingsDirectory + "/" + name); err != nil {
|
||||
log.Info("HandleRecordStream: something went wrong, " + err.Error())
|
||||
return
|
||||
}
|
||||
@@ -278,7 +275,7 @@ func defaultReserveMB(totalMB int64) int64 {
|
||||
// overall with pending=true, signalling the caller that it is about to drop an
|
||||
// un-uploaded recording as a last resort.
|
||||
func pickRecordingToCleanup(recordingsDirectory, cloudDirectory string) (string, bool, error) {
|
||||
entries, err := utils.ReadRecordingDirectory(recordingsDirectory)
|
||||
entries, err := os.ReadDir(recordingsDirectory)
|
||||
if err != nil {
|
||||
return "", false, err
|
||||
}
|
||||
@@ -287,7 +284,14 @@ func pickRecordingToCleanup(recordingsDirectory, cloudDirectory string) (string,
|
||||
var oldestSafeTime, oldestAnyTime time.Time
|
||||
|
||||
for _, entry := range entries {
|
||||
modTime := entry.ModTime()
|
||||
if entry.IsDir() {
|
||||
continue
|
||||
}
|
||||
info, err := entry.Info()
|
||||
if err != nil || !info.Mode().IsRegular() {
|
||||
continue
|
||||
}
|
||||
modTime := info.ModTime()
|
||||
|
||||
if oldestAnyName == "" || modTime.Before(oldestAnyTime) {
|
||||
oldestAnyName = entry.Name()
|
||||
@@ -445,6 +449,39 @@ func HandleRecordStream(queue *packets.Queue, configDirectory string, configurat
|
||||
// Cleanup muxer
|
||||
start = false
|
||||
|
||||
// Update the name of the recording with the duration.
|
||||
// We will update the name of the recording with the duration in milliseconds.
|
||||
if mp4Video.VideoTotalDuration > 0 {
|
||||
duration := mp4Video.VideoTotalDuration
|
||||
// Update the name with the duration in milliseconds.
|
||||
startRecordingSeconds := startRecording / 1000 // convert to seconds
|
||||
startRecordingMilliseconds := startRecording % 1000 // convert to milliseconds
|
||||
s := strconv.FormatInt(startRecordingSeconds, 10) + "_" +
|
||||
strconv.Itoa(len(strconv.FormatInt(startRecordingMilliseconds, 10))) + "-" +
|
||||
strconv.FormatInt(startRecordingMilliseconds, 10) + "_" +
|
||||
config.Name + "_" +
|
||||
"0-0-0-0" + "_" + // region coordinates, we
|
||||
"-1" + "_" + // token
|
||||
strconv.FormatInt(int64(duration), 10) // + "_" + // duration of recording
|
||||
//utils.VERSION // version of the agent
|
||||
|
||||
oldName := name
|
||||
name = s + ".mp4"
|
||||
fullName = configDirectory + "/data/recordings/" + name
|
||||
log.Info("capture.main.HandleRecordStream(motiondetection): renamed file from: " + oldName + " to: " + name)
|
||||
|
||||
// Rename the file to the new name.
|
||||
err := os.Rename(
|
||||
configDirectory+"/data/recordings/"+oldName,
|
||||
configDirectory+"/data/recordings/"+s+".mp4")
|
||||
|
||||
if err != nil {
|
||||
log.Error("capture.main.HandleRecordStream(motiondetection): error renaming file: " + err.Error())
|
||||
}
|
||||
} else {
|
||||
log.Info("capture.main.HandleRecordStream(continuous): no video data recorded, not renaming file.")
|
||||
}
|
||||
|
||||
encrypted := false
|
||||
// Check if we need to encrypt the recording.
|
||||
if config.Encryption != nil && config.Encryption.Enabled == "true" && config.Encryption.Recordings == "true" && config.Encryption.SymmetricKey != "" {
|
||||
@@ -469,7 +506,7 @@ func HandleRecordStream(queue *packets.Queue, configDirectory string, configurat
|
||||
}
|
||||
}
|
||||
|
||||
queueRecordingForUpload(configDirectory, recordingUploadMetadata(name, config, startRecording, mp4Video, encrypted, "0-0-0-0", -1))
|
||||
queueRecordingForUpload(configDirectory, recordingUploadMetadata(name, config.Key, startRecording, mp4Video, encrypted))
|
||||
|
||||
recordingStatus = "idle"
|
||||
|
||||
@@ -494,17 +531,29 @@ func HandleRecordStream(queue *packets.Queue, configDirectory string, configurat
|
||||
|
||||
start = true
|
||||
|
||||
// timestamp_microseconds_instanceName_regionCoordinates_numberOfChanges_token
|
||||
// 1564859471_6-474162_oprit_577-283-727-375_1153_27.mp4
|
||||
// - Timestamp
|
||||
// - Size + - + microseconds
|
||||
// - device
|
||||
// - Region
|
||||
// - Number of changes
|
||||
// - Token
|
||||
|
||||
startRecording = pkt.CurrentTime
|
||||
name, err = utils.RecordingFileName(startRecording, config.Key)
|
||||
if err == nil {
|
||||
fullName, err = utils.CreateRecordingPath(configDirectory+"/data/recordings", name, time.UnixMilli(startRecording))
|
||||
}
|
||||
if err != nil {
|
||||
log.WithError(err).Error("capture: could not create continuous recording path")
|
||||
start = false
|
||||
pkt = nextPkt
|
||||
continue
|
||||
}
|
||||
startRecordingSeconds := startRecording / 1000 // convert to seconds
|
||||
startRecordingMilliseconds := startRecording % 1000 // convert to milliseconds
|
||||
s := strconv.FormatInt(startRecordingSeconds, 10) + "_" + // start timestamp in seconds
|
||||
strconv.Itoa(len(strconv.FormatInt(startRecordingMilliseconds, 10))) + "-" + // length of milliseconds
|
||||
strconv.FormatInt(startRecordingMilliseconds, 10) + "_" + // milliseconds
|
||||
config.Name + "_" + // device name
|
||||
"0-0-0-0" + "_" + // region coordinates, we will not use this for continuous recording
|
||||
"0" + "_" + // token
|
||||
"0" + "_" //+ // duration of recording in milliseconds
|
||||
//utils.VERSION // version of the agent
|
||||
|
||||
name = s + ".mp4"
|
||||
fullName = configDirectory + "/data/recordings/" + name
|
||||
|
||||
// Running...
|
||||
log.Info("capture.main.HandleRecordStream(continuous): recording started")
|
||||
@@ -560,6 +609,39 @@ func HandleRecordStream(queue *packets.Queue, configDirectory string, configurat
|
||||
// Cleanup muxer
|
||||
start = false
|
||||
|
||||
// Update the name of the recording with the duration.
|
||||
// We will update the name of the recording with the duration in milliseconds.
|
||||
if mp4Video.VideoTotalDuration > 0 {
|
||||
duration := mp4Video.VideoTotalDuration
|
||||
// Update the name with the duration in milliseconds.
|
||||
startRecordingSeconds := startRecording / 1000 // convert to seconds
|
||||
startRecordingMilliseconds := startRecording % 1000 // convert to milliseconds
|
||||
s := strconv.FormatInt(startRecordingSeconds, 10) + "_" +
|
||||
strconv.Itoa(len(strconv.FormatInt(startRecordingMilliseconds, 10))) + "-" +
|
||||
strconv.FormatInt(startRecordingMilliseconds, 10) + "_" +
|
||||
config.Name + "_" +
|
||||
"0-0-0-0" + "_" + // region coordinates, we
|
||||
"-1" + "_" + // token
|
||||
strconv.FormatInt(int64(duration), 10) // + "_" + // duration of recording
|
||||
//utils.VERSION // version of the agent
|
||||
|
||||
oldName := name
|
||||
name = s + ".mp4"
|
||||
fullName = configDirectory + "/data/recordings/" + name
|
||||
log.Info("capture.main.HandleRecordStream(motiondetection): renamed file from: " + oldName + " to: " + name)
|
||||
|
||||
// Rename the file to the new name.
|
||||
err := os.Rename(
|
||||
configDirectory+"/data/recordings/"+oldName,
|
||||
configDirectory+"/data/recordings/"+s+".mp4")
|
||||
|
||||
if err != nil {
|
||||
log.Error("capture.main.HandleRecordStream(motiondetection): error renaming file: " + err.Error())
|
||||
}
|
||||
} else {
|
||||
log.Info("capture.main.HandleRecordStream(continuous): no video data recorded, not renaming file.")
|
||||
}
|
||||
|
||||
encrypted := false
|
||||
// Check if we need to encrypt the recording.
|
||||
if config.Encryption != nil && config.Encryption.Enabled == "true" && config.Encryption.Recordings == "true" && config.Encryption.SymmetricKey != "" {
|
||||
@@ -584,7 +666,7 @@ func HandleRecordStream(queue *packets.Queue, configDirectory string, configurat
|
||||
}
|
||||
}
|
||||
|
||||
queueRecordingForUpload(configDirectory, recordingUploadMetadata(name, config, startRecording, mp4Video, encrypted, "0-0-0-0", -1))
|
||||
queueRecordingForUpload(configDirectory, recordingUploadMetadata(name, config.Key, startRecording, mp4Video, encrypted))
|
||||
|
||||
recordingStatus = "idle"
|
||||
|
||||
@@ -639,7 +721,17 @@ func HandleRecordStream(queue *packets.Queue, configDirectory string, configurat
|
||||
continue
|
||||
}
|
||||
|
||||
displayTimeSeconds := displayTime / 1000
|
||||
// timestamp_microseconds_instanceName_regionCoordinates_numberOfChanges_token
|
||||
// 1564859471_6-474162_oprit_577-283-727-375_1153_27.mp4
|
||||
// - Timestamp
|
||||
// - Size + - + microseconds
|
||||
// - device
|
||||
// - Region
|
||||
// - Number of changes
|
||||
// - Token
|
||||
|
||||
displayTimeSeconds := displayTime / 1000 // convert to seconds
|
||||
displayTimeMilliseconds := displayTime % 1000 // convert to milliseconds
|
||||
motionRectangleString := "0-0-0-0"
|
||||
if motion.Rectangle.X != 0 || motion.Rectangle.Y != 0 ||
|
||||
motion.Rectangle.Width != 0 || motion.Rectangle.Height != 0 {
|
||||
@@ -650,15 +742,17 @@ func HandleRecordStream(queue *packets.Queue, configDirectory string, configurat
|
||||
// Get the number of changes from the motion detection.
|
||||
numberOfChanges := motion.NumberOfChanges
|
||||
|
||||
name, err := utils.RecordingFileName(displayTime, config.Key)
|
||||
var fullName string
|
||||
if err == nil {
|
||||
fullName, err = utils.CreateRecordingPath(configDirectory+"/data/recordings", name, time.UnixMilli(displayTime))
|
||||
}
|
||||
if err != nil {
|
||||
log.WithError(err).Error("capture: could not create motion recording path")
|
||||
continue
|
||||
}
|
||||
s := strconv.FormatInt(displayTimeSeconds, 10) + "_" + // start timestamp in seconds
|
||||
strconv.Itoa(len(strconv.FormatInt(displayTimeMilliseconds, 10))) + "-" + // length of milliseconds
|
||||
strconv.FormatInt(displayTimeMilliseconds, 10) + "_" + // milliseconds
|
||||
config.Name + "_" + // device name
|
||||
motionRectangleString + "_" + // region coordinates, we will not use this for continuous recording
|
||||
strconv.Itoa(numberOfChanges) + "_" + // number of changes
|
||||
"0" // + "_" + // duration of recording in milliseconds
|
||||
//utils.VERSION // version of the agent
|
||||
|
||||
name := s + ".mp4"
|
||||
fullName := configDirectory + "/data/recordings/" + name
|
||||
|
||||
// Running...
|
||||
log.Info("capture.main.HandleRecordStream(motiondetection): recording started (" + name + ")" + " at " + strconv.FormatInt(displayTimeSeconds, 10) + " unix")
|
||||
@@ -786,6 +880,38 @@ func HandleRecordStream(queue *packets.Queue, configDirectory string, configurat
|
||||
communication.TrySendMotion(models.MotionDataPartial{Timestamp: time.Now().Unix(), NumberOfChanges: 100000000})
|
||||
}
|
||||
|
||||
// Update the name of the recording with the duration.
|
||||
// We will update the name of the recording with the duration in milliseconds.
|
||||
if mp4Video.VideoTotalDuration > 0 {
|
||||
duration := mp4Video.VideoTotalDuration
|
||||
|
||||
// Update the name with the duration in milliseconds.
|
||||
s := strconv.FormatInt(displayTimeSeconds, 10) + "_" +
|
||||
strconv.Itoa(len(strconv.FormatInt(displayTimeMilliseconds, 10))) + "-" +
|
||||
strconv.FormatInt(displayTimeMilliseconds, 10) + "_" +
|
||||
config.Name + "_" +
|
||||
motionRectangleString + "_" +
|
||||
strconv.Itoa(numberOfChanges) + "_" + // number of changes
|
||||
strconv.FormatInt(int64(duration), 10) // + "_" + // duration of recording in milliseconds
|
||||
//utils.VERSION // version of the agent
|
||||
|
||||
oldName := name
|
||||
name = s + ".mp4"
|
||||
fullName = configDirectory + "/data/recordings/" + name
|
||||
log.Info("capture.main.HandleRecordStream(motiondetection): renamed file from: " + oldName + " to: " + name)
|
||||
|
||||
// Rename the file to the new name.
|
||||
err := os.Rename(
|
||||
configDirectory+"/data/recordings/"+oldName,
|
||||
configDirectory+"/data/recordings/"+s+".mp4")
|
||||
|
||||
if err != nil {
|
||||
log.Error("capture.main.HandleRecordStream(motiondetection): error renaming file: " + err.Error())
|
||||
}
|
||||
} else {
|
||||
log.Info("capture.main.HandleRecordStream(motiondetection): no video data recorded, not renaming file.")
|
||||
}
|
||||
|
||||
encrypted := false
|
||||
// Check if we need to encrypt the recording.
|
||||
if config.Encryption != nil && config.Encryption.Enabled == "true" && config.Encryption.Recordings == "true" && config.Encryption.SymmetricKey != "" {
|
||||
@@ -810,7 +936,7 @@ func HandleRecordStream(queue *packets.Queue, configDirectory string, configurat
|
||||
}
|
||||
}
|
||||
|
||||
queueRecordingForUpload(configDirectory, recordingUploadMetadata(name, config, displayTime, mp4Video, encrypted, motionRectangleString, numberOfChanges))
|
||||
queueRecordingForUpload(configDirectory, recordingUploadMetadata(name, config.Key, displayTime, mp4Video, encrypted))
|
||||
|
||||
// Clean up the recording directory if necessary.
|
||||
CleanupRecordingDirectory(configDirectory, configuration)
|
||||
@@ -944,10 +1070,6 @@ func Base64Image(captureDevice *Capture, communication *models.Communication, co
|
||||
count++
|
||||
continue
|
||||
}
|
||||
} else {
|
||||
// A stopped run closes this queue permanently. Retrying its
|
||||
// cursor would spin on EOF even after the HTTP client leaves.
|
||||
break
|
||||
}
|
||||
} else {
|
||||
break
|
||||
@@ -991,10 +1113,6 @@ func JpegImage(captureDevice *Capture, communication *models.Communication) imag
|
||||
} else {
|
||||
break
|
||||
}
|
||||
} else {
|
||||
// A stopped run closes this queue permanently. Retrying its
|
||||
// cursor would spin on EOF even after the HTTP client leaves.
|
||||
break
|
||||
}
|
||||
} else {
|
||||
break
|
||||
|
||||
@@ -42,11 +42,10 @@ func TestQueueRecordingForUploadStoresFinalizedMetadata(t *testing.T) {
|
||||
}
|
||||
|
||||
mp4Video := &video.MP4{VideoTotalDuration: 20452, SampleCount: 613}
|
||||
config := models.Config{Key: "device-key", Name: "camera-name"}
|
||||
metadata := recordingUploadMetadata("1785934709414-device-key.mp4", config, 1785934709414, mp4Video, true, "1-2-3-4", 57)
|
||||
metadata := recordingUploadMetadata("recording.mp4", "device-key", 1785934709414, mp4Video, true)
|
||||
queueRecordingForUpload(configDirectory, metadata)
|
||||
|
||||
got, err := os.ReadFile(filepath.Join(configDirectory, "data", "cloud", "1785934709414-device-key.metadata"))
|
||||
got, err := os.ReadFile(filepath.Join(configDirectory, "data", "cloud", "recording.metadata"))
|
||||
if err != nil {
|
||||
t.Fatalf("read upload marker: %v", err)
|
||||
}
|
||||
@@ -55,8 +54,7 @@ func TestQueueRecordingForUploadStoresFinalizedMetadata(t *testing.T) {
|
||||
t.Fatalf("decode upload marker: %v", err)
|
||||
}
|
||||
expectedFPS := mp4Video.AverageFPS()
|
||||
if stored.FileName != "1785934709414-device-key.mp4" || stored.DeviceKey != "device-key" || stored.Timestamp != 1785934709414 || stored.Duration != 20452 || math.Abs(stored.FPS-expectedFPS) > 1e-9 || !stored.Encrypted ||
|
||||
stored.DeviceName != "camera-name" || stored.RegionCoordinates != "1-2-3-4" || stored.NumberOfChanges != "57" {
|
||||
if stored.FileName != "recording.mp4" || stored.DeviceKey != "device-key" || stored.Timestamp != 1785934709414 || stored.Duration != 20452 || math.Abs(stored.FPS-expectedFPS) > 1e-9 || !stored.Encrypted {
|
||||
t.Fatalf("upload marker = %+v", stored)
|
||||
}
|
||||
if stored.FPS == math.Floor(stored.FPS) {
|
||||
|
||||
@@ -1,68 +0,0 @@
|
||||
package capture
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/kerberos-io/agent/machinery/src/models"
|
||||
"github.com/kerberos-io/agent/machinery/src/packets"
|
||||
)
|
||||
|
||||
func TestSnapshotReturnsAfterQueueClosure(t *testing.T) {
|
||||
for _, stream := range []string{"main", "sub"} {
|
||||
for _, format := range []string{"jpeg", "base64"} {
|
||||
for _, alreadyClosed := range []bool{true, false} {
|
||||
name := stream + "/" + format + "/closed_during_read"
|
||||
if alreadyClosed {
|
||||
name = stream + "/" + format + "/already_closed"
|
||||
}
|
||||
t.Run(name, func(t *testing.T) {
|
||||
communication := &models.Communication{}
|
||||
run := models.NewAgentRun(context.Background(), communication, false)
|
||||
queue := packets.NewQueue()
|
||||
captureDevice := &Capture{}
|
||||
if stream == "main" {
|
||||
captureDevice.SetMainClient("rtsp://unused.invalid/main")
|
||||
run.SetMainQueue(queue)
|
||||
} else {
|
||||
captureDevice.SetSubClient("rtsp://unused.invalid/sub")
|
||||
run.SetSubQueue(queue)
|
||||
}
|
||||
if err := run.Activate(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer queue.Close()
|
||||
if alreadyClosed {
|
||||
queue.Close()
|
||||
}
|
||||
done := make(chan bool, 1)
|
||||
go func() {
|
||||
if format == "jpeg" {
|
||||
image := JpegImage(captureDevice, communication)
|
||||
done <- image.Rect.Empty()
|
||||
} else {
|
||||
done <- Base64Image(captureDevice, communication, &models.Configuration{}) == ""
|
||||
}
|
||||
}()
|
||||
if !alreadyClosed {
|
||||
select {
|
||||
case <-done:
|
||||
t.Fatal("snapshot returned before a frame or queue closure")
|
||||
case <-time.After(10 * time.Millisecond):
|
||||
}
|
||||
queue.Close()
|
||||
}
|
||||
select {
|
||||
case empty := <-done:
|
||||
if !empty {
|
||||
t.Fatal("closed empty queue produced a snapshot")
|
||||
}
|
||||
case <-time.After(time.Second):
|
||||
t.Fatal("snapshot did not return after its packet queue closed")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -125,7 +125,7 @@ func HandleUpload(configDirectory string, configuration *models.Configuration, c
|
||||
// Check if we need to remove the original recording
|
||||
// removeAfterUpload is set to false by default
|
||||
if config.RemoveAfterUpload != "false" {
|
||||
err := utils.RemoveRecording(configDirectory+"/data/recordings", fileName)
|
||||
err := os.Remove(configDirectory + "/data/recordings/" + fileName)
|
||||
if err != nil {
|
||||
log.Error("HandleUpload: " + err.Error())
|
||||
}
|
||||
@@ -541,23 +541,13 @@ func (state *heartbeatONVIFState) fetchEvents(device *goonvif.Device) ([]byte, b
|
||||
}
|
||||
}
|
||||
|
||||
// Fall back to querying the relay outputs and digital inputs directly when
|
||||
// no subscription could be created, and also when the subscriptions
|
||||
// produced no I/O events: some devices accept the subscription but reject
|
||||
// the pulls, or never publish the initial I/O state.
|
||||
noSubscription := !initialStateSubscriptionCreated && state.loopPullPoint == ""
|
||||
if noSubscription || heartbeatEventsListEmpty(onvifEventsList) {
|
||||
if !initialStateSubscriptionCreated && state.loopPullPoint == "" {
|
||||
return state.fallbackEvents(device), operationFailed
|
||||
}
|
||||
|
||||
return onvifEventsList, operationFailed
|
||||
}
|
||||
|
||||
func heartbeatEventsListEmpty(eventsList []byte) bool {
|
||||
trimmed := strings.TrimSpace(string(eventsList))
|
||||
return trimmed == "" || trimmed == "[]" || trimmed == "null"
|
||||
}
|
||||
|
||||
func getHeartbeatONVIFPayload(camera models.IPCamera, state *heartbeatONVIFState) heartbeatONVIFPayload {
|
||||
if state == nil {
|
||||
state = newHeartbeatONVIFState()
|
||||
@@ -1454,19 +1444,13 @@ func HandleRealtimeProcessing(processingCursor *packets.QueueCursor, configurati
|
||||
// @Summary Will verify the hub connectivity.
|
||||
// @Description Will verify the hub connectivity.
|
||||
// @Success 200 {object} models.APIResponse
|
||||
func VerifyHub(c *gin.Context, configuredHubURI string) {
|
||||
func VerifyHub(c *gin.Context) {
|
||||
|
||||
var config models.Config
|
||||
err := c.BindJSON(&config)
|
||||
|
||||
if err == nil {
|
||||
hubURI, err := configuredHubVerificationURI(config.HubURI, configuredHubURI)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusBadRequest, models.APIResponse{
|
||||
Data: "cloud.VerifyHub(): " + err.Error(),
|
||||
})
|
||||
return
|
||||
}
|
||||
hubURI := config.HubURI
|
||||
publicKey := config.HubKey
|
||||
privateKey := config.HubPrivateKey
|
||||
|
||||
@@ -1479,9 +1463,9 @@ func VerifyHub(c *gin.Context, configuredHubURI string) {
|
||||
tr := &http.Transport{
|
||||
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
|
||||
}
|
||||
client = &http.Client{Transport: tr, CheckRedirect: rejectHubVerificationRedirect}
|
||||
client = &http.Client{Transport: tr}
|
||||
} else {
|
||||
client = &http.Client{CheckRedirect: rejectHubVerificationRedirect}
|
||||
client = &http.Client{}
|
||||
}
|
||||
|
||||
resp, err := client.Do(req)
|
||||
@@ -1530,7 +1514,7 @@ func VerifyHub(c *gin.Context, configuredHubURI string) {
|
||||
// @Summary Will verify the persistence.
|
||||
// @Description Will verify the persistence.
|
||||
// @Success 200 {object} models.APIResponse
|
||||
func VerifyPersistence(c *gin.Context, configDirectory, configuredHubURI string) {
|
||||
func VerifyPersistence(c *gin.Context, configDirectory string) {
|
||||
|
||||
var config models.Config
|
||||
err := c.BindJSON(&config)
|
||||
@@ -1543,7 +1527,6 @@ func VerifyPersistence(c *gin.Context, configDirectory, configuredHubURI string)
|
||||
if config.HubURI == "" ||
|
||||
config.HubKey == "" ||
|
||||
config.HubPrivateKey == "" ||
|
||||
config.S3 == nil ||
|
||||
config.S3.Region == "" {
|
||||
msg := "cloud.VerifyPersistence(kerberoshub): Kerberos Hub not properly configured."
|
||||
log.Error(msg)
|
||||
@@ -1551,24 +1534,25 @@ func VerifyPersistence(c *gin.Context, configDirectory, configuredHubURI string)
|
||||
Data: msg,
|
||||
})
|
||||
} else {
|
||||
hubURI, err := configuredHubVerificationURI(config.HubURI, configuredHubURI)
|
||||
if err != nil {
|
||||
msg := "cloud.VerifyPersistence(kerberoshub): " + err.Error()
|
||||
log.Error(msg)
|
||||
c.JSON(http.StatusBadRequest, models.APIResponse{
|
||||
Data: msg,
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
req, err := http.NewRequest(http.MethodHead, hubURI+"/storage/upload", nil)
|
||||
// Open test-480p.mp4
|
||||
file, err := os.Open(configDirectory + "/data/test-480p.mp4")
|
||||
if err != nil {
|
||||
msg := "cloud.VerifyPersistence(kerberoshub): error creating Kerberos Hub upload authorization request: " + err.Error()
|
||||
msg := "cloud.VerifyPersistence(kerberoshub): error reading test-480p.mp4: " + err.Error()
|
||||
log.Error(msg)
|
||||
c.JSON(400, models.APIResponse{
|
||||
Data: msg,
|
||||
})
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
req, err := http.NewRequest("POST", config.HubURI+"/storage/upload", file)
|
||||
if err != nil {
|
||||
msg := "cloud.VerifyPersistence(kerberoshub): error reading Kerberos Hub HEAD request, " + config.HubURI + "/storage: " + err.Error()
|
||||
log.Error(msg)
|
||||
c.JSON(400, models.APIResponse{
|
||||
Data: msg,
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
timestamp := time.Now().Unix()
|
||||
@@ -1586,9 +1570,9 @@ func VerifyPersistence(c *gin.Context, configDirectory, configuredHubURI string)
|
||||
tr := &http.Transport{
|
||||
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
|
||||
}
|
||||
client = &http.Client{Transport: tr, CheckRedirect: rejectHubVerificationRedirect}
|
||||
client = &http.Client{Transport: tr}
|
||||
} else {
|
||||
client = &http.Client{CheckRedirect: rejectHubVerificationRedirect}
|
||||
client = &http.Client{}
|
||||
}
|
||||
|
||||
resp, err := client.Do(req)
|
||||
@@ -1604,17 +1588,14 @@ func VerifyPersistence(c *gin.Context, configDirectory, configuredHubURI string)
|
||||
Data: msg,
|
||||
})
|
||||
} else {
|
||||
msg := hubPersistenceVerificationError(resp.StatusCode)
|
||||
msg := "cloud.VerifyPersistence(kerberoshub): upload not allowed using the configured credentials"
|
||||
log.Error(msg)
|
||||
c.JSON(400, models.APIResponse{
|
||||
Data: msg,
|
||||
})
|
||||
}
|
||||
} else {
|
||||
msg := "cloud.VerifyPersistence(kerberoshub): Kerberos Hub upload authorization request failed"
|
||||
if err != nil {
|
||||
msg += ": " + err.Error()
|
||||
}
|
||||
msg := "cloud.VerifyPersistence(kerberoshub): Error creating Kerberos Hub request"
|
||||
log.Error(msg)
|
||||
c.JSON(400, models.APIResponse{
|
||||
Data: msg,
|
||||
@@ -1783,35 +1764,6 @@ func VerifyPersistence(c *gin.Context, configDirectory, configuredHubURI string)
|
||||
}
|
||||
}
|
||||
|
||||
func configuredHubVerificationURI(requested, configured string) (string, error) {
|
||||
requested = strings.TrimRight(strings.TrimSpace(requested), "/")
|
||||
configured = strings.TrimRight(strings.TrimSpace(configured), "/")
|
||||
if configured == "" {
|
||||
return "", errors.New("save the Hub URI before verifying it")
|
||||
}
|
||||
if requested != configured {
|
||||
return "", errors.New("the Hub URI does not match the saved configuration; save it before verifying")
|
||||
}
|
||||
return configured, nil
|
||||
}
|
||||
|
||||
func rejectHubVerificationRedirect(_ *http.Request, _ []*http.Request) error {
|
||||
return http.ErrUseLastResponse
|
||||
}
|
||||
|
||||
func hubPersistenceVerificationError(statusCode int) string {
|
||||
reason := "Kerberos Hub rejected upload authorization"
|
||||
switch statusCode {
|
||||
case http.StatusBadRequest:
|
||||
reason += ": the credentials or device/project assignment are invalid, or the storage limit was reached"
|
||||
case http.StatusUnauthorized:
|
||||
reason += ": the account has no active subscription or the subscription expired"
|
||||
case http.StatusForbidden:
|
||||
reason += ": the device is muted"
|
||||
}
|
||||
return "cloud.VerifyPersistence(kerberoshub): " + reason + " (HTTP " + strconv.Itoa(statusCode) + ")"
|
||||
}
|
||||
|
||||
// VerifySecondaryPersistence godoc
|
||||
// @Router /api/persistence/secondary/verify [post]
|
||||
// @ID verify-secondary-persistence
|
||||
|
||||
@@ -1,222 +1,22 @@
|
||||
package cloud
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/kerberos-io/agent/machinery/src/models"
|
||||
agentonvif "github.com/kerberos-io/agent/machinery/src/onvif"
|
||||
goonvif "github.com/kerberos-io/onvif"
|
||||
goonvifdevice "github.com/kerberos-io/onvif/device"
|
||||
goonvifptz "github.com/kerberos-io/onvif/ptz"
|
||||
goonvifxsd "github.com/kerberos-io/onvif/xsd/onvif"
|
||||
)
|
||||
|
||||
func TestVerifyPersistenceKerberosHubUsesAuthorizationHead(t *testing.T) {
|
||||
var requestMethod string
|
||||
var requestBody []byte
|
||||
var requestHeaders http.Header
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
requestMethod = r.Method
|
||||
requestHeaders = r.Header.Clone()
|
||||
requestBody, _ = io.ReadAll(r.Body)
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}))
|
||||
defer server.Close()
|
||||
|
||||
config := models.Config{
|
||||
Cloud: "kerberoshub",
|
||||
Key: "camera-1",
|
||||
Name: "front-door",
|
||||
HubURI: server.URL,
|
||||
HubKey: "project-public",
|
||||
HubPrivateKey: "project-private",
|
||||
S3: &models.S3{Region: "eu"},
|
||||
}
|
||||
payload, err := json.Marshal(config)
|
||||
if err != nil {
|
||||
t.Fatalf("json.Marshal() error = %v", err)
|
||||
}
|
||||
|
||||
gin.SetMode(gin.TestMode)
|
||||
recorder := httptest.NewRecorder()
|
||||
ctx, _ := gin.CreateTestContext(recorder)
|
||||
ctx.Request = httptest.NewRequest(http.MethodPost, "/api/persistence/verify", bytes.NewReader(payload))
|
||||
ctx.Request.Header.Set("Content-Type", "application/json")
|
||||
|
||||
VerifyPersistence(ctx, t.TempDir(), server.URL)
|
||||
|
||||
if recorder.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, body = %s", recorder.Code, recorder.Body.String())
|
||||
}
|
||||
if requestMethod != http.MethodHead {
|
||||
t.Fatalf("Hub request method = %q, want HEAD", requestMethod)
|
||||
}
|
||||
if len(requestBody) != 0 {
|
||||
t.Fatalf("Hub request body has %d bytes, want none", len(requestBody))
|
||||
}
|
||||
for header, want := range map[string]string{
|
||||
"X-Kerberos-Storage-Device": "camera-1",
|
||||
"X-Kerberos-Hub-PublicKey": "project-public",
|
||||
"X-Kerberos-Hub-PrivateKey": "project-private",
|
||||
"X-Kerberos-Hub-Region": "eu",
|
||||
"X-Kerberos-Storage-Capture": "IPCamera",
|
||||
} {
|
||||
if got := requestHeaders.Get(header); got != want {
|
||||
t.Errorf("%s = %q, want %q", header, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestVerifyPersistenceKerberosHubRejectsCrossHostRedirect(t *testing.T) {
|
||||
var redirectedHeaders http.Header
|
||||
target := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
redirectedHeaders = r.Header.Clone()
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}))
|
||||
defer target.Close()
|
||||
|
||||
var originalHeaders http.Header
|
||||
hub := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
originalHeaders = r.Header.Clone()
|
||||
http.Redirect(w, r, target.URL+"/storage/upload", http.StatusTemporaryRedirect)
|
||||
}))
|
||||
defer hub.Close()
|
||||
|
||||
config := models.Config{
|
||||
Cloud: "kerberoshub",
|
||||
Key: "camera-1",
|
||||
Name: "front-door",
|
||||
HubURI: hub.URL,
|
||||
HubKey: "project-public",
|
||||
HubPrivateKey: "project-private",
|
||||
S3: &models.S3{Region: "eu"},
|
||||
}
|
||||
payload, err := json.Marshal(config)
|
||||
if err != nil {
|
||||
t.Fatalf("json.Marshal() error = %v", err)
|
||||
}
|
||||
|
||||
gin.SetMode(gin.TestMode)
|
||||
recorder := httptest.NewRecorder()
|
||||
ctx, _ := gin.CreateTestContext(recorder)
|
||||
ctx.Request = httptest.NewRequest(http.MethodPost, "/api/persistence/verify", bytes.NewReader(payload))
|
||||
ctx.Request.Header.Set("Content-Type", "application/json")
|
||||
|
||||
VerifyPersistence(ctx, t.TempDir(), hub.URL)
|
||||
|
||||
if recorder.Code != http.StatusBadRequest {
|
||||
t.Fatalf("status = %d, want %d; body = %s", recorder.Code, http.StatusBadRequest, recorder.Body.String())
|
||||
}
|
||||
for _, header := range []string{"X-Kerberos-Hub-PublicKey", "X-Kerberos-Hub-PrivateKey"} {
|
||||
if originalHeaders.Get(header) == "" {
|
||||
t.Errorf("%s was not sent to the configured Hub", header)
|
||||
}
|
||||
if redirectedHeaders != nil {
|
||||
t.Errorf("redirect target received a request with %s=%q", header, redirectedHeaders.Get(header))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestVerifyPersistenceKerberosHubRejectsUnsavedURI(t *testing.T) {
|
||||
requestCount := 0
|
||||
untrusted := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||
requestCount++
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}))
|
||||
defer untrusted.Close()
|
||||
|
||||
config := models.Config{
|
||||
Cloud: "kerberoshub",
|
||||
Key: "camera-1",
|
||||
Name: "front-door",
|
||||
HubURI: untrusted.URL,
|
||||
HubKey: "project-public",
|
||||
HubPrivateKey: "project-private",
|
||||
S3: &models.S3{Region: "eu"},
|
||||
}
|
||||
payload, err := json.Marshal(config)
|
||||
if err != nil {
|
||||
t.Fatalf("json.Marshal() error = %v", err)
|
||||
}
|
||||
|
||||
gin.SetMode(gin.TestMode)
|
||||
recorder := httptest.NewRecorder()
|
||||
ctx, _ := gin.CreateTestContext(recorder)
|
||||
ctx.Request = httptest.NewRequest(http.MethodPost, "/api/persistence/verify", bytes.NewReader(payload))
|
||||
ctx.Request.Header.Set("Content-Type", "application/json")
|
||||
|
||||
VerifyPersistence(ctx, t.TempDir(), "https://saved-hub.example.com")
|
||||
|
||||
if recorder.Code != http.StatusBadRequest {
|
||||
t.Fatalf("status = %d, want %d; body = %s", recorder.Code, http.StatusBadRequest, recorder.Body.String())
|
||||
}
|
||||
if requestCount != 0 {
|
||||
t.Fatalf("untrusted Hub received %d requests, want none", requestCount)
|
||||
}
|
||||
}
|
||||
|
||||
func TestVerifyHubRejectsUnsavedURI(t *testing.T) {
|
||||
requestCount := 0
|
||||
untrusted := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||
requestCount++
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}))
|
||||
defer untrusted.Close()
|
||||
|
||||
config := models.Config{
|
||||
HubURI: untrusted.URL,
|
||||
HubKey: "project-public",
|
||||
HubPrivateKey: "project-private",
|
||||
}
|
||||
payload, err := json.Marshal(config)
|
||||
if err != nil {
|
||||
t.Fatalf("json.Marshal() error = %v", err)
|
||||
}
|
||||
|
||||
gin.SetMode(gin.TestMode)
|
||||
recorder := httptest.NewRecorder()
|
||||
ctx, _ := gin.CreateTestContext(recorder)
|
||||
ctx.Request = httptest.NewRequest(http.MethodPost, "/api/hub/verify", bytes.NewReader(payload))
|
||||
ctx.Request.Header.Set("Content-Type", "application/json")
|
||||
|
||||
VerifyHub(ctx, "https://saved-hub.example.com")
|
||||
|
||||
if recorder.Code != http.StatusBadRequest {
|
||||
t.Fatalf("status = %d, want %d; body = %s", recorder.Code, http.StatusBadRequest, recorder.Body.String())
|
||||
}
|
||||
if requestCount != 0 {
|
||||
t.Fatalf("untrusted Hub received %d requests, want none", requestCount)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHubPersistenceVerificationErrorExplainsStatus(t *testing.T) {
|
||||
for _, test := range []struct {
|
||||
status int
|
||||
want string
|
||||
}{
|
||||
{status: http.StatusBadRequest, want: "credentials or device/project assignment"},
|
||||
{status: http.StatusUnauthorized, want: "no active subscription"},
|
||||
{status: http.StatusForbidden, want: "device is muted"},
|
||||
{status: http.StatusBadGateway, want: "HTTP 502"},
|
||||
} {
|
||||
t.Run(http.StatusText(test.status), func(t *testing.T) {
|
||||
if got := hubPersistenceVerificationError(test.status); !strings.Contains(got, test.want) {
|
||||
t.Fatalf("hubPersistenceVerificationError(%d) = %q, want it to contain %q", test.status, got, test.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestHeartbeatFailureLogOmitsHubResponseBody(t *testing.T) {
|
||||
response := &http.Response{
|
||||
StatusCode: http.StatusBadRequest,
|
||||
@@ -790,60 +590,3 @@ func countString(values []string, want string) int {
|
||||
}
|
||||
return count
|
||||
}
|
||||
|
||||
// Some devices accept the subscription but reject the pulls, or never publish
|
||||
// the initial I/O state. The relay outputs and digital inputs must then be
|
||||
// queried directly instead of reporting no I/O at all.
|
||||
func TestHeartbeatONVIFPayloadFallsBackToDirectIOWhenSubscriptionsReturnNoEvents(t *testing.T) {
|
||||
restoreHeartbeatONVIFStubs(t)
|
||||
|
||||
device := newTestONVIFDevice()
|
||||
camera := models.IPCamera{
|
||||
ONVIFXAddr: "http://camera/onvif",
|
||||
ONVIFUsername: "operator",
|
||||
ONVIFPassword: "secret",
|
||||
}
|
||||
|
||||
state := newHeartbeatONVIFState()
|
||||
state.cameraConfiguration = camera
|
||||
state.cameraKey = heartbeatONVIFCameraKey(camera)
|
||||
state.device = device
|
||||
state.staticLoaded = true
|
||||
state.staticPayload.enabled = "true"
|
||||
|
||||
heartbeatCreatePullPointSubscription = func(*goonvif.Device) (string, error) {
|
||||
return "pull-point", nil
|
||||
}
|
||||
heartbeatGetEventMessages = func(*goonvif.Device, string) ([]agentonvif.ONVIFEvents, error) {
|
||||
return nil, errors.New("pull rejected")
|
||||
}
|
||||
heartbeatUnsubscribePullPoint = func(*goonvif.Device, string) error {
|
||||
return nil
|
||||
}
|
||||
var relayCalls, inputCalls int
|
||||
heartbeatGetRelayOutputs = func(*goonvif.Device) (goonvifdevice.GetRelayOutputsResponse, error) {
|
||||
relayCalls++
|
||||
return goonvifdevice.GetRelayOutputsResponse{RelayOutputs: []goonvifxsd.RelayOutput{{DeviceEntity: goonvifxsd.DeviceEntity{Token: "relay0"}}}}, nil
|
||||
}
|
||||
heartbeatGetDigitalInputs = func(*goonvif.Device) (goonvifdevice.GetDigitalInputsResponse, error) {
|
||||
inputCalls++
|
||||
return goonvifdevice.GetDigitalInputsResponse{DigitalInputs: []goonvifxsd.DigitalInput{{Token: "di0"}}}, nil
|
||||
}
|
||||
|
||||
payload := getHeartbeatONVIFPayload(camera, state)
|
||||
|
||||
if relayCalls != 1 || inputCalls != 1 {
|
||||
t.Fatalf("direct I/O queries: relay=%d inputs=%d, want 1 each", relayCalls, inputCalls)
|
||||
}
|
||||
var events []agentonvif.ONVIFEvents
|
||||
if err := json.Unmarshal(payload.eventsList, &events); err != nil {
|
||||
t.Fatalf("payload.eventsList is not valid JSON: %v", err)
|
||||
}
|
||||
got := map[string]string{}
|
||||
for _, event := range events {
|
||||
got[event.Key] = event.Type
|
||||
}
|
||||
if got["relay0"] != "output" || got["di0"] != "input" || len(got) != 2 {
|
||||
t.Fatalf("payload.eventsList = %s, want relay0 output and di0 input", payload.eventsList)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,195 +0,0 @@
|
||||
package cloud
|
||||
|
||||
import (
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
const (
|
||||
customHeadersManifestHeader = "X-Kerberos-Storage-Custom-Headers"
|
||||
maxCustomHeaders = 32
|
||||
maxCustomHeaderNameLength = 128
|
||||
maxCustomHeaderValueLength = 4096
|
||||
)
|
||||
|
||||
var disallowedCustomHeaders = map[string]struct{}{
|
||||
"authorization": {},
|
||||
"connection": {},
|
||||
"content-length": {},
|
||||
"content-type": {},
|
||||
"cookie": {},
|
||||
"expect": {},
|
||||
"host": {},
|
||||
"keep-alive": {},
|
||||
"proxy-authorization": {},
|
||||
"proxy-connection": {},
|
||||
"te": {},
|
||||
"trailer": {},
|
||||
"trace-id": {},
|
||||
"transfer-encoding": {},
|
||||
"tus-resumable": {},
|
||||
"upgrade": {},
|
||||
}
|
||||
|
||||
func parseCustomVaultHeaders(value string) (map[string]string, error) {
|
||||
if strings.TrimSpace(value) == "" {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
// Decode into pointers so JSON null values can be told apart from strings.
|
||||
var rawHeaders map[string]*string
|
||||
if err := json.Unmarshal([]byte(value), &rawHeaders); err != nil {
|
||||
return nil, fmt.Errorf("custom Vault headers must be a JSON object of string values: %w", err)
|
||||
}
|
||||
if rawHeaders == nil {
|
||||
return nil, errors.New("custom Vault headers must be a JSON object of string values")
|
||||
}
|
||||
if len(rawHeaders) > maxCustomHeaders {
|
||||
return nil, fmt.Errorf("custom Vault headers exceed the limit of %d", maxCustomHeaders)
|
||||
}
|
||||
headers := make(map[string]string, len(rawHeaders))
|
||||
seen := make(map[string]struct{}, len(rawHeaders))
|
||||
for name, rawValue := range rawHeaders {
|
||||
if rawValue == nil {
|
||||
return nil, fmt.Errorf("custom Vault header %q must be a string, not null", name)
|
||||
}
|
||||
value := *rawValue
|
||||
headers[name] = value
|
||||
if err := validateCustomVaultHeader(name, value); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
lowerName := strings.ToLower(name)
|
||||
if _, duplicate := seen[lowerName]; duplicate {
|
||||
return nil, fmt.Errorf("custom Vault header %q is duplicated with different casing", name)
|
||||
}
|
||||
seen[lowerName] = struct{}{}
|
||||
}
|
||||
return headers, nil
|
||||
}
|
||||
|
||||
func validateCustomVaultHeader(name, value string) error {
|
||||
if name == "" || len(name) > maxCustomHeaderNameLength || !isHTTPToken(name) {
|
||||
return fmt.Errorf("invalid custom Vault header name %q", name)
|
||||
}
|
||||
lowerName := strings.ToLower(name)
|
||||
if strings.HasPrefix(lowerName, "x-kerberos-") {
|
||||
return fmt.Errorf("custom Vault header %q uses the reserved X-Kerberos namespace", name)
|
||||
}
|
||||
if strings.HasPrefix(lowerName, "upload-") {
|
||||
return fmt.Errorf("custom Vault header %q uses the reserved tus Upload namespace", name)
|
||||
}
|
||||
if _, disallowed := disallowedCustomHeaders[lowerName]; disallowed {
|
||||
return fmt.Errorf("custom Vault header %q is reserved", name)
|
||||
}
|
||||
if len(value) > maxCustomHeaderValueLength {
|
||||
return fmt.Errorf("custom Vault header %q exceeds the %d byte value limit", name, maxCustomHeaderValueLength)
|
||||
}
|
||||
for _, character := range value {
|
||||
if character < 0x20 || character == 0x7f {
|
||||
return fmt.Errorf("custom Vault header %q contains an invalid value", name)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func isHTTPToken(value string) bool {
|
||||
if value == "" {
|
||||
return false
|
||||
}
|
||||
for _, character := range value {
|
||||
if (character >= 'a' && character <= 'z') ||
|
||||
(character >= 'A' && character <= 'Z') ||
|
||||
(character >= '0' && character <= '9') ||
|
||||
strings.ContainsRune("!#$%&'*+-.^_`|~", character) {
|
||||
continue
|
||||
}
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func setCustomVaultHeaders(headers http.Header, customHeaders map[string]string) error {
|
||||
if len(customHeaders) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
names := make([]string, 0, len(customHeaders))
|
||||
for name, value := range customHeaders {
|
||||
if err := validateCustomVaultHeader(name, value); err != nil {
|
||||
return err
|
||||
}
|
||||
names = append(names, name)
|
||||
// Assign directly so the configured name is not canonicalised by Set.
|
||||
headers[name] = []string{value}
|
||||
}
|
||||
sort.Strings(names)
|
||||
manifest, err := json.Marshal(names)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
headers.Set(customHeadersManifestHeader, base64.RawURLEncoding.EncodeToString(manifest))
|
||||
return nil
|
||||
}
|
||||
|
||||
const maxVaultRedirects = 10
|
||||
|
||||
// stripVaultHeadersOnCrossHostRedirect removes Vault credentials and the
|
||||
// customer-supplied metadata (custom headers, their manifest and the tus
|
||||
// Upload-Metadata that embeds them) before following a redirect to another host.
|
||||
func stripVaultHeadersOnCrossHostRedirect(req *http.Request, via []*http.Request) error {
|
||||
if len(via) >= maxVaultRedirects {
|
||||
return fmt.Errorf("stopped after %d redirects", maxVaultRedirects)
|
||||
}
|
||||
if len(via) == 0 || req.URL.Host == via[0].URL.Host {
|
||||
return nil
|
||||
}
|
||||
|
||||
customNames := customVaultHeaderNamesFromManifest(req.Header)
|
||||
for name := range req.Header {
|
||||
canonicalName := http.CanonicalHeaderKey(name)
|
||||
_, custom := customNames[strings.ToLower(name)]
|
||||
if custom || strings.HasPrefix(canonicalName, "X-Kerberos-") || canonicalName == "Upload-Metadata" {
|
||||
delete(req.Header, name)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func customVaultHeaderNamesFromManifest(headers http.Header) map[string]struct{} {
|
||||
encoded := headers.Get(customHeadersManifestHeader)
|
||||
if encoded == "" {
|
||||
return nil
|
||||
}
|
||||
manifest, err := base64.RawURLEncoding.DecodeString(encoded)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
var names []string
|
||||
if err := json.Unmarshal(manifest, &names); err != nil {
|
||||
return nil
|
||||
}
|
||||
lowerNames := make(map[string]struct{}, len(names))
|
||||
for _, name := range names {
|
||||
lowerNames[strings.ToLower(name)] = struct{}{}
|
||||
}
|
||||
return lowerNames
|
||||
}
|
||||
|
||||
func encodeCustomVaultHeaders(customHeaders map[string]string) (string, error) {
|
||||
if len(customHeaders) == 0 {
|
||||
return "", nil
|
||||
}
|
||||
encoded, err := json.Marshal(customHeaders)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if len(encoded) == 0 {
|
||||
return "", errors.New("custom Vault headers could not be encoded")
|
||||
}
|
||||
return string(encoded), nil
|
||||
}
|
||||
@@ -1,100 +0,0 @@
|
||||
package cloud
|
||||
|
||||
import (
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"reflect"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestCustomVaultHeadersAreValidatedAndAdded(t *testing.T) {
|
||||
customHeaders, err := parseCustomVaultHeaders(`{"site_id":"site-1","line_id":"line-2"}`)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
headers := http.Header{}
|
||||
if err := setCustomVaultHeaders(headers, customHeaders); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !reflect.DeepEqual(headers["site_id"], []string{"site-1"}) || !reflect.DeepEqual(headers["line_id"], []string{"line-2"}) {
|
||||
t.Fatalf("custom header names were not preserved: %#v", headers)
|
||||
}
|
||||
if _, canonicalised := headers["Site_id"]; canonicalised {
|
||||
t.Fatalf("custom header name was canonicalised: %#v", headers)
|
||||
}
|
||||
|
||||
encodedManifest, err := base64.RawURLEncoding.DecodeString(headers.Get(customHeadersManifestHeader))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var names []string
|
||||
if err := json.Unmarshal(encodedManifest, &names); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !reflect.DeepEqual(names, []string{"line_id", "site_id"}) {
|
||||
t.Fatalf("manifest = %#v", names)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCustomVaultHeadersRejectReservedAndInvalidValues(t *testing.T) {
|
||||
for _, value := range []string{
|
||||
`{"X-Kerberos-Storage-AccessKey":"secret"}`,
|
||||
`{"Authorization":"secret"}`,
|
||||
`{"site_id":"one\nInjected: value"}`,
|
||||
`{"bad header":"value"}`,
|
||||
`{"site_id":1}`,
|
||||
`{"Content-Type":"text/plain"}`,
|
||||
`{"site_id":"one","SITE_ID":"two"}`,
|
||||
`{"site_id":null}`,
|
||||
`{"site_id":"one\ttwo"}`,
|
||||
`{"Upload-Checksum":"sha1 abc"}`,
|
||||
`{"upload-concat":"final;/files/a"}`,
|
||||
`{"Upload-Defer-Length":"1"}`,
|
||||
`null`,
|
||||
} {
|
||||
if _, err := parseCustomVaultHeaders(value); err == nil {
|
||||
t.Fatalf("parseCustomVaultHeaders(%q) succeeded", value)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestVaultRedirectStripsCustomHeadersOnHostChange(t *testing.T) {
|
||||
newRedirect := func(t *testing.T, host string) (*http.Request, []*http.Request) {
|
||||
t.Helper()
|
||||
original, err := http.NewRequest(http.MethodPost, "https://vault.example.com/api/storage", nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
redirected, err := http.NewRequest(http.MethodPost, "https://"+host+"/api/storage", nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
redirected.Header.Set("X-Kerberos-Storage-AccessKey", "secret")
|
||||
redirected.Header.Set("Upload-Metadata", "custom_headers abc")
|
||||
redirected.Header.Set("Tus-Resumable", "1.0.0")
|
||||
if err := setCustomVaultHeaders(redirected.Header, map[string]string{"site_id": "site-1", "Line-Id": "line-2"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return redirected, []*http.Request{original}
|
||||
}
|
||||
|
||||
crossHost, via := newRedirect(t, "other.example.com")
|
||||
if err := stripVaultHeadersOnCrossHostRedirect(crossHost, via); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(crossHost.Header) != 1 || crossHost.Header.Get("Tus-Resumable") != "1.0.0" {
|
||||
t.Fatalf("cross-host redirect headers = %#v", crossHost.Header)
|
||||
}
|
||||
|
||||
sameHost, via := newRedirect(t, "vault.example.com")
|
||||
if err := stripVaultHeadersOnCrossHostRedirect(sameHost, via); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !reflect.DeepEqual(sameHost.Header["site_id"], []string{"site-1"}) ||
|
||||
sameHost.Header.Get("X-Kerberos-Storage-AccessKey") != "secret" ||
|
||||
sameHost.Header.Get(customHeadersManifestHeader) == "" {
|
||||
t.Fatalf("same-host redirect headers = %#v", sameHost.Header)
|
||||
}
|
||||
}
|
||||
@@ -5,13 +5,13 @@ package cloud
|
||||
import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"os"
|
||||
|
||||
"github.com/dropbox/dropbox-sdk-go-unofficial/v6/dropbox"
|
||||
"github.com/dropbox/dropbox-sdk-go-unofficial/v6/dropbox/files"
|
||||
"github.com/dropbox/dropbox-sdk-go-unofficial/v6/dropbox/users"
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/kerberos-io/agent/machinery/src/models"
|
||||
"github.com/kerberos-io/agent/machinery/src/utils"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
@@ -37,12 +37,14 @@ func UploadDropbox(configuration *models.Configuration, fileName string) (bool,
|
||||
// Upload to Dropbox
|
||||
log.Info("UploadDropbox: Uploading to Dropbox")
|
||||
log.Info("UploadDropbox: Upload started for " + fileName)
|
||||
fullname := "data/recordings/" + fileName
|
||||
|
||||
dConfig := dropbox.Config{
|
||||
Token: token,
|
||||
LogLevel: dropbox.LogInfo, // if needed, set the desired logging level. Default is off
|
||||
}
|
||||
|
||||
file, err := utils.OpenRecording("data/recordings", fileName)
|
||||
file, err := os.OpenFile(fullname, os.O_RDWR, 0755)
|
||||
if file != nil {
|
||||
defer func() {
|
||||
if cerr := file.Close(); cerr != nil {
|
||||
|
||||
@@ -9,7 +9,6 @@ import (
|
||||
"os"
|
||||
|
||||
"github.com/kerberos-io/agent/machinery/src/models"
|
||||
"github.com/kerberos-io/agent/machinery/src/utils"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
@@ -79,8 +78,10 @@ func UploadKerberosHub(configuration *models.Configuration, fileName string) (bo
|
||||
}).Info("Resumable Hub upload unavailable; using legacy upload")
|
||||
}
|
||||
|
||||
fullname := "data/recordings/" + fileName
|
||||
|
||||
// Check if we still have the file otherwise we abort the request.
|
||||
file, err := utils.OpenRecording("data/recordings", fileName)
|
||||
file, err := os.OpenFile(fullname, os.O_RDWR, 0755)
|
||||
if file != nil {
|
||||
defer file.Close()
|
||||
}
|
||||
|
||||
@@ -10,7 +10,6 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/kerberos-io/agent/machinery/src/models"
|
||||
"github.com/kerberos-io/agent/machinery/src/utils"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
@@ -36,23 +35,10 @@ func UploadKerberosVault(configuration *models.Configuration, fileName string) (
|
||||
// This can happen when the file was already removed (e.g. cleanup, or an
|
||||
// earlier successful upload). Skip it so the watcher drops the marker
|
||||
// instead of retrying indefinitely.
|
||||
filePath, err := utils.ResolveRecordingPath("data/recordings", fileName)
|
||||
info, err := os.Stat("data/recordings/" + fileName)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
log.Info("UploadKerberosVault: skipping " + fileName + ", file doesn't exist anymore")
|
||||
return false, false, nil
|
||||
}
|
||||
log.WithError(err).Error("UploadKerberosVault: could not resolve recording path")
|
||||
return false, true, err
|
||||
}
|
||||
info, err := os.Stat(filePath)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
log.Info("UploadKerberosVault: skipping " + fileName + ", file doesn't exist anymore")
|
||||
return false, false, nil
|
||||
}
|
||||
log.WithError(err).Error("UploadKerberosVault: could not stat recording")
|
||||
return false, true, err
|
||||
log.Info("UploadKerberosVault: skipping " + fileName + ", file doesn't exist anymore")
|
||||
return false, false, nil
|
||||
}
|
||||
if info.Size() == 0 {
|
||||
log.Warn("UploadKerberosVault: skipping " + fileName + ", recording is empty")
|
||||
@@ -192,11 +178,9 @@ func sendToVault(vault models.KStorage, publicKey, deviceKey, fileName, label, s
|
||||
// is sent as the body of a POST to {URI}/storage. Kept for backwards
|
||||
// compatibility with vault deployments that do not support resumable uploads.
|
||||
func uploadVaultLegacy(vault models.KStorage, publicKey, deviceKey, fileName, label string) (bool, bool, string, error) {
|
||||
customHeaders, err := parseCustomVaultHeaders(vault.CustomHeaders)
|
||||
if err != nil {
|
||||
return false, false, "", err
|
||||
}
|
||||
file, err := utils.OpenRecording("data/recordings", fileName)
|
||||
fullname := "data/recordings/" + fileName
|
||||
|
||||
file, err := os.Open(fullname)
|
||||
if file != nil {
|
||||
defer file.Close()
|
||||
}
|
||||
@@ -219,9 +203,6 @@ func uploadVaultLegacy(vault models.KStorage, publicKey, deviceKey, fileName, la
|
||||
}
|
||||
req.Header.Set("Content-Type", "video/mp4")
|
||||
setVaultHeaders(req.Header, vault, publicKey, deviceKey, fileName)
|
||||
if err := setCustomVaultHeaders(req.Header, customHeaders); err != nil {
|
||||
return false, false, "", err
|
||||
}
|
||||
setQueuedRecordingMetadataHeaders(req.Header, fileName)
|
||||
|
||||
client := newVaultHTTPClient(0)
|
||||
@@ -286,7 +267,7 @@ func newVaultHTTPClient(timeout time.Duration) *http.Client {
|
||||
transport.TLSClientConfig.InsecureSkipVerify = true
|
||||
}
|
||||
|
||||
client := &http.Client{Transport: transport, CheckRedirect: stripVaultHeadersOnCrossHostRedirect}
|
||||
client := &http.Client{Transport: transport}
|
||||
if timeout > 0 {
|
||||
client.Timeout = timeout
|
||||
}
|
||||
|
||||
@@ -2,7 +2,6 @@ package cloud
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"math"
|
||||
"net/http"
|
||||
"os"
|
||||
@@ -11,16 +10,12 @@ import (
|
||||
"strings"
|
||||
|
||||
"github.com/kerberos-io/agent/machinery/src/models"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
const recordingFPSHeader = "X-Kerberos-Storage-Fps"
|
||||
const recordingDurationHeader = "X-Kerberos-Storage-Duration"
|
||||
const recordingTimestampHeader = "X-Kerberos-Storage-Timestamp"
|
||||
const recordingEncryptedHeader = "X-Kerberos-Storage-Encrypted"
|
||||
const recordingDeviceNameHeader = "X-Kerberos-Storage-DeviceName"
|
||||
const recordingRegionHeader = "X-Kerberos-Storage-RegionCoordinates"
|
||||
const recordingChangesHeader = "X-Kerberos-Storage-NumberOfChanges"
|
||||
|
||||
// queuedRecordingFPS reads the FPS snapshot written into the upload marker
|
||||
// when the recording was finalized. Historical empty markers intentionally
|
||||
@@ -66,14 +61,6 @@ func decodeRecordingUploadMetadata(value []byte) (models.RecordingUploadMetadata
|
||||
}
|
||||
|
||||
func readRecordingUploadMetadata(fileName string) ([]byte, bool) {
|
||||
value, err := readRecordingUploadMetadataFile(fileName)
|
||||
if err != nil && !os.IsNotExist(err) {
|
||||
log.WithError(err).Error("Failed to read recording upload metadata")
|
||||
}
|
||||
return value, err == nil
|
||||
}
|
||||
|
||||
func readRecordingUploadMetadataFile(fileName string) ([]byte, error) {
|
||||
markerNames := []string{
|
||||
models.RecordingUploadMetadataFileName(fileName),
|
||||
filepath.Base(fileName),
|
||||
@@ -81,13 +68,10 @@ func readRecordingUploadMetadataFile(fileName string) ([]byte, error) {
|
||||
for _, markerName := range markerNames {
|
||||
value, err := os.ReadFile(filepath.Join("data", "cloud", markerName))
|
||||
if err == nil {
|
||||
return value, nil
|
||||
}
|
||||
if !os.IsNotExist(err) {
|
||||
return nil, fmt.Errorf("read recording upload metadata %s: %w", markerName, err)
|
||||
return value, true
|
||||
}
|
||||
}
|
||||
return nil, os.ErrNotExist
|
||||
return nil, false
|
||||
}
|
||||
|
||||
func setQueuedRecordingMetadataHeaders(header http.Header, fileName string) {
|
||||
@@ -95,18 +79,6 @@ func setQueuedRecordingMetadataHeaders(header http.Header, fileName string) {
|
||||
header.Set(recordingFPSHeader, fps)
|
||||
}
|
||||
if metadata, ok := queuedRecordingMetadata(fileName); ok {
|
||||
if metadata.DeviceKey != "" {
|
||||
header.Set("X-Kerberos-Storage-Device", metadata.DeviceKey)
|
||||
}
|
||||
if metadata.DeviceName != "" {
|
||||
header.Set(recordingDeviceNameHeader, metadata.DeviceName)
|
||||
}
|
||||
if metadata.RegionCoordinates != "" {
|
||||
header.Set(recordingRegionHeader, metadata.RegionCoordinates)
|
||||
}
|
||||
if metadata.NumberOfChanges != "" {
|
||||
header.Set(recordingChangesHeader, metadata.NumberOfChanges)
|
||||
}
|
||||
if metadata.Duration > 0 {
|
||||
header.Set(recordingDurationHeader, strconv.FormatUint(metadata.Duration, 10))
|
||||
}
|
||||
|
||||
+36
-104
@@ -2,20 +2,14 @@ package cloud
|
||||
|
||||
import (
|
||||
"crypto/tls"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"math"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/kerberos-io/agent/machinery/src/models"
|
||||
"github.com/kerberos-io/agent/machinery/src/utils"
|
||||
"github.com/minio/minio-go/v6"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
@@ -24,6 +18,15 @@ func UploadS3(configuration *models.Configuration, fileName string) (bool, bool,
|
||||
|
||||
config := configuration.Config
|
||||
|
||||
// timestamp_microseconds_instanceName_regionCoordinates_numberOfChanges_token
|
||||
// 1564859471_6-474162_oprit_577-283-727-375_1153_27.mp4
|
||||
// - Timestamp
|
||||
// - Size + - + microseconds
|
||||
// - device
|
||||
// - Region
|
||||
// - Number of changes
|
||||
// - Token
|
||||
|
||||
if config.S3 == nil {
|
||||
errorMessage := "UploadS3: Uploading Failed, as no settings found"
|
||||
log.Error(errorMessage)
|
||||
@@ -68,14 +71,24 @@ func UploadS3(configuration *models.Configuration, fileName string) (bool, bool,
|
||||
s3Client.SetCustomTransport(transport)
|
||||
}
|
||||
|
||||
objectName, userMetadata, err := s3RecordingUploadDetails(config, fileName, aws_access_key_id)
|
||||
if err != nil {
|
||||
log.WithError(err).Error("UploadS3: invalid recording upload metadata")
|
||||
return false, true, err
|
||||
fileParts := strings.Split(fileName, "_")
|
||||
if len(fileParts) == 1 {
|
||||
errorMessage := "UploadS3: " + fileName + " is not a valid name."
|
||||
log.Error(errorMessage)
|
||||
return false, true, errors.New(errorMessage)
|
||||
}
|
||||
|
||||
deviceKey := config.Key
|
||||
startRecording, _ := strconv.ParseInt(fileParts[0], 10, 64)
|
||||
devicename := fileParts[2]
|
||||
coordinates := fileParts[3]
|
||||
//numberOfChanges := fileParts[4]
|
||||
token, _ := strconv.Atoi(fileParts[5])
|
||||
|
||||
log.Info("UploadS3: Upload started for " + fileName)
|
||||
file, err := utils.OpenRecording("data/recordings", fileName)
|
||||
fullname := "data/recordings/" + fileName
|
||||
|
||||
file, err := os.OpenFile(fullname, os.O_RDWR, 0755)
|
||||
if file != nil {
|
||||
defer file.Close()
|
||||
}
|
||||
@@ -94,13 +107,23 @@ func UploadS3(configuration *models.Configuration, fileName string) (bool, bool,
|
||||
}
|
||||
|
||||
n, err := s3Client.PutObject(config.S3.Bucket,
|
||||
objectName,
|
||||
config.S3.Username+"/"+fileName,
|
||||
file,
|
||||
fileInfo.Size(),
|
||||
minio.PutObjectOptions{
|
||||
ContentType: "video/mp4",
|
||||
StorageClass: "ONEZONE_IA",
|
||||
UserMetadata: userMetadata,
|
||||
UserMetadata: map[string]string{
|
||||
"event-timestamp": strconv.FormatInt(startRecording, 10),
|
||||
"event-microseconds": deviceKey,
|
||||
"event-instancename": devicename,
|
||||
"event-regioncoordinates": coordinates,
|
||||
"event-numberofchanges": deviceKey,
|
||||
"event-token": strconv.Itoa(token),
|
||||
"productid": deviceKey,
|
||||
"publickey": aws_access_key_id,
|
||||
"uploadtime": "now",
|
||||
},
|
||||
})
|
||||
|
||||
if err != nil {
|
||||
@@ -112,94 +135,3 @@ func UploadS3(configuration *models.Configuration, fileName string) (bool, bool,
|
||||
return true, true, nil
|
||||
}
|
||||
}
|
||||
|
||||
func s3RecordingObjectName(username, deviceKey string, startRecording int64) string {
|
||||
return username + "/" + time.UnixMilli(startRecording).UTC().Format("2006/01/02") +
|
||||
"/" + strconv.FormatInt(startRecording, 10) + "-" + deviceKey + ".mp4"
|
||||
}
|
||||
|
||||
func s3RecordingUploadDetails(config models.Config, fileName, publicKey string) (string, map[string]string, error) {
|
||||
value, err := readRecordingUploadMetadataFile(fileName)
|
||||
if err != nil && !os.IsNotExist(err) {
|
||||
return "", nil, err
|
||||
}
|
||||
|
||||
deviceKey := config.Key
|
||||
var startRecording int64
|
||||
userMetadata := map[string]string{
|
||||
"publickey": publicKey,
|
||||
"uploadtime": "now",
|
||||
}
|
||||
marker := strings.TrimSpace(string(value))
|
||||
_, fpsErr := strconv.ParseFloat(marker, 64)
|
||||
if err == nil && marker != "" && fpsErr != nil {
|
||||
var metadata models.RecordingUploadMetadata
|
||||
if err := json.Unmarshal(value, &metadata); err != nil {
|
||||
return "", nil, fmt.Errorf("decode recording upload metadata for %s: %w", fileName, err)
|
||||
}
|
||||
if metadata.FileName != "" && metadata.FileName != filepath.Base(fileName) {
|
||||
return "", nil, fmt.Errorf("recording upload metadata filename does not match %s", fileName)
|
||||
}
|
||||
if metadata.Timestamp <= 0 {
|
||||
return "", nil, fmt.Errorf("invalid recording timestamp in upload metadata for %s", fileName)
|
||||
}
|
||||
deviceKey = metadata.DeviceKey
|
||||
startRecording = metadata.Timestamp
|
||||
userMetadata["event-instancename"] = config.Name
|
||||
if metadata.DeviceName != "" {
|
||||
userMetadata["event-instancename"] = metadata.DeviceName
|
||||
}
|
||||
if metadata.RegionCoordinates != "" {
|
||||
userMetadata["event-regioncoordinates"] = metadata.RegionCoordinates
|
||||
}
|
||||
if metadata.NumberOfChanges != "" {
|
||||
userMetadata["event-numberofchanges"] = metadata.NumberOfChanges
|
||||
}
|
||||
userMetadata["event-microseconds"] = strconv.FormatInt(metadata.Timestamp%1000*1000, 10)
|
||||
userMetadata["duration"] = strconv.FormatUint(metadata.Duration, 10)
|
||||
userMetadata["encrypted"] = strconv.FormatBool(metadata.Encrypted)
|
||||
if metadata.FPS > 0 && metadata.FPS <= 240 {
|
||||
userMetadata["fps"] = strconv.FormatFloat(metadata.FPS, 'f', -1, 64)
|
||||
}
|
||||
} else {
|
||||
// Empty and FPS-only markers predate structured recording metadata.
|
||||
fileParts := strings.Split(fileName, "_")
|
||||
if len(fileParts) < 6 {
|
||||
return "", nil, fmt.Errorf("%s is not a valid legacy recording name", fileName)
|
||||
}
|
||||
startRecording, err = strconv.ParseInt(fileParts[0], 10, 64)
|
||||
if err != nil {
|
||||
return "", nil, fmt.Errorf("invalid recording timestamp in %s: %w", fileName, err)
|
||||
}
|
||||
if startRecording <= 0 || startRecording > math.MaxInt64/1000 {
|
||||
return "", nil, fmt.Errorf("invalid recording timestamp in %s", fileName)
|
||||
}
|
||||
startRecording *= 1000
|
||||
_, precision, found := strings.Cut(fileParts[1], "-")
|
||||
if found {
|
||||
fraction, err := strconv.ParseInt(precision, 10, 64)
|
||||
if err != nil || fraction < 0 || fraction >= 1000000 {
|
||||
return "", nil, fmt.Errorf("invalid recording precision in %s", fileName)
|
||||
}
|
||||
if len(precision) > 3 {
|
||||
fraction /= 1000
|
||||
}
|
||||
startRecording += fraction
|
||||
}
|
||||
token, _ := strconv.Atoi(fileParts[5])
|
||||
userMetadata["event-microseconds"] = deviceKey
|
||||
userMetadata["event-instancename"] = fileParts[2]
|
||||
userMetadata["event-regioncoordinates"] = fileParts[3]
|
||||
userMetadata["event-numberofchanges"] = deviceKey
|
||||
userMetadata["event-token"] = strconv.Itoa(token)
|
||||
}
|
||||
if deviceKey == "" || strings.ContainsAny(deviceKey, "/\\") {
|
||||
return "", nil, errors.New("missing or invalid recording device key")
|
||||
}
|
||||
if startRecording <= 0 {
|
||||
return "", nil, fmt.Errorf("invalid recording timestamp for %s", fileName)
|
||||
}
|
||||
userMetadata["event-timestamp"] = strconv.FormatInt(startRecording/1000, 10)
|
||||
userMetadata["productid"] = deviceKey
|
||||
return s3RecordingObjectName(config.S3.Username, deviceKey, startRecording), userMetadata, nil
|
||||
}
|
||||
|
||||
@@ -1,225 +0,0 @@
|
||||
package cloud
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/kerberos-io/agent/machinery/src/models"
|
||||
)
|
||||
|
||||
func TestS3RecordingUploadDetailsUsesCheckpoint(t *testing.T) {
|
||||
for _, fileName := range []string{
|
||||
"1767355200123-checkpoint-key.mp4",
|
||||
"1564859471_6-474162_old-camera_0-0-0-0_1153_27.mp4",
|
||||
} {
|
||||
t.Run(fileName, func(t *testing.T) {
|
||||
withRecording(t, fileName, []byte("recording"))
|
||||
recorded := time.Date(2026, time.January, 2, 12, 0, 0, 123000000, time.UTC)
|
||||
metadata := models.RecordingUploadMetadata{
|
||||
FileName: fileName,
|
||||
DeviceKey: "checkpoint-key",
|
||||
Timestamp: recorded.UnixMilli(),
|
||||
Duration: 20452,
|
||||
FPS: 29.97,
|
||||
Encrypted: true,
|
||||
DeviceName: "checkpoint-camera",
|
||||
RegionCoordinates: "1-2-3-4",
|
||||
NumberOfChanges: "57",
|
||||
}
|
||||
payload, err := json.Marshal(metadata)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
withQueuedRecordingFPS(t, fileName, string(payload))
|
||||
config := models.Config{
|
||||
Key: "changed-config-key",
|
||||
Name: "camera-name",
|
||||
S3: &models.S3{Username: "user"},
|
||||
}
|
||||
objectName, userMetadata, err := s3RecordingUploadDetails(config, fileName, "public-key")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
wantObjectName := "user/2026/01/02/" + strconv.FormatInt(recorded.UnixMilli(), 10) + "-checkpoint-key.mp4"
|
||||
if objectName != wantObjectName {
|
||||
t.Fatalf("object name = %q, want %q", objectName, wantObjectName)
|
||||
}
|
||||
wantMetadata := map[string]string{
|
||||
"publickey": "public-key",
|
||||
"uploadtime": "now",
|
||||
"event-instancename": "checkpoint-camera",
|
||||
"event-regioncoordinates": "1-2-3-4",
|
||||
"event-numberofchanges": "57",
|
||||
"event-microseconds": "123000",
|
||||
"event-timestamp": strconv.FormatInt(recorded.Unix(), 10),
|
||||
"productid": "checkpoint-key",
|
||||
"duration": "20452",
|
||||
"fps": "29.97",
|
||||
"encrypted": "true",
|
||||
}
|
||||
if !reflect.DeepEqual(userMetadata, wantMetadata) {
|
||||
t.Fatalf("metadata = %#v, want %#v", userMetadata, wantMetadata)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestS3RecordingUploadDetailsLegacyMarkers(t *testing.T) {
|
||||
for _, marker := range []string{"missing", "empty", "fps-only", "old-marker-name"} {
|
||||
t.Run(marker, func(t *testing.T) {
|
||||
const fileName = "1564859471_6-474162_camera_0-0-0-0_1153_27.mp4"
|
||||
withRecording(t, fileName, []byte("recording"))
|
||||
switch marker {
|
||||
case "empty":
|
||||
withQueuedRecordingFPS(t, fileName, "")
|
||||
case "fps-only", "old-marker-name":
|
||||
withQueuedRecordingFPS(t, fileName, "29.97")
|
||||
if marker == "old-marker-name" {
|
||||
err := os.Rename(filepath.Join("data/cloud", models.RecordingUploadMetadataFileName(fileName)),
|
||||
filepath.Join("data/cloud", fileName))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
config := models.Config{Key: "legacy-key", S3: &models.S3{Username: "user"}}
|
||||
objectName, metadata, err := s3RecordingUploadDetails(config, fileName, "public-key")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if want := "user/2019/08/03/1564859471474-legacy-key.mp4"; objectName != want {
|
||||
t.Fatalf("object name = %q, want %q", objectName, want)
|
||||
}
|
||||
if metadata["productid"] != "legacy-key" || metadata["event-instancename"] != "camera" ||
|
||||
metadata["event-regioncoordinates"] != "0-0-0-0" || metadata["event-timestamp"] != "1564859471" {
|
||||
t.Fatalf("unexpected legacy metadata: %#v", metadata)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestS3RecordingUploadDetailsRejectsInvalidCheckpoint(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
marker string
|
||||
want string
|
||||
}{
|
||||
{"malformed JSON", `{"timestamp":`, "decode"},
|
||||
{"missing timestamp", `{"device_key":"camera"}`, "timestamp"},
|
||||
{"missing device key", `{"timestamp":1787015373000}`, "device key"},
|
||||
{"invalid device key", `{"timestamp":1787015373000,"device_key":"camera/key"}`, "device key"},
|
||||
{"mismatched filename", `{"timestamp":1787015373000,"device_key":"camera","filename":"other.mp4"}`, "filename"},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
const fileName = "1564859471_6-474162_camera_0-0-0-0_1153_27.mp4"
|
||||
withRecording(t, fileName, []byte("recording"))
|
||||
withQueuedRecordingFPS(t, fileName, tt.marker)
|
||||
config := models.Config{Key: "legacy-key", S3: &models.S3{Username: "user"}}
|
||||
_, _, err := s3RecordingUploadDetails(config, fileName, "public-key")
|
||||
if err == nil || !strings.Contains(err.Error(), tt.want) {
|
||||
t.Fatalf("error = %v, want error containing %q instead of legacy fallback", err, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestS3RecordingUploadDetailsReportsCheckpointReadError(t *testing.T) {
|
||||
const fileName = "1564859471_6-474162_camera_0-0-0-0_1153_27.mp4"
|
||||
withRecording(t, fileName, []byte("recording"))
|
||||
if err := os.MkdirAll(filepath.Join("data/cloud", models.RecordingUploadMetadataFileName(fileName)), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
config := models.Config{Key: "legacy-key", S3: &models.S3{Username: "user"}}
|
||||
_, _, err := s3RecordingUploadDetails(config, fileName, "public-key")
|
||||
if err == nil || !strings.Contains(err.Error(), "read recording upload metadata") {
|
||||
t.Fatalf("error = %v, want checkpoint read error instead of legacy fallback", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestS3RecordingObjectName(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
recorded time.Time
|
||||
directory string
|
||||
}{
|
||||
{
|
||||
name: "zero padded month and day",
|
||||
recorded: time.Date(2026, time.January, 2, 12, 0, 0, 0, time.UTC),
|
||||
directory: "2026/01/02",
|
||||
},
|
||||
{
|
||||
name: "UTC date across year boundary",
|
||||
recorded: time.Date(2026, time.January, 1, 1, 0, 0, 0, time.FixedZone("UTC+2", 2*60*60)),
|
||||
directory: "2025/12/31",
|
||||
},
|
||||
{
|
||||
name: "leap day",
|
||||
recorded: time.Date(2024, time.February, 29, 23, 59, 59, 0, time.UTC),
|
||||
directory: "2024/02/29",
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
const deviceKey = "camera-key-123"
|
||||
want := "user/" + tt.directory + "/" + strconv.FormatInt(tt.recorded.UnixMilli(), 10) + "-" + deviceKey + ".mp4"
|
||||
for attempt := 0; attempt < 2; attempt++ {
|
||||
if got := s3RecordingObjectName("user", deviceKey, tt.recorded.UnixMilli()); got != want {
|
||||
t.Fatalf("s3RecordingObjectName() = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestUploadS3RejectsInvalidRecordingNames(t *testing.T) {
|
||||
configuration := &models.Configuration{
|
||||
Config: models.Config{
|
||||
Key: "camera-key-123",
|
||||
S3: &models.S3{
|
||||
Publickey: "test-public-key",
|
||||
Secretkey: "test-secret-key",
|
||||
Region: "eu-west-1",
|
||||
},
|
||||
},
|
||||
}
|
||||
for _, fileName := range []string{
|
||||
"recording.mp4",
|
||||
"1787015373_camera.mp4",
|
||||
"invalid_6-474162_camera_0-0-0-0_1153_27.mp4",
|
||||
} {
|
||||
t.Run(fileName, func(t *testing.T) {
|
||||
uploaded, configured, err := UploadS3(configuration, fileName)
|
||||
if uploaded || !configured || err == nil {
|
||||
t.Fatalf("UploadS3() = (%v, %v, %v), want (false, true, error)", uploaded, configured, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestUploadS3RejectsInvalidDeviceKeys(t *testing.T) {
|
||||
for _, deviceKey := range []string{"", "camera/key", `camera\key`} {
|
||||
t.Run(deviceKey, func(t *testing.T) {
|
||||
configuration := &models.Configuration{
|
||||
Config: models.Config{
|
||||
Key: deviceKey,
|
||||
S3: &models.S3{
|
||||
Publickey: "test-public-key",
|
||||
Secretkey: "test-secret-key",
|
||||
Region: "eu-west-1",
|
||||
},
|
||||
},
|
||||
}
|
||||
uploaded, configured, err := UploadS3(configuration, "1787015373_6-474162_camera_0-0-0-0_1153_27.mp4")
|
||||
if uploaded || !configured || err == nil || !strings.Contains(err.Error(), "device key") {
|
||||
t.Fatalf("UploadS3() = (%v, %v, %v), want (false, true, device key error)", uploaded, configured, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -16,7 +16,6 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/kerberos-io/agent/machinery/src/models"
|
||||
"github.com/kerberos-io/agent/machinery/src/utils"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
@@ -107,7 +106,7 @@ func logTusUploadProgress(label string, offset, size int64, loggedBucket *int64)
|
||||
// creation request; it is empty on HEAD/PATCH/DELETE.
|
||||
type tusHeaderFunc func(h http.Header, fileName string)
|
||||
|
||||
// runTusUpload performs a resumable (tus) upload of a flat or dated recording.
|
||||
// runTusUpload performs a resumable (tus) upload of data/recordings/<fileName>
|
||||
// to baseURL, sending target-specific authentication/routing headers via
|
||||
// setHeaders on every request. It encapsulates the create/resume/chunk/finalize
|
||||
// state machine shared by the Kerberos Vault (direct) and Kerberos Hub (proxied)
|
||||
@@ -121,7 +120,9 @@ type tusHeaderFunc func(h http.Header, fileName string)
|
||||
// should fall back to the legacy single-POST upload (older deployments).
|
||||
// - body: a short message for logging.
|
||||
func runTusUpload(baseURL, metadata, fileName, label, slot string, setHeaders tusHeaderFunc) (uploaded bool, responded bool, supported bool, body string, err error) {
|
||||
file, ferr := utils.OpenRecording("data/recordings", fileName)
|
||||
fullname := "data/recordings/" + fileName
|
||||
|
||||
file, ferr := os.Open(fullname)
|
||||
if file != nil {
|
||||
defer file.Close()
|
||||
}
|
||||
@@ -140,6 +141,19 @@ func runTusUpload(baseURL, metadata, fileName, label, slot string, setHeaders tu
|
||||
size := info.Size()
|
||||
|
||||
client := newVaultHTTPClient(0)
|
||||
client.CheckRedirect = func(req *http.Request, via []*http.Request) error {
|
||||
if len(via) == 0 {
|
||||
return nil
|
||||
}
|
||||
if req.URL.Host != via[0].URL.Host {
|
||||
for k := range req.Header {
|
||||
if strings.HasPrefix(http.CanonicalHeaderKey(k), "X-Kerberos-") {
|
||||
req.Header.Del(k)
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
sidecar := tusSidecarPath(fileName, slot)
|
||||
uploadURL := loadTusResumeState(sidecar, baseURL)
|
||||
@@ -297,10 +311,6 @@ func runTusUpload(baseURL, metadata, fileName, label, slot string, setHeaders tu
|
||||
// X-Kerberos-Storage-* headers on every request and routing (directory/provider)
|
||||
// is additionally carried in the tus Upload-Metadata.
|
||||
func uploadVaultResumable(vault models.KStorage, publicKey, deviceKey, fileName, label, slot string) (bool, bool, bool, string, error) {
|
||||
customHeaders, err := parseCustomVaultHeaders(vault.CustomHeaders)
|
||||
if err != nil {
|
||||
return false, false, true, "", err
|
||||
}
|
||||
baseURL := strings.TrimRight(vault.URI, "/") + tusUploadPath
|
||||
metadataValues := map[string]string{
|
||||
"filename": fileName,
|
||||
@@ -311,22 +321,10 @@ func uploadVaultResumable(vault models.KStorage, publicKey, deviceKey, fileName,
|
||||
"cloudkey": publicKey,
|
||||
"fps": queuedRecordingFPS(fileName),
|
||||
}
|
||||
if encodedHeaders, err := encodeCustomVaultHeaders(customHeaders); err != nil {
|
||||
return false, false, true, "", err
|
||||
} else if encodedHeaders != "" {
|
||||
metadataValues["custom_headers"] = encodedHeaders
|
||||
}
|
||||
customRequestHeaders := http.Header{}
|
||||
if err := setCustomVaultHeaders(customRequestHeaders, customHeaders); err != nil {
|
||||
return false, false, true, "", err
|
||||
}
|
||||
addRecordingTusMetadata(metadataValues, fileName)
|
||||
metadata := encodeTusMetadata(metadataValues)
|
||||
setHeaders := func(h http.Header, fn string) {
|
||||
setVaultTusHeaders(h, vault, publicKey, deviceKey, fn)
|
||||
for name, values := range customRequestHeaders {
|
||||
h[name] = append([]string(nil), values...)
|
||||
}
|
||||
}
|
||||
return runTusUpload(baseURL, metadata, fileName, label, slot, setHeaders)
|
||||
}
|
||||
@@ -357,12 +355,6 @@ func addRecordingTusMetadata(values map[string]string, fileName string) {
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
if metadata.DeviceKey != "" {
|
||||
values["device"] = metadata.DeviceKey
|
||||
}
|
||||
values["device_name"] = metadata.DeviceName
|
||||
values["region_coordinates"] = metadata.RegionCoordinates
|
||||
values["number_of_changes"] = metadata.NumberOfChanges
|
||||
if metadata.Duration > 0 {
|
||||
values["duration"] = strconv.FormatUint(metadata.Duration, 10)
|
||||
}
|
||||
@@ -505,7 +497,6 @@ func setVaultTusHeaders(h http.Header, vault models.KStorage, publicKey, deviceK
|
||||
h.Set("X-Kerberos-Storage-Capture", "IPCamera")
|
||||
if fileName != "" {
|
||||
h.Set("X-Kerberos-Storage-FileName", fileName)
|
||||
setQueuedRecordingMetadataHeaders(h, fileName)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -522,7 +513,6 @@ func setHubTusHeaders(h http.Header, config *models.Config, fileName string) {
|
||||
h.Set("X-Kerberos-Storage-Capture", "IPCamera")
|
||||
if fileName != "" {
|
||||
h.Set("X-Kerberos-Storage-FileName", fileName)
|
||||
setQueuedRecordingMetadataHeaders(h, fileName)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -3,7 +3,6 @@ package cloud
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net"
|
||||
@@ -18,86 +17,8 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/kerberos-io/agent/machinery/src/models"
|
||||
"github.com/kerberos-io/agent/machinery/src/utils"
|
||||
)
|
||||
|
||||
func TestUploadKerberosVaultReadsDatedLocalRecordings(t *testing.T) {
|
||||
previousCount, previousTimeout := kstorageRetryCount, kstorageRetryTimeout
|
||||
t.Cleanup(func() { kstorageRetryCount, kstorageRetryTimeout = previousCount, previousTimeout })
|
||||
for _, transport := range []string{"resumable", "legacy"} {
|
||||
t.Run(transport, func(t *testing.T) {
|
||||
const fileName = "1564859471123-device-key.mp4"
|
||||
payload := bytes.Repeat([]byte("video"), 1024)
|
||||
withRecording(t, fileName, payload)
|
||||
datedPath, err := utils.CreateRecordingPath("data/recordings", fileName, time.Unix(1564859471, 0))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Rename(filepath.Join("data/recordings", fileName), datedPath); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
withQueuedRecordingFPS(t, fileName, `{"filename":"1564859471123-device-key.mp4","device_key":"device-key","timestamp":1564859471123,"duration":27,"fps":25,"device_name":"camera-name","region_coordinates":"1-2-3-4","number_of_changes":"57"}`)
|
||||
fake := newFakeTus()
|
||||
var legacyBody []byte
|
||||
var legacyHeaders http.Header
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if transport == "resumable" {
|
||||
fake.ServeHTTP(w, r)
|
||||
return
|
||||
}
|
||||
body, readErr := io.ReadAll(r.Body)
|
||||
if readErr != nil {
|
||||
t.Error(readErr)
|
||||
}
|
||||
legacyBody = body
|
||||
legacyHeaders = r.Header.Clone()
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}))
|
||||
defer server.Close()
|
||||
if transport == "legacy" {
|
||||
t.Setenv("AGENT_DISABLE_RESUMABLE_UPLOAD", "true")
|
||||
} else {
|
||||
t.Setenv("AGENT_DISABLE_RESUMABLE_UPLOAD", "false")
|
||||
}
|
||||
kstorageRetryCount, kstorageRetryTimeout = 0, 0
|
||||
vault := testVault(server.URL)
|
||||
config := &models.Configuration{Config: models.Config{
|
||||
Key: "changed-config-key",
|
||||
KStorage: &vault,
|
||||
KStorageSecondary: &models.KStorage{},
|
||||
}}
|
||||
uploaded, configured, err := UploadKerberosVault(config, fileName)
|
||||
if err != nil || !uploaded || !configured {
|
||||
t.Fatalf("UploadKerberosVault() = (%v, %v, %v), want success", uploaded, configured, err)
|
||||
}
|
||||
if transport == "resumable" {
|
||||
if got := fake.totalBytes(); got != int64(len(payload)) {
|
||||
t.Fatalf("uploaded %d bytes, want %d", got, len(payload))
|
||||
}
|
||||
posts := fake.requestsForMethod(http.MethodPost)
|
||||
if len(posts) != 1 {
|
||||
t.Fatalf("got %d creation requests, want 1", len(posts))
|
||||
}
|
||||
metadata := decodeTusMetadata(posts[0].header.Get("Upload-Metadata"))
|
||||
if metadata["filename"] != fileName || metadata["directory"] != "dir" || metadata["timestamp"] != "1564859471123" ||
|
||||
metadata["device"] != "device-key" || metadata["device_name"] != "camera-name" ||
|
||||
metadata["region_coordinates"] != "1-2-3-4" || metadata["number_of_changes"] != "57" {
|
||||
t.Fatalf("unexpected upload metadata: %#v", metadata)
|
||||
}
|
||||
} else if !bytes.Equal(legacyBody, payload) ||
|
||||
legacyHeaders.Get("X-Kerberos-Storage-FileName") != fileName ||
|
||||
legacyHeaders.Get("X-Kerberos-Storage-Directory") != "dir" ||
|
||||
legacyHeaders.Get("X-Kerberos-Storage-Device") != "device-key" ||
|
||||
legacyHeaders.Get(recordingDeviceNameHeader) != "camera-name" ||
|
||||
legacyHeaders.Get(recordingRegionHeader) != "1-2-3-4" ||
|
||||
legacyHeaders.Get(recordingChangesHeader) != "57" ||
|
||||
legacyHeaders.Get(recordingTimestampHeader) != "1564859471123" {
|
||||
t.Fatalf("unexpected legacy upload: %d bytes, headers %#v", len(legacyBody), legacyHeaders)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// fakeUpload tracks the state of a single resumable upload on the fake server.
|
||||
type fakeUpload struct {
|
||||
size int64
|
||||
@@ -383,9 +304,7 @@ func TestUploadVaultResumable_HappyPath(t *testing.T) {
|
||||
withRecording(t, fileName, payload)
|
||||
withQueuedRecordingFPS(t, fileName, `{"filename":"recording.mp4","device_key":"device-key","timestamp":1785934709414,"duration":20452,"fps":29.97}`)
|
||||
|
||||
vault := testVault(ts.URL)
|
||||
vault.CustomHeaders = `{"site_id":"site-1","equipment_id":"equipment-3"}`
|
||||
uploaded, responded, supported, _, err := uploadVaultResumable(vault, "pk", "dev", fileName, "test", "primary")
|
||||
uploaded, responded, supported, _, err := uploadVaultResumable(testVault(ts.URL), "pk", "dev", fileName, "test", "primary")
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
@@ -409,23 +328,6 @@ func TestUploadVaultResumable_HappyPath(t *testing.T) {
|
||||
if got := metadata["timestamp"]; got != "1785934709414" {
|
||||
t.Fatalf("POST metadata timestamp = %q, want %q", got, "1785934709414")
|
||||
}
|
||||
var customHeaders map[string]string
|
||||
if err := json.Unmarshal([]byte(metadata["custom_headers"]), &customHeaders); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if customHeaders["site_id"] != "site-1" || customHeaders["equipment_id"] != "equipment-3" {
|
||||
t.Fatalf("POST custom header metadata = %#v", customHeaders)
|
||||
}
|
||||
for _, method := range []string{http.MethodPost, http.MethodHead, http.MethodPatch} {
|
||||
for _, request := range srv.requestsForMethod(method) {
|
||||
if request.header.Get("site_id") != "site-1" || request.header.Get("equipment_id") != "equipment-3" {
|
||||
t.Fatalf("%s custom headers = %#v", method, request.header)
|
||||
}
|
||||
if request.header.Get(customHeadersManifestHeader) == "" {
|
||||
t.Fatalf("%s is missing the custom header manifest", method)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestQueuedRecordingFPSValidation(t *testing.T) {
|
||||
|
||||
@@ -964,7 +964,7 @@ func GetDashboard(c *gin.Context, configDirectory string, configuration *models.
|
||||
// All days stored in this agent.
|
||||
days := []string{}
|
||||
latestEvents := []models.Media{}
|
||||
files, err := utils.ReadRecordingDirectory(recordingDirectory)
|
||||
files, err := utils.ReadDirectory(recordingDirectory)
|
||||
if err == nil {
|
||||
events := utils.GetSortedDirectory(files)
|
||||
|
||||
@@ -975,8 +975,6 @@ func GetDashboard(c *gin.Context, configDirectory string, configuration *models.
|
||||
var eventFilter models.EventFilter
|
||||
eventFilter.NumberOfElements = 5
|
||||
latestEvents = utils.GetMediaFormatted(events, recordingDirectory, configuration, eventFilter) // will get 5 latest recordings.
|
||||
} else {
|
||||
log.WithError(err).Error("Could not list recordings for Agent status")
|
||||
}
|
||||
|
||||
c.JSON(200, gin.H{
|
||||
@@ -1009,7 +1007,7 @@ func GetLatestEvents(c *gin.Context, configDirectory string, configuration *mode
|
||||
eventFilter.NumberOfElements = 10
|
||||
}
|
||||
recordingDirectory := configDirectory + "/data/recordings"
|
||||
files, err := utils.ReadRecordingDirectory(recordingDirectory)
|
||||
files, err := utils.ReadDirectory(recordingDirectory)
|
||||
if err == nil {
|
||||
events := utils.GetSortedDirectory(files)
|
||||
// We will get all recordings from the directory (as defined by the filter).
|
||||
@@ -1038,7 +1036,7 @@ func GetLatestEvents(c *gin.Context, configDirectory string, configuration *mode
|
||||
// @Success 200
|
||||
func GetDays(c *gin.Context, configDirectory string, configuration *models.Configuration, communication *models.Communication) {
|
||||
recordingDirectory := configDirectory + "/data/recordings"
|
||||
files, err := utils.ReadRecordingDirectory(recordingDirectory)
|
||||
files, err := utils.ReadDirectory(recordingDirectory)
|
||||
if err == nil {
|
||||
events := utils.GetSortedDirectory(files)
|
||||
days := utils.GetDays(events, recordingDirectory, configuration)
|
||||
|
||||
@@ -315,15 +315,6 @@ func initConfigPointers(config *models.Config) {
|
||||
func applyAgentEnvVars(configuration *models.Configuration, prefix string, applyDefaults bool) {
|
||||
environmentVariables := os.Environ()
|
||||
|
||||
if prefix == "" {
|
||||
if deploymentName := os.Getenv("DEPLOYMENT_NAME"); deploymentName != "" {
|
||||
configuration.Config.Name = deploymentName
|
||||
if configuration.Config.FriendlyName == "" {
|
||||
configuration.Config.FriendlyName = deploymentName
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Initialize the configuration for some new fields.
|
||||
if configuration.Config.KStorageSecondary == nil {
|
||||
configuration.Config.KStorageSecondary = &models.KStorage{}
|
||||
@@ -612,9 +603,6 @@ func applyAgentEnvVars(configuration *models.Configuration, prefix string, apply
|
||||
case "AGENT_KERBEROSVAULT_DIRECTORY":
|
||||
configuration.Config.KStorage.Directory = value
|
||||
break
|
||||
case "AGENT_KERBEROSVAULT_CUSTOM_HEADERS":
|
||||
configuration.Config.KStorage.CustomHeaders = value
|
||||
break
|
||||
|
||||
/* Retry policy and timeout */
|
||||
case "AGENT_KERBEROSVAULT_MAX_RETRIES":
|
||||
@@ -646,9 +634,6 @@ func applyAgentEnvVars(configuration *models.Configuration, prefix string, apply
|
||||
case "AGENT_KERBEROSVAULT_SECONDARY_DIRECTORY":
|
||||
configuration.Config.KStorageSecondary.Directory = value
|
||||
break
|
||||
case "AGENT_KERBEROSVAULT_SECONDARY_CUSTOM_HEADERS":
|
||||
configuration.Config.KStorageSecondary.CustomHeaders = value
|
||||
break
|
||||
|
||||
/* When storing in dropbox */
|
||||
case "AGENT_DROPBOX_ACCESS_TOKEN":
|
||||
|
||||
@@ -3,7 +3,6 @@ package config
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
@@ -12,53 +11,6 @@ import (
|
||||
"go.mongodb.org/mongo-driver/mongo"
|
||||
)
|
||||
|
||||
func TestApplyAgentEnvVarsDeploymentName(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
prefix string
|
||||
deploymentName string
|
||||
agentName string
|
||||
initialName string
|
||||
initialDisplay string
|
||||
wantName string
|
||||
wantDisplay string
|
||||
}{
|
||||
{name: "deployment fallback", deploymentName: "camera-1", wantName: "camera-1", wantDisplay: "camera-1"},
|
||||
{name: "explicit display name", deploymentName: "camera-1", agentName: "Front Door", wantName: "camera-1", wantDisplay: "Front Door"},
|
||||
{name: "existing display name", deploymentName: "camera-1", initialDisplay: "Front Door", wantName: "camera-1", wantDisplay: "Front Door"},
|
||||
{name: "deployment overrides bundled identity", deploymentName: "camera-1", initialName: "default", wantName: "camera-1", wantDisplay: "camera-1"},
|
||||
{name: "no deployment", initialName: "existing", initialDisplay: "Existing Camera", wantName: "existing", wantDisplay: "Existing Camera"},
|
||||
{name: "standalone display name", agentName: "Front Door", wantDisplay: "Front Door"},
|
||||
{name: "global layer excludes deployment", prefix: "GLOBAL_", deploymentName: "camera-1"},
|
||||
}
|
||||
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
t.Setenv("DEPLOYMENT_NAME", test.deploymentName)
|
||||
t.Setenv(test.prefix+"AGENT_NAME", test.agentName)
|
||||
if test.agentName == "" {
|
||||
if err := os.Unsetenv(test.prefix + "AGENT_NAME"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
configuration := &models.Configuration{Config: models.Config{
|
||||
Name: test.initialName,
|
||||
FriendlyName: test.initialDisplay,
|
||||
}}
|
||||
initConfigPointers(&configuration.Config)
|
||||
|
||||
applyAgentEnvVars(configuration, test.prefix, false)
|
||||
|
||||
if got := configuration.Config.Name; got != test.wantName {
|
||||
t.Errorf("Name = %q, want %q", got, test.wantName)
|
||||
}
|
||||
if got := configuration.Config.FriendlyName; got != test.wantDisplay {
|
||||
t.Errorf("FriendlyName = %q, want %q", got, test.wantDisplay)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestApplyAgentEnvVarsPixelChangeThresholdDefault(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
@@ -88,22 +40,6 @@ func TestApplyAgentEnvVarsPixelChangeThresholdDefault(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestApplyAgentEnvVarsVaultCustomHeaders(t *testing.T) {
|
||||
t.Setenv("AGENT_KERBEROSVAULT_CUSTOM_HEADERS", `{"site_id":"site-1"}`)
|
||||
t.Setenv("AGENT_KERBEROSVAULT_SECONDARY_CUSTOM_HEADERS", `{"site_id":"site-2"}`)
|
||||
configuration := &models.Configuration{}
|
||||
initConfigPointers(&configuration.Config)
|
||||
|
||||
applyAgentEnvVars(configuration, "", false)
|
||||
|
||||
if got := configuration.Config.KStorage.CustomHeaders; got != `{"site_id":"site-1"}` {
|
||||
t.Fatalf("primary custom headers = %q", got)
|
||||
}
|
||||
if got := configuration.Config.KStorageSecondary.CustomHeaders; got != `{"site_id":"site-2"}` {
|
||||
t.Fatalf("secondary custom headers = %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func intPointer(value int) *int {
|
||||
return &value
|
||||
}
|
||||
|
||||
@@ -187,7 +187,6 @@ type KStorage struct {
|
||||
SecretAccessKey string `json:"secret_access_key,omitempty" bson:"secret_access_key,omitempty"`
|
||||
Provider string `json:"provider,omitempty" bson:"provider,omitempty"`
|
||||
Directory string `json:"directory,omitempty" bson:"directory,omitempty"`
|
||||
CustomHeaders string `json:"custom_headers,omitempty" bson:"custom_headers,omitempty"`
|
||||
MaxRetries int `json:"max_retries,omitempty" bson:"max_retries,omitempty"`
|
||||
Timeout int `json:"timeout,omitempty" bson:"timeout,omitempty"`
|
||||
}
|
||||
|
||||
@@ -11,15 +11,12 @@ const RecordingUploadMetadataExtension = ".metadata"
|
||||
// with a recording. New optional fields can be added without changing the queue
|
||||
// mechanism or breaking older agents.
|
||||
type RecordingUploadMetadata struct {
|
||||
FileName string `json:"filename"`
|
||||
DeviceKey string `json:"device_key"`
|
||||
Timestamp int64 `json:"timestamp"` // Unix milliseconds.
|
||||
Duration uint64 `json:"duration"` // Milliseconds.
|
||||
FPS float64 `json:"fps,omitempty"`
|
||||
Encrypted bool `json:"encrypted,omitempty"`
|
||||
DeviceName string `json:"device_name,omitempty"`
|
||||
RegionCoordinates string `json:"region_coordinates,omitempty"`
|
||||
NumberOfChanges string `json:"number_of_changes,omitempty"`
|
||||
FileName string `json:"filename"`
|
||||
DeviceKey string `json:"device_key"`
|
||||
Timestamp int64 `json:"timestamp"` // Unix milliseconds.
|
||||
Duration uint64 `json:"duration"` // Milliseconds.
|
||||
FPS float64 `json:"fps,omitempty"`
|
||||
Encrypted bool `json:"encrypted,omitempty"`
|
||||
}
|
||||
|
||||
// RecordingUploadMetadataFileName returns the queue marker name associated
|
||||
|
||||
+243
-326
@@ -5,9 +5,7 @@ import (
|
||||
"encoding/json"
|
||||
"encoding/xml"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"math"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
@@ -132,7 +130,7 @@ func HandleONVIFActions(configuration *models.Configuration, communication *mode
|
||||
|
||||
if onvifAction.Action == "absolute-move" {
|
||||
|
||||
// Move the camera to the saved absolute position.
|
||||
// We will move the camera to zero position.
|
||||
x := ptzAction.X
|
||||
y := ptzAction.Y
|
||||
z := ptzAction.Z
|
||||
@@ -143,13 +141,42 @@ func HandleONVIFActions(configuration *models.Configuration, communication *mode
|
||||
// Log functions
|
||||
log.Debug("onvif.HandleONVIFActions(): functions: " + strings.Join(functions, ", "))
|
||||
|
||||
err = MoveToPosition(device, configurations, token, x, y, z)
|
||||
// Check if we need to use absolute or continuous move
|
||||
/*canAbsoluteMove := false
|
||||
canContinuousMove := false
|
||||
|
||||
if len(functions) > 0 {
|
||||
for _, function := range functions {
|
||||
if function == "AbsolutePanTiltMove" || function == "AbsoluteZoomMove" {
|
||||
canAbsoluteMove = true
|
||||
} else if function == "ContinuousPanTiltMove" || function == "ContinuousZoomMove" {
|
||||
canContinuousMove = true
|
||||
}
|
||||
}
|
||||
}*/
|
||||
|
||||
// Ideally we should be able to use the AbsolutePanTiltMove function, but it looks like
|
||||
// the current detection through GetPTZFuntionsFromDevice is not working properly. Therefore we will fallback
|
||||
// on the ContinuousPanTiltMove function which is more compatible with more cameras.
|
||||
err = AbsolutePanTiltMoveFake(device, configurations, token, x, y, z)
|
||||
if err != nil {
|
||||
log.Error("onvif.HandleONVIFActions() - MoveToPosition: " + err.Error())
|
||||
log.Debug("onvif.HandleONVIFActions() - AbsolutePanTitleMoveFake: " + err.Error())
|
||||
} else {
|
||||
log.Info("onvif.HandleONVIFActions() - MoveToPosition: successfully moved camera.")
|
||||
log.Info("onvif.HandleONVIFActions() - AbsolutePanTitleMoveFake: successfully moved camera.")
|
||||
}
|
||||
|
||||
/*if canAbsoluteMove {
|
||||
err = AbsolutePanTiltMove(device, configurations, token, x, y, z)
|
||||
if err != nil {
|
||||
log.Error("HandleONVIFActions (AbsolutePanTitleMove): " + err.Error())
|
||||
}
|
||||
} else if canContinuousMove {
|
||||
err = AbsolutePanTiltMoveFake(device, configurations, token, x, y, z)
|
||||
if err != nil {
|
||||
log.Error("HandleONVIFActions (AbsolutePanTitleMoveFake): " + err.Error())
|
||||
}
|
||||
}*/
|
||||
|
||||
} else if onvifAction.Action == "preset" {
|
||||
|
||||
// Execute the preset
|
||||
@@ -444,20 +471,16 @@ func handleONVIFResponse(operation string, response *http.Response, requestErr e
|
||||
}
|
||||
|
||||
func AbsolutePanTiltMove(device *onvif.Device, configuration ptz.GetConfigurationsResponse, token xsdonvif.ReferenceToken, pan float64, tilt float64, zoom float64) error {
|
||||
if len(configuration.PTZConfiguration) == 0 {
|
||||
return errors.New("camera returned no PTZ configuration")
|
||||
}
|
||||
ptzConfiguration := configuration.PTZConfiguration[0]
|
||||
|
||||
absolutePantiltVector := xsdonvif.Vector2D{
|
||||
X: pan,
|
||||
Y: tilt,
|
||||
Space: ptzConfiguration.DefaultAbsolutePantTiltPositionSpace,
|
||||
Space: configuration.PTZConfiguration[0].DefaultAbsolutePantTiltPositionSpace,
|
||||
}
|
||||
|
||||
absoluteZoomVector := xsdonvif.Vector1D{
|
||||
X: zoom,
|
||||
Space: ptzConfiguration.DefaultAbsoluteZoomPositionSpace,
|
||||
Space: configuration.PTZConfiguration[0].DefaultAbsoluteZoomPositionSpace,
|
||||
}
|
||||
|
||||
resp, err := device.CallMethod(ptz.AbsoluteMove{
|
||||
@@ -470,325 +493,240 @@ func AbsolutePanTiltMove(device *onvif.Device, configuration ptz.GetConfiguratio
|
||||
return handleONVIFResponse("absolute_move", resp, err)
|
||||
}
|
||||
|
||||
type ptzMoveClient interface {
|
||||
GetPosition() (xsdonvif.PTZVector, error)
|
||||
AbsoluteMove(ptzTarget) error
|
||||
ContinuousMove(ptzVelocity) error
|
||||
Stop() error
|
||||
}
|
||||
// This function will simulate the AbsolutePanTiltMove function.
|
||||
// However the AboslutePanTiltMove function is not working on all cameras.
|
||||
// So we'll use the ContinuousMove function to simulate the AbsolutePanTiltMove function using the position polling.
|
||||
func AbsolutePanTiltMoveFake(device *onvif.Device, configuration ptz.GetConfigurationsResponse, token xsdonvif.ReferenceToken, pan float64, tilt float64, zoom float64) error {
|
||||
position, err := GetPosition(device, token)
|
||||
if position.PanTilt.X >= pan-0.01 && position.PanTilt.X <= pan+0.01 && position.PanTilt.Y >= tilt-0.01 && position.PanTilt.Y <= tilt+0.01 && position.Zoom.X >= zoom-0.01 && position.Zoom.X <= zoom+0.01 {
|
||||
log.Debug("AbsolutePanTiltMoveFake: already at position")
|
||||
} else {
|
||||
|
||||
type ptzTarget struct {
|
||||
Pan float64
|
||||
Tilt float64
|
||||
Zoom float64
|
||||
}
|
||||
// The speed of panning, the higher the faster we'll pan the camera
|
||||
// value is a range between 0 and 1.
|
||||
speed := 0.6
|
||||
wait := 100 * time.Millisecond
|
||||
|
||||
type ptzVelocity struct {
|
||||
Pan float64
|
||||
Tilt float64
|
||||
Zoom float64
|
||||
}
|
||||
// We'll move quickly to the position (might be inaccurate)
|
||||
err = ZoomOutCompletely(device, configuration, token)
|
||||
err = PanUntilPosition(device, configuration, token, pan, zoom, speed, wait)
|
||||
err = TiltUntilPosition(device, configuration, token, tilt, zoom, speed, wait)
|
||||
|
||||
type ptzMoveSupport struct {
|
||||
Absolute bool
|
||||
Continuous bool
|
||||
}
|
||||
// Now we'll move a bit slower to make sure we are ok (will be more accurate)
|
||||
speed = 0.1
|
||||
wait = 200 * time.Millisecond
|
||||
|
||||
type ptzMoveOptions struct {
|
||||
PanTolerance float64
|
||||
TiltTolerance float64
|
||||
ZoomTolerance float64
|
||||
NativeTimeout time.Duration
|
||||
ContinuousTimeout time.Duration
|
||||
NoProgressTimeout time.Duration
|
||||
PollInterval time.Duration
|
||||
ProgressEpsilon float64
|
||||
VelocityGain float64
|
||||
MinVelocity float64
|
||||
MaxVelocity float64
|
||||
VelocityChange float64
|
||||
}
|
||||
err = PanUntilPosition(device, configuration, token, pan, zoom, speed, wait)
|
||||
err = TiltUntilPosition(device, configuration, token, tilt, zoom, speed, wait)
|
||||
err = ZoomUntilPosition(device, configuration, token, zoom, speed, wait)
|
||||
|
||||
var defaultPTZMoveOptions = ptzMoveOptions{
|
||||
PanTolerance: 0.01,
|
||||
TiltTolerance: 0.005,
|
||||
ZoomTolerance: 0.005,
|
||||
NativeTimeout: 8 * time.Second,
|
||||
ContinuousTimeout: 15 * time.Second,
|
||||
NoProgressTimeout: 3 * time.Second,
|
||||
PollInterval: 150 * time.Millisecond,
|
||||
ProgressEpsilon: 0.001,
|
||||
VelocityGain: 1.5,
|
||||
MinVelocity: 0.08,
|
||||
MaxVelocity: 0.7,
|
||||
VelocityChange: 0.05,
|
||||
}
|
||||
|
||||
type devicePTZMoveClient struct {
|
||||
device *onvif.Device
|
||||
configuration xsdonvif.PTZConfiguration
|
||||
token xsdonvif.ReferenceToken
|
||||
}
|
||||
|
||||
func (client *devicePTZMoveClient) GetPosition() (xsdonvif.PTZVector, error) {
|
||||
return GetPosition(client.device, client.token)
|
||||
}
|
||||
|
||||
func (client *devicePTZMoveClient) AbsoluteMove(target ptzTarget) error {
|
||||
configuration := ptz.GetConfigurationsResponse{
|
||||
PTZConfiguration: []xsdonvif.PTZConfiguration{client.configuration},
|
||||
return err
|
||||
}
|
||||
return AbsolutePanTiltMove(client.device, configuration, client.token, target.Pan, target.Tilt, target.Zoom)
|
||||
return err
|
||||
}
|
||||
|
||||
func (client *devicePTZMoveClient) ContinuousMove(velocity ptzVelocity) error {
|
||||
panTilt := xsdonvif.Vector2D{
|
||||
X: velocity.Pan,
|
||||
Y: velocity.Tilt,
|
||||
Space: client.configuration.DefaultContinuousPanTiltVelocitySpace,
|
||||
func ZoomOutCompletely(device *onvif.Device, configuration ptz.GetConfigurationsResponse, token xsdonvif.ReferenceToken) error {
|
||||
// Zoom out completely!!!
|
||||
zoomOut := xsdonvif.Vector1D{
|
||||
X: -1,
|
||||
Space: configuration.PTZConfiguration[0].DefaultContinuousZoomVelocitySpace,
|
||||
}
|
||||
zoom := xsdonvif.Vector1D{
|
||||
X: velocity.Zoom,
|
||||
Space: client.configuration.DefaultContinuousZoomVelocitySpace,
|
||||
}
|
||||
resp, err := client.device.CallMethod(ptz.ContinuousMove{
|
||||
ProfileToken: &client.token,
|
||||
Velocity: ptz.Speed{
|
||||
PanTilt: &panTilt,
|
||||
Zoom: &zoom,
|
||||
_, err := device.CallMethod(ptz.ContinuousMove{
|
||||
ProfileToken: &token,
|
||||
Velocity: xsdonvif.PTZSpeedZoom{
|
||||
Zoom: zoomOut,
|
||||
},
|
||||
})
|
||||
return handleONVIFResponse("continuous_move_to_position", resp, err)
|
||||
}
|
||||
if err != nil {
|
||||
log.Error("ZoomOutCompletely: " + err.Error())
|
||||
}
|
||||
|
||||
func (client *devicePTZMoveClient) Stop() error {
|
||||
resp, err := client.device.CallMethod(ptz.Stop{
|
||||
ProfileToken: client.token,
|
||||
PanTilt: true,
|
||||
for {
|
||||
position, _ := GetPosition(device, token)
|
||||
if position.Zoom.X == 0 {
|
||||
break
|
||||
}
|
||||
time.Sleep(250 * time.Millisecond)
|
||||
}
|
||||
|
||||
_, err = device.CallMethod(ptz.Stop{
|
||||
ProfileToken: token,
|
||||
Zoom: true,
|
||||
})
|
||||
return handleONVIFResponse("move_to_position_stop", resp, err)
|
||||
}
|
||||
|
||||
func MoveToPosition(device *onvif.Device, configuration ptz.GetConfigurationsResponse, token xsdonvif.ReferenceToken, pan float64, tilt float64, zoom float64) error {
|
||||
if len(configuration.PTZConfiguration) == 0 {
|
||||
return errors.New("camera returned no PTZ configuration")
|
||||
}
|
||||
ptzConfiguration := configuration.PTZConfiguration[0]
|
||||
client := &devicePTZMoveClient{
|
||||
device: device,
|
||||
configuration: ptzConfiguration,
|
||||
token: token,
|
||||
}
|
||||
support := ptzMoveSupport{
|
||||
Absolute: ptzConfiguration.DefaultAbsolutePantTiltPositionSpace != nil &&
|
||||
ptzConfiguration.DefaultAbsoluteZoomPositionSpace != nil,
|
||||
Continuous: ptzConfiguration.DefaultContinuousPanTiltVelocitySpace != nil &&
|
||||
ptzConfiguration.DefaultContinuousZoomVelocitySpace != nil,
|
||||
}
|
||||
return moveToPosition(client, support, ptzTarget{Pan: pan, Tilt: tilt, Zoom: zoom}, defaultPTZMoveOptions)
|
||||
}
|
||||
|
||||
// AbsolutePanTiltMoveFake preserves the continuous-move compatibility path for callers that explicitly request it.
|
||||
func AbsolutePanTiltMoveFake(device *onvif.Device, configuration ptz.GetConfigurationsResponse, token xsdonvif.ReferenceToken, pan float64, tilt float64, zoom float64) error {
|
||||
if len(configuration.PTZConfiguration) == 0 {
|
||||
return errors.New("camera returned no PTZ configuration")
|
||||
}
|
||||
ptzConfiguration := configuration.PTZConfiguration[0]
|
||||
client := &devicePTZMoveClient{
|
||||
device: device,
|
||||
configuration: ptzConfiguration,
|
||||
token: token,
|
||||
}
|
||||
support := ptzMoveSupport{
|
||||
Continuous: ptzConfiguration.DefaultContinuousPanTiltVelocitySpace != nil &&
|
||||
ptzConfiguration.DefaultContinuousZoomVelocitySpace != nil,
|
||||
}
|
||||
return moveToPosition(client, support, ptzTarget{Pan: pan, Tilt: tilt, Zoom: zoom}, defaultPTZMoveOptions)
|
||||
}
|
||||
|
||||
func moveToPosition(client ptzMoveClient, support ptzMoveSupport, target ptzTarget, options ptzMoveOptions) error {
|
||||
if err := validatePTZTarget(target); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
position, err := client.GetPosition()
|
||||
if err != nil {
|
||||
return fmt.Errorf("read initial PTZ position: %w", err)
|
||||
log.Error("ZoomOutCompletely: " + err.Error())
|
||||
}
|
||||
_, _, reached, err := ptzTargetError(position, target, options)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if reached {
|
||||
return nil
|
||||
}
|
||||
|
||||
var nativeErr error
|
||||
if support.Absolute {
|
||||
if err := client.AbsoluteMove(target); err != nil {
|
||||
nativeErr = fmt.Errorf("native absolute move: %w", err)
|
||||
} else if err := waitForPTZTarget(client, target, options.NativeTimeout, options); err != nil {
|
||||
nativeErr = fmt.Errorf("verify native absolute move: %w", err)
|
||||
} else {
|
||||
return nil
|
||||
}
|
||||
|
||||
if err := client.Stop(); err != nil {
|
||||
nativeErr = errors.Join(nativeErr, fmt.Errorf("stop native absolute move: %w", err))
|
||||
}
|
||||
log.WithError(nativeErr).Warn("Native ONVIF absolute move failed; using continuous-move fallback")
|
||||
}
|
||||
|
||||
if !support.Continuous {
|
||||
if nativeErr != nil {
|
||||
return errors.Join(nativeErr, errors.New("camera does not advertise complete continuous PTZ support"))
|
||||
}
|
||||
return errors.New("camera does not advertise a supported absolute or continuous PTZ movement method")
|
||||
}
|
||||
|
||||
if err := moveContinuouslyToPTZTarget(client, target, options); err != nil {
|
||||
if nativeErr != nil {
|
||||
return errors.Join(nativeErr, fmt.Errorf("continuous-move fallback: %w", err))
|
||||
}
|
||||
return fmt.Errorf("continuous move: %w", err)
|
||||
}
|
||||
return nil
|
||||
return err
|
||||
}
|
||||
|
||||
func waitForPTZTarget(client ptzMoveClient, target ptzTarget, timeout time.Duration, options ptzMoveOptions) error {
|
||||
deadline := time.Now().Add(timeout)
|
||||
lastProgress := time.Now()
|
||||
bestDistance := math.Inf(1)
|
||||
func PanUntilPosition(device *onvif.Device, configuration ptz.GetConfigurationsResponse, token xsdonvif.ReferenceToken, pan float64, zoom float64, speed float64, wait time.Duration) error {
|
||||
position, err := GetPosition(device, token)
|
||||
|
||||
for {
|
||||
position, err := client.GetPosition()
|
||||
if err != nil {
|
||||
return fmt.Errorf("read PTZ position: %w", err)
|
||||
}
|
||||
_, distance, reached, err := ptzTargetError(position, target, options)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if reached {
|
||||
return nil
|
||||
if position.PanTilt.X >= pan-0.01 && position.PanTilt.X <= pan+0.01 {
|
||||
|
||||
} else {
|
||||
|
||||
// We'll need to determine if we need to move CW or CCW.
|
||||
// Check the current position and compare it with the desired position.
|
||||
directionX := speed
|
||||
if position.PanTilt.X > pan {
|
||||
directionX = speed * -1
|
||||
}
|
||||
|
||||
panTiltVector := xsdonvif.Vector2D{
|
||||
X: directionX,
|
||||
Y: 0,
|
||||
Space: configuration.PTZConfiguration[0].DefaultContinuousPanTiltVelocitySpace,
|
||||
}
|
||||
resp, err := device.CallMethod(ptz.ContinuousMove{
|
||||
ProfileToken: &token,
|
||||
Velocity: xsdonvif.PTZSpeedPanTilt{
|
||||
PanTilt: panTiltVector,
|
||||
},
|
||||
})
|
||||
err = handleONVIFResponse("continuous_pan", resp, err)
|
||||
|
||||
// While moving we'll check if we reached the desired position.
|
||||
// or if we overshot the desired position.
|
||||
|
||||
// Break after 3seconds
|
||||
now := time.Now()
|
||||
if bestDistance-distance >= options.ProgressEpsilon {
|
||||
bestDistance = distance
|
||||
lastProgress = now
|
||||
}
|
||||
if !now.Before(deadline) {
|
||||
return fmt.Errorf("target not reached within %s", timeout)
|
||||
}
|
||||
if now.Sub(lastProgress) >= options.NoProgressTimeout {
|
||||
return fmt.Errorf("PTZ position made no progress for %s", options.NoProgressTimeout)
|
||||
}
|
||||
time.Sleep(options.PollInterval)
|
||||
}
|
||||
}
|
||||
|
||||
func moveContinuouslyToPTZTarget(client ptzMoveClient, target ptzTarget, options ptzMoveOptions) (err error) {
|
||||
defer func() {
|
||||
if stopErr := client.Stop(); stopErr != nil {
|
||||
err = errors.Join(err, fmt.Errorf("stop continuous move: %w", stopErr))
|
||||
}
|
||||
}()
|
||||
|
||||
deadline := time.Now().Add(options.ContinuousTimeout)
|
||||
lastProgress := time.Now()
|
||||
bestDistance := math.Inf(1)
|
||||
var lastVelocity ptzVelocity
|
||||
haveVelocity := false
|
||||
|
||||
for {
|
||||
position, positionErr := client.GetPosition()
|
||||
if positionErr != nil {
|
||||
return fmt.Errorf("read PTZ position: %w", positionErr)
|
||||
}
|
||||
delta, distance, reached, targetErr := ptzTargetError(position, target, options)
|
||||
if targetErr != nil {
|
||||
return targetErr
|
||||
}
|
||||
if reached {
|
||||
return nil
|
||||
}
|
||||
|
||||
now := time.Now()
|
||||
if bestDistance-distance >= options.ProgressEpsilon {
|
||||
bestDistance = distance
|
||||
lastProgress = now
|
||||
}
|
||||
if !now.Before(deadline) {
|
||||
return fmt.Errorf("target not reached within %s", options.ContinuousTimeout)
|
||||
}
|
||||
if now.Sub(lastProgress) >= options.NoProgressTimeout {
|
||||
return fmt.Errorf("PTZ position made no progress for %s", options.NoProgressTimeout)
|
||||
}
|
||||
|
||||
velocity := ptzVelocity{
|
||||
Pan: proportionalPTZVelocity(delta.Pan, options.PanTolerance, options),
|
||||
Tilt: proportionalPTZVelocity(delta.Tilt, options.TiltTolerance, options),
|
||||
Zoom: proportionalPTZVelocity(delta.Zoom, options.ZoomTolerance, options),
|
||||
}
|
||||
if !haveVelocity || ptzVelocityChanged(lastVelocity, velocity, options.VelocityChange) {
|
||||
if moveErr := client.ContinuousMove(velocity); moveErr != nil {
|
||||
return fmt.Errorf("set continuous PTZ velocity: %w", moveErr)
|
||||
for {
|
||||
position, _ := GetPosition(device, token)
|
||||
if position.PanTilt.X == -1 || position.PanTilt.X == 1 || (directionX > 0 && position.PanTilt.X >= pan) || (directionX < 0 && position.PanTilt.X <= pan) || (position.PanTilt.X >= pan-0.01 && position.PanTilt.X <= pan+0.01) {
|
||||
break
|
||||
}
|
||||
lastVelocity = velocity
|
||||
haveVelocity = true
|
||||
if time.Since(now) > 3*time.Second {
|
||||
break
|
||||
}
|
||||
time.Sleep(wait)
|
||||
}
|
||||
time.Sleep(options.PollInterval)
|
||||
}
|
||||
}
|
||||
|
||||
func ptzTargetError(position xsdonvif.PTZVector, target ptzTarget, options ptzMoveOptions) (ptzVelocity, float64, bool, error) {
|
||||
if position.PanTilt == nil || position.Zoom == nil {
|
||||
return ptzVelocity{}, 0, false, errors.New("camera returned an incomplete PTZ position")
|
||||
}
|
||||
delta := ptzVelocity{
|
||||
Pan: target.Pan - position.PanTilt.X,
|
||||
Tilt: target.Tilt - position.PanTilt.Y,
|
||||
Zoom: target.Zoom - position.Zoom.X,
|
||||
}
|
||||
reached := math.Abs(delta.Pan) <= options.PanTolerance &&
|
||||
math.Abs(delta.Tilt) <= options.TiltTolerance &&
|
||||
math.Abs(delta.Zoom) <= options.ZoomTolerance
|
||||
distance := math.Abs(delta.Pan) + math.Abs(delta.Tilt) + math.Abs(delta.Zoom)
|
||||
return delta, distance, reached, nil
|
||||
}
|
||||
_, err = device.CallMethod(ptz.Stop{
|
||||
ProfileToken: token,
|
||||
PanTilt: true,
|
||||
Zoom: true,
|
||||
})
|
||||
|
||||
func proportionalPTZVelocity(delta float64, tolerance float64, options ptzMoveOptions) float64 {
|
||||
if math.Abs(delta) <= tolerance {
|
||||
return 0
|
||||
}
|
||||
speed := math.Abs(delta) * options.VelocityGain
|
||||
speed = math.Max(options.MinVelocity, math.Min(speed, options.MaxVelocity))
|
||||
return math.Copysign(speed, delta)
|
||||
}
|
||||
|
||||
func ptzVelocityChanged(previous ptzVelocity, next ptzVelocity, threshold float64) bool {
|
||||
return math.Abs(previous.Pan-next.Pan) >= threshold ||
|
||||
math.Abs(previous.Tilt-next.Tilt) >= threshold ||
|
||||
math.Abs(previous.Zoom-next.Zoom) >= threshold
|
||||
}
|
||||
|
||||
func validatePTZTarget(target ptzTarget) error {
|
||||
values := []struct {
|
||||
name string
|
||||
value float64
|
||||
}{
|
||||
{name: "pan", value: target.Pan},
|
||||
{name: "tilt", value: target.Tilt},
|
||||
{name: "zoom", value: target.Zoom},
|
||||
}
|
||||
for _, value := range values {
|
||||
if math.IsNaN(value.value) || math.IsInf(value.value, 0) {
|
||||
return fmt.Errorf("invalid PTZ %s target", value.name)
|
||||
if err != nil {
|
||||
log.Error("ContinuousPanTiltMove (Pan): " + err.Error())
|
||||
}
|
||||
}
|
||||
return nil
|
||||
return err
|
||||
}
|
||||
|
||||
func TiltUntilPosition(device *onvif.Device, configuration ptz.GetConfigurationsResponse, token xsdonvif.ReferenceToken, tilt float64, zoom float64, speed float64, wait time.Duration) error {
|
||||
position, err := GetPosition(device, token)
|
||||
|
||||
if position.PanTilt.Y >= tilt-0.005 && position.PanTilt.Y <= tilt+0.005 {
|
||||
|
||||
} else {
|
||||
|
||||
// We'll need to determine if we need to move CW or CCW.
|
||||
// Check the current position and compare it with the desired position.
|
||||
directionY := speed
|
||||
if position.PanTilt.Y > tilt {
|
||||
directionY = speed * -1
|
||||
}
|
||||
|
||||
panTiltVector := xsdonvif.Vector2D{
|
||||
X: 0,
|
||||
Y: directionY,
|
||||
Space: configuration.PTZConfiguration[0].DefaultContinuousPanTiltVelocitySpace,
|
||||
}
|
||||
|
||||
velocity := xsdonvif.PTZSpeedPanTilt{
|
||||
PanTilt: panTiltVector,
|
||||
}
|
||||
|
||||
resp, err := device.CallMethod(ptz.ContinuousMove{
|
||||
ProfileToken: &token,
|
||||
Velocity: velocity,
|
||||
})
|
||||
err = handleONVIFResponse("continuous_tilt", resp, err)
|
||||
|
||||
// While moving we'll check if we reached the desired position.
|
||||
// or if we overshot the desired position.
|
||||
|
||||
// Break after 3seconds
|
||||
now := time.Now()
|
||||
for {
|
||||
position, _ := GetPosition(device, token)
|
||||
if position.PanTilt.Y == -1 || position.PanTilt.Y == 1 || (directionY > 0 && position.PanTilt.Y >= tilt) || (directionY < 0 && position.PanTilt.Y <= tilt) || (position.PanTilt.Y >= tilt-0.005 && position.PanTilt.Y <= tilt+0.005) {
|
||||
break
|
||||
}
|
||||
if time.Since(now) > 3*time.Second {
|
||||
break
|
||||
}
|
||||
time.Sleep(wait)
|
||||
}
|
||||
|
||||
_, err = device.CallMethod(ptz.Stop{
|
||||
ProfileToken: token,
|
||||
PanTilt: true,
|
||||
Zoom: true,
|
||||
})
|
||||
|
||||
if err != nil {
|
||||
log.Error("ContinuousPanTiltMove (Tilt): " + err.Error())
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func ZoomUntilPosition(device *onvif.Device, configuration ptz.GetConfigurationsResponse, token xsdonvif.ReferenceToken, zoom float64, speed float64, wait time.Duration) error {
|
||||
position, err := GetPosition(device, token)
|
||||
|
||||
if position.Zoom.X >= zoom-0.005 && position.Zoom.X <= zoom+0.005 {
|
||||
|
||||
} else {
|
||||
|
||||
// We'll need to determine if we need to move CW or CCW.
|
||||
// Check the current position and compare it with the desired position.
|
||||
directionZ := speed
|
||||
if position.Zoom.X > zoom {
|
||||
directionZ = speed * -1
|
||||
}
|
||||
|
||||
zoomVector := xsdonvif.Vector1D{
|
||||
X: directionZ,
|
||||
Space: configuration.PTZConfiguration[0].DefaultContinuousZoomVelocitySpace,
|
||||
}
|
||||
resp, err := device.CallMethod(ptz.ContinuousMove{
|
||||
ProfileToken: &token,
|
||||
Velocity: xsdonvif.PTZSpeedZoom{
|
||||
Zoom: zoomVector,
|
||||
},
|
||||
})
|
||||
err = handleONVIFResponse("continuous_zoom_to_position", resp, err)
|
||||
|
||||
// While moving we'll check if we reached the desired position.
|
||||
// or if we overshot the desired position.
|
||||
|
||||
// Break after 3seconds
|
||||
now := time.Now()
|
||||
for {
|
||||
position, _ := GetPosition(device, token)
|
||||
if position.Zoom.X == -1 || position.Zoom.X == 1 || (directionZ > 0 && position.Zoom.X >= zoom) || (directionZ < 0 && position.Zoom.X <= zoom) || (position.Zoom.X >= zoom-0.005 && position.Zoom.X <= zoom+0.005) {
|
||||
break
|
||||
}
|
||||
if time.Since(now) > 3*time.Second {
|
||||
break
|
||||
}
|
||||
time.Sleep(wait)
|
||||
}
|
||||
|
||||
_, err = device.CallMethod(ptz.Stop{
|
||||
ProfileToken: token,
|
||||
PanTilt: true,
|
||||
Zoom: true,
|
||||
})
|
||||
|
||||
if err != nil {
|
||||
log.Error("ContinuousPanTiltMove (Zoom): " + err.Error())
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func ContinuousPanTilt(device *onvif.Device, configuration ptz.GetConfigurationsResponse, token xsdonvif.ReferenceToken, pan float64, tilt float64) error {
|
||||
@@ -957,10 +895,6 @@ func GetPTZFunctionsFromDevice(configurations ptz.GetConfigurationsResponse) ([]
|
||||
canZoom := false
|
||||
canPanTilt := false
|
||||
|
||||
if len(configurations.PTZConfiguration) == 0 {
|
||||
return functions, canZoom, canPanTilt
|
||||
}
|
||||
|
||||
if configurations.PTZConfiguration[0].DefaultAbsolutePantTiltPositionSpace != nil {
|
||||
functions = append(functions, "AbsolutePanTiltMove")
|
||||
canPanTilt = true
|
||||
@@ -1108,23 +1042,6 @@ func CreatePullPointSubscription(dev *onvif.Device) (string, error) {
|
||||
return pullPointAdress, err
|
||||
}
|
||||
|
||||
const (
|
||||
wsAddressingNamespace = "http://www.w3.org/2005/08/addressing"
|
||||
pullMessagesAction = "http://www.onvif.org/ver10/events/wsdl/PullPointSubscription/PullMessagesRequest"
|
||||
unsubscribeAction = "http://docs.oasis-open.org/wsn/bw-2/SubscriptionManager/UnsubscribeRequest"
|
||||
)
|
||||
|
||||
// subscriptionAddressingHeader builds the WS-Addressing Action and To headers
|
||||
// for a request sent to a pull-point subscription. Some devices identify the
|
||||
// subscription from wsa:To rather than the request URL, and reject requests
|
||||
// without it as NotAuthorized.
|
||||
func subscriptionAddressingHeader(subscriptionAddress string, action string) string {
|
||||
var address bytes.Buffer
|
||||
_ = xml.EscapeText(&address, []byte(subscriptionAddress))
|
||||
return `<wsa:Action xmlns:wsa="` + wsAddressingNamespace + `">` + action + `</wsa:Action>` +
|
||||
`<wsa:To xmlns:wsa="` + wsAddressingNamespace + `">` + address.String() + `</wsa:To>`
|
||||
}
|
||||
|
||||
func UnsubscribePullPoint(dev *onvif.Device, pullPointAddress string) error {
|
||||
|
||||
// Unsubscribe from the device
|
||||
@@ -1138,7 +1055,7 @@ func UnsubscribePullPoint(dev *onvif.Device, pullPointAddress string) error {
|
||||
return err
|
||||
}
|
||||
|
||||
res, err := dev.SendSoapWithHeader(pullPointAddress, string(requestBody), subscriptionAddressingHeader(pullPointAddress, unsubscribeAction))
|
||||
res, err := dev.SendSoap(pullPointAddress, string(requestBody))
|
||||
if err != nil {
|
||||
log.WithError(err).WithFields(log.Fields{
|
||||
"component": "onvif",
|
||||
@@ -1220,7 +1137,7 @@ func GetEventMessages(dev *onvif.Device, pullPointAddress string) ([]ONVIFEvents
|
||||
log.Error("onvif.main.GetEventMessages(pullMessages): " + err.Error())
|
||||
return eventsArray, err
|
||||
}
|
||||
res, err := dev.SendSoapWithHeader(string(subscriptionURI), string(requestBody), subscriptionAddressingHeader(subscriptionURI, pullMessagesAction))
|
||||
res, err := dev.SendSoap(string(subscriptionURI), string(requestBody))
|
||||
if err != nil {
|
||||
log.Error("onvif.main.GetEventMessages(pullMessages): " + err.Error())
|
||||
return eventsArray, err
|
||||
|
||||
@@ -1,264 +0,0 @@
|
||||
package onvif
|
||||
|
||||
import (
|
||||
"encoding/xml"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/kerberos-io/onvif/ptz"
|
||||
xsdonvif "github.com/kerberos-io/onvif/xsd/onvif"
|
||||
)
|
||||
|
||||
type fakePTZMoveClient struct {
|
||||
current ptzTarget
|
||||
target ptzTarget
|
||||
lastVelocity ptzVelocity
|
||||
absoluteErr error
|
||||
continuousErr error
|
||||
stopErr error
|
||||
ignoreAbsolute bool
|
||||
stalled bool
|
||||
incompletePosition bool
|
||||
positionErr error
|
||||
positionErrAtCall int
|
||||
positionCalls int
|
||||
absoluteCalls int
|
||||
continuousVelocities []ptzVelocity
|
||||
stopCalls int
|
||||
moving bool
|
||||
}
|
||||
|
||||
func (client *fakePTZMoveClient) GetPosition() (xsdonvif.PTZVector, error) {
|
||||
client.positionCalls++
|
||||
if client.positionErr != nil && client.positionCalls == client.positionErrAtCall {
|
||||
return xsdonvif.PTZVector{}, client.positionErr
|
||||
}
|
||||
if client.moving && !client.stalled {
|
||||
client.current.Pan = moveFakeAxis(client.current.Pan, client.target.Pan, client.lastVelocity.Pan)
|
||||
client.current.Tilt = moveFakeAxis(client.current.Tilt, client.target.Tilt, client.lastVelocity.Tilt)
|
||||
client.current.Zoom = moveFakeAxis(client.current.Zoom, client.target.Zoom, client.lastVelocity.Zoom)
|
||||
}
|
||||
|
||||
panTilt := &xsdonvif.Vector2D{X: client.current.Pan, Y: client.current.Tilt}
|
||||
if client.incompletePosition {
|
||||
return xsdonvif.PTZVector{PanTilt: panTilt}, nil
|
||||
}
|
||||
return xsdonvif.PTZVector{
|
||||
PanTilt: panTilt,
|
||||
Zoom: &xsdonvif.Vector1D{X: client.current.Zoom},
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (client *fakePTZMoveClient) AbsoluteMove(target ptzTarget) error {
|
||||
client.absoluteCalls++
|
||||
client.target = target
|
||||
if client.absoluteErr != nil {
|
||||
return client.absoluteErr
|
||||
}
|
||||
if !client.ignoreAbsolute {
|
||||
client.current = target
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (client *fakePTZMoveClient) ContinuousMove(velocity ptzVelocity) error {
|
||||
if client.continuousErr != nil {
|
||||
return client.continuousErr
|
||||
}
|
||||
client.lastVelocity = velocity
|
||||
client.continuousVelocities = append(client.continuousVelocities, velocity)
|
||||
client.moving = true
|
||||
return nil
|
||||
}
|
||||
|
||||
func (client *fakePTZMoveClient) Stop() error {
|
||||
client.stopCalls++
|
||||
client.moving = false
|
||||
return client.stopErr
|
||||
}
|
||||
|
||||
func moveFakeAxis(current float64, target float64, velocity float64) float64 {
|
||||
step := velocity * 0.5
|
||||
if step == 0 {
|
||||
return current
|
||||
}
|
||||
next := current + step
|
||||
if (velocity > 0 && next >= target) || (velocity < 0 && next <= target) {
|
||||
return target
|
||||
}
|
||||
return next
|
||||
}
|
||||
|
||||
func testPTZMoveOptions() ptzMoveOptions {
|
||||
options := defaultPTZMoveOptions
|
||||
options.NativeTimeout = 50 * time.Millisecond
|
||||
options.ContinuousTimeout = 100 * time.Millisecond
|
||||
options.NoProgressTimeout = 10 * time.Millisecond
|
||||
options.PollInterval = time.Millisecond
|
||||
options.ProgressEpsilon = 0.0001
|
||||
options.VelocityChange = 0.01
|
||||
return options
|
||||
}
|
||||
|
||||
func TestMoveToPositionUsesVerifiedNativeMove(t *testing.T) {
|
||||
target := ptzTarget{Pan: 0.5, Tilt: -0.25, Zoom: 0.4}
|
||||
client := &fakePTZMoveClient{}
|
||||
|
||||
err := moveToPosition(client, ptzMoveSupport{Absolute: true, Continuous: true}, target, testPTZMoveOptions())
|
||||
if err != nil {
|
||||
t.Fatalf("moveToPosition() error = %v", err)
|
||||
}
|
||||
if client.absoluteCalls != 1 {
|
||||
t.Fatalf("absolute calls = %d, want 1", client.absoluteCalls)
|
||||
}
|
||||
if len(client.continuousVelocities) != 0 {
|
||||
t.Fatalf("continuous calls = %d, want 0", len(client.continuousVelocities))
|
||||
}
|
||||
}
|
||||
|
||||
func TestMoveToPositionFallsBackToSimultaneousContinuousMove(t *testing.T) {
|
||||
target := ptzTarget{Pan: 0.5, Tilt: -0.25, Zoom: 0.4}
|
||||
client := &fakePTZMoveClient{
|
||||
target: target,
|
||||
absoluteErr: errors.New("absolute move rejected"),
|
||||
}
|
||||
|
||||
err := moveToPosition(client, ptzMoveSupport{Absolute: true, Continuous: true}, target, testPTZMoveOptions())
|
||||
if err != nil {
|
||||
t.Fatalf("moveToPosition() error = %v", err)
|
||||
}
|
||||
if len(client.continuousVelocities) == 0 {
|
||||
t.Fatal("continuous fallback was not used")
|
||||
}
|
||||
first := client.continuousVelocities[0]
|
||||
if first.Pan == 0 || first.Tilt == 0 || first.Zoom == 0 {
|
||||
t.Fatalf("first fallback velocity = %+v, want all axes moving", first)
|
||||
}
|
||||
if client.stopCalls != 2 {
|
||||
t.Fatalf("stop calls = %d, want native cleanup and fallback cleanup", client.stopCalls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMoveToPositionFallsBackWhenNativeMoveMakesNoProgress(t *testing.T) {
|
||||
target := ptzTarget{Pan: 0.3, Tilt: 0.2, Zoom: 0.1}
|
||||
client := &fakePTZMoveClient{
|
||||
target: target,
|
||||
ignoreAbsolute: true,
|
||||
}
|
||||
options := testPTZMoveOptions()
|
||||
options.NoProgressTimeout = 3 * time.Millisecond
|
||||
|
||||
err := moveToPosition(client, ptzMoveSupport{Absolute: true, Continuous: true}, target, options)
|
||||
if err != nil {
|
||||
t.Fatalf("moveToPosition() error = %v", err)
|
||||
}
|
||||
if len(client.continuousVelocities) == 0 {
|
||||
t.Fatal("continuous fallback was not used")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMoveToPositionStopsAfterContinuousStatusError(t *testing.T) {
|
||||
target := ptzTarget{Pan: 0.3, Tilt: 0.2, Zoom: 0.1}
|
||||
client := &fakePTZMoveClient{
|
||||
target: target,
|
||||
positionErr: errors.New("status unavailable"),
|
||||
positionErrAtCall: 3,
|
||||
}
|
||||
|
||||
err := moveToPosition(client, ptzMoveSupport{Continuous: true}, target, testPTZMoveOptions())
|
||||
if err == nil || !strings.Contains(err.Error(), "status unavailable") {
|
||||
t.Fatalf("moveToPosition() error = %v, want status error", err)
|
||||
}
|
||||
if client.stopCalls != 1 {
|
||||
t.Fatalf("stop calls = %d, want 1", client.stopCalls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMoveToPositionStopsWhenContinuousMoveIsRejected(t *testing.T) {
|
||||
target := ptzTarget{Pan: 0.3, Tilt: 0.2, Zoom: 0.1}
|
||||
client := &fakePTZMoveClient{
|
||||
target: target,
|
||||
continuousErr: errors.New("continuous move rejected"),
|
||||
}
|
||||
|
||||
err := moveToPosition(client, ptzMoveSupport{Continuous: true}, target, testPTZMoveOptions())
|
||||
if err == nil || !strings.Contains(err.Error(), "continuous move rejected") {
|
||||
t.Fatalf("moveToPosition() error = %v, want movement error", err)
|
||||
}
|
||||
if client.stopCalls != 1 {
|
||||
t.Fatalf("stop calls = %d, want 1", client.stopCalls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMoveToPositionReturnsNoProgressErrorAndStops(t *testing.T) {
|
||||
target := ptzTarget{Pan: 0.3, Tilt: 0.2, Zoom: 0.1}
|
||||
client := &fakePTZMoveClient{
|
||||
target: target,
|
||||
stalled: true,
|
||||
}
|
||||
options := testPTZMoveOptions()
|
||||
options.NoProgressTimeout = 3 * time.Millisecond
|
||||
|
||||
err := moveToPosition(client, ptzMoveSupport{Continuous: true}, target, options)
|
||||
if err == nil || !strings.Contains(err.Error(), "made no progress") {
|
||||
t.Fatalf("moveToPosition() error = %v, want no-progress error", err)
|
||||
}
|
||||
if client.stopCalls != 1 {
|
||||
t.Fatalf("stop calls = %d, want 1", client.stopCalls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMoveToPositionReturnsDeadlineErrorAndStops(t *testing.T) {
|
||||
target := ptzTarget{Pan: 100, Tilt: 100, Zoom: 100}
|
||||
client := &fakePTZMoveClient{target: target}
|
||||
options := testPTZMoveOptions()
|
||||
options.ContinuousTimeout = 3 * time.Millisecond
|
||||
options.NoProgressTimeout = 50 * time.Millisecond
|
||||
|
||||
err := moveToPosition(client, ptzMoveSupport{Continuous: true}, target, options)
|
||||
if err == nil || !strings.Contains(err.Error(), "target not reached within") {
|
||||
t.Fatalf("moveToPosition() error = %v, want deadline error", err)
|
||||
}
|
||||
if client.stopCalls != 1 {
|
||||
t.Fatalf("stop calls = %d, want 1", client.stopCalls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMoveToPositionRejectsIncompletePosition(t *testing.T) {
|
||||
client := &fakePTZMoveClient{incompletePosition: true}
|
||||
|
||||
err := moveToPosition(client, ptzMoveSupport{Continuous: true}, ptzTarget{Pan: 0.1}, testPTZMoveOptions())
|
||||
if err == nil || !strings.Contains(err.Error(), "incomplete PTZ position") {
|
||||
t.Fatalf("moveToPosition() error = %v, want incomplete-position error", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCombinedContinuousMoveVelocityIncludesEveryAxis(t *testing.T) {
|
||||
request := ptz.ContinuousMove{
|
||||
Velocity: ptz.Speed{
|
||||
PanTilt: &xsdonvif.Vector2D{X: 0.5, Y: -0.25},
|
||||
Zoom: &xsdonvif.Vector1D{X: 0.2},
|
||||
},
|
||||
}
|
||||
|
||||
encoded, err := xml.Marshal(request)
|
||||
if err != nil {
|
||||
t.Fatalf("xml.Marshal() error = %v", err)
|
||||
}
|
||||
xmlBody := string(encoded)
|
||||
if !strings.Contains(xmlBody, `PanTilt x="0.5" y="-0.25"`) {
|
||||
t.Fatalf("ContinuousMove XML = %s, want pan and tilt", xmlBody)
|
||||
}
|
||||
if !strings.Contains(xmlBody, `Zoom x="0.2"`) {
|
||||
t.Fatalf("ContinuousMove XML = %s, want zoom", xmlBody)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGetPTZFunctionsHandlesMissingConfiguration(t *testing.T) {
|
||||
functions, canZoom, canPanTilt := GetPTZFunctionsFromDevice(ptz.GetConfigurationsResponse{})
|
||||
if len(functions) != 0 || canZoom || canPanTilt {
|
||||
t.Fatalf("GetPTZFunctionsFromDevice() = %v, %t, %t; want empty capabilities", functions, canZoom, canPanTilt)
|
||||
}
|
||||
}
|
||||
@@ -1,88 +0,0 @@
|
||||
package onvif
|
||||
|
||||
import (
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"github.com/kerberos-io/onvif"
|
||||
)
|
||||
|
||||
// newSubscriptionTestDevice starts a fake ONVIF device and returns a connected
|
||||
// device plus the request bodies it received on the subscription endpoint.
|
||||
func newSubscriptionTestDevice(t *testing.T, response string) (*onvif.Device, string, func() []string) {
|
||||
t.Helper()
|
||||
|
||||
var mu sync.Mutex
|
||||
var subscriptionBodies []string
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
body, _ := io.ReadAll(r.Body)
|
||||
if r.URL.Path == "/onvif/subscription" {
|
||||
mu.Lock()
|
||||
subscriptionBodies = append(subscriptionBodies, string(body))
|
||||
mu.Unlock()
|
||||
_, _ = w.Write([]byte(response))
|
||||
return
|
||||
}
|
||||
_, _ = w.Write([]byte(`<Envelope><Body/></Envelope>`))
|
||||
}))
|
||||
t.Cleanup(srv.Close)
|
||||
|
||||
dev, err := onvif.NewDevice(onvif.DeviceParams{Xaddr: strings.TrimPrefix(srv.URL, "http://")})
|
||||
if err != nil {
|
||||
t.Fatalf("NewDevice() error = %v", err)
|
||||
}
|
||||
subscriptionAddress := srv.URL + "/onvif/subscription?subscription=7&idx=2"
|
||||
return dev, subscriptionAddress, func() []string {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
return append([]string(nil), subscriptionBodies...)
|
||||
}
|
||||
}
|
||||
|
||||
func assertSubscriptionAddressing(t *testing.T, body string, action string) {
|
||||
t.Helper()
|
||||
|
||||
for _, want := range []string{
|
||||
`<wsa:Action xmlns:wsa="http://www.w3.org/2005/08/addressing">` + action + `</wsa:Action>`,
|
||||
`<wsa:To xmlns:wsa="http://www.w3.org/2005/08/addressing">`,
|
||||
`/onvif/subscription?subscription=7&idx=2</wsa:To>`,
|
||||
} {
|
||||
if !strings.Contains(body, want) {
|
||||
t.Fatalf("request body does not contain %q:\n%s", want, body)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Some devices identify a pull-point subscription by wsa:To and reject
|
||||
// PullMessages without it as NotAuthorized.
|
||||
func TestGetEventMessagesSendsSubscriptionAddressingHeaders(t *testing.T) {
|
||||
dev, subscriptionAddress, bodies := newSubscriptionTestDevice(t, `<Envelope><Body><PullMessagesResponse/></Body></Envelope>`)
|
||||
|
||||
if _, err := GetEventMessages(dev, subscriptionAddress); err != nil {
|
||||
t.Fatalf("GetEventMessages() error = %v", err)
|
||||
}
|
||||
|
||||
received := bodies()
|
||||
if len(received) != 1 {
|
||||
t.Fatalf("subscription requests = %d, want 1", len(received))
|
||||
}
|
||||
assertSubscriptionAddressing(t, received[0], pullMessagesAction)
|
||||
}
|
||||
|
||||
func TestUnsubscribePullPointSendsSubscriptionAddressingHeaders(t *testing.T) {
|
||||
dev, subscriptionAddress, bodies := newSubscriptionTestDevice(t, `<Envelope><Body><UnsubscribeResponse/></Body></Envelope>`)
|
||||
|
||||
if err := UnsubscribePullPoint(dev, subscriptionAddress); err != nil {
|
||||
t.Fatalf("UnsubscribePullPoint() error = %v", err)
|
||||
}
|
||||
|
||||
received := bodies()
|
||||
if len(received) != 1 {
|
||||
t.Fatalf("subscription requests = %d, want 1", len(received))
|
||||
}
|
||||
assertSubscriptionAddressing(t, received[0], unsubscribeAction)
|
||||
}
|
||||
@@ -40,9 +40,6 @@ func NewQueue() *Queue {
|
||||
}
|
||||
|
||||
func (self *Queue) SetMaxGopCount(n int) {
|
||||
if n < 1 {
|
||||
n = 1
|
||||
}
|
||||
self.lock.Lock()
|
||||
self.maxgopcount = n
|
||||
self.lock.Unlock()
|
||||
@@ -100,31 +97,20 @@ func (self *Queue) WritePacket(pkt Packet) (err error) {
|
||||
self.lock.Lock()
|
||||
|
||||
self.buf.Push(pkt)
|
||||
isVideoKeyframe := pkt.Idx == int8(self.videoidx) && pkt.IsKeyFrame
|
||||
if isVideoKeyframe {
|
||||
if pkt.Idx == int8(self.videoidx) && pkt.IsKeyFrame {
|
||||
self.curgopcount++
|
||||
}
|
||||
|
||||
for self.curgopcount > self.maxgopcount && self.buf.Count > 0 {
|
||||
for self.buf.Count > 0 {
|
||||
oldest := self.buf.Pop()
|
||||
if oldest.Idx == int8(self.videoidx) && oldest.IsKeyFrame {
|
||||
self.curgopcount--
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
// Discard the dependent frames from the evicted GOP as well. Keeping
|
||||
// them would let an overrun cursor resume on a P-frame whose reference
|
||||
// keyframe is no longer in the queue.
|
||||
for self.buf.Count > 0 {
|
||||
oldest := self.buf.Get(self.buf.Head)
|
||||
if oldest.Idx == int8(self.videoidx) && oldest.IsKeyFrame {
|
||||
break
|
||||
}
|
||||
self.buf.Pop()
|
||||
}
|
||||
for self.curgopcount >= self.maxgopcount && self.buf.Count > 1 {
|
||||
pkt := self.buf.Pop()
|
||||
if pkt.Idx == int8(self.videoidx) && pkt.IsKeyFrame {
|
||||
self.curgopcount--
|
||||
}
|
||||
if self.curgopcount < self.maxgopcount {
|
||||
break
|
||||
}
|
||||
}
|
||||
//println("shrink", self.curgopcount, self.maxgopcount, self.buf.Head, self.buf.Tail, "count", self.buf.Count, "size", self.buf.Size)
|
||||
|
||||
self.cond.Broadcast()
|
||||
|
||||
|
||||
@@ -107,102 +107,3 @@ func TestQueueCursorReadPacketContextDoesNotAllocateWhenPacketIsAvailable(t *tes
|
||||
t.Fatalf("ReadPacketContext() allocations = %.2f, want 0", allocations)
|
||||
}
|
||||
}
|
||||
|
||||
func TestQueueEvictionKeepsKeyframeAtHead(t *testing.T) {
|
||||
queue := NewQueue()
|
||||
if err := queue.WriteHeader([]Stream{{Index: 0, IsVideo: true}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
queue.SetMaxGopCount(2)
|
||||
|
||||
for _, packet := range []Packet{
|
||||
{Idx: 0, IsVideo: true, IsKeyFrame: true, Data: []byte("IDR-0")},
|
||||
{Idx: 1, IsAudio: true, Data: []byte("audio-0")},
|
||||
{Idx: 0, IsVideo: true, Data: []byte("P-0")},
|
||||
{Idx: 0, IsVideo: true, IsKeyFrame: true, Data: []byte("IDR-1")},
|
||||
{Idx: 0, IsVideo: true, Data: []byte("P-1")},
|
||||
{Idx: 0, IsVideo: true, IsKeyFrame: true, Data: []byte("IDR-2")},
|
||||
} {
|
||||
if err := queue.WritePacket(packet); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
oldest, err := queue.Oldest().ReadPacket()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !oldest.IsKeyFrame || string(oldest.Data) != "IDR-1" {
|
||||
t.Fatalf("oldest packet = %q (keyframe=%t), want IDR-1 keyframe", oldest.Data, oldest.IsKeyFrame)
|
||||
}
|
||||
}
|
||||
|
||||
func TestQueueWithSingleGOPRetainsCurrentKeyframe(t *testing.T) {
|
||||
queue := NewQueue()
|
||||
if err := queue.WriteHeader([]Stream{{Index: 0, IsVideo: true}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
queue.SetMaxGopCount(1)
|
||||
|
||||
for _, packet := range []Packet{
|
||||
{Idx: 0, IsVideo: true, IsKeyFrame: true, Data: []byte("IDR-0")},
|
||||
{Idx: 0, IsVideo: true, Data: []byte("P-0")},
|
||||
{Idx: 0, IsVideo: true, IsKeyFrame: true, Data: []byte("IDR-1")},
|
||||
} {
|
||||
if err := queue.WritePacket(packet); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
oldest, err := queue.Oldest().ReadPacket()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !oldest.IsKeyFrame || string(oldest.Data) != "IDR-1" {
|
||||
t.Fatalf("oldest packet = %q (keyframe=%t), want IDR-1 keyframe", oldest.Data, oldest.IsKeyFrame)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLaggingQueueCursorResumesAtKeyframe(t *testing.T) {
|
||||
queue := NewQueue()
|
||||
if err := queue.WriteHeader([]Stream{{Index: 0, IsVideo: true}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
queue.SetMaxGopCount(2)
|
||||
|
||||
for _, packet := range []Packet{
|
||||
{Idx: 0, IsVideo: true, IsKeyFrame: true, Data: []byte("IDR-0")},
|
||||
{Idx: 0, IsVideo: true, Data: []byte("P-0")},
|
||||
{Idx: 0, IsVideo: true, IsKeyFrame: true, Data: []byte("IDR-1")},
|
||||
} {
|
||||
if err := queue.WritePacket(packet); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
cursor := queue.Oldest()
|
||||
first, err := cursor.ReadPacket()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if string(first.Data) != "IDR-0" {
|
||||
t.Fatalf("first packet = %q, want IDR-0", first.Data)
|
||||
}
|
||||
|
||||
for _, packet := range []Packet{
|
||||
{Idx: 0, IsVideo: true, Data: []byte("P-1")},
|
||||
{Idx: 0, IsVideo: true, IsKeyFrame: true, Data: []byte("IDR-2")},
|
||||
} {
|
||||
if err := queue.WritePacket(packet); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
resumed, err := cursor.ReadPacket()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !resumed.IsKeyFrame || string(resumed.Data) != "IDR-1" {
|
||||
t.Fatalf("resumed packet = %q (keyframe=%t), want IDR-1 keyframe", resumed.Data, resumed.IsKeyFrame)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,81 +0,0 @@
|
||||
package http
|
||||
|
||||
import (
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/kerberos-io/agent/machinery/src/encryption"
|
||||
"github.com/kerberos-io/agent/machinery/src/models"
|
||||
)
|
||||
|
||||
func TestFilesServesFlatAndDatedRecordings(t *testing.T) {
|
||||
gin.SetMode(gin.TestMode)
|
||||
for _, layout := range []string{"flat", "dated"} {
|
||||
for _, encrypted := range []bool{false, true} {
|
||||
name := layout + "-plain"
|
||||
if encrypted {
|
||||
name = layout + "-encrypted"
|
||||
}
|
||||
t.Run(name, func(t *testing.T) {
|
||||
const fileName = "1767355200123-device-key.mp4"
|
||||
directory := t.TempDir()
|
||||
relativePath := fileName
|
||||
if layout == "dated" {
|
||||
relativePath = "2026/01/02/" + fileName
|
||||
}
|
||||
filePath := filepath.Join(directory, "data/recordings", relativePath)
|
||||
if err := os.MkdirAll(filepath.Dir(filePath), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
config := &models.Configuration{Config: models.Config{Encryption: &models.Encryption{}}}
|
||||
content := []byte("recording-payload")
|
||||
if encrypted {
|
||||
config.Config.Encryption.Recordings = "true"
|
||||
config.Config.Encryption.SymmetricKey = "0123456789abcdef0123456789abcdef"
|
||||
ciphertext, err := encryption.AesEncrypt(content, config.Config.Encryption.SymmetricKey)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
content = []byte(ciphertext)
|
||||
}
|
||||
if err := os.WriteFile(filePath, content, 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
router := gin.New()
|
||||
router.GET("/file/*filepath", func(c *gin.Context) { Files(c, directory, config) })
|
||||
for _, reference := range []string{relativePath, fileName} {
|
||||
response := httptest.NewRecorder()
|
||||
router.ServeHTTP(response, httptest.NewRequest("GET", "/file/"+reference, nil))
|
||||
if response.Code != 200 || response.Body.String() != "recording-payload" {
|
||||
t.Fatalf("GET %s = %d %q", reference, response.Code, response.Body.String())
|
||||
}
|
||||
if got := response.Header().Get("Content-Type"); got != "video/mp4" {
|
||||
t.Fatalf("Content-Type = %q, want video/mp4", got)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestFilesRejectsMissingAndOutsideRecordings(t *testing.T) {
|
||||
directory := t.TempDir()
|
||||
if err := os.MkdirAll(filepath.Join(directory, "data/recordings"), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(directory, "data/outside.mp4"), []byte("not a recording"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
router := gin.New()
|
||||
router.GET("/file/*filepath", func(c *gin.Context) { Files(c, directory, &models.Configuration{}) })
|
||||
for _, reference := range []string{"missing.mp4", "../outside.mp4", "2026/01/02/missing.mp4"} {
|
||||
response := httptest.NewRecorder()
|
||||
router.ServeHTTP(response, httptest.NewRequest("GET", "/file/"+reference, nil))
|
||||
if response.Code != 404 {
|
||||
t.Fatalf("GET %s = %d, want 404", reference, response.Code)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -65,12 +65,12 @@ func AddRoutes(r *gin.Engine, authMiddleware *jwt.GinJWTMiddleware, configDirect
|
||||
|
||||
// Will verify the hub settings.
|
||||
api.POST("/hub/verify", func(c *gin.Context) {
|
||||
cloud.VerifyHub(c, configuration.Config.HubURI)
|
||||
cloud.VerifyHub(c)
|
||||
})
|
||||
|
||||
// Will verify the persistence settings.
|
||||
api.POST("/persistence/verify", func(c *gin.Context) {
|
||||
cloud.VerifyPersistence(c, configDirectory, configuration.Config.HubURI)
|
||||
cloud.VerifyPersistence(c, configDirectory)
|
||||
})
|
||||
|
||||
// Will verify the secondary persistence settings.
|
||||
|
||||
@@ -5,7 +5,6 @@ import (
|
||||
"io"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
jwt "github.com/appleboy/gin-jwt/v2"
|
||||
@@ -20,7 +19,6 @@ import (
|
||||
"github.com/kerberos-io/agent/machinery/src/capture"
|
||||
"github.com/kerberos-io/agent/machinery/src/encryption"
|
||||
"github.com/kerberos-io/agent/machinery/src/models"
|
||||
"github.com/kerberos-io/agent/machinery/src/utils"
|
||||
swaggerFiles "github.com/swaggo/files"
|
||||
ginSwagger "github.com/swaggo/gin-swagger"
|
||||
)
|
||||
@@ -162,9 +160,9 @@ func requestLogger() gin.HandlerFunc {
|
||||
func Files(c *gin.Context, configDirectory string, configuration *models.Configuration) {
|
||||
|
||||
// Get File
|
||||
filePath, err := utils.ResolveRecordingPath(configDirectory+"/data/recordings", strings.TrimPrefix(c.Param("filepath"), "/"))
|
||||
filePath := configDirectory + "/data/recordings" + c.Param("filepath")
|
||||
_, err := os.Open(filePath)
|
||||
if err != nil {
|
||||
log.WithError(err).Warn("Could not resolve recording for playback")
|
||||
c.JSON(404, gin.H{"error": "File not found"})
|
||||
return
|
||||
}
|
||||
|
||||
+80
-45
@@ -191,32 +191,51 @@ func GetMediaFormatted(files []os.FileInfo, recordingDirectory string, configura
|
||||
count := 0
|
||||
for _, file := range files {
|
||||
fileName := file.Name()
|
||||
recordedAt, err := RecordingStartTime(fileName)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
timestamp := recordedAt.Unix()
|
||||
if eventFilter.TimestampOffsetStart > 0 && timestamp < eventFilter.TimestampOffsetStart {
|
||||
continue
|
||||
}
|
||||
if eventFilter.TimestampOffsetEnd > 0 && timestamp >= eventFilter.TimestampOffsetEnd {
|
||||
continue
|
||||
}
|
||||
loc, _ := time.LoadLocation(configuration.Config.Timezone)
|
||||
localTime := recordedAt.In(loc)
|
||||
filePaths = append(filePaths, models.Media{
|
||||
Key: fileName,
|
||||
Path: recordingDirectory + "/" + fileName,
|
||||
CameraName: configuration.Config.Name,
|
||||
CameraKey: configuration.Config.Key,
|
||||
Day: localTime.Format("02-01-2006"),
|
||||
ShortDay: localTime.Format("Jan _2"),
|
||||
Time: localTime.Format("15:04:05"),
|
||||
Timestamp: strconv.FormatInt(timestamp, 10),
|
||||
})
|
||||
count++
|
||||
if eventFilter.NumberOfElements > 0 && count >= eventFilter.NumberOfElements {
|
||||
break
|
||||
fileParts := strings.Split(fileName, "_")
|
||||
if len(fileParts) == 6 {
|
||||
timestamp := fileParts[0]
|
||||
timestampInt, err := strconv.ParseInt(timestamp, 10, 64)
|
||||
if err == nil {
|
||||
|
||||
if eventFilter.TimestampOffsetStart > 0 {
|
||||
// TimestampOffsetStart represents the newest lower bound to include.
|
||||
if timestampInt < eventFilter.TimestampOffsetStart {
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
// If we have an offset we will check if we should skip or not
|
||||
if eventFilter.TimestampOffsetEnd > 0 {
|
||||
// Medias are sorted from new to older. TimestampOffsetEnd holds the oldest
|
||||
// timestamp of the previous batch of events. By doing this check, we make sure
|
||||
// to skip the previous batch.
|
||||
if timestampInt >= eventFilter.TimestampOffsetEnd {
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
loc, _ := time.LoadLocation(configuration.Config.Timezone)
|
||||
time := time.Unix(timestampInt, 0).In(loc)
|
||||
day := time.Format("02-01-2006")
|
||||
timeString := time.Format("15:04:05")
|
||||
shortDay := time.Format("Jan _2")
|
||||
|
||||
media := models.Media{
|
||||
Key: fileName,
|
||||
Path: recordingDirectory + "/" + fileName,
|
||||
CameraName: configuration.Config.Name,
|
||||
CameraKey: configuration.Config.Key,
|
||||
Day: day,
|
||||
ShortDay: shortDay,
|
||||
Time: timeString,
|
||||
Timestamp: timestamp,
|
||||
}
|
||||
filePaths = append(filePaths, media)
|
||||
count = count + 1
|
||||
if eventFilter.NumberOfElements > 0 && count >= eventFilter.NumberOfElements {
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return filePaths
|
||||
@@ -225,12 +244,18 @@ func GetMediaFormatted(files []os.FileInfo, recordingDirectory string, configura
|
||||
func GetDays(files []os.FileInfo, recordingDirectory string, configuration *models.Configuration) []string {
|
||||
days := []string{}
|
||||
for _, file := range files {
|
||||
recordedAt, err := RecordingStartTime(file.Name())
|
||||
if err != nil {
|
||||
continue
|
||||
fileName := file.Name()
|
||||
fileParts := strings.Split(fileName, "_")
|
||||
if len(fileParts) == 6 {
|
||||
timestamp := fileParts[0]
|
||||
timestampInt, err := strconv.ParseInt(timestamp, 10, 64)
|
||||
if err == nil {
|
||||
loc, _ := time.LoadLocation(configuration.Config.Timezone)
|
||||
time := time.Unix(timestampInt, 0).In(loc)
|
||||
day := time.Format("02-01-2006")
|
||||
days = append(days, day)
|
||||
}
|
||||
}
|
||||
loc, _ := time.LoadLocation(configuration.Config.Timezone)
|
||||
days = append(days, recordedAt.In(loc).Format("02-01-2006"))
|
||||
}
|
||||
uniqueDays := Unique(days)
|
||||
return uniqueDays
|
||||
@@ -253,13 +278,10 @@ func NumberOfFilesInDirectory(path string) int {
|
||||
return len(files)
|
||||
}
|
||||
|
||||
// NumberOfMP4sInDirectory counts MP4 files, including dated subdirectories.
|
||||
// NumberOfMP4sInDirectory returns the count of all files with mp4 extension in current directory
|
||||
func NumberOfMP4sInDirectory(path string) int {
|
||||
files, err := ReadRecordingDirectory(path)
|
||||
if err != nil {
|
||||
log.WithError(err).Error("Could not count recordings")
|
||||
return 0
|
||||
}
|
||||
pattern := filepath.Join(path, "*.mp4")
|
||||
files, _ := filepath.Glob(pattern)
|
||||
return len(files)
|
||||
}
|
||||
|
||||
@@ -387,13 +409,25 @@ func Decrypt(directoryOrFile string, symmetricKey []byte) {
|
||||
|
||||
var files []string
|
||||
if fileInfo.IsDir() {
|
||||
dir, err := ReadRecordingDirectory(directoryOrFile)
|
||||
// Create decrypted directory
|
||||
err = os.MkdirAll(directoryOrFile+"/decrypted", 0755)
|
||||
if err != nil {
|
||||
log.Fatal(err.Error())
|
||||
return
|
||||
}
|
||||
dir, err := os.ReadDir(directoryOrFile)
|
||||
if err != nil {
|
||||
log.Fatal(err.Error())
|
||||
return
|
||||
}
|
||||
for _, file := range dir {
|
||||
files = append(files, filepath.Join(directoryOrFile, file.Name()))
|
||||
// Check if file is not a directory
|
||||
if !file.IsDir() {
|
||||
// Check if an mp4 file
|
||||
if strings.HasSuffix(file.Name(), ".mp4") {
|
||||
files = append(files, directoryOrFile+"/"+file.Name())
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
files = append(files, directoryOrFile)
|
||||
@@ -415,12 +449,13 @@ func Decrypt(directoryOrFile string, symmetricKey []byte) {
|
||||
return
|
||||
}
|
||||
|
||||
outputDirectory := filepath.Join(filepath.Dir(file), "decrypted")
|
||||
if err := os.MkdirAll(outputDirectory, 0o755); err != nil {
|
||||
log.Fatal(err.Error())
|
||||
return
|
||||
}
|
||||
err = os.WriteFile(filepath.Join(outputDirectory, filepath.Base(file)), []byte(decrypted), 0644)
|
||||
// Write decrypted content to file with appended .decrypted
|
||||
// Get filename split by / and get last element.
|
||||
fileParts := strings.Split(file, "/")
|
||||
fileName := fileParts[len(fileParts)-1]
|
||||
pathToFile := strings.Join(fileParts[:len(fileParts)-1], "/")
|
||||
|
||||
err = os.WriteFile(pathToFile+"/decrypted/"+fileName, []byte(decrypted), 0644)
|
||||
if err != nil {
|
||||
log.Fatal(err.Error())
|
||||
return
|
||||
|
||||
@@ -1,136 +0,0 @@
|
||||
package utils
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
func RecordingFileName(timestamp int64, deviceKey string) (string, error) {
|
||||
if timestamp <= 0 {
|
||||
return "", fmt.Errorf("invalid recording timestamp %d", timestamp)
|
||||
}
|
||||
if deviceKey == "" || strings.ContainsAny(deviceKey, "/\\") {
|
||||
return "", fmt.Errorf("missing or invalid recording device key")
|
||||
}
|
||||
return strconv.FormatInt(timestamp, 10) + "-" + deviceKey + ".mp4", nil
|
||||
}
|
||||
|
||||
func RecordingStartTime(name string) (time.Time, error) {
|
||||
base := filepath.Base(name)
|
||||
if !strings.HasSuffix(base, ".mp4") {
|
||||
return time.Time{}, fmt.Errorf("invalid recording filename %q", name)
|
||||
}
|
||||
timestamp, deviceKey, short := strings.Cut(strings.TrimSuffix(base, ".mp4"), "-")
|
||||
if short && deviceKey != "" {
|
||||
if millis, err := strconv.ParseInt(timestamp, 10, 64); err == nil && millis > 0 {
|
||||
return time.UnixMilli(millis), nil
|
||||
}
|
||||
}
|
||||
parts := strings.Split(base, "_")
|
||||
if len(parts) == 6 {
|
||||
seconds, err := strconv.ParseInt(parts[0], 10, 64)
|
||||
if err == nil && seconds > 0 {
|
||||
return time.Unix(seconds, 0), nil
|
||||
}
|
||||
}
|
||||
return time.Time{}, fmt.Errorf("invalid recording filename %q", name)
|
||||
}
|
||||
|
||||
func CreateRecordingPath(directory, fileName string, recordedAt time.Time) (string, error) {
|
||||
if !filepath.IsLocal(fileName) || fileName == "." || filepath.Base(fileName) != fileName || strings.ContainsAny(fileName, "/\\") {
|
||||
return "", fmt.Errorf("invalid recording filename %q", fileName)
|
||||
}
|
||||
if recordedAt.UnixMilli() <= 0 {
|
||||
return "", fmt.Errorf("invalid recording start time %s", recordedAt)
|
||||
}
|
||||
directory = filepath.Join(directory, recordedAt.UTC().Format("2006/01/02"))
|
||||
if err := os.MkdirAll(directory, 0o755); err != nil {
|
||||
return "", fmt.Errorf("create recording directory: %w", err)
|
||||
}
|
||||
return filepath.Join(directory, fileName), nil
|
||||
}
|
||||
|
||||
// ResolveRecordingPath accepts relative paths from listings and historical
|
||||
// basename-only references from upload markers and playback URLs.
|
||||
func ResolveRecordingPath(directory, name string) (string, error) {
|
||||
if !filepath.IsLocal(name) {
|
||||
return "", fmt.Errorf("invalid recording path %q", name)
|
||||
}
|
||||
filePath := filepath.Join(directory, name)
|
||||
info, err := os.Stat(filePath)
|
||||
if os.IsNotExist(err) && filepath.Base(name) == name {
|
||||
recordedAt, parseErr := RecordingStartTime(name)
|
||||
if parseErr == nil {
|
||||
filePath = filepath.Join(directory, recordedAt.UTC().Format("2006/01/02"), name)
|
||||
info, err = os.Stat(filePath)
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if !info.Mode().IsRegular() {
|
||||
return "", fmt.Errorf("recording %q is not a regular file", name)
|
||||
}
|
||||
return filePath, nil
|
||||
}
|
||||
|
||||
func OpenRecording(directory, name string) (*os.File, error) {
|
||||
filePath, err := ResolveRecordingPath(directory, name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return os.Open(filePath)
|
||||
}
|
||||
|
||||
func RemoveRecording(directory, name string) error {
|
||||
filePath, err := ResolveRecordingPath(directory, name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return os.Remove(filePath)
|
||||
}
|
||||
|
||||
type recordingFileInfo struct {
|
||||
os.FileInfo
|
||||
relativePath string
|
||||
}
|
||||
|
||||
func (file recordingFileInfo) Name() string {
|
||||
return file.relativePath
|
||||
}
|
||||
|
||||
// ReadRecordingDirectory returns MP4s with root-relative names, preserving the
|
||||
// file information used by the existing sorting and media formatting helpers.
|
||||
func ReadRecordingDirectory(directory string) ([]os.FileInfo, error) {
|
||||
files := []os.FileInfo{}
|
||||
err := filepath.WalkDir(directory, func(filePath string, entry fs.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if entry.IsDir() && entry.Name() == "decrypted" && filePath != directory {
|
||||
return filepath.SkipDir
|
||||
}
|
||||
if !entry.Type().IsRegular() || filepath.Ext(entry.Name()) != ".mp4" {
|
||||
return nil
|
||||
}
|
||||
info, err := entry.Info()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
name, err := filepath.Rel(directory, filePath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
files = append(files, recordingFileInfo{FileInfo: info, relativePath: filepath.ToSlash(name)})
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return files, nil
|
||||
}
|
||||
@@ -1,235 +0,0 @@
|
||||
package utils
|
||||
|
||||
import (
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strconv"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/kerberos-io/agent/machinery/src/encryption"
|
||||
"github.com/kerberos-io/agent/machinery/src/models"
|
||||
)
|
||||
|
||||
func TestShortRecordingNamePreservesMillisecondsAndDeviceKey(t *testing.T) {
|
||||
const timestamp = int64(1767355200123)
|
||||
const deviceKey = "device-key_with-hyphens"
|
||||
name, err := RecordingFileName(timestamp, deviceKey)
|
||||
if err != nil || name != "1767355200123-device-key_with-hyphens.mp4" {
|
||||
t.Fatalf("RecordingFileName() = %q, %v", name, err)
|
||||
}
|
||||
other, err := RecordingFileName(timestamp+1, deviceKey)
|
||||
if err != nil || other == name {
|
||||
t.Fatalf("recordings within the same second must have distinct names: %q, %v", other, err)
|
||||
}
|
||||
directory := t.TempDir()
|
||||
filePath, err := CreateRecordingPath(directory, name, time.UnixMilli(timestamp))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if want := filepath.Join(directory, "2026/01/02", name); filePath != want {
|
||||
t.Fatalf("path = %q, want %q", filePath, want)
|
||||
}
|
||||
if err := os.WriteFile(filePath, []byte("video"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
resolved, err := ResolveRecordingPath(directory, name)
|
||||
if err != nil || resolved != filePath {
|
||||
t.Fatalf("ResolveRecordingPath() = %q, %v", resolved, err)
|
||||
}
|
||||
files, err := ReadRecordingDirectory(directory)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
config := &models.Configuration{Config: models.Config{Timezone: "UTC"}}
|
||||
media := GetMediaFormatted(files, directory, config, models.EventFilter{TimestampOffsetStart: timestamp / 1000})
|
||||
if len(media) != 1 || media[0].Key != "2026/01/02/"+name || media[0].Timestamp != "1767355200" {
|
||||
t.Fatalf("unexpected short-name media: %+v", media)
|
||||
}
|
||||
if days := GetDays(files, directory, config); !reflect.DeepEqual(days, []string{"02-01-2026"}) {
|
||||
t.Fatalf("unexpected short-name days: %v", days)
|
||||
}
|
||||
if err := RemoveRecording(directory, name); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, invalidKey := range []string{"", "camera/key", `camera\key`} {
|
||||
if _, err := RecordingFileName(timestamp, invalidKey); err == nil {
|
||||
t.Errorf("accepted invalid device key %q", invalidKey)
|
||||
}
|
||||
}
|
||||
if _, err := RecordingFileName(0, deviceKey); err == nil {
|
||||
t.Fatal("accepted invalid recording timestamp")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCreateRecordingPathUsesRecordingUTCDate(t *testing.T) {
|
||||
directory := t.TempDir()
|
||||
recordedAt := time.Date(2026, time.January, 1, 1, 0, 0, 0, time.FixedZone("UTC+2", 2*60*60))
|
||||
name := strconv.FormatInt(recordedAt.Unix(), 10) + "_3-123_camera_0-0-0-0_1_1000.mp4"
|
||||
filePath, err := CreateRecordingPath(directory, name, recordedAt)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if want := filepath.Join(directory, "2025/12/31", name); filePath != want {
|
||||
t.Fatalf("path = %q, want %q", filePath, want)
|
||||
}
|
||||
if err := os.WriteFile(filePath, []byte("video"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, reference := range []string{name, "2025/12/31/" + name} {
|
||||
got, err := ResolveRecordingPath(directory, reference)
|
||||
if err != nil || got != filePath {
|
||||
t.Fatalf("ResolveRecordingPath(%q) = %q, %v; want %q", reference, got, err, filePath)
|
||||
}
|
||||
file, err := OpenRecording(directory, reference)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
content, err := io.ReadAll(file)
|
||||
file.Close()
|
||||
if err != nil || string(content) != "video" {
|
||||
t.Fatalf("recording content = %q, %v", content, err)
|
||||
}
|
||||
}
|
||||
if err := RemoveRecording(directory, name); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := os.Stat(filePath); !os.IsNotExist(err) {
|
||||
t.Fatalf("recording still exists after removal: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRecordingPathsKeepFlatFilesAccessible(t *testing.T) {
|
||||
directory := t.TempDir()
|
||||
name := "legacy-recording.mp4"
|
||||
filePath := filepath.Join(directory, name)
|
||||
if err := os.WriteFile(filePath, []byte("legacy"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := ResolveRecordingPath(directory, name)
|
||||
if err != nil || got != filePath {
|
||||
t.Fatalf("ResolveRecordingPath() = %q, %v; want %q", got, err, filePath)
|
||||
}
|
||||
if err := RemoveRecording(directory, name); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRecordingPathsReportInvalidAndMissingPaths(t *testing.T) {
|
||||
directory := t.TempDir()
|
||||
for _, name := range []string{"", ".", "..", "../outside.mp4", "/outside.mp4", "nested/recording.mp4", `nested\recording.mp4`} {
|
||||
if _, err := CreateRecordingPath(directory, name, time.Now()); err == nil {
|
||||
t.Errorf("CreateRecordingPath(%q) should reject invalid filename", name)
|
||||
}
|
||||
}
|
||||
for _, name := range []string{"", ".", "../outside.mp4", "/outside.mp4"} {
|
||||
if _, err := ResolveRecordingPath(directory, name); err == nil {
|
||||
t.Errorf("ResolveRecordingPath(%q) should reject invalid path", name)
|
||||
}
|
||||
}
|
||||
for _, name := range []string{"missing.mp4", "1700000000_missing.mp4", "2026/01/01/missing.mp4"} {
|
||||
if _, err := ResolveRecordingPath(directory, name); !os.IsNotExist(err) {
|
||||
t.Errorf("ResolveRecordingPath(%q) error = %v, want not-exist", name, err)
|
||||
}
|
||||
}
|
||||
blocked := filepath.Join(directory, "not-a-directory")
|
||||
if err := os.WriteFile(blocked, nil, 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := CreateRecordingPath(blocked, "recording.mp4", time.Now()); err == nil {
|
||||
t.Fatal("expected directory creation error")
|
||||
}
|
||||
if _, err := CreateRecordingPath(directory, "recording.mp4", time.Time{}); err == nil {
|
||||
t.Fatal("expected invalid recording start time error")
|
||||
}
|
||||
if _, err := ReadRecordingDirectory(filepath.Join(directory, "missing")); !os.IsNotExist(err) {
|
||||
t.Fatalf("ReadRecordingDirectory() error = %v, want not-exist", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReadRecordingDirectoryIncludesFlatAndDatedMedia(t *testing.T) {
|
||||
directory := t.TempDir()
|
||||
names := []string{
|
||||
"1700000000_6_7_8_9_10.mp4",
|
||||
"2026/01/02/1767355200_6_7_8_9_10.mp4",
|
||||
}
|
||||
for index, name := range names {
|
||||
filePath := filepath.Join(directory, name)
|
||||
if err := os.MkdirAll(filepath.Dir(filePath), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filePath, []byte("video"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
modified := time.Unix(int64(index+1), 0)
|
||||
if err := os.Chtimes(filePath, modified, modified); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(directory, "2026/01/02/ignore.txt"), nil, 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Join(directory, "2026/01/02/decrypted"), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(directory, "2026/01/02/decrypted/output.mp4"), nil, 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
files, err := ReadRecordingDirectory(directory)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(files) != 2 || NumberOfMP4sInDirectory(directory) != 2 {
|
||||
t.Fatalf("expected 2 MP4 recordings, got %d", len(files))
|
||||
}
|
||||
config := &models.Configuration{Config: models.Config{Timezone: "UTC", Name: "camera", Key: "device"}}
|
||||
files = GetSortedDirectory(files)
|
||||
media := GetMediaFormatted(files, directory, config, models.EventFilter{})
|
||||
if len(media) != 2 || media[0].Key != names[1] || media[1].Key != names[0] {
|
||||
t.Fatalf("unexpected media ordering or relative keys: %+v", media)
|
||||
}
|
||||
if media[0].Path != filepath.Join(directory, names[1]) || media[0].Timestamp != "1767355200" {
|
||||
t.Fatalf("unexpected nested media path or timestamp: %+v", media[0])
|
||||
}
|
||||
if got, want := GetDays(files, directory, config), []string{"02-01-2026", "14-11-2023"}; !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("days = %v, want %v", got, want)
|
||||
}
|
||||
filtered := GetMediaFormatted(files, directory, config, models.EventFilter{TimestampOffsetStart: 1767355200})
|
||||
if len(filtered) != 1 || filtered[0].Key != names[1] {
|
||||
t.Fatalf("unexpected filtered nested media: %+v", filtered)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDecryptDirectoryIncludesDatedRecordingsAndSkipsOutputs(t *testing.T) {
|
||||
directory := t.TempDir()
|
||||
key := []byte("0123456789abcdef0123456789abcdef")
|
||||
payload := []byte("recording-payload")
|
||||
ciphertext, err := encryption.AesEncrypt(payload, string(key))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
names := []string{"legacy.mp4", "2026/01/02/recording.mp4"}
|
||||
for _, name := range names {
|
||||
filePath := filepath.Join(directory, name)
|
||||
if err := os.MkdirAll(filepath.Dir(filePath), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filePath, []byte(ciphertext), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
Decrypt(directory, key)
|
||||
Decrypt(directory, key)
|
||||
for _, name := range names {
|
||||
output := filepath.Join(directory, filepath.Dir(name), "decrypted", filepath.Base(name))
|
||||
content, err := os.ReadFile(output)
|
||||
if err != nil || string(content) != string(payload) {
|
||||
t.Fatalf("decrypted %q = %q, %v", name, content, err)
|
||||
}
|
||||
}
|
||||
if got := NumberOfMP4sInDirectory(directory); got != len(names) {
|
||||
t.Fatalf("recording count = %d, want %d (excluding decrypted outputs)", got, len(names))
|
||||
}
|
||||
}
|
||||
@@ -223,8 +223,6 @@
|
||||
"kerberosvault_description_accesskey": "The access key of your Vault account.",
|
||||
"kerberosvault_secretkey": "Secret key",
|
||||
"kerberosvault_description_secretkey": "The secret key of your Vault account.",
|
||||
"kerberosvault_customheaders": "Custom upload headers",
|
||||
"kerberosvault_description_customheaders": "A JSON object of custom HTTP headers, for example {\"site_id\":\"site-1\",\"line_id\":\"line-2\"}. These values are included as metadata in Vault integration messages.",
|
||||
"kerberosvault_maxretries": "Max retries",
|
||||
"kerberosvault_description_maxretries": "The maximum number of retries to upload a recording.",
|
||||
"kerberosvault_timeout": "Timeout",
|
||||
|
||||
+3
-4
@@ -1,6 +1,5 @@
|
||||
const { hostname, host, protocol } = window.location;
|
||||
const websocketprotocol = protocol === 'http:' ? 'ws:' : 'wss:';
|
||||
const devAgentPort = process.env.REACT_APP_AGENT_API_PORT || '8082';
|
||||
|
||||
// Uncomment this when using codespaces or other special DNS names (which you can't control)
|
||||
// replace this with the DNS name of the kerberos agent server (the codespace url)
|
||||
@@ -10,9 +9,9 @@ const dev = {
|
||||
ENV: 'dev',
|
||||
// Comment the below lines, when using codespaces or other special DNS names (which you can't control)
|
||||
HOSTNAME: hostname,
|
||||
API_URL: `${protocol}//${hostname}:${devAgentPort}/api`,
|
||||
URL: `${protocol}//${hostname}:${devAgentPort}`,
|
||||
WS_URL: `${websocketprotocol}//${hostname}:${devAgentPort}/ws`,
|
||||
API_URL: `${protocol}//${hostname}:8080/api`,
|
||||
URL: `${protocol}//${hostname}:8080`,
|
||||
WS_URL: `${websocketprotocol}//${hostname}:8080/ws`,
|
||||
MODE: window['env']['mode'],
|
||||
// Uncomment, and comment the above lines, when using codespaces or other special DNS names (which you can't control)
|
||||
// HOSTNAME: externalHost,
|
||||
|
||||
@@ -2542,28 +2542,6 @@ export class Settings extends React.Component {
|
||||
)
|
||||
}
|
||||
/>
|
||||
<Input
|
||||
noPadding
|
||||
label={t(
|
||||
'settings.persistence.kerberosvault_customheaders'
|
||||
)}
|
||||
placeholder={t(
|
||||
'settings.persistence.kerberosvault_description_customheaders'
|
||||
)}
|
||||
value={
|
||||
config.kstorage
|
||||
? config.kstorage.custom_headers || ''
|
||||
: ''
|
||||
}
|
||||
onChange={(value) =>
|
||||
this.onUpdateField(
|
||||
'kstorage',
|
||||
'custom_headers',
|
||||
value,
|
||||
config.kstorage
|
||||
)
|
||||
}
|
||||
/>
|
||||
<Input
|
||||
noPadding
|
||||
label={t(
|
||||
@@ -2837,28 +2815,6 @@ export class Settings extends React.Component {
|
||||
)
|
||||
}
|
||||
/>
|
||||
<Input
|
||||
noPadding
|
||||
label={t(
|
||||
'settings.persistence.kerberosvault_customheaders'
|
||||
)}
|
||||
placeholder={t(
|
||||
'settings.persistence.kerberosvault_description_customheaders'
|
||||
)}
|
||||
value={
|
||||
config.kstorage_secondary
|
||||
? config.kstorage_secondary.custom_headers || ''
|
||||
: ''
|
||||
}
|
||||
onChange={(value) =>
|
||||
this.onUpdateField(
|
||||
'kstorage_secondary',
|
||||
'custom_headers',
|
||||
value,
|
||||
config.kstorage_secondary
|
||||
)
|
||||
}
|
||||
/>
|
||||
</BlockBody>
|
||||
<BlockFooter>
|
||||
<Button
|
||||
|
||||
Reference in New Issue
Block a user