Skip to content
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 6 additions & 22 deletions kubernetes/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -42,22 +42,14 @@ CHART_DEBUG?=0
# they will be automatically passed to the Helm chart during installation
# Chart debug: set CHART_DEBUG=1 to enable chartdebug=true in Helm deployment

# ITEP constants
# REGISTRY=registry.test-public-maestro.edgeorch.net/scenescape
CHARTREPO=registry.test-public-maestro.edgeorch.net/chartrepo/scenescape
# HELM_REPO_USERNAME=<harbor_username>
# HELM_REPO_PASSWORD=<harbor_cli_secret>

# start kind, then install SceneScape with helm
default: install-deps clean-kind kind build-all-tests install
default: install-deps clean-kind kind build-all install

# publish to ITEP by building, packaging, then pushing
# must set appropriate REGISTRY, CHARTREPO, HELM_REPO_USERNAME and HELM_REPO_PASSWORD constants
build-and-package-all: build-all install-package-deps package
build-and-package: build-all package

# build init-images, scenescape images and push everything to a registry, then generate Chart.yaml
build-all-tests: build-all
build-all: build push chart.yaml
# build scenescape images and push everything to a registry, then generate Chart.yaml
build-all: build push

kind: generate-kind-yaml start-kind install-cert-manager

Expand Down Expand Up @@ -162,17 +154,9 @@ push-all: push
chart.yaml:
sed -e "s|{VERSION}|$(VERSION)|g" template/Chart.template > scenescape-chart/Chart.yaml

# packaging dependencies
install-package-deps:
helm plugin install https://github.com/chartmuseum/helm-push || true
helm repo add itep_harbor https://$(CHARTREPO)

# packages and pushes the helm chart
# must set HELM_REPO_USERNAME and HELM_REPO_PASSWORD variables
package: copy-files
PACKAGE=$$(helm package scenescape-chart/ | awk '{print $$NF}'); \
helm cm-push $$PACKAGE itep_harbor -u=$(HELM_REPO_USERNAME) -p=$(HELM_REPO_PASSWORD); \
rm $$PACKAGE
package: copy-files chart.yaml
helm package scenescape-chart/

# Query what's in the registry (local)
list-registry:
Expand Down
144 changes: 36 additions & 108 deletions kubernetes/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# Intel® SceneScape on Kubernetes

Intel® SceneScape Kubernetes helm chart

## Overview

This folder contains the helm chart to run Intel® SceneScape on Kubernetes.
Expand All @@ -10,9 +8,14 @@ This readme goes through a minimal setup for running this on your local developm

Advanced users intending to deploy this in production will have to change the default chart values or modify the templates.

## Quick start
There are 2 main ways to install Intel® SceneScape on Kubernetes:

1. [All-in-one (Kind + Registry + SceneScape)](#all-in-one) - for development and testing using locally built SceneScape images
2. [SceneScape only](#scenescape-only) - for existing clusters and published SceneScape images

## All-in-one

The easiest way to install Intel® SceneScape on Kubernetes is with a single command:
The easiest way to install Intel® SceneScape on [Kind](https://kind.sigs.k8s.io/) is with a single command:

```sh
KUBERNETES=1 ./deploy.sh
Expand All @@ -21,6 +24,7 @@ KUBERNETES=1 ./deploy.sh
This will:

- Start a kind cluster and local registry (if needed)
- Installs [Cert Manager](https://cert-manager.io/)
- Build and push all required images
- Autogenerate a strong admin UI password (SUPASS)
- Deploy Intel® SceneScape to the cluster using Helm
Expand All @@ -30,83 +34,46 @@ When the webUI is up, log in with `admin` and the autogenerated password. By def
> **Note:** You can retrieve the generated admin password at any time with:
>
> ```sh
> helm get values scenescape-release-1 -n scenescape
> helm get values scenescape -n scenescape
> ```

Save this password for future logins. You can change the admin password later via the web UI after logging in.

## Advanced Installation Options

### 1. Using Makefile Targets Directly

You can use the Makefile targets for more control, automation, or development. This approach lets you run each step individually or as a sequence.
### Useful targets

#### Recommended workflow

#### 1. **Set custom passwords:**

Set the `SUPASS` environment variable before running the `install` target to specify your own admin password for web application:

```sh
export SUPASS=your_custom_password
```

Set the `PGPASS` environment variable before running the `install` target to specify your own admin password for postgres database:
- Install SceneScape: `make -C kubernetes install`
- Uninstall (leave kind cluster running): `make -C kubernetes uninstall`
- Remove all: `make -C kubernetes clean-all`

```sh
export PGPASS=your_custom_password
```
## SceneScape Only

**Important:** If you omit setting these passwords, installation will fail.
If you already have a Kubernetes cluster you can use the Helm chart directly.

**How to generate strong passwords:**
**Prerequisites:**

```sh
export SUPASS=$(openssl rand -base64 48 | tr -dc 'A-Za-z0-9!@#$%^&*()_+-=[]{}|;:,.<>?/~' | head -c 24)
export PGPASS=$(openssl rand -base64 48 | tr -dc 'A-Za-z0-9!@#$%^&*()_+-=[]{}|;:,.<>?/~' | head -c 16)
```
Install [Cert Manager](https://cert-manager.io/) in your cluster.

#### 2. **Deploy Scenescape:**
**Install with a custom admin password:**

```sh
make -C kubernetes install-deps clean-kind kind build-all install
helm install scenescape scenescape-chart -n <NAMESPACE> --create-namespace \
--set supass=<YOUR_ADMIN_PASSWORD> \
--set pgserver.password=<YOUR_POSTGRES_PASSWORD>
```

This will:

- Install required dependencies (`kind`, `kubectl`, `k9s`, `helm`)
- Remove any previous kind cluster and registry
- Start a new kind cluster and local registry
- Build and push all required images to the local registry
- Deploy Intel® SceneScape to the cluster using Helm

**Other useful targets:**

- Stop: `make -C kubernetes uninstall`
- Remove all: `make -C kubernetes clean-all`

---

### 2. Using the Helm Chart Directly (For Existing Kubernetes Clusters)

If you already have a Kubernetes cluster and want to deploy Intel® SceneScape without the Makefile or `deploy.sh`, you can use the Helm chart directly.

**Install with a custom admin password:**
Optionally, prepare updated [values file](scenescape-chart/values.yaml) and save it as `values-custom.yaml`.

```sh
helm install scenescape-release-1 scenescape-chart -n scenescape --create-namespace \
--set supass=your_custom_password \
--set pgserver.password=your_custom_password
helm install scenescape scenescape-chart -n <NAMESPACE> --create-namespace \
--set supass=<YOUR_ADMIN_PASSWORD> \
--set pgserver.password=<YOUR_POSTGRES_PASSWORD> \
--values values-custom.yaml
```

- The `supass` value sets the admin password for the web UI. **If you do not set `supass`, installation will fail.**
- The `pgserver.password` value sets the admin password for the Postgres database. **If you do not set `pgserver.password`, installation will fail.**
- You can set other values with `--set` or a custom `values.yaml` file.

**To uninstall:**

```sh
helm uninstall scenescape-release-1 -n scenescape
helm uninstall scenescape -n <NAMESPACE>
```

## Environment Variables
Expand Down Expand Up @@ -135,6 +102,15 @@ These values ensure that all internal cluster communication, including between p

The proxy settings will be automatically detected and passed to all Intel® SceneScape containers as environment variables.

### NodePort Services

By default, Intel® SceneScape exposes its services using ClusterIP type services. If you want to expose them using NodePort services instead, set the following chart value:

```yaml
nodePort:
enabled: true
```

### Chart Debug Mode

To enable Helm chart debugging (useful for troubleshooting deployment issues):
Expand All @@ -156,51 +132,3 @@ make -C kubernetes install
```

This enables additional testing components and configurations.

## Detailed steps and explanation

Run from the project directory (e.g. ~/scenescape)

1. Start up a kind cluster and a local registry.
```console
$ make -C kubernetes kind
```
This uses the template files in kubernetes/template and generates yaml files for kind cluster configuration. It then starts up a registry container, a kind cluster container and adds them to the same Docker network so they can communicate. Run `generate-kind-yaml` and `start-kind` targets separately if you want to keep your edited yaml files.
Leave the kind cluster running or omit this step if you have your own cluster and registry ready.
2. Build Intel® SceneScape images and init-images, then push everything to the local registry.
```console
$ make -C kubernetes build-all
```
3. Install the Intel® SceneScape release with helm.
```console
$ make -C kubernetes install
```
4. Verify that Intel® SceneScape is running.
```console
kubectl get pods -n scenescape -w
# alternative TUI
k9s
```
5. Uninstall the Intel® SceneScape release.
```console
$ make -C kubernetes uninstall
```

### Additional notes

- Additionally, to remove the kind cluster, use the `clean-kind` target. The kind registry isn't removed so the images are cached if you wish to pull from it again. To also remove the kind registry, use the `clean-kind-registry` target.
- Use the `clean-all` target to remove all containers.
- **WARNING: Intel® SceneScape data isn't persisted, uninstalling the release will lead to data loss.**
- **NON-SUDO USERS**: The `default` target will run the `install-deps` target to ensure that the `kind`, `kubectl`, `k9s` and `helm` binaries are available and install them to /usr/local/bin with sudo. If your user does not have sudo access, check the comments for the `install-deps` target and edit the Makefile accordingly.

## FAQ

- How do I verify that everything is working properly?
Run `k9s` and check that the Intel® SceneScape pods are `Ready` and in the `Running` status. If they're stuck in an error status, refer to the steps in Troubleshooting.

## Troubleshooting

- If the scene controller does not seem to be running (no dots moving in the scene), restart the scene deployment.
- If your pods can't pull the images, check to see whether the registry container is on the same docker network as the kind cluster container.
Troubleshoot by running `docker inspect kind`. If they are not, run `docker network connect "kind" "kind-registry"`.
- If you can't access the Intel® SceneScape webUI, make sure Intel® SceneScape on Docker isn't running.
70 changes: 2 additions & 68 deletions kubernetes/scenescape-chart/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,72 +2,6 @@

Scene-based AI software framework.

## Overview
## Docs

Intel® SceneScape is a software framework that enables spatial awareness by integrating data from cameras and other sensors into scenes. It simplifies application development by providing near-real-time, actionable data about the state of the scene, including what, when, and where objects are present, along with their sensed attributes and environment. This scene-based approach makes it easy to incorporate and fuse sensor inputs, enabling analysis of past events, monitoring of current activities, and prediction of future outcomes from scene data.

Even with a single camera, transitioning to a scene paradigm offers significant advantages. Applications are written against the scene data directly, allowing for flexibility in modifying the sensor setup. You can move, modify, remove, or add cameras and sensors without changing your application or business logic. As you enhance your sensor array, the data quality improves, leading to better insights and decisions without altering your underlying application logic.

Intel® SceneScape turns raw sensor data into actionable insights by representing objects, people, and vehicles within a scene. Applications can access this information to make informed decisions, such as identifying safety hazards, detecting equipment issues, managing queues, correcting product placements, or responding to emergencies.

## How It Works

Intel® SceneScape uses advanced AI algorithms and hardware to process data from cameras and sensors, maintaining a dynamic scene graph that includes 3D spatial information and time-based changes. This enables developers to write applications that interact with a digital version of the environment in near real-time, allowing for responsive and adaptive application behavior based on the latest sensor data.

The framework leverages the Intel® Distribution of OpenVINO™ toolkit to efficiently handle sensor data, enabling developers to write applications that can be deployed across various Intel® hardware accelerators like CPUs, GPUs, VPUs, FPGAs, and GNAs. This ensures optimized performance and scalability.

A key goal of Intel® SceneScape is to make writing applications and business logic faster, simpler, and easier. By defining each scene with a fixed local coordinate system, spatial context is provided to sensor data. Scenes can represent various environments, such as buildings, ships, aircraft, or campuses, and can be linked to a global geographical coordinate system if needed. Intel® SceneScape manages:

- Multiple scenes, each with its own coordinate system.
- A single parent scene for each sensor at any given time.
- The precise location and orientation of cameras and sensors within the scene, stored in the Intel® SceneScape database. This information is crucial for interpreting sensor data correctly.
- Compatibility with glTF scene graph representations.

Intel® SceneScape is built on a collection of containerized services that work together to deliver comprehensive functionality, ensuring seamless integration and operation.

![SceneScape architecture diagram](https://github.com/open-edge-platform/scenescape/blob/main/docs/user-guide/images/architecture.png)
Figure 1: Architecture Diagram

### Scene controller

System which maintains the current state of the scene, including tracked objects, cameras, and sensors.

### DLStreamer Pipeline Server

Deep Learning Streamer Pipeline Server (DL Streamer Pipeline Server) is a Python-based, interoperable containerized microservice for easy development and deployment of video analytics pipelines. It is built on top of GStreamer and Deep Learning Streamer (DL Streamer) , providing video ingestion and deep learning inferencing functionalities.

### Auto Camera Calibration

Computes camera parameters utilizing known priors and camera feed.

### MQTT broker

Mosquitto MQTT broker which acts as the primary message bus connecting sensors, internal components, and applications, including the web interface.

### Web server

Apache web server providing a Django-based web UI which allows users to view updates to the scene graph and manage scenes, cameras, sensors, and analytics. It also serves the Intel® SceneScape REST API.

### NTP server

Time server which maintains the reference clock and keeps clients in sync.

### SQL database

PostgreSQL database server which stores static information used by the web UI and the scene controller. No video or object location data is stored by Intel® SceneScape.

## Configuration

### Proxy Settings

If you're deploying Intel® SceneScape in an environment that requires proxy access to external resources, use the following best-practice values for `noProxy`:

```yaml
httpProxy: "http://your-proxy-server:port"
httpsProxy: "https://your-proxy-server:port"
noProxy: "localhost,127.0.0.1,.local,.svc,.svc.cluster.local,10.96.0.0/12,10.244.0.0/16,172.17.0.0/16"
```

For a detailed explanation of what to put in `no_proxy` and why, see the [Proxy Configuration section in the top-level README](../README.md#proxy-configuration).

These settings will be applied to all Intel® SceneScape containers as environment variables, enabling them to access external resources through your corporate proxy.
See the [Documentation](https://docs.openedgeplatform.intel.com/2025.1/scenescape/index.html) for more information.
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Release.Name }}-broker
namespace: {{ .Release.Namespace }}
data:
mosquitto-secure.conf: |
{{ .Files.Get "files/broker/mosquitto-secure.conf" | indent 4 }}
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Release.Name }}-broker
namespace: {{ .Release.Namespace }}
labels:
app: {{ .Release.Name }}-broker
spec:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ .Release.Name }}-datasets-pvc
namespace: {{ .Release.Namespace }}
annotations:
"helm.sh/hook": pre-install
"helm.sh/hook-weight": "{{ int .Values.pvc.hookWeight }}"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Release.Name }}-queuing-cams
namespace: {{ .Release.Namespace }}
labels:
app: {{ .Release.Name }}-queuing-cams
spec:
Expand All @@ -24,7 +23,7 @@ spec:
runAsGroup: 0
initContainers:
- name: wait-for-rtsp
image: curlimages/curl:latest
image: {{ .Values.dlspsWaitImage.repository }}/{{ .Values.dlspsWaitImage.name }}:{{ .Values.dlspsWaitImage.tag }}
command: ["sh", "-c", "until nc -z mediaserver 8554; do echo waiting for mediaserver; sleep 2; done"]
securityContext:
{{ include "defaultContainerSecurityContext" . | indent 10 }}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Release.Name }}-retail-cams
namespace: {{ .Release.Namespace }}
labels:
app: {{ .Release.Name }}-retail-cams
spec:
Expand All @@ -24,7 +23,7 @@ spec:
runAsGroup: 0
initContainers:
- name: wait-for-rtsp
image: curlimages/curl:latest
image: {{ .Values.dlspsWaitImage.repository }}/{{ .Values.dlspsWaitImage.name }}:{{ .Values.dlspsWaitImage.tag }}
command: ["sh", "-c", "until nc -z mediaserver 8554; do echo waiting for mediaserver; sleep 2; done"]
securityContext:
{{ include "defaultContainerSecurityContext" . | indent 10 }}
Expand Down
Loading
Loading