Skip to content

docs: add notes about distributed deployment to installation #919

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
daniel-weisse merged 3 commits into from
Jan 12, 2026
Merged

docs: add notes about distributed deployment to installation #919

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1d8e658
docs: add notes about distributed deployment to installation
daniel-weisse Jan 7, 2026
5bf504e
fixup! docs: add notes about distributed deployment to installation
daniel-weisse Jan 12, 2026
b80013e
docs: backport to v1.9
daniel-weisse Jan 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
135 changes: 105 additions & 30 deletions docs/docs/deployment/kubernetes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,17 +89,44 @@
Use MarbleRun's [CLI](../reference/cli.md) that facilitates the administrative tasks.
You can install MarbleRun using the CLI as follows:

* For a cluster with SGX support:
<Tabs groupId="installation">

```bash
marblerun install --domain=mycluster.uksouth.cloudapp.azure.com
```
<TabItem value="distributed" label="Distributed">

* For a cluster without SGX support:
```bash
marblerun install --distributed-deployment --domain=mycluster.uksouth.cloudapp.azure.com
```

The distributed deployment mode allows you to scale the MarbleRun Coordinator across different nodes on your cluster.

Check warning on line 100 in docs/docs/deployment/kubernetes.md

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[Microsoft.Vocab] Verify your use of 'allows' with the A-Z word list.

Raw Output:
{"message": "[Microsoft.Vocab] Verify your use of 'allows' with the A-Z word list.", "location": {"path": "docs/docs/deployment/kubernetes.md", "range": {"start": {"line": 100, "column": 33}}}, "severity": "INFO"}
This ensures high availability of MarbleRun's control plane and is the recommended deployment mode for production use cases.

See [the key management documentation](../architecture/security.md#distributed-coordinator) for more details on how distributed MarbleRun deployments manage their state.

</TabItem>

<TabItem value="standalone" label="Standalone">

```bash
marblerun install --domain=mycluster.uksouth.cloudapp.azure.com
```

The standalone installation deploys a single MarbleRun Coordinator.
Please note that moving the Pod to a different node requires [manual recovery](../workflows/recover-coordinator.md).
Consider using a distributed deployment if availability, scaling, or automatic recovery are a concern.

</TabItem>

<TabItem value="simulation" label="Simulation Mode">

```bash
marblerun install --domain=mycluster.uksouth.cloudapp.azure.com --simulation
```
```bash
marblerun install --domain=mycluster.uksouth.cloudapp.azure.com --simulation
```

MarbleRun's simulation mode deploys a single Coordinator instance without the need for SGX support.
Since the Coordinator isn't actually running in an SGX enclave, this mode must only be used for evaluation and testing purposes.

Check warning on line 126 in docs/docs/deployment/kubernetes.md

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[Microsoft.Passive] 'be used' looks like passive voice.

Raw Output:
{"message": "[Microsoft.Passive] 'be used' looks like passive voice.", "location": {"path": "docs/docs/deployment/kubernetes.md", "range": {"start": {"line": 126, "column": 85}}}, "severity": "INFO"}

</TabItem>
</Tabs>

This command will pull the latest Helm chart from [our repository](https:/helm.edgeless.systems) and manages the installation of said chart.

Expand Down Expand Up @@ -132,30 +159,64 @@

Update the hostname with your cluster's FQDN.

* For a cluster with SGX support:
<Tabs groupId="installation">

```bash
helm install marblerun edgeless/marblerun \
--create-namespace \
-n marblerun \
--set coordinator.hostname=mycluster.uksouth.cloudapp.azure.com \
--set marbleInjector.start=true \
--set marbleInjector.useCertManager=true
```
<TabItem value="distributed" label="Distributed">

* For a cluster without SGX support:
```bash
helm install marblerun edgeless/marblerun \
--create-namespace \
-n marblerun \
--set coordinator.replicas=3 \
--set coordinator.distributedDeployment=true \
--set coordinator.hostname=mycluster.uksouth.cloudapp.azure.com \
--set marbleInjector.start=true \
--set marbleInjector.useCertManager=true
```

```bash
helm install marblerun edgeless/marblerun \
--create-namespace \
-n marblerun \
--set coordinator.resources=null \
--set coordinator.simulation=1 \
--set tolerations=null \
--set coordinator.hostname=mycluster.uksouth.cloudapp.azure.com \
--set marbleInjector.start=true \
--set marbleInjector.useCertManager=true
```
The distributed deployment mode allows you to scale the MarbleRun Coordinator across different nodes on your cluster.

Check warning on line 177 in docs/docs/deployment/kubernetes.md

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[Microsoft.Vocab] Verify your use of 'allows' with the A-Z word list.

Raw Output:
{"message": "[Microsoft.Vocab] Verify your use of 'allows' with the A-Z word list.", "location": {"path": "docs/docs/deployment/kubernetes.md", "range": {"start": {"line": 177, "column": 33}}}, "severity": "INFO"}
This ensures high availability of MarbleRun's control plane and is the recommended deployment mode for production use cases.

See [the key management documentation](../architecture/security.md#distributed-coordinator) for more details on how distributed MarbleRun deployments manage their state.

</TabItem>

<TabItem value="standalone" label="Standalone">

```bash
helm install marblerun edgeless/marblerun \
--create-namespace \
-n marblerun \
--set coordinator.hostname=mycluster.uksouth.cloudapp.azure.com \
--set marbleInjector.start=true \
--set marbleInjector.useCertManager=true
```

The standalone installation deploys a single MarbleRun Coordinator.
Please note that moving the Pod to a different node requires [manual recovery](../workflows/recover-coordinator.md).
Consider using a distributed deployment if availability, scaling, or automatic recovery are a concern.

</TabItem>

<TabItem value="simulation" label="Simulation Mode">

```bash
helm install marblerun edgeless/marblerun \
--create-namespace \
-n marblerun \
--set coordinator.resources=null \
--set coordinator.simulation=1 \
--set tolerations=null \
--set coordinator.hostname=mycluster.uksouth.cloudapp.azure.com \
--set marbleInjector.start=true \
--set marbleInjector.useCertManager=true
```

MarbleRun's simulation mode deploys a single Coordinator instance without the need for SGX support.
Since the Coordinator isn't actually running in an SGX enclave, this mode must only be used for evaluation and testing purposes.

Check warning on line 216 in docs/docs/deployment/kubernetes.md

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[Microsoft.Passive] 'be used' looks like passive voice.

Raw Output:
{"message": "[Microsoft.Passive] 'be used' looks like passive voice.", "location": {"path": "docs/docs/deployment/kubernetes.md", "range": {"start": {"line": 216, "column": 85}}}, "severity": "INFO"}

</TabItem>
</Tabs>

By default `coordinator.hostname` is set to `localhost`.
The domain is used as the CommonName in the Coordinator's TLS certificate.
Expand Down Expand Up @@ -198,13 +259,27 @@
marblerun install --dcap-pccs-url <PCCS_URL> --dcap-secure-cert <TRUE/FALSE>
```

or pass the path to a custom DCAP config file using `--dcap-qcnl-config-file`:

```bash
marblerun install --dcap-qcnl-config-file=/path/to/sgx_qcnl.conf
```

* Using Helm

```bash
helm install marblerun edgeless/marblerun \
--create-namespace \
-n marblerun \
--set coordinator.hostname=mycluster.uksouth.cloudapp.azure.com \
--set dcap.pccsUrl=<PCCS_URL> \
--set dcap.useSecureCert=<TRUE/FALSE>
```

or pass the path to a custom DCAP config to Helm:

```bash
helm install marblerun edgeless/marblerun \
--create-namespace \
-n marblerun \
--set-file dcap.qcnlConfig=/path/to/sgx_qcnl.conf
```
57 changes: 55 additions & 2 deletions docs/docs/reference/coordinator.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@

#### Returns

* `recoveryKeys` object
* `RecoverySecrets` object

An optional field that will be present if the manifest contains `RecoveryKeys`.
Key-value mapping of strings to strings, where the key matches each supplied key from `RecoveryKeys` in the manifest,
Expand All @@ -175,7 +175,7 @@
{
"status": "success",
"data": {
"recoveryKeys": {
"RecoverySecrets": {
"recoveryKey1": "ZW5jcnlwdGVkUmVjb3ZlcnlTZWNyZXQxCg==",
"recoveryKey2": "ZW5jcnlwdGVkUmVjb3ZlcnlTZWNyZXQyCg=="
}
Expand Down Expand Up @@ -407,6 +407,45 @@
</TabItem>
</Tabs>

## Retrieve Coordinator recovery public key

<Tabs groupId="apiVersion">
<TabItem value="v2" label="v2">

```http
GET /api/v2/recover/public-key
```

Retrieve a public key for encryption of recovery secrets.
Use the returned key to encrypt your decrypted recovery secret before sending it to a Coordinator for recovery.
See [the offline recovery secret signing workflow](../workflows/recover-coordinator.md#offline-recovery-secret-signing) for more details on how the endpoint is used.

Check warning on line 421 in docs/docs/reference/coordinator.md

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[Microsoft.Passive] 'is used' looks like passive voice.

Raw Output:
{"message": "[Microsoft.Passive] 'is used' looks like passive voice.", "location": {"path": "docs/docs/reference/coordinator.md", "range": {"start": {"line": 421, "column": 158}}}, "severity": "INFO"}

#### Returns

* `algorithm` string

Algorithm of the key, for example `RSA`.
Currently, only RSA is supported.

Check warning on line 428 in docs/docs/reference/coordinator.md

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[Microsoft.Passive] 'is supported' looks like passive voice.

Raw Output:
{"message": "[Microsoft.Passive] 'is supported' looks like passive voice.", "location": {"path": "docs/docs/reference/coordinator.md", "range": {"start": {"line": 428, "column": 25}}}, "severity": "INFO"}

* `ephemeralPublicKey` string

DER encoded public key

Example response:

```JSON
{
"status": "success",
"data": {
"algorithm": "RSA",
"ephemeralPublicKey": "MIICIjANBgkqhkiG9...EAAQ=="
}
}
```

</TabItem>
</Tabs>

## Retrieve MarbleRun managed secrets

<Tabs groupId="apiVersion">
Expand Down Expand Up @@ -907,6 +946,13 @@

An array of user IDs that haven't yet acknowledged the update.

* `recoverySecrets` object

An optional field that will be present if the manifest was successfully applied,
and the new manifest's `RecoveryKeys` differ from the previous one.

Check warning on line 952 in docs/docs/reference/coordinator.md

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[Microsoft.ComplexWords] Consider using 'earlier' instead of 'previous'.

Raw Output:
{"message": "[Microsoft.ComplexWords] Consider using 'earlier' instead of 'previous'.", "location": {"path": "docs/docs/reference/coordinator.md", "range": {"start": {"line": 952, "column": 59}}}, "severity": "INFO"}
Key-value mapping of strings to strings, where the key matches each supplied key from `RecoveryKeys` in the new manifest,
and the value is the base64 encoded encrypted recovery secret.

Example response:

```json
Expand Down Expand Up @@ -1015,6 +1061,13 @@

An array of user IDs that haven't yet acknowledged the update.

* `recoverySecrets` object

An optional field that will be present if the manifest was successfully applied,
and the new manifest's `RecoveryKeys` differ from the previous one.

Check warning on line 1067 in docs/docs/reference/coordinator.md

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[Microsoft.ComplexWords] Consider using 'earlier' instead of 'previous'.

Raw Output:
{"message": "[Microsoft.ComplexWords] Consider using 'earlier' instead of 'previous'.", "location": {"path": "docs/docs/reference/coordinator.md", "range": {"start": {"line": 1067, "column": 59}}}, "severity": "INFO"}
Key-value mapping of strings to strings, where the key matches each supplied key from `RecoveryKeys` in the new manifest,
and the value is the base64 encoded encrypted recovery secret.

Example response:

```JSON
Expand Down
3 changes: 3 additions & 0 deletions docs/styles/config/vocabularies/edgeless/accept.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ collateral
config
CPU
crypto
DCAP
DER
Dockerfile
emojivoto
Env
Expand All @@ -33,6 +35,7 @@ precheck
premain
Redis
runtime
SGX
subcommand
Tensorflow
toleration
Expand Down
Loading