Skip to content

Commit c4740c1

Browse files
committed
cleaning up
Signed-off-by: Fellipe Resende <[email protected]>
1 parent 53e19be commit c4740c1

File tree

2 files changed

+22
-19
lines changed

2 files changed

+22
-19
lines changed

content/en/docs/components/spark-operator/user-guide/notebooks-spark-operator.md

Lines changed: 22 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -19,14 +19,16 @@ The following diagram illustrates how the components work together:
1919
class="mt-3 mb-3 border rounded">
2020
</img>
2121

22+
---
23+
2224
## Overview
2325

2426
In a typical Kubeflow setup, users access JupyterLab Notebooks through the central dashboard. These notebooks can now be configured to run PySpark code remotely through kernels managed by Jupyter Enterprise Gateway (JEG).
2527

2628
Behind the scenes:
2729

2830
1. JEG receives execution requests from notebooks.
29-
2. JEG creates `SparkApplication` Custom Resources.
31+
2. JEG creates and submits `SparkApplication` Custom Resources.
3032
3. The Spark Operator handles the lifecycle of Spark driver and executor pods in Kubernetes.
3133

3234
This architecture enables scalable, elastic execution of big data or distributed ML workloads.
@@ -36,12 +38,13 @@ This architecture enables scalable, elastic execution of big data or distributed
3638
- A running Kubeflow deployment with Notebook Controller enabled
3739
- Spark Operator installed and configured in the cluster
3840
- Helm installed locally
39-
- (Optional) Minikube for local development or testing
4041

4142
---
4243

4344
## Step 1: Deploy Enterprise Gateway
4445

46+
This step creates a dedicated Kubernetes namespace (enterprise-gateway) and sets up a local persistent volume and claim using hostPath.
47+
4548
Begin by creating the necessary storage resources. Save the following manifest as `enterprise-gateway-storage.yaml`:
4649

4750
```yaml
@@ -80,28 +83,15 @@ spec:
8083
storage: 1Gi
8184

8285
```
83-
Apply it:
8486

85-
This step creates a dedicated Kubernetes namespace (enterprise-gateway) and sets up a local persistent volume and claim using hostPath.
87+
Apply it:
8688
```yaml
8789
kubectl apply -f enterprise-gateway-storage.yaml
8890
```
89-
Then deploy Enterprise Gateway using Helm:
91+
Now we will be deploying Jupyter Enterpise Gateway with support for remote kernel management and persistent kernelspec storage.
9092

91-
The command below uses a YAML file named enterprise-gateway-helm.yaml, which includes an example configuration shown below.
93+
Save the following manifest as `enterprise-gateway-helm.yaml` which will be used as the basic configuration for the gateway.
9294

93-
```yaml
94-
helm upgrade --install enterprise-gateway \
95-
https://github.com/jupyter-server/enterprise_gateway/releases/download/v3.2.3/jupyter_enterprise_gateway_helm-3.2.3.tar.gz \
96-
--namespace enterprise-gateway \
97-
--values enterprise-gateway-helm.yaml \
98-
--create-namespace \
99-
--wait
100-
101-
```
102-
Example configuration yaml:
103-
104-
Save the following manifest as `enterprise-gateway-helm.yaml`.
10595
```yaml
10696
image: elyra/enterprise-gateway:3.2.3
10797
imagePullPolicy: Always
@@ -128,7 +118,20 @@ kip:
128118
defaultContainerRegistry: quay.io
129119

130120
```
131-
This deploys JEG with remote kernel management and persistent kernelspec storage.
121+
122+
Then deploy Enterprise Gateway using Helm:
123+
124+
The command below uses a YAML file named enterprise-gateway-helm.yaml, which includes an example configuration shown above.
125+
126+
```yaml
127+
helm upgrade --install enterprise-gateway \
128+
https://github.com/jupyter-server/enterprise_gateway/releases/download/v3.2.3/jupyter_enterprise_gateway_helm-3.2.3.tar.gz \
129+
--namespace enterprise-gateway \
130+
--values enterprise-gateway-helm.yaml \
131+
--create-namespace \
132+
--wait
133+
134+
```
132135

133136
## Step 2: Configure the Notebook to connect to the Jupyter Gateway
134137

93.3 KB
Loading

0 commit comments

Comments
 (0)