JupyterHub Deployment on Nova

This README will walk you through deploying JupyterHub on the Nova kubernetes cluster, for the purpose of running code notebooks for ML/AI model development.

Values

config.yaml contains all of the Helm values necessary for deployment. Please review them to confirm that they meet your needs.

Installation in k8s cluster

Add the JupyterHub helm chart repository

helm repo add jupyterhub https://hub.jupyter.org/helm-chart/
helm repo update

Install the helm chart, changing <NAMESPACE> to your namespace of choice and <RELEASE NAME> to a unique helm release name:

helm upgrade --cleanup-on-fail \
  --install <RELEASE NAME> jupyterhub/jupyterhub \
  --namespace <NAMESPACE> \
  --create-namespace \
  --version=3.3.8 \
  --values config.yaml

Port Forwarding

1. Open a separate terminal on your desktop
2. Use ssh to port forward a connection to your k8s cluster. This may look like ssh -D 1080 -q -N <user>@<cluster_address>
3. Leave the terminal open while using JupyterHub (see next sections)

Using k9s

K9s (note the "9" rather than "8") is a neat command-line tool for managing your Kubernetes cluster resources without typing out long kubectl commands. We find k9s to be a convenient tool for tasks like this one: getting a helm chart deployed, monitoring its status, and port-forwarding relevant services to your local computer. Please note that k9s connects to your cluster using your kubeconfig files. If you have not yet configured your kubeconfig, please contact your cluster administrator for access.

1. Install k9s
2. Run k9s in your terminal to launch k9s
3. Type :ns, then hit enter to list the namespaces on your kubernetes cluster
4. Navigate using your arrow keys, and select the "jupyterhub" namespace by hitting enter. You will see a list of pods in your selected namespace.
5. View Service resources in the current namespace by typing :Service and hitting enter
6. Use the arrow keys to hover over the "proxy-public" service
7. Port-forward a the service using the shortcut Shift + F and navigating the popup using your arrow keys and enter.
8. The service (in this case JupyterHub) will be accessible in your web browser at your chosen port, e.g. http://localhost:8000

Using JupyterHub for AI/ML

1. Log in with username admin, password admin
2. Choose the pytorch container from the list
3. Create or download (e.g. this one) a code notebook that utilizes pytorch to train an AI/ML model
4. The notebook runs!