Skip to content

Commit 373b185

Browse files
authored
Add recurring run demo (#917)
* Add recurring run demo
1 parent 881853e commit 373b185

File tree

7 files changed

+465
-0
lines changed

7 files changed

+465
-0
lines changed

demos/README.md

+4
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,7 @@ that's suitable for presentation to public audiences.
1111

1212
* [Simple pipeline](simple_pipeline/): highlights the use of pipelines and
1313
hyperparameter tuning on a GKE cluster with node autoprovisioning.
14+
15+
* [Recurring Run](recurring_run/): A simple demo that illustrates how to use
16+
the Kubeflow Pipelines SDK to provision [recurring
17+
runs](https://www.kubeflow.org/docs/components/pipelines/concepts/run/).

demos/recurring/README.md

+27
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Kubeflow demo - Recurring runs with the KFP SDK
2+
3+
## 1. Setup your environment
4+
5+
This demo assumes that you have a functioning Kubeflow Pipelines deployment. If
6+
not, follow the instructions
7+
[here](https://www.kubeflow.org/docs/components/pipelines/installation/) and
8+
[here](https://www.kubeflow.org/docs/components/pipelines/sdk/install-sdk/).
9+
10+
This demo has been verified to work with:
11+
- KFP version `1.7.1`
12+
- KFP SDK version `1.8.11`
13+
14+
Activate the conda environment you created following the above steps.
15+
16+
Create a Jupyter kernel for your conda environment.
17+
18+
```bash
19+
ipython kernel install --name "kfp" --user
20+
```
21+
22+
## 2. Run the KFP SDK script
23+
24+
Step through the provided [notebook](recurring.ipynb) to create a recurring run
25+
using the KFP SDK. Make sure to select the `kfp` kernel that you created
26+
earlier.
27+

demos/recurring/component.yaml

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
name: Download
2+
inputs:
3+
- {name: Url, type: URI}
4+
metadata:
5+
annotations:
6+
author: Alexander Perlman <[email protected]>
7+
implementation:
8+
container:
9+
image: alpine/curl
10+
command:
11+
- sh
12+
- -exc
13+
- |
14+
url="$0"
15+
path='/tmp/script'
16+
curl "$url" -o "$path"
17+
chmod 700 "$path"
18+
/bin/sh "$path"
19+
- inputValue: Url

demos/recurring/download.yaml

+56
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
apiVersion: argoproj.io/v1alpha1
2+
kind: Workflow
3+
metadata:
4+
generateName: pipeline-
5+
annotations: {pipelines.kubeflow.org/kfp_sdk_version: 1.8.11, pipelines.kubeflow.org/pipeline_compilation_time: '2022-01-31T21:51:10.599476',
6+
pipelines.kubeflow.org/pipeline_spec: '{"inputs": [{"name": "url"}], "name": "Pipeline"}'}
7+
labels: {pipelines.kubeflow.org/kfp_sdk_version: 1.8.11}
8+
spec:
9+
entrypoint: pipeline
10+
templates:
11+
- name: download
12+
container:
13+
args: []
14+
command:
15+
- sh
16+
- -exc
17+
- |
18+
url="$0"
19+
path='/tmp/script'
20+
curl "$url" -o "$path"
21+
chmod 700 "$path"
22+
/bin/sh "$path"
23+
- '{{inputs.parameters.url}}'
24+
image: alpine/curl
25+
inputs:
26+
parameters:
27+
- {name: url}
28+
metadata:
29+
annotations: {author: Alexander Perlman <[email protected]>, pipelines.kubeflow.org/component_spec: '{"implementation":
30+
{"container": {"command": ["sh", "-exc", "url=\"$0\"\npath=''/tmp/script''\ncurl
31+
\"$url\" -o \"$path\"\nchmod 700 \"$path\"\n/bin/sh \"$path\"\n", {"inputValue":
32+
"Url"}], "image": "alpine/curl"}}, "inputs": [{"name": "Url", "type": "URI"}],
33+
"metadata": {"annotations": {"author": "Alexander Perlman <[email protected]>"}},
34+
"name": "Download"}', pipelines.kubeflow.org/component_ref: '{"digest":
35+
"1bb47e384d056817b16202398d1e5fc8ce02daf1e40f69e3103218402c05437b", "url":
36+
"https://raw.githubusercontent.com/droctothorpe/examples/master/demos/recurring/component.yaml"}',
37+
pipelines.kubeflow.org/arguments.parameters: '{"Url": "{{inputs.parameters.url}}"}'}
38+
labels:
39+
pipelines.kubeflow.org/kfp_sdk_version: 1.8.11
40+
pipelines.kubeflow.org/pipeline-sdk-type: kfp
41+
pipelines.kubeflow.org/enable_caching: "true"
42+
- name: pipeline
43+
inputs:
44+
parameters:
45+
- {name: url}
46+
dag:
47+
tasks:
48+
- name: download
49+
template: download
50+
arguments:
51+
parameters:
52+
- {name: url, value: '{{inputs.parameters.url}}'}
53+
arguments:
54+
parameters:
55+
- {name: url}
56+
serviceAccountName: pipeline-runner

0 commit comments

Comments
 (0)