Add TESK_API_TASKMASTER_STORAGE_CLASS, to be able to change the type of volume created in tasks

Author: lvarin

README.md

@@ -0,0 +1,96 @@
+# Rclone
+
+For this template we are using the official [rclone docker image](https://hub.docker.com/r/rclone/rclone) in docker hub.
+
+In order to install this version of the template into a namespace do:
+
+```bash
+$ oc create -f rclone.yaml
+template.template.openshift.io/rclone created
+```
+
+Then use the template from the web interface, or from the command line:
+
+```bash
+$ oc process rclone \
+    ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX \
+    SECRET_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX \
+    PVC=existing_pvc \
+    BUCKET_DIR=existing_bucket/existing/path
+```
+
+## Command line tests
+
+In `rclone.conf`, replace the credentials (`${ACCESSKEY}` and `${SECRETKEY}`) and the endpoint (`${S3HOST}`) with their appropriate values.
+
+```bash
+$ cp rclone.conf ~/.config/rclone/
+$ docker run --rm \
+    --volume ~/.config/rclone:/config/rclone \
+    --volume ~/data:/data:shared \
+    --user $(id -u):$(id -g) \
+    --volume /etc/passwd:/etc/passwd:ro --volume /etc/group:/etc/group:ro \
+    --device /dev/fuse --cap-add SYS_ADMIN --security-opt apparmor:unconfined \
+    rclone/rclone \
+    listremotes
+```
+
+Should output:
+
+```bash
+default:
+```
+
+Then, in order to list the buckets in the project:
+
+```bash
+$ docker run --rm \
+    --volume ~/.config/rclone:/config/rclone \
+    --volume ~/data:/data:shared \
+    --user $(id -u):$(id -g) \
+    --volume /etc/passwd:/etc/passwd:ro --volume /etc/group:/etc/group:ro \
+    --device /dev/fuse --cap-add SYS_ADMIN --security-opt apparmor:unconfined \
+    rclone/rclone \
+    lsd default:
+```
+
+With this information, it is possible to mount a remote S3 remote bucket into a local folder:
+
+```bash
+$ sudo chcon -Rt svirt_sandbox_file_t ~/data
+$ sudo docker run --rm \
+    --volume ~/.config/rclone:/config/rclone \
+    --volume ~/data:/data:shared \
+    --device /dev/fuse --cap-add SYS_ADMIN \
+    rclone/rclone mount default:28022020-gonzalez /data/home
+```
+
+Meanwhile the docker command is running, you can read the files in the bucket from a different terminal (as root):
+
+```bash
+sudo ls ~/data
+```
+
+## Obtain the S3 credentials
+
+We will use the command line to obtain the credentials. First install the command line client:
+
+```bash
+pip install python-openstackclient
+```
+
+Then go to [Access & Security](https://pouta.csc.fi/dashboard/project/access_and_security/), download the OpenStack RC File v2.0, and source it:
+
+```bash
+$ source ~/Downloads/project_XXXXXXX-openrc.sh
+Please enter your OpenStack Password for project project_2001316 as user <USER>:
+
+```
+
+Finally you can list the credentials:
+
+```bash
+openstack ec2 credentials list -f yaml
+```
+
+We are interested in `Access:` and `Secret:`.

rclone.conf

@@ -0,0 +1,9 @@
+[default]
+type = s3
+provider = Other
+env_auth = false
+access_key_id = ${ACCESSKEY}
+secret_access_key = ${SECRETKEY}
+endpoint = ${S3HOST}
+location_constraint = regionOne
+

rclone.yaml

@@ -0,0 +1,107 @@
+#
+# https://hub.docker.com/r/rclone/rclone
+#
+apiVersion: template.openshift.io/v1
+kind: Template
+labels:
+  template: rclone
+message: rclone template
+metadata:
+  annotations:
+    description: Deploys a rclone Cronjob to synchronize a PVC into a S3 bucket/folder
+    iconClass: "fa fa-cubes"
+    openshift.io/display-name: rclone
+    openshift.io/provider-display-name: CSC
+  name: rclone
+objects:
+- apiVersion: batch/v1beta1
+  kind: CronJob
+  metadata:
+    labels:
+      app: rclone
+    name: rclone
+  spec:
+    schedule: ${SCHEDULE}
+    jobTemplate:
+      spec:
+        template:
+          metadata:
+            labels:
+              parent: "rclone"
+          spec:
+            containers:
+            - name: rclone
+              image: rclone/rclone:1
+              command: ["rclone", "sync", "-P", "/mnt", "default:${BUCKET_DIR}"]
+              volumeMounts:
+              - mountPath: /.rclone.conf
+                name: rclone-conf
+                subPath: rclone.conf
+              - mountPath: /mnt
+                name: pvc
+            restartPolicy: OnFailure
+            volumes:
+            - name: rclone-conf
+              secret:
+                secretName: rclone-conf
+                defaultMode: 420
+                items:
+                  - key: rclone.conf
+                    path: rclone.conf
+            - name: pvc
+              persistentVolumeClaim:
+                claimName: ${PVC}
+- apiVersion: v1
+  kind: Secret
+  metadata:
+    name: rclone-conf
+    labels:
+      app: rclone
+  type: Opaque
+  stringData:
+    "rclone.conf": |-
+      [default]
+      type = s3
+      provider = Other
+      env_auth = false
+      access_key_id = ${ACCESS_KEY}
+      secret_access_key = ${SECRET_KEY}
+      endpoint = ${S3HOST}
+      location_constraint = regionOne
+
+parameters:
+  - name: ACCESS_KEY
+    required: true
+    description: Access key
+    displayName: Access key
+    from: '[a-zA-Z0-9]{32}'
+
+  - name: SECRET_KEY
+    description: "Secret key"
+    displayName: Secret key
+    required: true
+    from: '[a-zA-Z0-9]{32}'
+
+  - name: PVC
+    description: "Name on the persistent volume claim to synchronize from"
+    displayName: PersistentVolumeClaim
+    required: true
+
+  - name: BUCKET_DIR
+    description: "Bucket and directory to synchronize to. The format should be <bucket>/path/path2"
+    displayName: Bucket and directory
+    required: true
+
+  - name: S3HOST
+    description: "Host name for the S3 server"
+    displayName: Host name
+    required: true
+    value: "object.pouta.csc.fi"
+
+  - name: SCHEDULE
+    description: >
+      When to run the cronjob, see https://en.wikipedia.org/wiki/Cron for reference
+    displayName: Schedule
+    required: true
+    value: "30 * * * *"
+