Switch to the authentication token-based runner creation workflow

Starting in GitLab v15.10 (and enabled by default in v16.0), the runner
creation uses a workflow based on authentication tokens instead of
registration tokens. This introduces a new runner property named
system_id that needs to be passed to the /jobs/request API.

Extend the runner with a new system_id parameter (passed through the
environment variable 'RUNNER_SYSTEM_ID'. Additionall update the
documentation in README.md to explain the new workflow.

Signed-off-by: Laurent Pinchart <[email protected]>

Author: pinchartl

README.md

@@ -9,42 +9,64 @@ A gitlab runner implementation intended to bridge gitlab to
 The lava gitlab runner only requires network (https) access to both the gitlab
 server and the lava server(s), so can pretty much run anywhere.
 
-The runner requires a runner token to connect to the gitlab server and pick up
-jobs. The lava url and token are provided by the gitlab-ci jobs so don't have
-to be configured on the runner.
+The runner requires a runner authentication token to connect to the gitlab
+server and pick up jobs. The lava url and token are provided by the gitlab-ci
+jobs so don't have to be configured on the runner.
+
+## Creating a new runner
+
+A new runner must be created on the GitLab server. This can be done using the
+[runner creation API](https://docs.gitlab.com/ee/api/users.html#create-a-runner-linked-to-a-user),
+or manually in the GitLab
+[runner management web interface](https://docs.gitlab.com/ee/ci/runners/runners_scope.html).
+Make sure to follow the runner creation with an authentication token workflow,
+as the registration token workflow is deprecated.
+
+One key parameter provided when creating the runner is `run_untagged=false` (or
+leaving the `Run untagged jobs` box unchecked in the web interface), which will
+make the runner *only* pickup jobs which matches its tags. This is important to
+prevent the runner from picking up "normal" jobs which it will not be able to
+process.
 
-## Registering a new runner
+When the runner is created GitLab provides an authentication token starting
+with `glrt-`. This token should be provided to the runner for its GitLab
+connection, along with a system ID that identifies the machine on which the
+runner is executed.
 
-The runner cannot register itself with the gitlab server, so this has to be
-done by hand using the gitlab
-[runner registration API](https://docs.gitlab.com/ee/api/runners.html#register-a-new-runner).
+The system ID should be a unique string. GitLab doesn't currently require any
+particular formatting, but it is recommended to follow the way the official
+`gitlab-runner` creates system IDs:
 
-The registration token can be retrieved from the runners section in the Gitlab
-administration area. With that token the runner can be register using a curl
-command like:
-```
-curl --request POST "https://GITLAB_URL/api/v4/runners"  \
-  --form "description=Lava runner" \
-  --form "run_untagged=false" \
-  --form "tag_list=lava-runner" \
-  --form "token=REGISTRATION_TOKEN"
-```
+- Deriving it from the machine ID, found in `/var/lib/dbus/machine-id` or
+  `/etc/machine-id`, but concatenating the machine ID with the string
+  "gitlab-runner", taking the first 12 characters of its SHA256 hash in hex
+  form, and prepending it with `s_`.
 
-As a response to this command a new token for the registered runner will be
-provided, this token should be provided to the runner for it's gitlab
-connection.
+- Generating a random 12-character string using letters and digits
+  (`[a-z][A-Z][0-9]`), and prepending it with `r_`.
 
-One thing to key parameter provided here is `run_untagged=false`, which will
-make the runner *only* pickup jobs which matches its tag. This is important to
-prevent the runner from picking up "normal" jobs which it will not be able to
-process.
+In either case the system ID should be recorded in a persistent storage, along
+with the authentication token, and be passed to the `Runner::new()` function.
+
+The token can be verified using a curl command like:
+
+```shell
+curl --request POST "https://GITLAB_URL/api/v4/runners/verify"  \
+  --form "token=AUTHENTICATION_TOKEN" \
+  --form "system_id=SYSTEM_ID"
+```
+
+This step is optional. If performed, it will pre-register the system ID with
+the GitLab server. Otherwise the system ID will be registered the first time
+the runner pings for jobs.
 
 ## Running the runner from the source tree
 
 The runner can be build using Cargo like any Rust program. To run it the url to
-the gitlab server and the token have to be either provided on the command line
-or in the `GITLAB_URL` and `GITLAB_TOKEN` environment variables. For example to
-run directly from source `cargo run https://gitlab.myserver.org RUNNER_TOKEN`.
+the gitlab server, the token and the system ID have to be either provided on
+the command line or in the `GITLAB_URL`, `GITLAB_TOKEN` and `RUNNER_SYSTEM_ID`
+environment variables. For example to run directly from source `cargo run
+https://gitlab.myserver.org RUNNER_TOKEN SYSTEM_ID`.
 
 ## Running from a docker image
 

src/main.rs

@@ -88,6 +88,8 @@ struct Opts {
     server: Url,
     #[structopt(env = "GITLAB_TOKEN")]
     token: String,
+    #[structopt(env = "RUNNER_SYSTEM_ID")]
+    system_id: String,
     #[structopt(short, long, env = "RUNNER_LOG")]
     log: Option<String>,
     #[structopt(
@@ -927,7 +929,7 @@ async fn main() {
     let dir = tempfile::tempdir().unwrap();
 
     let (mut runner, layer) =
-        Runner::new_with_layer(opts.server, opts.token, dir.path().to_path_buf());
+        Runner::new_with_layer(opts.server, opts.token, opts.system_id, dir.path().to_path_buf());
 
     let log_targets: filter::Targets = if let Some(log) = opts.log {
         log.parse().unwrap()