parallelize tests on GHA

[evgeni]

alt to #9920 and #9919

[evgeni]

I really wish GH would allow include to create a new axis…

[evgeni]

I wonder if gha-matrix should gain a base-matrix param, which is used a base and then matrix.json is joined therein?

>>> import json
>>> base_matrix_input = '{"task":["a","b","c"]}'
>>> matrix_json = '{"ruby":["2.7","3.0"]}'
>>> matrix = json.loads(base_matrix_input)
>>> matrix.update(json.loads(matrix_json))
>>> matrix
{'task': ['a', 'b', 'c'], 'ruby': ['2.7', '3.0']}

[ekohl]

Right now we set up node as well, but I don't think it's actually needed for these tasks. It is for some though, which we should add. Kind of related to what I started in theforeman/actions#18.

For foreman.git it'll be simple: we can assume that package.json is present. For plugins we can't, but your approach here leads to a thought: how would the matrix look if we take tasks and their dependencies into account. Would it be easier, or only add more complexity?

[ekohl]

Another thought: in gha-puppet we have a dummy task at the end which we can add as a required step to verify the whole matrix. Today we don't have the Jenkins task as a required step, but we could add that. Having a single workflow makes that easier so I too prefer this approach to the other ones.

[evgeni]

Right now we set up node as well, but I don't think it's actually needed for these tasks. It is for some though, which we should add. Kind of related to what I started in theforeman/actions#18.

For foreman.git it'll be simple: we can assume that package.json is present. For plugins we can't, but your approach here leads to a thought: how would the matrix look if we take tasks and their dependencies into account. Would it be easier, or only add more complexity?

Is the question "How do we avoid the cost of setting up Node and installing NPM dependencies for the branches that do not test JS things (= all of the existing ones, as we do not call webpack:compile and test:integration)?" OR "How do we avoid that cost for plugins that do not have Webpack/JS at all"?
The former requires knowledge about the dependencies of the task while the later requires knowledge about the dependencies of the plugin.
I don't think we can easily map task dependencies in the matrix, without it getting too wild. (We can hard code a few common task names, but not sure it's worth it)
For the plugin deps, going by package.json is probably a good thing as you did in actions#18.

[evgeni]

Another thought: in gha-puppet we have a dummy task at the end which we can add as a required step to verify the whole matrix. Today we don't have the Jenkins task as a required step, but we could add that. Having a single workflow makes that easier so I too prefer this approach to the other ones.

I can take a stab at this, let's see if GHA likes colons in ids.

Edit: no colons needs, phew

[ekohl]

We could introduce rake tasks that fail if no package-lock.json file is present (or some other marker). IMHO it shouldn't call npm install because in CI you may want to ensure a cache, but at least be explicit about dependencies.

[evgeni]

Looks like what you wanted?

[ekohl]

I think we should include the task in the Gemfile.lock filename. It's likely it'll be exactly the same in each task, but not guaranteed. Other than that, 👍 on this.

I debated dropping Node and making these ruby-only tests, but that's for a follow up.

[evgeni]

Error: Artifact name is not valid: Gemfile-test:units-ruby-2.7-node-14-pg-12.lock. Contains the following character: Colon :

Meh

I've taken it out again. GH has no good way to do string substitution inside their templating language :/

[ekohl]

I've taken it out again. GH has no good way to do string substitution inside their templating language :/

"yay"

[evgeni]

But it also produces only one artifact:

[ekohl]

I suspect you each one overwrites the other. https://github.com/theforeman/foreman/actions/runs/6980908724?pr=9923 does produce multiple, but that only has a simple definition right now.

[evgeni]

Probably. Shall we merge as-is?