Skip to content

Commit bad589e

Browse files
daniel-hutaodaniel-hutao
authored
Merge pull request #419 from merico-dev/docs-e2e-global-variables
docs/tests: add docs and e2e tests for variables
2 parents 34696e9 + 013a91a commit bad589e

File tree

5 files changed

+195
-12
lines changed

5 files changed

+195
-12
lines changed

‎.github/workflows/e2e-test.yml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -62,13 +62,13 @@ jobs:
6262
run: |
6363
aws eks update-kubeconfig --region ap-southeast-1 --name dtm-test
6464
- name: apply
65-
run: ./dtm apply -f ./test/e2e/yaml/e2e-config.yaml -y
65+
run: ./dtm apply -f ./test/e2e/yaml/e2e-config.yaml --var-file ./test/e2e/yaml/e2e-variables.yaml -y
6666
- name: apply twice
67-
run: ./dtm apply -f ./test/e2e/yaml/e2e-config.yaml -y
67+
run: ./dtm apply -f ./test/e2e/yaml/e2e-config.yaml --var-file ./test/e2e/yaml/e2e-variables.yaml -y
6868
- name: check if pod is ready
6969
run: while [[ $(kubectl get pods -l app=dtm-e2e-go -o 'jsonpath={..status.conditions[?(@.type=="Ready")].status}') != "True" ]]; do echo "pod not ready yet..."; sleep 3; done
7070
timeout-minutes: 10
7171
- name: verify
72-
run: ./dtm verify -f ./test/e2e/yaml/e2e-config.yaml
72+
run: ./dtm verify -f ./test/e2e/yaml/e2e-config.yaml --var-file ./test/e2e/yaml/e2e-variables.yaml
7373
- name: clean
74-
run: ./dtm delete -f ./test/e2e/yaml/e2e-config.yaml -y
74+
run: ./dtm delete -f ./test/e2e/yaml/e2e-config.yaml --var-file ./test/e2e/yaml/e2e-variables.yaml -y

‎docs/config.md

Lines changed: 176 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,176 @@
1+
# Config
2+
3+
This document summarizes the config file for DevStream.
4+
5+
DevStream uses a single YAML file to store your DevOps toolchain configuration.
6+
7+
## Default Config File
8+
9+
By default, `dtm` tries to use `./config.yaml` (under your current directory.)
10+
11+
## Specifying a Config File Explicitly
12+
13+
You can override the default value with `-f` or `--config-file`. Examples:
14+
15+
```shell
16+
dtm apply -f path/to/your/config.yaml
17+
dtm apply --config-file path/to/your/config.yaml
18+
```
19+
20+
## Config File Content
21+
22+
The config file only contains:
23+
24+
- Only one section (at the moment), which is `tools`.
25+
- `tools` is a list of dictionaries.
26+
- Each dictionary defines a DevOps "tool" which is managed by a DevStream plugin
27+
- Each dictionary (tool) has the following mandatory fields:
28+
- `name`: the name of the tool, string, without underscore
29+
- `plugin`: the plugin to be used
30+
- you can have duplicated `name` in one config file, you can also have duplicated `plugin` in one config file, but the `name + plugin` combination must be unique in one config file
31+
- Each dictionary (tool) has an optional field which is `options`, which in turn is a dictionary containing parameters for that specific plugin. For plugins' parameters, see [plugins](./plugins.md).
32+
- Each directory (tool) has an optional field which is `dependsOn`. Continue reading for detail about dependencies.
33+
34+
## Example Config File
35+
36+
`config.yaml`:
37+
38+
```yaml
39+
tools:
40+
- name: go-webapp-repo
41+
plugin: github-repo-scaffolding-golang
42+
options:
43+
org: devstream-io
44+
repo: dtm-e2e-go
45+
branch: main
46+
image_repo: dtme2etest/dtm-e2e-go
47+
```
48+
49+
## Dependencies
50+
51+
If you want tool A to be installed before tool B, you can let tool B depend on tool A.
52+
53+
The syntax for dependency is:
54+
55+
```yaml
56+
dependsOn: ["NAME1.PLUGIN1"]'
57+
```
58+
59+
Since `dependsOn` is a list, a tool can have multiple dependencies:
60+
61+
```
62+
dependsOn: [ "NAME1.PLUGIN1", "NAME2.PLUGIN2", "..."]
63+
```
64+
65+
In the following example, tool "go-webapp-repo" (using plugin github-repo-scaffolding-golang) will be installed before tool "golang-demo-actions" (using plugin githubactions-golang):
66+
67+
```yaml
68+
tools:
69+
- name: go-webapp-repo
70+
plugin: github-repo-scaffolding-golang
71+
options:
72+
org: devstream-io
73+
repo: dtm-e2e-go
74+
branch: main
75+
image_repo: dtme2etest/dtm-e2e-go
76+
- name: golang-demo-actions
77+
plugin: githubactions-golang
78+
dependsOn: ["go-webapp-repo.github-repo-scaffolding-golang"]
79+
options:
80+
org: ${{go-webapp-repo.github-repo-scaffolding-golang.outputs.org}}
81+
repo: ${{go-webapp-repo.github-repo-scaffolding-golang.outputs.repo}}
82+
language:
83+
name: go
84+
version: "1.17"
85+
branch: main
86+
build:
87+
enable: True
88+
test:
89+
enable: True
90+
coverage:
91+
enable: True
92+
```
93+
94+
## Variables
95+
96+
To not repeat yourself (DRY,) we can define some variables in a var file and use the vars in the config file.
97+
98+
### Default Variable File
99+
100+
The default var file is located at `./variables.yaml`. If this file doesn't exist, and no user-specified var file is provided, the config won't be rendered with any variables, apparently.
101+
102+
### Specifying a Variable File Explicitly
103+
104+
To override the default location of the variables file, use the `--var-file` option:
105+
106+
```shell
107+
dtm apply -f path/to/your/config.yaml --var-file path/to/your/variables.yaml
108+
```
109+
110+
### Variable File Content
111+
112+
The var file is a YAML file containing key-value pairs. Example:
113+
114+
"variables.yaml":
115+
116+
```yaml
117+
gitlabUser: ironcore864
118+
defaultBranch: main
119+
gitlabCIGolangTemplate: https://gitlab.com/ironcore864/go-hello-world/-/raw/main/go.tpl
120+
```
121+
122+
At the moment, nested/composite values (for example, the value is a list/dictionary) are not supported yet.
123+
124+
### Using a Variables File
125+
126+
To use these variables in a config file, use the following syntax:
127+
128+
```yaml
129+
[[ varNameHere ]]
130+
```
131+
132+
### Example Config File with the Use of Variables
133+
134+
`variables.yaml`:
135+
136+
```yaml
137+
gitlabUser: ironcore864
138+
defaultBranch: main
139+
gitlabCIGolangTemplate: https://gitlab.com/ironcore864/go-hello-world/-/raw/main/go.tpl
140+
```
141+
142+
Example config with the variables defined in the above `variables.yaml`:
143+
144+
`config.yaml`:
145+
146+
```yaml
147+
tools:
148+
- name: myapp
149+
plugin: gitlabci-generic
150+
options:
151+
pathWithNamespace: [[ gitlabUser ]]/go-hello-world
152+
branch: [[ defaultBranch ]]
153+
templateURL: [[ gitlabCIGolangTemplate ]]
154+
templateVariables:
155+
App: hello
156+
```
157+
158+
DevStream will render the config with the provided var file. After rendering, the config above is equivalent to the following content:
159+
160+
```yaml
161+
tools:
162+
- name: myapp
163+
plugin: gitlabci-generic
164+
options:
165+
pathWithNamespace: ironcore864/go-hello-world
166+
branch: main
167+
templateURL: https://gitlab.com/ironcore864/go-hello-world/-/raw/main/go.tpl
168+
templateVariables:
169+
App: hello
170+
```
171+
172+
```{toctree}
173+
---
174+
maxdepth: 1
175+
---
176+
```

‎docs/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ maxdepth: 1
88
---
99
overview
1010
quickstart_en
11+
config
1112
architecture
1213
core_concepts
1314
output

‎test/e2e/yaml/e2e-config.yaml

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,10 @@ tools:
22
- name: go-webapp-repo
33
plugin: github-repo-scaffolding-golang
44
options:
5-
org: devstream-io
6-
repo: dtm-e2e-go
7-
branch: main
8-
image_repo: dtme2etest/dtm-e2e-go
5+
org: [[ githubOrganization ]]
6+
repo: [[ repoName ]]
7+
branch: [[ defaultBranch ]]
8+
image_repo: [[ dockerRegistryUserName ]]/[[ repoName ]]
99
- name: golang-demo-actions
1010
plugin: githubactions-golang
1111
dependsOn: ["go-webapp-repo.github-repo-scaffolding-golang"]
@@ -15,7 +15,7 @@ tools:
1515
language:
1616
name: go
1717
version: "1.17"
18-
branch: main
18+
branch: [[ defaultBranch ]]
1919
build:
2020
enable: True
2121
test:
@@ -34,17 +34,17 @@ tools:
3434
chart:
3535
chart_name: argo/argo-cd
3636
release_name: argocd
37-
namespace: argocd
37+
namespace: [[ argocdNameSpace ]]
3838
wait: true
39-
timeout: 10m
39+
timeout: [[ argocdDeployTimeout ]]
4040
upgradeCRDs: true
4141
- name: go-webapp-argocd-deploy
4242
plugin: argocdapp
4343
dependsOn: ["argocd.argocd", "go-webapp-repo.github-repo-scaffolding-golang"]
4444
options:
4545
app:
4646
name: ${{go-webapp-repo.github-repo-scaffolding-golang.outputs.repo}}
47-
namespace: argocd
47+
namespace: [[ argocdNameSpace ]]
4848
destination:
4949
server: https://kubernetes.default.svc
5050
namespace: default

‎test/e2e/yaml/e2e-variables.yaml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
defaultBranch: main
2+
githubOrganization: devstream-io
3+
repoName: dtm-e2e-go
4+
dockerRegistryUserName: dtme2etest
5+
argocdNameSpace: argocd
6+
argocdDeployTimeout: 10m

0 commit comments

Comments
 (0)