Skip to content

Commit b55b9bb

Browse files
reaperznrhodesn
reaperzn
authored and
rhodesn
committed
Updated manual tests (#184)
* improved testing template in HACKING.md * improved testing template layout in HACKING.md Adjusted the guideline before the template Corrected some grammar errors * Added instructions to downgrade package in .deb based distros * Added a check for man auter.conf * minor fixes as per @LukeHandle
1 parent fe7a44a commit b55b9bb

File tree

1 file changed

+269
-70
lines changed

1 file changed

+269
-70
lines changed

‎HACKING.md

+269-70
Original file line numberDiff line numberDiff line change
@@ -9,14 +9,25 @@ auter --enable # touches /var/lib/auter/enabled, prints "auter enabled"
99
auter --status # prints "auter is currently enabled and not running"
1010
auter --help # shows help message
1111
auter --version # prints "auter VERSIONNUMBER"
12-
man auter # check the contents of the man page
12+
man auter # check the contents of the man page reflect the relevant changes
13+
man auter.conf # if relevant, check the contents of the man page reflect the changes
1314
```
1415

15-
If you don't have any updates available do this:
16+
If you don't have any updates available, try downgrade a package. Normally openssl has multiple versions available. If multiple packages are not available in the base repo try enable one of the archive/vault repos:
1617

18+
For RPM based distros try:
1719
```sh
18-
yum install zsh
19-
yum downgrade zsh
20+
yum --showduplicates list curl
21+
yum downgrade curl libcurl
22+
```
23+
For deb based distros try (Note: You may need to also downgrade dependencies):
24+
```
25+
# This will give you a list of packages that have multiple versions available in the repos:
26+
for PKG in $(dpkg --list | awk '{print $2}'); do VERSIONS="$(apt-cache showpkg $PKG | awk '/Versions:/,/^Reverse Depends:/ {if ($1 ~ /^[0-9]/) print $1}' | xargs)";[[ $(echo ${VERSIONS} | wc -w) -gt 1 ]] && echo "$PKG -% $VERSIONS";done | column -t -s "%"
27+
28+
apt-get update
29+
apt-get install <PACKAGE>=<VERSION>
30+
2031
```
2132

2233
Use the following to setup auter's pre/post scripts:
@@ -25,7 +36,7 @@ Use the following to setup auter's pre/post scripts:
2536
echo 'logger custom pre prep script ran' > /etc/auter/pre-prep.d/pre_prep_script
2637
echo 'logger custom post prep script ran' > /etc/auter/post-prep.d/post_prep_script
2738
echo 'logger custom pre apply script ran' > /etc/auter/pre-apply.d/pre_apply_script
28-
echo 'logger custom post apply script ran' > /etc/auter/post-apply.d/pre_apply_script
39+
echo 'logger custom post apply script ran' > /etc/auter/post-apply.d/post_apply_script
2940
echo 'logger custom pre reboot script ran' > /etc/auter/pre-reboot.d/pre_reboot_script
3041
echo 'logger custom post reboot script ran' > /etc/auter/post-reboot.d/post_reboot_script
3142
chmod +x /etc/auter/*.d/*script
@@ -36,80 +47,268 @@ chmod +x /etc/auter/*.d/*script
3647
1. Test the commands manually in a normal shell session.
3748
2. Test for both positive and negative outcomes.
3849
3. Test the effects of the change in the function. Again, if possible, test success and failure conditions
39-
4. Test the all functionality in auter that is affected by they code changes.
50+
4. Test all functionality in auter that is affected by the code changes.
4051

4152
##### Testing template
4253

43-
This should be completed in all pull requests before being merged.
54+
This MUST be completed for all supported distrubutions in all pull requests to the master branch before being merged.
4455

4556
```md
57+
# <OS test version>
58+
# Steps taken to create install file:
59+
```
60+
61+
```
62+
---
63+
# Test 1: Basic auter status tests
64+
1) Do a fresh install of auter
65+
2) Execute: `auter --status`
66+
Checks:
67+
- __[ pass/fail ]__ Exit code 0
68+
- __[ pass/fail ]__ Prints "auter is currently enabled and not running" to screen
69+
- __[ pass/fail ]__ /var/lib/auter/enabled exists
70+
71+
3) Execute: `auter --disable`
72+
Checks:
73+
- __[ pass/fail ]__ Exit code 0
74+
- __[ pass/fail ]__ Prints "INFO: auter disabled" to screen
75+
- __[ pass/fail ]__ /var/lib/auter/enabled does not exist
76+
77+
4) Execute: `auter --status`
78+
Checks:
79+
- __[ pass/fail ]__ Exit code 0
80+
- __[ pass/fail ]__ Prints "auter is currently disabled" to screen
81+
- __[ pass/fail ]__ /var/lib/auter/enabled does not exist
82+
83+
5) Execute: `auter --enable`
84+
- __[ pass/fail ]__ Exit code 0
85+
- __[ pass/fail ]__ Prints "INFO: auter enabled" to screen
86+
- __[ pass/fail ]__ /var/lib/auter/enabled exists
87+
88+
6) Execute: `auter --help`
89+
- __[ pass/fail ]__ Exit code 0
90+
- __[ pass/fail ]__ Prints the auter help to screen
91+
92+
7) Execute: `auter` (no arguments provided)
93+
- __[ pass/fail ]__ Exit code 0
94+
- __[ pass/fail ]__ Prints the auter help to screen
95+
96+
97+
8) Execute: `auter --version`
98+
- __[ pass/fail ]__ Exit code 0
99+
- __[ pass/fail ]__ Prints the correct auter version
100+
---
101+
102+
# Test 2: Manual process - default config [ PASS/FAIL ]
46103
### Config settings
104+
1) `egrep -v "^$|^#" /etc/auter/auter.conf`
105+
```
106+
107+
```
108+
2) Prepare custom scripts: Execute:
109+
```
110+
echo 'logger custom pre prep script ran' > /etc/auter/pre-prep.d/pre_prep_script
111+
echo 'logger custom post prep script ran' > /etc/auter/post-prep.d/post_prep_script
112+
echo 'logger custom pre apply script ran' > /etc/auter/pre-apply.d/pre_apply_script
113+
echo 'logger custom post apply script ran' > /etc/auter/post-apply.d/post_apply_script
114+
echo 'logger custom pre reboot script ran' > /etc/auter/pre-reboot.d/pre_reboot_script
115+
echo 'logger custom post reboot script ran' > /etc/auter/post-reboot.d/post_reboot_script
116+
chmod +x /etc/auter/*.d/*script
117+
```
118+
3) Execute: `auter --prep`
119+
- __[ pass/fail ]__ prints the following block to stdout:
120+
```
121+
INFO: Running with: /usr/bin/auter --prep
122+
INFO: Running in an interactive shell, disabling all random sleeps
123+
INFO: Running Pre-Prep script /etc/auter/pre-prep.d/pre_prep_script
124+
INFO: Updates downloaded
125+
INFO: Running Post-Prep script /etc/auter/post-prep.d/post_prep_script
126+
```
127+
- __[ pass/fail ]__ **_/var/lib/auter/last-prep-output-default_** contains yum download-only output
128+
- __[ pass/fail ]__ updates downloaded to **_/var/cache/yum/..._**
129+
- __[ pass/fail ]__ pre/post prep scripts ran successfully with messages logged to syslog
130+
##### Output from prep:
131+
```
132+
133+
```
134+
4) Execute `auter --apply`
135+
- __[ pass/fail ]__ prints the following block to stdout: Note: For debian based distros there will not be a transaction ID
136+
```
137+
INFO: Running with: /usr/bin/auter --apply
138+
INFO: Running in an interactive shell, disabling all random sleeps
139+
INFO: Running Pre-Apply script /etc/auter/pre-apply.d/pre_apply_script
140+
INFO: Applying updates
141+
INFO: Running Post-Apply script /etc/auter/post-apply.d/pre_apply_script
142+
INFO: Updates complete (yum Transaction ID : <ID>). You may need to reboot for some updates to take effect
143+
INFO: Auter successfully ran at <TIMESTAMP>
144+
```
145+
- __[ pass/fail ]__ expected updates were applied. Check **_/var/log/apt/history.log_** or **_/var/log/yum.log_**
146+
- __[ pass/fail ]__ **_/var/lib/auter/last-apply-output-default_** contains update info
147+
- __[ pass/fail ]__ no upates available after running. Check `yum update <<<n` or `apt-get --just-print upgrade`
148+
- __[ pass/fail ]__ pre/post apply scripts ran successfully, messages logged to syslog
149+
- __[ pass/fail ]__ no mail is sent to the root user with the stdout from auter
150+
##### Output from apply:
151+
```
152+
153+
```
154+
5) Execute 'auter --reboot`
155+
- __[ pass/fail ]__ prints the following block to stdout:
156+
```
157+
INFO: Running with: /usr/bin/auter --reboot
158+
INFO: Running in an interactive shell, disabling all random sleeps
159+
INFO: Running Pre-Reboot script /etc/auter/pre-reboot.d/pre_reboot_script
160+
INFO: Adding post-reboot-hook to run scripts under /etc/auter/post-reboot.d to /etc/cron.d/auter-postreboot-default
161+
INFO: Rebooting server
162+
```
163+
- __[ pass/fail ]__ reboot scheduled in 2 minutes time
164+
- __[ pass/fail ]__ pre reboot script ran successfully, messages logged to syslog
165+
- __[ pass/fail ]__ **_/etc/cron.d/auter-postreboot-default_** contains `@reboot root /usr/bin/auter --postreboot --config /etc/auter/auter.conf`
166+
- __[ pass/fail ]__ **_/etc/cron.d/auter-postreboot-default_** has the correct permissions: `root:root:0644`
167+
##### Output from reboot:
168+
```
169+
170+
```
171+
6) After server has rebooted:
172+
- __[ pass/fail ]__ Server actually rebooted: Execute `uptime`
173+
- __[ pass/fail ]__ **_ /etc/cron.d/auter-postreboot-default _** has been removed after the has fully completed the startup process
174+
- __[ pass/fail ]__ post reboot script ran successfully, messages logged to syslog
175+
##### Full auter logs:
176+
Execute: `egrep "auter|custom" /var/log/messages` or `egrep "auter|custom" /var/log/syslog` or `journalctl -S today | egrep "auter|custom"`
177+
```
178+
179+
```
180+
---
181+
182+
# Test 3: Updates via cron - default config with `--stdout` option [ PASS/FAIL ]
183+
### Config settings
184+
1) `egrep -v "^$|^#" /etc/auter/auter.conf`
185+
```
186+
187+
```
188+
2) Prepare custom scripts: Execute:
189+
```
190+
echo 'logger custom pre prep script ran' > /etc/auter/pre-prep.d/pre_prep_script
191+
echo 'logger custom post prep script ran' > /etc/auter/post-prep.d/post_prep_script
192+
echo 'logger custom pre apply script ran' > /etc/auter/pre-apply.d/pre_apply_script
193+
echo 'logger custom post apply script ran' > /etc/auter/post-apply.d/post_apply_script
194+
echo 'logger custom pre reboot script ran' > /etc/auter/pre-reboot.d/pre_reboot_script
195+
echo 'logger custom post reboot script ran' > /etc/auter/post-reboot.d/post_reboot_script
196+
chmod +x /etc/auter/*.d/*script
197+
```
198+
3) Adjust the MAXDELAY value to avoid extented sleep times
199+
```
200+
sed -i 's/MAXDELAY.*$/MAXDELAY="60"/g' /etc/auter/auter.conf
201+
```
202+
4) Schedule a cron job for prep to run in 5 minutes and watch the logs:
203+
```
204+
echo "$(date --date="5 minutes" +%_M" "%_H" "%d" "%_m" *") root $(which auter) --prep --stdout" > /etc/cron.d/auter-prep
205+
tail -n0 -f /var/log/messages or tail -f /var/log/syslog or journalctl -f
206+
```
207+
After auter has completed the prep:
208+
- __[ pass/fail ]__ Expected logs:
209+
```
210+
auter: INFO: Running with: /usr/bin/auter --prep --stdout
211+
auter: INFO: Running Pre-Prep script /etc/auter/pre-prep.d/pre_prep_script
212+
root: custom pre prep script ran
213+
auter: INFO: Updates downloaded
214+
auter: INFO: Running Post-Prep script /etc/auter/post-prep.d/post_prep_scrip
215+
root: custom post prep script ran
216+
```
217+
- __[ pass/fail ]__ **_/var/lib/auter/last-prep-output-default_** contains yum download-only output
218+
- __[ pass/fail ]__ updates downloaded to **_/var/cache/yum/…_** or **_/var/cache/apt/archives/..._**
219+
- __[ pass/fail ]__ pre/post prep scripts ran successfully with messages logged to syslog
220+
- __[ pass/fail ]__ mail sent to root user with stdout output from auter. Debian will log stdout to syslog rather than mail
221+
Output from logs:
222+
```
223+
224+
```
225+
5) Schedule a cron job for apply to run in 5 minutes and watch the logs:
226+
```
227+
echo "$(date --date="5 minutes" +%_M" "%_H" "%d" "%_m" *") root $(which auter) --apply --stdout" > /etc/cron.d/auter-apply
228+
tail -n0 -f /var/log/messages or tail -f /var/log/syslog or journalctl -f
229+
```
230+
After auter has completed the apply: Note: For debian based distros there will not be a transaction ID
231+
- __[ pass/fail ]__ Expected logs:
232+
```
233+
auter: INFO: Running with: /usr/bin/auter --apply --stdout
234+
auter: INFO: Running Pre-Apply script /etc/auter/pre-apply.d/pre_apply_script
235+
root: custom pre apply script ran
236+
auter: INFO: Applying updates
237+
auter: INFO: Running Post-Apply script /etc/auter/post-apply.d/pre_apply_script
238+
root: custom post apply script ran
239+
auter: INFO: Updates complete (yum Transaction ID : 86). You may need to reboot for some updates to take effect
240+
auter: INFO: Auter successfully ran at 2018-07-18T14:59:24+0000
241+
```
242+
- __[ pass/fail ]__ no updates available after running
243+
- __[ pass/fail ]__ pre/post scripts ran successfully with messages logged to syslog
244+
- __[ pass/fail ]__ mail sent to root user with stdout output from auter. Debian will log stdout to syslog rather than mail
245+
Output from logs:
246+
```
247+
248+
```
249+
6) Schedule a cron job to run auter --reboot in 5 minutes and watch the logs:
250+
```
251+
echo "$(date --date="5 minutes" +%_M" "%_H" "%d" "%_m" *") root $(which auter) --reboot --stdout" > /etc/cron.d/auter-reboot
252+
tail -n0 -f /var/log/messages or tail -f /var/log/syslog or journalctl -f
253+
```
254+
- __[ pass/fail ]__ Expected logs:
255+
```
256+
INFO: Running with: /usr/bin/auter --reboot --stdout
257+
INFO: Running Pre-Reboot script /etc/auter/pre-reboot.d/98-configsnap-pre-reboot
258+
INFO: Running Pre-Reboot script /etc/auter/pre-reboot.d/pre_reboot_script
259+
custom pre reboot script ran
260+
INFO: Adding post-reboot-hook to run scripts under /etc/auter/post-reboot.d to /etc/cron.d/auter-postreboot-default
261+
INFO: Rebooting server
262+
```
263+
- __[ pass/fail ]__ pre-reboot scripts ran successfully
264+
- __[ pass/fail ]__ Wall message is printed
265+
- __[ pass/fail ]__ mail sent to root user with stdout output from auter. Debian will log stdout to syslog rather than mail
266+
- __[ pass/fail ]__ Server reboots
267+
Output from logs:
268+
```
269+
270+
```
271+
7) After the server has booted, it may take up to 2 minutes for auter logs to appear. watch the logs:
272+
```
273+
egrep "auter:|custom" /var/log/messages | awk '/auter --reboot/,0'
274+
or
275+
egrep "auter:|custom" /var/log/syslog | awk '/auter --reboot/,0'
276+
or
277+
journalctl -S today | egrep auter|custom | awk '/auter --reboot/,0'
278+
```
279+
- __[ pass/fail ]__ Expected logs:
280+
```
281+
auter: INFO: Running with: /usr/bin/auter --reboot --stdout
282+
auter: INFO: Running Pre-Reboot script /etc/auter/pre-reboot.d/pre_reboot_script
283+
root: custom pre reboot script ran
284+
auter: INFO: Adding post-reboot-hook to run scripts under /etc/auter/post-reboot.d to /etc/cron.d/auter-postreboot-default
285+
auter: INFO: Rebooting server
286+
auter: INFO: Running with: /usr/bin/auter --postreboot --config /etc/auter/auter.conf
287+
auter: INFO: Removed post-reboot hook: /etc/cron.d/auter-postreboot-default
288+
auter: INFO: Running Post-Reboot script /etc/auter/post-reboot.d/post_reboot_script
289+
root: custom post reboot script ran
290+
```
291+
- __[ pass/fail ]__ post-reboot scripts ran successfully
292+
- __[ pass/fail ]__ output from auter also mailed to the root user on CentOS, output logged to syslog on Fedora
293+
Output from logs:
294+
```
295+
296+
```
297+
#### new functionality testing
298+
### Config settings
299+
1) `egrep -v "^$|^#" /etc/auter/auter.conf`
300+
```
47301

48-
AUTOREBOOT="no"
49-
ONLYINSTALLFROMPREP="yes"
50-
PREDOWNLOADUPDATES=yes
51-
52-
### <OS test version>
53-
54-
#### auter status
55-
56-
[ pass/fail ] auter --status
57-
- Check: /var/lib/auter/enabled exists
58-
59-
[ pass/fail ] auter --disable
60-
- Check: /var/lib/auter/enabled does not exist
61-
62-
[ pass/fail ] auter --enable
63-
- Check: /var/lib/auter/enabled exists
64-
65-
[ pass/fail ] auter --help
66-
- Check: same output when running 'auter' without arguments
67-
68-
[ pass/fail ] auter --version
69-
- Check: prints 'auter VERSION'
70-
71-
#### update manually
72-
73-
[ pass/fail ] auter --prep
74-
- Check: prints "INFO: Running with: ./auter --prep Updates downloaded" to stdout
75-
- Check: /var/lib/auter/last-prep-default contains update info
76-
- Check: updates downloaded to /var/cache/auter/default
77-
- Check: pre/post prep scripts ran successfully with messages logged to syslog
78-
79-
[ pass/fail ] auter --apply
80-
- Check: prints "INFO: Running with: /usr/bin/auter --apply; Applying updates; Updates complete, you may need to reboot for some updates to take effect" to stdout
81-
- Check: expected updates were applied using 'yum history info' or 'dnf history info'
82-
- Check: /var/lib/auter/last-update-default contains update info
83-
- Check: no upates available after running
84-
- Check: pre/post apply scripts ran successfully, messages logged to syslog
85-
- Check: no mail is sent to the root user with the stdout from auter
86-
87-
[ pass/fail ] auter --reboot
88-
- Check: reboot scheduled in 2 minutes time
89-
- Check: prints "INFO: Running with: ./auter --reboot; Rebooting server" followed by shutdown message to stdout
90-
- Check: 5 minutes after reboot is complete pre/post reboot scripts ran successfully with messages logged to syslog
91-
92-
#### updates via cron
93-
94-
[ pass/fail ] auter --prep --stdout
95-
- Check: updates downloaded to /var/cache/auter/default
96-
- Check: pre/post prep scripts ran successfully, messages logged to syslog
97-
- Check: output from auter also mailed to root user on CentOS boxes, output logged to syslog on Fedora systems
98-
99-
[ pass/fail ] auter --apply --stdout
100-
- Check: no updates available after running
101-
- Check: pre/post scripts ran successfully with messages logged to syslog
102-
- Check: output from auter also mailed to root user on CentOS boxes, output logged to syslog on Fedora systems
103-
104-
[ pass/fail ] auter --reboot --stdout
105-
- Check: server rebooted after 2 minutes
106-
- Check: pre/post scripts ran successfully
107-
- Check: output from auter also mailed to the root user on CentOS, output logged to syslog on Fedora
302+
```
303+
2) Details of new feature
108304

109-
#### new functionality testing
305+
3) Test command
306+
Expected outome:
307+
```
110308

111-
[ pass/fail ] <test new functionality added>
112-
- Check:
309+
```
310+
4) Next Test:
311+
etc...
113312
```
114313

115314
#### Documentation

0 commit comments

Comments
 (0)