Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add usage+help to all scripts #331

Open
wants to merge 41 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
9eaa9f5
t: Add test for help of openqa-label-known-issues
okurz Jul 22, 2024
165b4f8
openqa-label-known-issues: Add argument parsing
okurz Jul 23, 2024
6ba6ead
backlog-check-wip-limit: Add usage+help
okurz Jul 23, 2024
09f7b6a
backlog-set-due-date: Add usage+help
okurz Jul 23, 2024
60b594c
monitor-openqa_job: Add usage+help
okurz Jul 23, 2024
1ec6e90
obs-check-devel-openqa-leap-extra-dependencies: Add usage+help
okurz Jul 23, 2024
842a969
obs-check-package-origin: Add usage+help
okurz Jul 23, 2024
ed77bda
openqa-advanced-retrigger-jobs: Add usage+help
okurz Jul 23, 2024
0b48d10
openqa-incompletes-stats: Add usage+help
okurz Jul 23, 2024
2e69af6
openqa-investigate: Add usage+help
okurz Jul 23, 2024
563ddc4
openqa-investigate-multi: Add usage+help
okurz Jul 23, 2024
bd70d13
openqa-label-known-issues-and-investigate-hook: Add usage+help
okurz Jul 23, 2024
8e4c59f
openqa-label-known-issues-hook: Add usage+help
okurz Jul 23, 2024
6af4dd8
openqa-label-known-issues-multi: Add usage+help
okurz Jul 23, 2024
d15e9ce
openqa-monitor-incompletes: Add usage+help
okurz Jul 23, 2024
47783ff
openqa-monitor-investigation-candidates: Add usage+help
okurz Jul 23, 2024
cb357ca
openqa-query-for-job-label: Add usage+help
okurz Jul 23, 2024
92bda47
openqa-schedule-mm-ping-test: Add usage+help
okurz Jul 23, 2024
d38eb77
os-autoinst-obs-auto-submit: Add usage+help
okurz Jul 23, 2024
9bd19ce
os-autoinst-testapi-usage: Add usage+help
okurz Jul 23, 2024
ffc2193
reboot-stability-check: Add usage+help
okurz Jul 23, 2024
e24499a
trigger-openqa_in_openqa: Add usage+help
okurz Jul 23, 2024
ba70c56
trigger-pflash-test: Add usage+help
okurz Jul 23, 2024
955f617
monitor-openqa_job: Allow to import for testability and extendability
okurz Jul 23, 2024
0f472bc
obs-check-devel-openqa-leap-extra-dependencies: Allow to import for t…
okurz Jul 23, 2024
3b1285f
obs-check-package-origin: Allow to import for testability and extenda…
okurz Jul 23, 2024
a67e19a
openqa-advanced-retrigger-jobs: Allow to import for testability and e…
okurz Jul 23, 2024
2f71372
openqa-incompletes-stats: Allow to import for testability and extenda…
okurz Jul 23, 2024
c6f5bc1
openqa-investigate-multi: Allow to import for testability and extenda…
okurz Jul 23, 2024
43576b1
openqa-label-known-issues: Allow to import for testability and extend…
okurz Jul 23, 2024
995f31b
openqa-label-known-issues-and-investigate-hook: Allow to import for t…
okurz Jul 23, 2024
fbcd436
openqa-label-known-issues-hook: Allow to import for testability and e…
okurz Jul 23, 2024
9d8a9b2
openqa-label-known-issues-multi: Allow to import for testability and …
okurz Jul 23, 2024
efc133c
openqa-monitor-incompletes: Allow to import for testability and exten…
okurz Jul 23, 2024
2ee3cbf
openqa-monitor-investigation-candidates: Allow to import for testabil…
okurz Jul 23, 2024
00fa80a
openqa-query-for-job-label: Allow to import for testability and exten…
okurz Jul 23, 2024
210795c
openqa-schedule-mm-ping-test: Allow to import for testability and ext…
okurz Jul 23, 2024
532aad8
os-autoinst-obs-auto-submit: Allow to import for testability and exte…
okurz Jul 23, 2024
7881226
os-autoinst-testapi-usage: Allow to import for testability and extend…
okurz Jul 23, 2024
e925b2d
trigger-openqa_in_openqa: Allow to import for testability and extenda…
okurz Jul 23, 2024
a3722ac
trigger-pflash-test: Allow to import for testability and extendability
okurz Jul 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 33 additions & 3 deletions backlog-check-wip-limit
Original file line number Diff line number Diff line change
@@ -1,9 +1,39 @@
#!/bin/sh -e
#!/bin/bash -e
redmine_api_key="${redmine_api_key:?"Need redmine API key"}"
Martchus marked this conversation as resolved.
Show resolved Hide resolved
host="${host:-"https://progress.opensuse.org"}"
query_id="${query_id:-400}"
ticket_limit="${ticket_limit:-200}"
wip_limit="${wip_limit:-10}"
status="${status:-"In Progress"}"
tickets=$(curl -s -H "X-Redmine-API-Key: $redmine_api_key" "https://progress.opensuse.org/issues.json?query_id=$query_id&limit=$ticket_limit" | jq -r '.issues | .[] | select(.status.name=="In Progress") | .id')
test "$(echo "$tickets" | wc -l)" -le "$wip_limit"

usage() {
cat << EOF
Usage: $0 [OPTIONS]

Checks the number of tickets in a redmine query against a "WIP-Limit".

Martchus marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

@asdil12 asdil12 Jul 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please document at least the most important env vars and what happens if check fails (exit code=1?)

Options:
-h, --help display this help
EOF
exit "$1"
}

main() {
opts=$(getopt -o h -l help -n "$0" -- "$@") || usage 1
eval set -- "$opts"
while true; do
case "$1" in
-h | --help) usage 0 ;;
--)
shift
break
;;
*) break ;;
esac
done

tickets=$(curl -s -H "X-Redmine-API-Key: $redmine_api_key" "https://progress.opensuse.org/issues.json?query_id=$query_id&limit=$ticket_limit" | jq -r '.issues | .[] | select(.status.name=="In Progress") | .id')
test "$(echo "$tickets" | wc -l)" -le "$wip_limit"
}

caller 0 > /dev/null || main "$@"
88 changes: 58 additions & 30 deletions backlog-set-due-date
Original file line number Diff line number Diff line change
@@ -1,14 +1,4 @@
#!/bin/sh -e

# This script sets the due date on tickets in Redmine based on
# specified conditions
#
# If you want to run this for testing there are two options
# 1. Run the script without updating the tickets
# dry_run=1 ./backlog-set-due-date
#
# 2. Like #1 but also read the tickets from a file
# dry_run=1 ./backlog-set-due-date <issues-json-file>
#!/bin/bash -e

dry_run="${dry_run:-"0"}"
query_id="${query_id:-230}"
Expand All @@ -17,22 +7,60 @@ host="${host:-"https://progress.opensuse.org"}"
status="${status:-"In Progress"}"
duration="${duration:-"14 days"}"
priority="${priority:-"Low"}"
issues=$(mktemp -t backlog-set-due-date-XXXX)
jquery=".issues | .[] | select(.priority.name!=\"$priority\" and .due_date==null and .assigned_to!=null and .status.name==\"$status\")"

if [ $# -eq 1 ] && [ -f "$1" ] && [ "$dry_run" = "1" ]; then
jq -r "$jquery" "$1" > "$issues"
else
redmine_api_key="${redmine_api_key:?"Need redmine API key"}"
curl -sS -H "X-Redmine-API-Key: $redmine_api_key" "$host/issues.json?query_id=$query_id&limit=$ticket_limit" | jq -r "$jquery" > "$issues"
fi

[ "$dry_run" = "1" ] && prefix="echo"

for id in $(jq .id "$issues"); do
due_date=$(date -d"+$duration" +%Y-%m-%d)
echo "Updating ticket $id, new due date setup to $due_date"
$prefix curl -v -H "X-Redmine-API-Key: $redmine_api_key" -H 'Content-Type: application/json' -X PUT \
-d "{\"issue\": {\"due_date\": \"$due_date\", \"notes\": \"Setting due date based on mean cycle time of SUSE QE Tools\"}}" \
"$host/issues/$id.json"
done

usage() {
cat << EOF
Usage: $0 [OPTIONS]

This script sets the due date on tickets in Redmine based on
specified conditions.

If you want to run this for testing there are two options:
1. Run the script without updating the tickets
dry_run=1 ./backlog-set-due-date

2. Like #1 but also read the tickets from a file
dry_run=1 ./backlog-set-due-date <issues-json-file>

Options:
-h, --help display this help
EOF
exit "$1"
}

main() {
opts=$(getopt -o h -l help -n "$0" -- "$@") || usage 1
eval set -- "$opts"
while true; do
case "$1" in
-h | --help) usage 0 ;;
--)
shift
break
;;
*) break ;;
esac
done

issues=$(mktemp -t backlog-set-due-date-XXXX)
jquery=".issues | .[] | select(.priority.name!=\"$priority\" and .due_date==null and .assigned_to!=null and .status.name==\"$status\")"

if [ $# -eq 1 ] && [ -f "$1" ] && [ "$dry_run" = "1" ]; then
jq -r "$jquery" "$1" > "$issues"
else
redmine_api_key="${redmine_api_key:?"Need redmine API key"}"
curl -sS -H "X-Redmine-API-Key: $redmine_api_key" "$host/issues.json?query_id=$query_id&limit=$ticket_limit" | jq -r "$jquery" > "$issues"
fi

[ "$dry_run" = "1" ] && prefix="echo"

for id in $(jq .id "$issues"); do
due_date=$(date -d"+$duration" +%Y-%m-%d)
echo "Updating ticket $id, new due date setup to $due_date"
$prefix curl -v -H "X-Redmine-API-Key: $redmine_api_key" -H 'Content-Type: application/json' -X PUT \
-d "{\"issue\": {\"due_date\": \"$due_date\", \"notes\": \"Setting due date based on mean cycle time of SUSE QE Tools\"}}" \
"$host/issues/$id.json"
done
}

caller 0 > /dev/null || main "$@"
107 changes: 67 additions & 40 deletions monitor-openqa_job
Original file line number Diff line number Diff line change
@@ -1,12 +1,5 @@
#!/bin/bash -ex

# Monitor an openQA job by polling the status of a job over the API.
#
# Continuously polls the openQA API for the status of the specified jobs until
# all jobs finish. After the jobs have finished this program exits with an exit
# code corresponding to the jobs' result. For example all jobs pass it would exit
# this program with exit code 0 for success; otherwise with 1.

set -euo pipefail

# configuration variables with defaults.
Expand All @@ -18,38 +11,72 @@ obs_component="${obs_component:-"package"}"
obs_package_name="${1:-""}"
comment_on_obs=${comment_on_obs:-}

# shellcheck source=/dev/null
. "$(dirname "$0")"/_common

[[ -f job_post_response ]] || (echo "Need job response status file 'job_post_response'" && exit 2)
declare -A failed_versions
failed_jobs=()
for job_id in $(job_ids job_post_response); do
echo "Waiting for job $job_id to finish"
while sleep "${sleep_time}"; do
job_state=$($openqa_cli api --host "$host" jobs/"$job_id" | jq -r '.job.state')
[[ $job_state = 'done' ]] && break
usage() {
cat << EOF
Usage: $0 [OPTIONS]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add an example here on how to invoke this script (how to supply job id(s?)).


Monitor an openQA job by polling the status of a job over the API.

Continuously polls the openQA API for the status of the specified jobs until
all jobs finish. After the jobs have finished this program exits with an exit
code corresponding to the jobs' result. For example all jobs pass it would
exit this program with exit code 0 for success; otherwise with 1.

Options:
-h, --help display this help
EOF
exit "$1"
}

main() {
opts=$(getopt -o h -l help -n "$0" -- "$@") || usage 1
eval set -- "$opts"
while true; do
case "$1" in
-h | --help) usage 0 ;;
--)
shift
break
;;
*) break ;;
esac
done
response=$($openqa_cli api --host "$host" jobs/"$job_id")
result=$(echo "$response" | jq -r '.job.result')
echo "Result of job $job_id: $result"
if [[ $result != 'passed' ]] && [[ -n $obs_package_name ]]; then
version=$(echo "$response" | jq -r '.job.settings.VERSION')
failed_versions[$version]=1
failed_jobs+=("$job_id")
fi
done

[[ ${failed_jobs[*]} ]] || exit 0
[[ $comment_on_obs ]] || exit 1

osc api "/comments/$obs_component/$obs_package_name" | grep id= | sed -n 's/.*id="\([^"]*\)">.*test.* failed.*/\1/p' | while read -r id; do

# shellcheck source=/dev/null
. "$(dirname "$0")"/_common

[[ -f job_post_response ]] || (echo "Need job response status file 'job_post_response'" && exit 2)
declare -A failed_versions
failed_jobs=()
for job_id in $(job_ids job_post_response); do
echo "Waiting for job $job_id to finish"
while sleep "${sleep_time}"; do
job_state=$($openqa_cli api --host "$host" jobs/"$job_id" | jq -r '.job.state')
[[ $job_state = 'done' ]] && break
done
response=$($openqa_cli api --host "$host" jobs/"$job_id")
result=$(echo "$response" | jq -r '.job.result')
echo "Result of job $job_id: $result"
if [[ $result != 'passed' ]] && [[ -n $obs_package_name ]]; then
version=$(echo "$response" | jq -r '.job.settings.VERSION')
failed_versions[$version]=1
failed_jobs+=("$job_id")
fi
done

[[ ${failed_jobs[*]} ]] || exit 0
[[ $comment_on_obs ]] || exit 1

osc api "/comments/$obs_component/$obs_package_name" | grep id= | sed -n 's/.*id="\([^"]*\)">.*test.* failed.*/\1/p' | while read -r id; do
osc api -X DELETE /comment/"$id"
done
comment="openQA-in-openQA test(s) failed (job IDs: ${failed_jobs[*]}), see https://openqa.opensuse.org/tests/overview?"
for version in "${!failed_versions[@]}"; do
comment+="version=${version}&"
done
comment+="groupid=$openqa_groupid"
osc api --data="$comment" -X POST "/comments/${obs_component}/$obs_package_name"
exit 1
done
comment="openQA-in-openQA test(s) failed (job IDs: ${failed_jobs[*]}), see https://openqa.opensuse.org/tests/overview?"
for version in "${!failed_versions[@]}"; do
comment+="version=${version}&"
done
comment+="groupid=$openqa_groupid"
osc api --data="$comment" -X POST "/comments/${obs_component}/$obs_package_name"
exit 1
}

caller 0 > /dev/null || main "$@"
51 changes: 40 additions & 11 deletions obs-check-devel-openqa-leap-extra-dependencies
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,45 @@

osc="osc --apiurl https://api.opensuse.org"

for project in $($osc search --project -s 'devel:openQA:Leap:' | grep -o '^devel:openQA:Leap:.*'); do
for package in $($osc list "$project"); do
$osc api "/comments/package/$project/$package" \
| yq -pxml -oy '.comments.comment |= ([] + .) | .comments.comment[].+content' \
| grep -q "Reason for linking:" \
|| {
echo "No reason for $project/$package, consider adding a comment or remove the link (see poo#128087 for details)" >&2
problem=1
}
usage() {
cat << EOF
Usage: $0 [OPTIONS]

Checks devel:openQA:Leap: OBS project dependencies.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please be a bit more descriptive here.
Even reading the source code I'm still not sure what exactly this script does.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, me neither

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If nobody knows what this does, we might want to delete the script.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the script is actively used within https://gitlab.suse.de/openqa/scripts-ci/


Options:
-h, --help display this help
EOF
exit "$1"
}

main() {
opts=$(getopt -o h -l help -n "$0" -- "$@") || usage 1
eval set -- "$opts"
while true; do
case "$1" in
-h | --help) usage 0 ;;
--)
shift
break
;;
*) break ;;
esac
done
done

test -z "$problem"
for project in $($osc search --project -s 'devel:openQA:Leap:' | grep -o '^devel:openQA:Leap:.*'); do
for package in $($osc list "$project"); do
$osc api "/comments/package/$project/$package" \
| yq -pxml -oy '.comments.comment |= ([] + .) | .comments.comment[].+content' \
| grep -q "Reason for linking:" \
|| {
echo "No reason for $project/$package, consider adding a comment or remove the link (see poo#128087 for details)" >&2
problem=1
}
done
done

test -z "$problem"
}

caller 0 > /dev/null || main "$@"
55 changes: 41 additions & 14 deletions obs-check-package-origin
Original file line number Diff line number Diff line change
@@ -1,11 +1,5 @@
#!/bin/bash -e

# Motivation: https://progress.opensuse.org/issues/154723
# 1. fetch requirements from package spec file,
# 2. look for package names for such requirements
# 3. search all relevant instances where the package is maintained
# 4. fetch version info from spec file and from Tumbleweed repo

osc="${osc:-"osc --apiurl https://api.opensuse.org"}"

BLUE="\e[94m"
Expand Down Expand Up @@ -155,12 +149,45 @@ list_versions() {
done | sort | column -t -N Origin,Version,TW-Version
}

init_check
[ $# -eq 0 ] && {
info "Usage: $0 package-name ..."
exit 99
usage() {
cat << EOF
Usage: $0 [OPTIONS]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please include positional argument(s) in usage.


Motivation: https://progress.opensuse.org/issues/154723
1. fetch requirements from package spec file,
2. look for package names for such requirements
3. search all relevant instances where the package is maintained
4. fetch version info from spec file and from Tumbleweed repo

Options:
-h, --help display this help
EOF
exit "$1"
}

main() {
opts=$(getopt -o h -l help -n "$0" -- "$@") || usage 1
eval set -- "$opts"
while true; do
case "$1" in
-h | --help) usage 0 ;;
--)
shift
break
;;
*) break ;;
esac
done

init_check
[ $# -eq 0 ] && {
info "Usage: $0 package-name ..."
exit 99
}
while (($#)); do
list_versions "$1"
shift
done
}
while (($#)); do
list_versions "$1"
shift
done

caller 0 > /dev/null || main "$@"
Loading
Loading