Skip to content

Commit d91ee92

Browse files
committed
Add stale framework reference checks
1 parent 23e8f0e commit d91ee92

5 files changed

Lines changed: 92 additions & 3 deletions

File tree

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
name: Validate framework registry
2+
3+
on:
4+
pull_request:
5+
push:
6+
branches:
7+
- main
8+
schedule:
9+
- cron: "17 8 * * 1"
10+
workflow_dispatch:
11+
12+
jobs:
13+
validate-framework-registry:
14+
runs-on: ubuntu-latest
15+
steps:
16+
- name: Check out repository
17+
uses: actions/checkout@v4
18+
19+
- name: Validate framework registry shape
20+
run: ruby scripts/validate_framework_registry.rb
21+
22+
- name: Report stale framework references
23+
run: ruby scripts/validate_framework_registry.rb --stale --max-age-days 365

CONTRIBUTING.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -170,6 +170,7 @@ If your contribution adds or changes framework references, update
170170

171171
```bash
172172
ruby scripts/validate_framework_registry.rb
173+
ruby scripts/validate_framework_registry.rb --stale --max-age-days 365
173174
```
174175

175176
If your contribution changes CI/CD examples, update

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -123,6 +123,7 @@ Validate framework provenance, versions, owners, and review dates with:
123123

124124
```bash
125125
ruby scripts/validate_framework_registry.rb
126+
ruby scripts/validate_framework_registry.rb --stale --max-age-days 365
126127
```
127128

128129
CI/CD examples for GitHub Actions, GitLab CI, Azure DevOps, Jenkins,

docs/framework-reference-registry.md

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,5 +27,14 @@ Validate the registry locally with:
2727
ruby scripts/validate_framework_registry.rb
2828
```
2929

30-
Future stale-framework checks should use `date_reviewed` and `owner` from this
31-
registry instead of scraping individual skill files.
30+
Report references whose `date_reviewed` value is older than the review policy
31+
window with:
32+
33+
```bash
34+
ruby scripts/validate_framework_registry.rb --stale --max-age-days 365
35+
```
36+
37+
The scheduled `Validate framework registry` workflow runs this stale-reference
38+
report weekly. Owners should refresh stale entries by confirming the source URL,
39+
updating `version` when the upstream framework has changed, and setting a new
40+
`date_reviewed`.

scripts/validate_framework_registry.rb

Lines changed: 56 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,15 @@
22
# frozen_string_literal: true
33

44
require "yaml"
5+
require "date"
56

67
ROOT = File.expand_path("..", __dir__)
78
REGISTRY_PATH = File.join(ROOT, "data", "frameworks.yaml")
89
REQUIRED_TOP_LEVEL = %w[schema_version last_reviewed required_families references].freeze
910
REQUIRED_ENTRY_FIELDS = %w[id family name version url date_reviewed owner aliases].freeze
1011
REQUIRED_FAMILIES = %w[OWASP NIST MITRE CIS CVSS SSVC EPSS SLSA CycloneDX SPDX].freeze
1112
DATE_PATTERN = /\A\d{4}-\d{2}-\d{2}\z/
13+
DEFAULT_MAX_AGE_DAYS = 365
1214

1315
def rel(path)
1416
path.delete_prefix("#{ROOT}#{File::SEPARATOR}")
@@ -28,6 +30,56 @@ def validate_string(value, label, errors)
2830
errors << "#{label} must be a non-empty string" unless value.is_a?(String) && !value.empty?
2931
end
3032

33+
def usage
34+
warn "Usage: ruby scripts/validate_framework_registry.rb [--stale] [--max-age-days DAYS] [--as-of YYYY-MM-DD]"
35+
exit 2
36+
end
37+
38+
def parse_date_argument(value)
39+
Date.iso8601(value)
40+
rescue Date::Error
41+
usage
42+
end
43+
44+
def parse_args(argv)
45+
options = {
46+
stale: false,
47+
max_age_days: DEFAULT_MAX_AGE_DAYS,
48+
as_of: Date.today
49+
}
50+
51+
until argv.empty?
52+
case argv.shift
53+
when "--stale"
54+
options[:stale] = true
55+
when "--max-age-days"
56+
value = argv.shift
57+
usage unless value&.match?(/\A\d+\z/)
58+
options[:max_age_days] = value.to_i
59+
when "--as-of"
60+
value = argv.shift
61+
usage unless value
62+
options[:as_of] = parse_date_argument(value)
63+
else
64+
usage
65+
end
66+
end
67+
68+
options
69+
end
70+
71+
def validate_staleness(entry, prefix, options, errors)
72+
return unless options[:stale] && entry["date_reviewed"].to_s.match?(DATE_PATTERN)
73+
74+
reviewed = Date.iso8601(entry["date_reviewed"])
75+
age_days = (options[:as_of] - reviewed).to_i
76+
return if age_days <= options[:max_age_days]
77+
78+
errors << "#{prefix}: #{entry['id']} reviewed #{age_days} days ago; owner #{entry['owner']} must refresh by #{options[:max_age_days]} days"
79+
end
80+
81+
options = parse_args(ARGV.dup)
82+
3183
errors = []
3284
registry = load_registry(errors)
3385

@@ -81,6 +133,8 @@ def validate_string(value, label, errors)
81133
errors << "#{prefix}.date_reviewed must use YYYY-MM-DD"
82134
end
83135

136+
validate_staleness(entry, prefix, options, errors)
137+
84138
aliases = entry["aliases"]
85139
unless aliases.is_a?(Array) && !aliases.empty?
86140
errors << "#{prefix}.aliases must be a non-empty array"
@@ -105,7 +159,8 @@ def validate_string(value, label, errors)
105159
end
106160

107161
if errors.empty?
108-
puts "OK: validated framework registry with #{references.size} reference(s)."
162+
stale_suffix = options[:stale] ? " and no references older than #{options[:max_age_days]} days as of #{options[:as_of]}" : ""
163+
puts "OK: validated framework registry with #{references.size} reference(s)#{stale_suffix}."
109164
else
110165
puts "FAIL: framework registry validation failed."
111166
errors.each { |error| puts " - #{error}" }

0 commit comments

Comments
 (0)