Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Code owners for the GenAI-Data-Security-Initiative monorepo.
# Docs: https://docs.github.com/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners
#
# Owners are requested for review automatically when matching files change.

# DSGAI scanner tool subproject
/dsgai_scanner_tool/ @emmanuelgjr
45 changes: 45 additions & 0 deletions .github/workflows/scanner-lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: scanner-lint

# Path-filtered lint for the DSGAI scanner subproject only, to keep monorepo
# CI noise down. This is the pattern all later scanner CI follows.
on:
push:
branches: [main]
paths:
- 'dsgai_scanner_tool/**'
- '.github/workflows/scanner-lint.yml'
pull_request:
paths:
- 'dsgai_scanner_tool/**'
- '.github/workflows/scanner-lint.yml'

permissions:
contents: read

jobs:
lint:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

- name: shellcheck integrations shell scripts
run: |
sudo apt-get update -qq && sudo apt-get install -y shellcheck
# -S warning: report warnings and errors. Known dsgai-secret-scan.sh
# issues are suppressed inline with TODO(PR-10) markers until PR-10.
find dsgai_scanner_tool/integrations -name '*.sh' -print0 \
| xargs -0 -r shellcheck -S warning

- name: yamllint scanner YAML
run: |
python -m pip install --quiet yamllint
yamllint -d "{extends: relaxed, rules: {line-length: disable, document-start: disable, new-lines: disable, truthy: {check-keys: false}}}" \
dsgai_scanner_tool/

- name: Markdown link check (scanner docs — relative links + anchors)
run: |
# Deterministic internal-link check: verifies relative file links and
# in-repo heading anchors resolve. External URLs are intentionally not
# fetched (flaky in CI); that is a separate concern.
python dsgai_scanner_tool/scripts/check_md_links.py dsgai_scanner_tool
6 changes: 6 additions & 0 deletions dsgai_scanner_tool/.gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Force LF for scripts so Windows checkouts (core.autocrlf=true) can't ship
# CRLF that breaks shellcheck / bash on Linux CI runners.
*.sh text eol=lf
*.py text eol=lf
*.yml text eol=lf
*.yaml text eol=lf
8 changes: 7 additions & 1 deletion dsgai_scanner_tool/CHANGES_v0.3.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,15 @@ dates are ISO-8601. The previous line is recorded in [`CHANGES_v0.2.md`](CHANGES
- Contributor infrastructure: `[scanner]` GitHub issue-form templates (false-positive,
false-negative, new-rule, bug), scanner `CONTRIBUTING.md`, public `ROADMAP.md`, and
this changelog scaffold. (PR-01)
- Repo hygiene: **Non-goals** section in the README, root `CODEOWNERS` for
`dsgai_scanner_tool/`, path-filtered `scanner-lint.yml` CI (shellcheck + yamllint +
internal-link check), a deterministic markdown internal-link checker
(`scripts/check_md_links.py`), and a `.gitattributes` forcing LF on scripts/YAML so
Windows checkouts can't ship CRLF that breaks Linux CI. (PR-02)

### Changed
- _nothing yet_
- `DSGAI-samplereport.png` compressed from ~5.0 MB to ~0.35 MB (14×) as an interim fix;
full regeneration from the fixture app lands in PR-09. (PR-02)

### Fixed
- _nothing yet_
Binary file modified dsgai_scanner_tool/DSGAI-samplereport.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
18 changes: 18 additions & 0 deletions dsgai_scanner_tool/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -408,6 +408,24 @@ When proposing new scan patterns:
2. Validate the PCRE pattern with `rg --pcre2 'pattern' .` against a real repo
3. For VALUE-BEARING patterns, prove the value never escapes by inspecting `DSGAI-scan.json` after a test run

See [`CONTRIBUTING.md`](CONTRIBUTING.md) for the full contributor guide and the
[`ROADMAP.md`](ROADMAP.md) for what's planned.

## Non-goals

To keep the scanner maintainable and trustworthy, some things are deliberately out of scope:

- **We will not reimplement general-purpose secret scanning.** We ship a gitleaks rule
pack instead (see `integrations/gitleaks/`) and lean on battle-tested tooling for
entropy-based detection.
- **We will not become a general-purpose SAST tool.** Scope is the 21 DSGAI controls and
the GenAI-specific patterns behind them — not every code smell in a repo.
- **We will not add rules without fixture test cases.** A rule with no positive *and*
negative test has no defined precision, so it doesn't merge.
- **We will not accept changes that weaken the redaction guarantees.** Value-bearing
matches never enter a report, checkpoint, or persisted tool call — that property is
non-negotiable.

---

## License
Expand Down
91 changes: 91 additions & 0 deletions dsgai_scanner_tool/scripts/check_md_links.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
#!/usr/bin/env python3
"""Deterministic internal-link checker for the DSGAI scanner docs.

Verifies that relative markdown links point at files that exist, and that
in-file `#anchor` fragments match a heading in the target document. External
(http/https/mailto) links are intentionally not fetched — that is flaky in CI
and a separate concern. Exit 1 if any internal link is broken.

Usage: python check_md_links.py <dir-or-file> [more...]
"""
import re, sys, unicodedata
from pathlib import Path

LINK_RE = re.compile(r'(?<!\!)\[[^\]]*\]\(([^)]+)\)')
HEADING_RE = re.compile(r'^(#{1,6})\s+(.*?)\s*#*\s*$')


def slug(text: str) -> str:
"""GitHub-style heading -> anchor slug."""
text = unicodedata.normalize('NFKD', text)
# strip markdown inline formatting and links
text = re.sub(r'`([^`]*)`', r'\1', text)
text = re.sub(r'\[([^\]]*)\]\([^)]*\)', r'\1', text)
text = re.sub(r'[*_~]', '', text)
text = text.lower()
text = re.sub(r'[^\w\s-]', '', text)
text = text.strip().replace(' ', '-')
return text


def anchors_of(path: Path) -> set:
out = set()
if not path.exists():
return out
for line in path.read_text(encoding='utf-8', errors='replace').splitlines():
m = HEADING_RE.match(line)
if m:
out.add(slug(m.group(2)))
return out


def collect_md(targets):
files = []
for t in targets:
p = Path(t)
if p.is_dir():
files += sorted(p.rglob('*.md'))
elif p.suffix == '.md':
files.append(p)
return files


def main():
targets = sys.argv[1:] or ['.']
files = collect_md(targets)
anchor_cache = {}
broken = []
for f in files:
text = f.read_text(encoding='utf-8', errors='replace')
for m in LINK_RE.finditer(text):
target = m.group(1).strip()
if target.startswith('<') and target.endswith('>'):
target = target[1:-1]
# skip external and pure-anchor-to-external schemes
if re.match(r'^[a-z]+://', target) or target.startswith('mailto:'):
continue
path_part, _, frag = target.partition('#')
if path_part == '':
# same-file anchor
dest = f
else:
dest = (f.parent / path_part).resolve()
if not dest.exists():
broken.append(f"{f}: missing file -> {target}")
continue
if frag:
if dest not in anchor_cache:
anchor_cache[dest] = anchors_of(dest)
if slug(frag) not in anchor_cache[dest]:
broken.append(f"{f}: missing anchor -> {target}")
if broken:
print("Broken internal links:")
for b in broken:
print(" " + b)
return 1
print(f"OK: {len(files)} markdown files, all internal links resolve.")
return 0


if __name__ == '__main__':
sys.exit(main())
Loading