Automated review of dependencies with few maintainers aka "Bernies"

[sbp]

Some of ATR's dependencies have few maintainers or very slow release cycles. These dependencies are most at risk of social engineering attacks, and such attack would be far less likely to be discovered than attacks on dependencies with many maintainers and users. We should add some automated review pipelines, in addition to doing manual review on upgrades.

[dave2wave]

I think a first step would be to make a list of dependencies, their source repositories, and package ecosystem. With that we can know what scanners we should use.

[dave2wave]

Also Jarek could be a resource about how airflow handles dependencies.

[andrewmusselman]

I've asked him for some guidance.

[potiuk]

This is what I am going to use scrutineer for. Previously I had some deterministic tooling that I use for first version of Airflow Beach Cleaning https://docs.google.com/spreadsheets/d/1IT8PMEhtvhwSgH9ksXl97F-NJKez0wR7gTgP6NiMcJk/edit?gid=0#gid=0 where I had a deterministic python script looking at OSS scorecard + some manual classification and "quick eyeball` assesment.

But I hit the "Scaling" wall when I was doing it - it was very time consuming and manual - even if parts were possible to automate. Only the AI + Agents opened a way to make it way better.

So my plan is:

a) use scrutineer to scan them -> including detecting insecure setup
b) use it to reach out to those deps (scrutineer integrates with ecosyste.ms for dependency data + maintainer contacts etc.
c) generate PRs to fix issues found (with AI)
d) see if they are resonding
e) choose: (f)ix, (f)ork, (f)orego

The AI /Agentic tooling as of 2 months allows doing it at scale. This is basically what I am doing (in a small scale) to learn and see how people respond (experiment) with I was first doing it semi-manually, then I asked my agent to do it for me in several actions, and finall apache/infrastructure-actions#807 @dave2wave just merged a SKILL that is supposed to do exactly this.

With Scrutineer - a lot of SKILLS for that are already there - but I will want to experiment and add SKILLs that are specifically targetted to walk the dependency tree, starting interaction with the maintainer and tracking that, so I do not have to do it manually - I will generally want most of it to happen semi-automatically (following the principles of https://github.com/apache/airflow-steward ) - where all "write" shoudl be always reviews and confirmed by Human... But it has good chance to scale - I will likely build some of that even directly in "!Steward" -> as a reusable skill for other maintainers.

So eventually .. one way for ATR to do it will be to "adopt" "!steward" and use skills from there.

[asf-tooling]

Automated triage — analyzed at main@ab610b23

Type: discussion • Classification: no_action • Confidence: high
Application domain(s): sbom_and_supply_chain, infrastructure_and_shared

Summary

This is an open-ended discussion about adding automated security review pipelines for ATR's own dependencies that have few maintainers. @potiuk has proposed using 'scrutineer' (an AI/agentic tool) to scan dependencies, detect insecure setups, and follow a 'fix, fork, forego' framework, referencing existing external work in apache/infrastructure-actions (PR #807) and apache/airflow-steward. The team is still in exploratory mode - @andrewmusselman has reached out to the Airflow team for guidance, and no concrete code change for this repository has been agreed upon yet.

Proposed approach

No code change is actionable at this time. The discussion is converging on adopting external tooling (scrutineer/airflow-steward) rather than building custom review pipelines inside the ATR codebase. @potiuk's plan involves using scrutineer to scan dependencies, integrate with ecosyste.ms for maintainer data, and generate fix PRs with AI - all of which would be orchestrated externally via 'skills' in apache/infrastructure-actions.

The immediate next step identified by @dave2wave is to create an inventory of ATR's dependencies, their source repositories, and package ecosystems. This would inform which scanners and tools to apply. Until the team converges on a specific tool or approach, no code change to this repository is warranted.

Open questions

• Will scrutineer be adopted as the primary tool, or will additional/alternative approaches be considered?
• Should an initial dependency inventory be tracked as a separate task or document within this repo?
• What specific metrics (maintainer count, release cadence, OpenSSF Scorecard scores) will be used to flag at-risk dependencies?
• Will the outcome be a GitHub Actions workflow in this repo, or entirely external orchestration via infrastructure-actions/steward?

The agent reviewed this issue and is not proposing patches in this run. Review the existing-code citations and open questions above before deciding next steps.

Files examined

• .asf.yaml
• .github/PULL_REQUEST_TEMPLATE.md
• .github/dependabot.yml
• .github/labeler.yml
• .github/linters/.markdown-lint.yml
• .github/workflows/allowlistchecker.yml
• .github/workflows/analyze.yml
• .github/workflows/build.yml

---

Draft from a triage agent. A human reviewer should validate before merging any change. The agent did not run tests or verify diffs apply.

[dave2wave]

Essentially this is a search for potential and actual "Bernies". There was discussion today on the A-O OSCE meeting.

@potiuk could we schedule a demo of your use of scrutinizer for sometime after June 10?

[dave2wave]

https://github.com/andrew/weekend-at-bernies/tree/main/owners

[potiuk]

Essentially this is a search for potential and actual "Bernies". There was discussion today on the A-O OSCE meeting.

@potiuk could we schedule a demo of your use of scrutinizer for sometime after June 10?

Sure. I have not done much with it yet, but after 10 June, I am likely to be already quite advanced. Note 14-18 I am on a family trip with my nephew in NY, and later I am attending UN Headquarters Open Source Week 19-25 so it would be most convenient some time before.

[dave2wave]

There may be useful data in this repository: https://github.com/andrew/weekend-at-bernies