omnigent 0.8.1 - #19
Closed
omnigent-ci[bot] wants to merge 1 commit into
Closed
Conversation
fanzeyi
added a commit
that referenced
this pull request
Aug 5, 2026
The first version of this workflow was exploitable. It is a `workflow_run` job,
so it runs in the BASE repo context holding `contents: write` even when the
test-bot run it reacts to came from a fork PR -- and this repo is public with
tests.yml on `pull_request`, so anyone can trigger that run.
`head_branch` on a fork PR is the contributor's own branch name, so the
`auto/formula/*` prefix was not a security control: a stranger could fork, push
`auto/formula/v9.9.9` containing only a modified Formula/omnigent.rb, and pass
every gate. The formula-only diff check does not help, because a hostile formula
IS a formula-only diff -- a formula is executable Ruby plus arbitrary download
URLs -- and the bottle-artifact check does not help either, because building a
hostile formula still produces bottles. The result would have been published as
the bottle every `brew install omnigent` user receives.
Now trust is established before anything runs:
* `head_repository.full_name == github.repository` in the job `if`, so fork
runs never start the job at all, plus an `isCrossRepository` assert in-step;
* the PR author must be the omnigent-ci App (tolerating gh's `app/` prefix,
which is what it actually reports -- comparing against the bare slug would
have made this a permanent silent skip);
* `workflow_run.head_sha` must still equal the PR head, because `brew pr-pull`
resolves the PR's CURRENT head rather than the SHA that was tested, so a
green run followed by a force-push would otherwise publish untested code.
Also SHA-pin the three Homebrew/actions references instead of `@main`: this job
has `contents: write` and decides what users install, so a moving tag is a
supply-chain vector. Pinned to 18fcb8e3, the same commit omnigent-ai/omnigent
pins.
Anything that fails a gate falls through to the existing manual `pr-pull` label,
where a human reviews the diff first. Replayed against real PRs: #19
(app/omnigent-ci) is admitted; #17 and #18 (human-authored) skip to the manual
path.
Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>
fanzeyi
added a commit
that referenced
this pull request
Aug 5, 2026
* ci: auto-publish generated formula bumps when bottles exist
omnigent's release automation opens `auto/formula/<tag>` here on every release,
but landing it still needs a human to notice the PR and apply `pr-pull`. That is
why the tap froze at 0.2.0 while PyPI moved to 0.5.1, and why 0.8.1 sat unbottled
for two days. Auto-publish those PRs once the build has demonstrably produced
bottles.
Runs `brew pr-pull` directly rather than adding the `pr-pull` label, because a
label applied with the default GITHUB_TOKEN does not start a new workflow run, so
publish.yml would never fire. The label path is untouched and still works for
humans.
Gated on evidence, not on a green tick, because a green tick here does not mean
the formula built. `brew test-bot` sets ignore_failures for any formula that is
neither newly added nor already bottled at its current version
(lib/tests/formulae.rb) -- every version bump in this tap, since the bottle block
is only added afterwards -- so a failed `brew install --build-bottle` still exits
0 and merely prints "Warning: N failed step ignored!". 0.7.0, 0.8.0 and 0.8.1 all
reported green that way with no bottle.
Three guards, each checked against real runs of this repo:
1. bottle artifacts must exist and be non-trivial. upload-artifact uses
if-no-files-found: warn, so a build that produced no bottle uploads nothing.
2. no job log may contain "failed step ignored!".
3. the diff must be exactly Formula/omnigent.rb, so anything touching
workflows or other files keeps the manual review gate.
Verified against run 30964311208 (the generated 0.8.1, bottles on macos-15 and
macos-26) -> both gates pass; and run 30856043824 (the original 0.8.1, which
reported three green checks and never built) -> both gates reject. The scope
guard admits #19 (Formula/omnigent.rb only) and excludes #17, which also touched
.github/workflows/tests.yml.
Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>
* ci: harden auto pr-pull against fork PRs and force-pushes
The first version of this workflow was exploitable. It is a `workflow_run` job,
so it runs in the BASE repo context holding `contents: write` even when the
test-bot run it reacts to came from a fork PR -- and this repo is public with
tests.yml on `pull_request`, so anyone can trigger that run.
`head_branch` on a fork PR is the contributor's own branch name, so the
`auto/formula/*` prefix was not a security control: a stranger could fork, push
`auto/formula/v9.9.9` containing only a modified Formula/omnigent.rb, and pass
every gate. The formula-only diff check does not help, because a hostile formula
IS a formula-only diff -- a formula is executable Ruby plus arbitrary download
URLs -- and the bottle-artifact check does not help either, because building a
hostile formula still produces bottles. The result would have been published as
the bottle every `brew install omnigent` user receives.
Now trust is established before anything runs:
* `head_repository.full_name == github.repository` in the job `if`, so fork
runs never start the job at all, plus an `isCrossRepository` assert in-step;
* the PR author must be the omnigent-ci App (tolerating gh's `app/` prefix,
which is what it actually reports -- comparing against the bare slug would
have made this a permanent silent skip);
* `workflow_run.head_sha` must still equal the PR head, because `brew pr-pull`
resolves the PR's CURRENT head rather than the SHA that was tested, so a
green run followed by a force-push would otherwise publish untested code.
Also SHA-pin the three Homebrew/actions references instead of `@main`: this job
has `contents: write` and decides what users install, so a moving tag is a
supply-chain vector. Pinned to 18fcb8e3, the same commit omnigent-ai/omnigent
pins.
Anything that fails a gate falls through to the existing manual `pr-pull` label,
where a human reviews the diff first. Replayed against real PRs: #19
(app/omnigent-ci) is admitted; #17 and #18 (human-authored) skip to the manual
path.
Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>
* ci: request review on formula bumps instead of auto-publishing
Replace the auto pr-pull job with one that asks a maintainer to look, and tells
them whether the formula actually built. Publishing stays manual: a human reviews
the resource diff and applies `pr-pull`, exactly as today.
The problem worth solving was never that applying the label is hard, it was that
nobody notices the PR -- the tap froze at 0.2.0 while PyPI moved to 0.5.1, and
0.8.1 sat unbottled after its formula was fixed. A review request fixes that
without handing a robot the ability to publish what users install.
It also removes the risk the auto-publishing version carried. That was a
`workflow_run` job holding `contents: write`, which on a public repo whose
tests.yml runs on `pull_request` meant anyone could reach it: `head_branch` on a
fork PR is the contributor's own branch name, so an `auto/formula/*` prefix
authenticates nothing, a hostile formula is still a formula-only diff, and
building one still produces bottles. This job holds no `contents` write and uses
no third-party actions, so the worst a hostile PR gets from it is a review
request on itself.
The verdict is the useful part, because a green check here does not mean the
formula built. `brew test-bot` sets ignore_failures for any formula that is
neither newly added nor already bottled at its current version
(lib/tests/formulae.rb) -- every version bump in this tap -- so a failed
`brew install --build-bottle` still exits 0 and merely prints
"Warning: N failed step ignored!". The comment therefore reports bottle artifacts
(the only trustworthy proof of a build), any swallowed failures, and whether the
tested SHA is still the head.
Dry-run against real runs of this repo:
30964311208 (generated 0.8.1) -> "bottles produced for bottles_macos-15,
bottles_macos-26, no swallowed failures", review requested;
30856043824 (original 0.8.1, three green checks, never built) -> "not safe to
publish: no bottle artifacts; test-bot (macos-15) and (macos-26) ignored a
failed step", with an explicit do-not-label warning.
Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>
* ci: use an unguessable delimiter for the verdict step output
`problems` embeds job and artifact names. With a fixed heredoc delimiter, a
crafted name containing that delimiter could close the block early and inject
further step outputs — including ok=true, which is what decides whether the
comment says the bump is safe to publish. Only same-repo App-authored PRs reach
this step today, so this is defence in depth rather than a live hole.
Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>
* ci: fix brew style shellcheck findings in the review workflow
`brew style` on a tap does not only lint Ruby: it runs shellcheck and actionlint
over the `run:` blocks of .github/workflows too, so this workflow failed the
`brew test-bot --only-tap-syntax` step. Two findings, both mine:
SC2129 -- consecutive `echo ... >> "$GITHUB_OUTPUT"` lines should be one
grouped redirect.
SC2016 -- the comment bodies were built with single-quoted printf formats
containing markdown code spans, and shellcheck reads those backticks as
command substitution.
Replaced the printf calls with unquoted heredocs, which need no single quotes and
read better, using \` for literal backticks so they stay markdown rather than
becoming command substitution. Bodies now go to a file under $RUNNER_TEMP and are
passed via --body-file.
Reproduced locally with `brew style omnigent-ai/tap` against the real linter
(3 files inspected, no offenses detected) and rendered both comment variants to
confirm the code spans survive.
Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>
* ci: indent the heredoc bodies so the workflow is valid YAML
The heredoc bodies and their EOF terminators sat at column 0, which ends the
`run: |` block scalar. YAML then tried to parse the body as YAML and hit `@`, a
reserved indicator:
formula-bump-review.yml:156:0: could not parse as YAML: found character that
cannot start any token [syntax-check]
Indent them to the block-scalar level instead. YAML strips that common
indentation, so bash still receives the terminator at column 0 and the heredoc
works.
I had claimed `brew style` was clean before pushing the previous commit. It was
not: I grepped its output for shellcheck/actionlint/offenses, and the failing line
is tagged [syntax-check], so my own filter hid it. Verified properly this time --
full `brew style` output (3 files inspected, no offenses detected, exit 0), the
YAML parsed and the comment step extracted and executed with a stubbed `gh` to
confirm the rendered markdown has no stray indentation.
Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>
---------
Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bumps the omnigent formula to 0.8.1.
Regenerates the stable
url/sha256and everyresourcestanza from the PyPI dependency tree ofomnigent==0.8.1(resolved withuv pip compilefor macOS arm + intel), spliced into the hand-tuned template inomnigent-ai/omnigent(.github/scripts/homebrew/omnigent.rb.template). The structural parts (depends_on,install,test) are unchanged.Once
brew test-botbuilds the bottles, label this PRpr-pullso the tap'sbrew pr-pullworkflow commits thebottle doblock and merges.Generated by
omnigent-ai/omnigent.github/workflows/homebrew-tap-pr.ymlon the v0.8.1 release.