Skip to content

DO NOT MERGE: verify generate_formula.py output builds - #18

Closed
fanzeyi wants to merge 1 commit into
mainfrom
verify/generator-output
Closed

DO NOT MERGE: verify generate_formula.py output builds#18
fanzeyi wants to merge 1 commit into
mainfrom
verify/generator-output

Conversation

@fanzeyi

@fanzeyi fanzeyi commented Aug 4, 2026

Copy link
Copy Markdown
Member

Verification only -- will be closed, not merged.

Formula/omnigent.rb is verbatim output of generate_formula.py from omnigent-ai/omnigent, rather than the hand-patched formula in #17.

Why: #17 proves a hand-edited formula builds. It does not prove the generator that regenerates this formula on every release produces something buildable -- and that generator is what protects future releases from repeating the bazel / pendulum / dylib-ID failures. Comparing generator output to #17 showed the same 100 packages with an identical sdist/wheel split and identical non-comment code, but a static comparison cannot prove it compiles.

Includes the tests.yml gate from #17 so a swallowed brew install --build-bottle failure cannot report green.

cursor-sdk, fastapi and uvicorn differ from #17: this was generated through a PyPI mirror that lags upstream, whereas CI resolves from pypi.org. Expected drift, not a regression.

Verification only. Formula/omnigent.rb here is verbatim output of
omnigent-ai/omnigent .github/scripts/homebrew/generate_formula.py, not the
hand-patched formula in #17. PR #17 proved a hand-edited formula builds; this
proves the generator that will regenerate it on every release produces the same
buildable result, which is the part that actually protects future releases.

Includes the tests.yml gate from #17 so a build failure cannot report green.

Three resources differ from #17 (cursor-sdk, fastapi, uvicorn) because this was
generated through a PyPI mirror that lags upstream; CI resolves from pypi.org and
would pin the newer ones. That is expected drift, not a regression.

Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>
fanzeyi added a commit to omnigent-ai/omnigent that referenced this pull request Aug 4, 2026
## Related issue

Closes #866

## Summary

The `omnigent` Homebrew formula has not built since 0.7.0, and the tap reported
green anyway, so 0.7.0, 0.8.0 and 0.8.1 all merged with no bottle — every user
compiled from source, and CEL policies silently did not work.

- **Root cause was the CEL migration.** #2970 swapped `cel-expr-python` for
  `cel-python` on the premise that it is pure Python. It is not: `cel-python`
  hard-depends on `google-re2`, whose sdist runs `bazel build` whenever
  `GITHUB_ACTIONS` is set. The bazel dependency moved rather than disappeared.
- **Pin compiled extensions to upstream wheels.** `generate_formula.py` gains
  `WHEEL_REQUIRED` / `PREFER_WHEEL` / `PURE_WHEEL` with abi3 and universal2
  handling, so grpcio (by far the most expensive build), protobuf, regex,
  uvloop, httptools, argon2-cffi-bindings, markupsafe, pyyaml, zstandard and
  google-re2 stop being compiled. Native wheels rank above pure-Python ones, so
  protobuf keeps its upb build instead of the slow fallback.
- **jiter, tiktoken and watchfiles keep building from source.** Their maturin
  wheels carry no Mach-O install-name padding, so Homebrew relocation fails with
  "Failed changing dylib ID" (#866). `pendulum` can go neither way — its wheel
  cannot be relocated and its sdist does not link on 3.14 (pyo3 leaves
  `_Py_NoneStruct` undefined) — so it takes the pure-Python wheel, which ships
  no extension module at all.
- **A dropped dependency is now an error, not a warning.** A missing sdist used
  to be skipped silently, yielding a formula whose venv lacked an import;
  `--allow-no-sdist` is the explicit waiver. The formula test also asserts
  `import re2, celpy`, since omnigent imports celpy behind `try/except
  ImportError` and would otherwise disable policies silently.
- **Delete `update-homebrew.yml`.** It raced `homebrew-tap-pr.yml` on the same
  `release: published` event and asserted on hand-maintained stanzas the
  template no longer emits, so it failed on every run. Its one worthwhile part
  moves into `homebrew-tap-pr.yml`: an admin/maintain gate on manual dispatch
  (it writes to another repo with an App token), plus
  `persist-credentials: false`. Its nightly `schedule` is deliberately NOT
  carried over -- that cron only existed because `brew
  update-python-resources` resolves through pip's `--uploaded-prior-to=P1D`
  window and so could never see a same-day release. The generator runs `uv pip
  compile --no-config` straight against PyPI, so the blindness it worked around
  no longer exists, and a nightly regeneration would just burn a runner to
  print "nothing to do".

## Test Plan

Verified by building the generated formula in the tap, not by inspection.

- `omnigent-ai/homebrew-tap#18` contains **verbatim output of this
  `generate_formula.py`** and bottled successfully on macos-15 and macos-26
  (run 30944428771, `bottles_macos-15` / `bottles_macos-26` ≈ 37 MB each). This
  is the check that matters: it proves the generator — not a hand-edit —
  produces a buildable formula, so the next release regenerates something that
  works.
- `omnigent-ai/homebrew-tap#17` carries the same fix for the shipped 0.8.1
  formula and is green on all three runners, with `brew test` running
  `import re2, celpy`. Inspected the bottle: `celpy/__init__.py`,
  `re2/_re2.cpython-314-darwin.so`, and a relocated
  `jiter/jiter.cpython-314-darwin.so`.
- Audited every pinned wheel by replaying Homebrew's own operation,
  `install_name_tool -id <Cellar path>` against each extracted `.so`, so the
  wheel/source split is evidence-based rather than guessed.
- `python3.12 -m py_compile`, `ruff check`, `ruff format --check`, `brew style`
  (no offenses), `ruby -c`, plus stubbed-PyPI unit checks of the new failure
  paths (missing sdist is fatal, `--allow-no-sdist` waives it, abi3 accepted,
  free-threaded `cp314t` rejected).
- Confirmed generator output matches the green formula: same 100 resources,
  identical sdist/wheel split, no non-comment differences.

## Demo

N/A — release tooling, no user-visible UI.

## Type of change

- [x] Bug fix
- [ ] Feature
- [ ] UI / frontend change
- [ ] Refactor / chore
- [ ] Docs
- [ ] Test / CI
- [ ] Breaking change

## Test coverage

- [ ] Unit tests added / updated
- [ ] Integration tests added / updated
- [ ] E2E tests added / updated
- [x] Manual verification completed
- [ ] Existing tests cover this change
- [ ] Not applicable

## Coverage notes

The generator has no test suite in this repo, and its real contract — "the
emitted formula builds under Homebrew on macOS" — cannot be asserted here. It is
covered instead by building the generated formula on the tap's `brew test-bot`
matrix (homebrew-tap#18, bottles produced on macos-15 and macos-26). The two new
generator failure paths were exercised locally against stubbed PyPI metadata,
and every wheel pin was verified relocatable with `install_name_tool`.

## Changelog

`brew install omnigent` works again, and installs prebuilt wheels instead of
compiling grpcio and friends from source.

Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>
@fanzeyi

fanzeyi commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

Verification complete — closing unmerged as intended.

Generator output built and bottled on both arm runners (run 30936446679 → 30944428771): bottles_macos-15 and bottles_macos-26, ~37 MB each, with the Fail on ignored build failures gate passing on all three jobs. That was the point of this branch: #17 proved a hand-edited formula builds, this proved the generator that regenerates it every release produces the same buildable result.

The fix itself ships in #17 (tap) and omnigent-ai/omnigent#4080 (generator + template). Nothing here should land — it pins cursor-sdk, fastapi and uvicorn to older versions because it was generated through a lagging PyPI mirror.

@fanzeyi fanzeyi closed this Aug 4, 2026
@fanzeyi
fanzeyi deleted the verify/generator-output branch August 4, 2026 23:25
fanzeyi added a commit to omnigent-ai/omnigent that referenced this pull request Aug 4, 2026
Closes #866

The `omnigent` Homebrew formula has not built since 0.7.0, and the tap reported
green anyway, so 0.7.0, 0.8.0 and 0.8.1 all merged with no bottle — every user
compiled from source, and CEL policies silently did not work.

- **Root cause was the CEL migration.** #2970 swapped `cel-expr-python` for
  `cel-python` on the premise that it is pure Python. It is not: `cel-python`
  hard-depends on `google-re2`, whose sdist runs `bazel build` whenever
  `GITHUB_ACTIONS` is set. The bazel dependency moved rather than disappeared.
- **Pin compiled extensions to upstream wheels.** `generate_formula.py` gains
  `WHEEL_REQUIRED` / `PREFER_WHEEL` / `PURE_WHEEL` with abi3 and universal2
  handling, so grpcio (by far the most expensive build), protobuf, regex,
  uvloop, httptools, argon2-cffi-bindings, markupsafe, pyyaml, zstandard and
  google-re2 stop being compiled. Native wheels rank above pure-Python ones, so
  protobuf keeps its upb build instead of the slow fallback.
- **jiter, tiktoken and watchfiles keep building from source.** Their maturin
  wheels carry no Mach-O install-name padding, so Homebrew relocation fails with
  "Failed changing dylib ID" (#866). `pendulum` can go neither way — its wheel
  cannot be relocated and its sdist does not link on 3.14 (pyo3 leaves
  `_Py_NoneStruct` undefined) — so it takes the pure-Python wheel, which ships
  no extension module at all.
- **A dropped dependency is now an error, not a warning.** A missing sdist used
  to be skipped silently, yielding a formula whose venv lacked an import;
  `--allow-no-sdist` is the explicit waiver. The formula test also asserts
  `import re2, celpy`, since omnigent imports celpy behind `try/except
  ImportError` and would otherwise disable policies silently.
- **Delete `update-homebrew.yml`.** It raced `homebrew-tap-pr.yml` on the same
  `release: published` event and asserted on hand-maintained stanzas the
  template no longer emits, so it failed on every run. Its one worthwhile part
  moves into `homebrew-tap-pr.yml`: an admin/maintain gate on manual dispatch
  (it writes to another repo with an App token), plus
  `persist-credentials: false`. Its nightly `schedule` is deliberately NOT
  carried over -- that cron only existed because `brew
  update-python-resources` resolves through pip's `--uploaded-prior-to=P1D`
  window and so could never see a same-day release. The generator runs `uv pip
  compile --no-config` straight against PyPI, so the blindness it worked around
  no longer exists, and a nightly regeneration would just burn a runner to
  print "nothing to do".

Verified by building the generated formula in the tap, not by inspection.

- `omnigent-ai/homebrew-tap#18` contains **verbatim output of this
  `generate_formula.py`** and bottled successfully on macos-15 and macos-26
  (run 30944428771, `bottles_macos-15` / `bottles_macos-26` ≈ 37 MB each). This
  is the check that matters: it proves the generator — not a hand-edit —
  produces a buildable formula, so the next release regenerates something that
  works.
- `omnigent-ai/homebrew-tap#17` carries the same fix for the shipped 0.8.1
  formula and is green on all three runners, with `brew test` running
  `import re2, celpy`. Inspected the bottle: `celpy/__init__.py`,
  `re2/_re2.cpython-314-darwin.so`, and a relocated
  `jiter/jiter.cpython-314-darwin.so`.
- Audited every pinned wheel by replaying Homebrew's own operation,
  `install_name_tool -id <Cellar path>` against each extracted `.so`, so the
  wheel/source split is evidence-based rather than guessed.
- `python3.12 -m py_compile`, `ruff check`, `ruff format --check`, `brew style`
  (no offenses), `ruby -c`, plus stubbed-PyPI unit checks of the new failure
  paths (missing sdist is fatal, `--allow-no-sdist` waives it, abi3 accepted,
  free-threaded `cp314t` rejected).
- Confirmed generator output matches the green formula: same 100 resources,
  identical sdist/wheel split, no non-comment differences.

N/A — release tooling, no user-visible UI.

- [x] Bug fix
- [ ] Feature
- [ ] UI / frontend change
- [ ] Refactor / chore
- [ ] Docs
- [ ] Test / CI
- [ ] Breaking change

- [ ] Unit tests added / updated
- [ ] Integration tests added / updated
- [ ] E2E tests added / updated
- [x] Manual verification completed
- [ ] Existing tests cover this change
- [ ] Not applicable

The generator has no test suite in this repo, and its real contract — "the
emitted formula builds under Homebrew on macOS" — cannot be asserted here. It is
covered instead by building the generated formula on the tap's `brew test-bot`
matrix (homebrew-tap#18, bottles produced on macos-15 and macos-26). The two new
generator failure paths were exercised locally against stubbed PyPI metadata,
and every wheel pin was verified relocatable with `install_name_tool`.

`brew install omnigent` works again, and installs prebuilt wheels instead of
compiling grpcio and friends from source.

Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>
fanzeyi added a commit to omnigent-ai/omnigent that referenced this pull request Aug 5, 2026
Closes #866

The `omnigent` Homebrew formula has not built since 0.7.0, and the tap reported
green anyway, so 0.7.0, 0.8.0 and 0.8.1 all merged with no bottle — every user
compiled from source, and CEL policies silently did not work.

- **Root cause was the CEL migration.** #2970 swapped `cel-expr-python` for
  `cel-python` on the premise that it is pure Python. It is not: `cel-python`
  hard-depends on `google-re2`, whose sdist runs `bazel build` whenever
  `GITHUB_ACTIONS` is set. The bazel dependency moved rather than disappeared.
- **Pin compiled extensions to upstream wheels.** `generate_formula.py` gains
  `WHEEL_REQUIRED` / `PREFER_WHEEL` / `PURE_WHEEL` with abi3 and universal2
  handling, so grpcio (by far the most expensive build), protobuf, regex,
  uvloop, httptools, argon2-cffi-bindings, markupsafe, pyyaml, zstandard and
  google-re2 stop being compiled. Native wheels rank above pure-Python ones, so
  protobuf keeps its upb build instead of the slow fallback.
- **jiter, tiktoken and watchfiles keep building from source.** Their maturin
  wheels carry no Mach-O install-name padding, so Homebrew relocation fails with
  "Failed changing dylib ID" (#866). `pendulum` can go neither way — its wheel
  cannot be relocated and its sdist does not link on 3.14 (pyo3 leaves
  `_Py_NoneStruct` undefined) — so it takes the pure-Python wheel, which ships
  no extension module at all.
- **A dropped dependency is now an error, not a warning.** A missing sdist used
  to be skipped silently, yielding a formula whose venv lacked an import;
  `--allow-no-sdist` is the explicit waiver. The formula test also asserts
  `import re2, celpy`, since omnigent imports celpy behind `try/except
  ImportError` and would otherwise disable policies silently.
- **Delete `update-homebrew.yml`.** It raced `homebrew-tap-pr.yml` on the same
  `release: published` event and asserted on hand-maintained stanzas the
  template no longer emits, so it failed on every run. Its one worthwhile part
  moves into `homebrew-tap-pr.yml`: an admin/maintain gate on manual dispatch
  (it writes to another repo with an App token), plus
  `persist-credentials: false`. Its nightly `schedule` is deliberately NOT
  carried over -- that cron only existed because `brew
  update-python-resources` resolves through pip's `--uploaded-prior-to=P1D`
  window and so could never see a same-day release. The generator runs `uv pip
  compile --no-config` straight against PyPI, so the blindness it worked around
  no longer exists, and a nightly regeneration would just burn a runner to
  print "nothing to do".

Verified by building the generated formula in the tap, not by inspection.

- `omnigent-ai/homebrew-tap#18` contains **verbatim output of this
  `generate_formula.py`** and bottled successfully on macos-15 and macos-26
  (run 30944428771, `bottles_macos-15` / `bottles_macos-26` ≈ 37 MB each). This
  is the check that matters: it proves the generator — not a hand-edit —
  produces a buildable formula, so the next release regenerates something that
  works.
- `omnigent-ai/homebrew-tap#17` carries the same fix for the shipped 0.8.1
  formula and is green on all three runners, with `brew test` running
  `import re2, celpy`. Inspected the bottle: `celpy/__init__.py`,
  `re2/_re2.cpython-314-darwin.so`, and a relocated
  `jiter/jiter.cpython-314-darwin.so`.
- Audited every pinned wheel by replaying Homebrew's own operation,
  `install_name_tool -id <Cellar path>` against each extracted `.so`, so the
  wheel/source split is evidence-based rather than guessed.
- `python3.12 -m py_compile`, `ruff check`, `ruff format --check`, `brew style`
  (no offenses), `ruby -c`, plus stubbed-PyPI unit checks of the new failure
  paths (missing sdist is fatal, `--allow-no-sdist` waives it, abi3 accepted,
  free-threaded `cp314t` rejected).
- Confirmed generator output matches the green formula: same 100 resources,
  identical sdist/wheel split, no non-comment differences.

N/A — release tooling, no user-visible UI.

- [x] Bug fix
- [ ] Feature
- [ ] UI / frontend change
- [ ] Refactor / chore
- [ ] Docs
- [ ] Test / CI
- [ ] Breaking change

- [ ] Unit tests added / updated
- [ ] Integration tests added / updated
- [ ] E2E tests added / updated
- [x] Manual verification completed
- [ ] Existing tests cover this change
- [ ] Not applicable

The generator has no test suite in this repo, and its real contract — "the
emitted formula builds under Homebrew on macOS" — cannot be asserted here. It is
covered instead by building the generated formula on the tap's `brew test-bot`
matrix (homebrew-tap#18, bottles produced on macos-15 and macos-26). The two new
generator failure paths were exercised locally against stubbed PyPI metadata,
and every wheel pin was verified relocatable with `install_name_tool`.

`brew install omnigent` works again, and installs prebuilt wheels instead of
compiling grpcio and friends from source.

Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>
fanzeyi added a commit that referenced this pull request Aug 5, 2026
The first version of this workflow was exploitable. It is a `workflow_run` job,
so it runs in the BASE repo context holding `contents: write` even when the
test-bot run it reacts to came from a fork PR -- and this repo is public with
tests.yml on `pull_request`, so anyone can trigger that run.

`head_branch` on a fork PR is the contributor's own branch name, so the
`auto/formula/*` prefix was not a security control: a stranger could fork, push
`auto/formula/v9.9.9` containing only a modified Formula/omnigent.rb, and pass
every gate. The formula-only diff check does not help, because a hostile formula
IS a formula-only diff -- a formula is executable Ruby plus arbitrary download
URLs -- and the bottle-artifact check does not help either, because building a
hostile formula still produces bottles. The result would have been published as
the bottle every `brew install omnigent` user receives.

Now trust is established before anything runs:

  * `head_repository.full_name == github.repository` in the job `if`, so fork
    runs never start the job at all, plus an `isCrossRepository` assert in-step;
  * the PR author must be the omnigent-ci App (tolerating gh's `app/` prefix,
    which is what it actually reports -- comparing against the bare slug would
    have made this a permanent silent skip);
  * `workflow_run.head_sha` must still equal the PR head, because `brew pr-pull`
    resolves the PR's CURRENT head rather than the SHA that was tested, so a
    green run followed by a force-push would otherwise publish untested code.

Also SHA-pin the three Homebrew/actions references instead of `@main`: this job
has `contents: write` and decides what users install, so a moving tag is a
supply-chain vector. Pinned to 18fcb8e3, the same commit omnigent-ai/omnigent
pins.

Anything that fails a gate falls through to the existing manual `pr-pull` label,
where a human reviews the diff first. Replayed against real PRs: #19
(app/omnigent-ci) is admitted; #17 and #18 (human-authored) skip to the manual
path.

Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>
fanzeyi added a commit that referenced this pull request Aug 5, 2026
* ci: auto-publish generated formula bumps when bottles exist

omnigent's release automation opens `auto/formula/<tag>` here on every release,
but landing it still needs a human to notice the PR and apply `pr-pull`. That is
why the tap froze at 0.2.0 while PyPI moved to 0.5.1, and why 0.8.1 sat unbottled
for two days. Auto-publish those PRs once the build has demonstrably produced
bottles.

Runs `brew pr-pull` directly rather than adding the `pr-pull` label, because a
label applied with the default GITHUB_TOKEN does not start a new workflow run, so
publish.yml would never fire. The label path is untouched and still works for
humans.

Gated on evidence, not on a green tick, because a green tick here does not mean
the formula built. `brew test-bot` sets ignore_failures for any formula that is
neither newly added nor already bottled at its current version
(lib/tests/formulae.rb) -- every version bump in this tap, since the bottle block
is only added afterwards -- so a failed `brew install --build-bottle` still exits
0 and merely prints "Warning: N failed step ignored!". 0.7.0, 0.8.0 and 0.8.1 all
reported green that way with no bottle.

Three guards, each checked against real runs of this repo:

  1. bottle artifacts must exist and be non-trivial. upload-artifact uses
     if-no-files-found: warn, so a build that produced no bottle uploads nothing.
  2. no job log may contain "failed step ignored!".
  3. the diff must be exactly Formula/omnigent.rb, so anything touching
     workflows or other files keeps the manual review gate.

Verified against run 30964311208 (the generated 0.8.1, bottles on macos-15 and
macos-26) -> both gates pass; and run 30856043824 (the original 0.8.1, which
reported three green checks and never built) -> both gates reject. The scope
guard admits #19 (Formula/omnigent.rb only) and excludes #17, which also touched
.github/workflows/tests.yml.

Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>

* ci: harden auto pr-pull against fork PRs and force-pushes

The first version of this workflow was exploitable. It is a `workflow_run` job,
so it runs in the BASE repo context holding `contents: write` even when the
test-bot run it reacts to came from a fork PR -- and this repo is public with
tests.yml on `pull_request`, so anyone can trigger that run.

`head_branch` on a fork PR is the contributor's own branch name, so the
`auto/formula/*` prefix was not a security control: a stranger could fork, push
`auto/formula/v9.9.9` containing only a modified Formula/omnigent.rb, and pass
every gate. The formula-only diff check does not help, because a hostile formula
IS a formula-only diff -- a formula is executable Ruby plus arbitrary download
URLs -- and the bottle-artifact check does not help either, because building a
hostile formula still produces bottles. The result would have been published as
the bottle every `brew install omnigent` user receives.

Now trust is established before anything runs:

  * `head_repository.full_name == github.repository` in the job `if`, so fork
    runs never start the job at all, plus an `isCrossRepository` assert in-step;
  * the PR author must be the omnigent-ci App (tolerating gh's `app/` prefix,
    which is what it actually reports -- comparing against the bare slug would
    have made this a permanent silent skip);
  * `workflow_run.head_sha` must still equal the PR head, because `brew pr-pull`
    resolves the PR's CURRENT head rather than the SHA that was tested, so a
    green run followed by a force-push would otherwise publish untested code.

Also SHA-pin the three Homebrew/actions references instead of `@main`: this job
has `contents: write` and decides what users install, so a moving tag is a
supply-chain vector. Pinned to 18fcb8e3, the same commit omnigent-ai/omnigent
pins.

Anything that fails a gate falls through to the existing manual `pr-pull` label,
where a human reviews the diff first. Replayed against real PRs: #19
(app/omnigent-ci) is admitted; #17 and #18 (human-authored) skip to the manual
path.

Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>

* ci: request review on formula bumps instead of auto-publishing

Replace the auto pr-pull job with one that asks a maintainer to look, and tells
them whether the formula actually built. Publishing stays manual: a human reviews
the resource diff and applies `pr-pull`, exactly as today.

The problem worth solving was never that applying the label is hard, it was that
nobody notices the PR -- the tap froze at 0.2.0 while PyPI moved to 0.5.1, and
0.8.1 sat unbottled after its formula was fixed. A review request fixes that
without handing a robot the ability to publish what users install.

It also removes the risk the auto-publishing version carried. That was a
`workflow_run` job holding `contents: write`, which on a public repo whose
tests.yml runs on `pull_request` meant anyone could reach it: `head_branch` on a
fork PR is the contributor's own branch name, so an `auto/formula/*` prefix
authenticates nothing, a hostile formula is still a formula-only diff, and
building one still produces bottles. This job holds no `contents` write and uses
no third-party actions, so the worst a hostile PR gets from it is a review
request on itself.

The verdict is the useful part, because a green check here does not mean the
formula built. `brew test-bot` sets ignore_failures for any formula that is
neither newly added nor already bottled at its current version
(lib/tests/formulae.rb) -- every version bump in this tap -- so a failed
`brew install --build-bottle` still exits 0 and merely prints
"Warning: N failed step ignored!". The comment therefore reports bottle artifacts
(the only trustworthy proof of a build), any swallowed failures, and whether the
tested SHA is still the head.

Dry-run against real runs of this repo:
  30964311208 (generated 0.8.1) -> "bottles produced for bottles_macos-15,
    bottles_macos-26, no swallowed failures", review requested;
  30856043824 (original 0.8.1, three green checks, never built) -> "not safe to
    publish: no bottle artifacts; test-bot (macos-15) and (macos-26) ignored a
    failed step", with an explicit do-not-label warning.

Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>

* ci: use an unguessable delimiter for the verdict step output

`problems` embeds job and artifact names. With a fixed heredoc delimiter, a
crafted name containing that delimiter could close the block early and inject
further step outputs — including ok=true, which is what decides whether the
comment says the bump is safe to publish. Only same-repo App-authored PRs reach
this step today, so this is defence in depth rather than a live hole.

Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>

* ci: fix brew style shellcheck findings in the review workflow

`brew style` on a tap does not only lint Ruby: it runs shellcheck and actionlint
over the `run:` blocks of .github/workflows too, so this workflow failed the
`brew test-bot --only-tap-syntax` step. Two findings, both mine:

  SC2129 -- consecutive `echo ... >> "$GITHUB_OUTPUT"` lines should be one
    grouped redirect.
  SC2016 -- the comment bodies were built with single-quoted printf formats
    containing markdown code spans, and shellcheck reads those backticks as
    command substitution.

Replaced the printf calls with unquoted heredocs, which need no single quotes and
read better, using \` for literal backticks so they stay markdown rather than
becoming command substitution. Bodies now go to a file under $RUNNER_TEMP and are
passed via --body-file.

Reproduced locally with `brew style omnigent-ai/tap` against the real linter
(3 files inspected, no offenses detected) and rendered both comment variants to
confirm the code spans survive.

Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>

* ci: indent the heredoc bodies so the workflow is valid YAML

The heredoc bodies and their EOF terminators sat at column 0, which ends the
`run: |` block scalar. YAML then tried to parse the body as YAML and hit `@`, a
reserved indicator:

    formula-bump-review.yml:156:0: could not parse as YAML: found character that
    cannot start any token [syntax-check]

Indent them to the block-scalar level instead. YAML strips that common
indentation, so bash still receives the terminator at column 0 and the heredoc
works.

I had claimed `brew style` was clean before pushing the previous commit. It was
not: I grepped its output for shellcheck/actionlint/offenses, and the failing line
is tagged [syntax-check], so my own filter hid it. Verified properly this time --
full `brew style` output (3 files inspected, no offenses detected, exit 0), the
YAML parsed and the comment step extracted and executed with a stubbed `gh` to
confirm the rendered markdown has no stray indentation.

Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>

---------

Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>
nicky-isaacs-awoo added a commit to DataDog/omnigent that referenced this pull request Aug 13, 2026
…ent-ai#4080)

Closes omnigent-ai#866

The `omnigent` Homebrew formula has not built since 0.7.0, and the tap reported
green anyway, so 0.7.0, 0.8.0 and 0.8.1 all merged with no bottle — every user
compiled from source, and CEL policies silently did not work.

- **Root cause was the CEL migration.** omnigent-ai#2970 swapped `cel-expr-python` for
  `cel-python` on the premise that it is pure Python. It is not: `cel-python`
  hard-depends on `google-re2`, whose sdist runs `bazel build` whenever
  `GITHUB_ACTIONS` is set. The bazel dependency moved rather than disappeared.
- **Pin compiled extensions to upstream wheels.** `generate_formula.py` gains
  `WHEEL_REQUIRED` / `PREFER_WHEEL` / `PURE_WHEEL` with abi3 and universal2
  handling, so grpcio (by far the most expensive build), protobuf, regex,
  uvloop, httptools, argon2-cffi-bindings, markupsafe, pyyaml, zstandard and
  google-re2 stop being compiled. Native wheels rank above pure-Python ones, so
  protobuf keeps its upb build instead of the slow fallback.
- **jiter, tiktoken and watchfiles keep building from source.** Their maturin
  wheels carry no Mach-O install-name padding, so Homebrew relocation fails with
  "Failed changing dylib ID" (omnigent-ai#866). `pendulum` can go neither way — its wheel
  cannot be relocated and its sdist does not link on 3.14 (pyo3 leaves
  `_Py_NoneStruct` undefined) — so it takes the pure-Python wheel, which ships
  no extension module at all.
- **A dropped dependency is now an error, not a warning.** A missing sdist used
  to be skipped silently, yielding a formula whose venv lacked an import;
  `--allow-no-sdist` is the explicit waiver. The formula test also asserts
  `import re2, celpy`, since omnigent imports celpy behind `try/except
  ImportError` and would otherwise disable policies silently.
- **Delete `update-homebrew.yml`.** It raced `homebrew-tap-pr.yml` on the same
  `release: published` event and asserted on hand-maintained stanzas the
  template no longer emits, so it failed on every run. Its one worthwhile part
  moves into `homebrew-tap-pr.yml`: an admin/maintain gate on manual dispatch
  (it writes to another repo with an App token), plus
  `persist-credentials: false`. Its nightly `schedule` is deliberately NOT
  carried over -- that cron only existed because `brew
  update-python-resources` resolves through pip's `--uploaded-prior-to=P1D`
  window and so could never see a same-day release. The generator runs `uv pip
  compile --no-config` straight against PyPI, so the blindness it worked around
  no longer exists, and a nightly regeneration would just burn a runner to
  print "nothing to do".

Verified by building the generated formula in the tap, not by inspection.

- `omnigent-ai/homebrew-tap#18` contains **verbatim output of this
  `generate_formula.py`** and bottled successfully on macos-15 and macos-26
  (run 30944428771, `bottles_macos-15` / `bottles_macos-26` ≈ 37 MB each). This
  is the check that matters: it proves the generator — not a hand-edit —
  produces a buildable formula, so the next release regenerates something that
  works.
- `omnigent-ai/homebrew-tap#17` carries the same fix for the shipped 0.8.1
  formula and is green on all three runners, with `brew test` running
  `import re2, celpy`. Inspected the bottle: `celpy/__init__.py`,
  `re2/_re2.cpython-314-darwin.so`, and a relocated
  `jiter/jiter.cpython-314-darwin.so`.
- Audited every pinned wheel by replaying Homebrew's own operation,
  `install_name_tool -id <Cellar path>` against each extracted `.so`, so the
  wheel/source split is evidence-based rather than guessed.
- `python3.12 -m py_compile`, `ruff check`, `ruff format --check`, `brew style`
  (no offenses), `ruby -c`, plus stubbed-PyPI unit checks of the new failure
  paths (missing sdist is fatal, `--allow-no-sdist` waives it, abi3 accepted,
  free-threaded `cp314t` rejected).
- Confirmed generator output matches the green formula: same 100 resources,
  identical sdist/wheel split, no non-comment differences.

N/A — release tooling, no user-visible UI.

- [x] Bug fix
- [ ] Feature
- [ ] UI / frontend change
- [ ] Refactor / chore
- [ ] Docs
- [ ] Test / CI
- [ ] Breaking change

- [ ] Unit tests added / updated
- [ ] Integration tests added / updated
- [ ] E2E tests added / updated
- [x] Manual verification completed
- [ ] Existing tests cover this change
- [ ] Not applicable

The generator has no test suite in this repo, and its real contract — "the
emitted formula builds under Homebrew on macOS" — cannot be asserted here. It is
covered instead by building the generated formula on the tap's `brew test-bot`
matrix (homebrew-tap#18, bottles produced on macos-15 and macos-26). The two new
generator failure paths were exercised locally against stubbed PyPI metadata,
and every wheel pin was verified relocatable with `install_name_tool`.

`brew install omnigent` works again, and installs prebuilt wheels instead of
compiling grpcio and friends from source.

Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>

Co-authored-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>
nicky-isaacs-awoo added a commit to DataDog/omnigent that referenced this pull request Aug 13, 2026
…ent-ai#4080)

Closes omnigent-ai#866

The `omnigent` Homebrew formula has not built since 0.7.0, and the tap reported
green anyway, so 0.7.0, 0.8.0 and 0.8.1 all merged with no bottle — every user
compiled from source, and CEL policies silently did not work.

- **Root cause was the CEL migration.** omnigent-ai#2970 swapped `cel-expr-python` for
  `cel-python` on the premise that it is pure Python. It is not: `cel-python`
  hard-depends on `google-re2`, whose sdist runs `bazel build` whenever
  `GITHUB_ACTIONS` is set. The bazel dependency moved rather than disappeared.
- **Pin compiled extensions to upstream wheels.** `generate_formula.py` gains
  `WHEEL_REQUIRED` / `PREFER_WHEEL` / `PURE_WHEEL` with abi3 and universal2
  handling, so grpcio (by far the most expensive build), protobuf, regex,
  uvloop, httptools, argon2-cffi-bindings, markupsafe, pyyaml, zstandard and
  google-re2 stop being compiled. Native wheels rank above pure-Python ones, so
  protobuf keeps its upb build instead of the slow fallback.
- **jiter, tiktoken and watchfiles keep building from source.** Their maturin
  wheels carry no Mach-O install-name padding, so Homebrew relocation fails with
  "Failed changing dylib ID" (omnigent-ai#866). `pendulum` can go neither way — its wheel
  cannot be relocated and its sdist does not link on 3.14 (pyo3 leaves
  `_Py_NoneStruct` undefined) — so it takes the pure-Python wheel, which ships
  no extension module at all.
- **A dropped dependency is now an error, not a warning.** A missing sdist used
  to be skipped silently, yielding a formula whose venv lacked an import;
  `--allow-no-sdist` is the explicit waiver. The formula test also asserts
  `import re2, celpy`, since omnigent imports celpy behind `try/except
  ImportError` and would otherwise disable policies silently.
- **Delete `update-homebrew.yml`.** It raced `homebrew-tap-pr.yml` on the same
  `release: published` event and asserted on hand-maintained stanzas the
  template no longer emits, so it failed on every run. Its one worthwhile part
  moves into `homebrew-tap-pr.yml`: an admin/maintain gate on manual dispatch
  (it writes to another repo with an App token), plus
  `persist-credentials: false`. Its nightly `schedule` is deliberately NOT
  carried over -- that cron only existed because `brew
  update-python-resources` resolves through pip's `--uploaded-prior-to=P1D`
  window and so could never see a same-day release. The generator runs `uv pip
  compile --no-config` straight against PyPI, so the blindness it worked around
  no longer exists, and a nightly regeneration would just burn a runner to
  print "nothing to do".

Verified by building the generated formula in the tap, not by inspection.

- `omnigent-ai/homebrew-tap#18` contains **verbatim output of this
  `generate_formula.py`** and bottled successfully on macos-15 and macos-26
  (run 30944428771, `bottles_macos-15` / `bottles_macos-26` ≈ 37 MB each). This
  is the check that matters: it proves the generator — not a hand-edit —
  produces a buildable formula, so the next release regenerates something that
  works.
- `omnigent-ai/homebrew-tap#17` carries the same fix for the shipped 0.8.1
  formula and is green on all three runners, with `brew test` running
  `import re2, celpy`. Inspected the bottle: `celpy/__init__.py`,
  `re2/_re2.cpython-314-darwin.so`, and a relocated
  `jiter/jiter.cpython-314-darwin.so`.
- Audited every pinned wheel by replaying Homebrew's own operation,
  `install_name_tool -id <Cellar path>` against each extracted `.so`, so the
  wheel/source split is evidence-based rather than guessed.
- `python3.12 -m py_compile`, `ruff check`, `ruff format --check`, `brew style`
  (no offenses), `ruby -c`, plus stubbed-PyPI unit checks of the new failure
  paths (missing sdist is fatal, `--allow-no-sdist` waives it, abi3 accepted,
  free-threaded `cp314t` rejected).
- Confirmed generator output matches the green formula: same 100 resources,
  identical sdist/wheel split, no non-comment differences.

N/A — release tooling, no user-visible UI.

- [x] Bug fix
- [ ] Feature
- [ ] UI / frontend change
- [ ] Refactor / chore
- [ ] Docs
- [ ] Test / CI
- [ ] Breaking change

- [ ] Unit tests added / updated
- [ ] Integration tests added / updated
- [ ] E2E tests added / updated
- [x] Manual verification completed
- [ ] Existing tests cover this change
- [ ] Not applicable

The generator has no test suite in this repo, and its real contract — "the
emitted formula builds under Homebrew on macOS" — cannot be asserted here. It is
covered instead by building the generated formula on the tap's `brew test-bot`
matrix (homebrew-tap#18, bottles produced on macos-15 and macos-26). The two new
generator failure paths were exercised locally against stubbed PyPI metadata,
and every wheel pin was verified relocatable with `install_name_tool`.

`brew install omnigent` works again, and installs prebuilt wheels instead of
compiling grpcio and friends from source.

Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>

Co-authored-by: Zeyi (Rice) Fan <zeyi.f@databricks.com>

Co-authored-by: Nick Isaacs <nick.isaacs@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant