Upstream-readiness hardening: restore CI gate, schemas, smoke test, attestation, consistency CI - #19
Open
AIwork4me wants to merge 13 commits into
Open
Upstream-readiness hardening: restore CI gate, schemas, smoke test, attestation, consistency CI#19AIwork4me wants to merge 13 commits into
AIwork4me wants to merge 13 commits into
Conversation
Commit 6a64af5 renamed reproducibility.lock.yaml -> REPRO.yaml (pure rename) but did not update consumers, so: - scripts/check_repo.py loaded reproducibility.lock.yaml -> None -> _current_vlm_overall(None) raised AttributeError -> the CI gate CRASHED (rc=1) on main, and the pytest suite errored (test_upstream_commits_pinned_in_lock, test_official_reference_verified, test_modelcard_lock_agreement, and the clean_on_repo integration). - ~13 docs/cards still pointed at the dead filename (broken links). Fix: check_repo reads REPRO.yaml (LOCK_FILENAME constant); guard _current_vlm_overall against a missing lock so the gate reports cleanly instead of crashing. Replace every dead-filename reference with REPRO.yaml. Gate now clean (rc=0); 177 tests pass. docs/superpowers/ frozen design records left untouched. Co-Authored-By: Claude <noreply@anthropic.com>
reproduce.md / reproduce.pipeline.md (added in 6a64af5) cited commands that do not exist in the package CLI: - `python -m mineru_rocm runner` / `python -m mineru_rocm serve-vlm` — no __main__.py, no serve-vlm subcommand (cli.py has predict/score/doctor/ validate/manifest/canary). - `mineru-rocm predict pipeline` (positional) + `--img-dir` / `--out-dir`. Real CLI is `predict --backend {pipeline|vlm-vllm|vlm-llamacpp} --gt-json --images-dir --pred-dir` (+ `--server-url`/`--api-model-name` for VLM); VLM is served via `examples/serve_vlm_vllm.sh`. Also dropped the hardcoded `/root/datasets/...` image dir in favour of env-var placeholders, and added the MinerU version boundary + attestation pointer. examples/serve_vlm_vllm.sh: MODEL_DIR now derives from $HF_HOME / ~/.cache/huggingface (override via MODEL_DIR) instead of a hardcoded /root/.cache path; fails clearly if the snapshot is absent. src/mineru_rocm/backends/vlm.py: _DEFAULT_VLM_MODEL_PATH (hardcoded /root) replaced with _default_vlm_model_path() derived from the HF cache; the graceful fallback to the HF repo id is preserved. No consumer parsed the reproduce-doc front-matter, so the rewrite is safe. Co-Authored-By: Claude <noreply@anthropic.com>
Make the MinerU version support boundary explicit and unambiguous. REPRO.yaml: add a structured `validated_software` block — mineru 3.4.4 (commit 0dfc9460) validated with its scope; mineru_vl_utils 1.0.5 validated; mineru_4 explicitly `not_validated` with a no-extrapolation note; cli_backend_names lists validated (pipeline/vlm-vllm/vlm-llamacpp) vs sample-only (vlm-transformers). Wording: add a "Version boundary" statement to README.md, README.zh-CN.md, and docs/reproducibility.md; add a `mineru_software` field (mineru=3.4.4, mineru_4=not_validated) to model_card.json and model_card.pipeline.json. Audit: docs/audit/upstream-readiness-audit.md records the full current-state audit (version drift, dead-filename refs, fabricated CLI, stale Windows-VLM status, byte-exact vs not_recorded, CI gaps) as the basis for this branch, distinguishing confirmed facts from cannot-confirm-now items. Co-Authored-By: Claude <noreply@anthropic.com>
…ording Stale Windows-HIP VLM status (contradicted by Phase 2, commit 339a5e5) fixed in docs/hardware-matrix.md, docs/architecture.md, docs/how-it-works.md, README.zh-CN.md (VLM + pipeline tables + narrative + Known Gaps), and the docs/known-gaps.md self-contradiction ("verified" vs "community"). Windows-HIP VLM is now consistently `community-tested` (vlm-llamacpp, 95.46). README.md: repaired corrupted results block (a stray \x0b control byte had eaten "`v" of "vlm-llamacpp"; run-stats paragraph was duplicated; pipeline table rows were orphaned) and gave the pipeline its own results subsection. Reproducibility claims scoped to the evidence: "byte-exact" -> "SHA-256-pinned where recorded" in README/README.zh-CN/docs/reproducibility.md; REPRO.yaml now carries a documented `reproducibility.level: artifact-manifested` with `levels_documented` + `limitations` (table_sha256 / canary GT / hip_visible_devices / VLM drift / unpinned upstream anchor). Release provenance refreshed to the latest existing tag v1.1.0 (commit 339a5e5, tag object 04be8fe…). Support wording unified to the documented enum (community-tested / independently-reproduced / experimental / not-tested / unsupported): model cards migrated to `community-tested`; the divergent model_card.windows-hip.json (community_badge + singular platform) rewritten to the shared schema_version/platforms/badge shape; disclaimer added to both READMEs + the hardware matrix with a status legend. CHANGELOG historical entries left intact (timestamped records + the literal `source: verified` field name). Co-Authored-By: Claude <noreply@anthropic.com>
schemas/repro.schema.json: structural contract for REPRO.yaml — required sections, the annotated-tag/release commit/tag-object SHA patterns, the documented reproducibility.level enum (environment-pinned / artifact-manifested / prediction-reproduced / score-reproduced / byte-exact), and the MinerU version boundary (mineru.validated + mineru_4.status enum). schemas/model-card.schema.json: unified field model for every model_card*.json — schema_version/model_id/platforms/badge/omnidocbench_version/overall/ artifacts/mineru_software, with the support-status enum (community-tested / independently-reproduced / experimental / not-tested / unsupported) on badge values and known-platform enums on platforms + badge keys. 'verified' / 'community_badge' / 'community-wanted' are no longer valid. scripts/validate_schemas.py: jsonschema gate + cross-field checks the schema language cannot express (every *_sha256 value is 64-hex or 'not_recorded'; badge keys are known platforms; mineru.validated=true ⇒ mineru_4 not_validated). Errors name file/field/expected/actual + a fix hint. pyproject.toml [dev]: add jsonschema>=4. tests/test_validate_schemas.py: clean-on-repo + negative cases (bad SHA, undefined status, bad level enum). Co-Authored-By: Claude <noreply@anthropic.com>
scripts/generate_evidence_manifest.py: deterministic SHA-256 manifest of the COMMITTED evidence (result bundles, model cards, REPRO.yaml, eval config). The lock pins external inputs; this pins the in-repo artifacts so load-bearing evidence drift is detectable. Repo-relative paths only (no /root/...); files sorted before hashing for cross-machine stability; missing critical files exit 1 (never silently skipped); records algorithm/tool/git_commit/generated_at. --canonical (default curated set) | --file | --dir | --out | --format --check (exit 1 on any sha/path drift vs the committed manifest) evidence/evidence-manifest.json: the generated manifest for the 44 canonical evidence files (regenerable; stable across runs). tests/test_generate_evidence _manifest.py covers stability, missing-file failure, and drift detection. Co-Authored-By: Claude <noreply@anthropic.com>
docs/reproducibility-attestation.md defines the only valid support-status terms
(community-tested / independently-reproduced / experimental / not-tested /
unsupported) and the independence levels (repeat run / second-machine /
internal-independent / third-party-independent). It records who ran what:
- primary_run: AIwork4me maintainer (repo operator), gfx1100/W7900, ROCm 7.2 —
predictions + scores regenerated by the operator; independence_level =
'repeat run' (self-attested).
- secondary_run: SAME maintainer/team on a different machine (Strix Halo,
Windows HIP) — 'second-machine reproduction', NOT independent.
- third-party independent reproduction: NOT performed.
States explicitly that the 95.46->95.56 VLM change is a re-scoring of the same
predictions (same team), not new inference and not independent reproduction;
and that no clean-environment reproduction is automated. Therefore every result
stays `community-tested`; promotion to `independently-reproduced` requires
appending a third-party entry here.
REPRO.yaml: add a structured `reproduction` block mirroring the doc
(primary_run / secondary_run / internal_independent_reproduction /
third_party_independent_reproduction). schemas/repro.schema.json: require
`reproduction` with its key sub-fields. Evidence manifest regenerated.
Co-Authored-By: Claude <noreply@anthropic.com>
scripts/smoke_test.py: layered install-confirmation that needs no full
OmniDocBench run.
L0 environment (OS/python/mineru_rocm/torch+HIP/GPU/cuda; informational)
L1 device availability (torch.cuda.is_available/count/name)
L2 owned CLI (mineru-rocm --help + doctor; advisory upstream mineru, warns
if != validated 3.4.4)
L3 minimal inference via the GPU-free `smoke` backend (the adapter contract)
L4 optional VLM HTTP health check (--vlm-url; no vLLM/llama.cpp assumption)
--check-only skips GPU + inference (green on no-GPU CI, never claims a GPU run);
--json for machine output; repo-relative --img-dir/--out-dir only (no /root).
Documents why torch.cuda.* reports the AMD GPU on a ROCm PyTorch.
docs/smoke-test.md: level table, path policy, result interpretation, and the
MinerU 3.4.4 boundary. tests/test_smoke_test.py covers --check-only, --help,
L0/L3/L4 unit behavior. evidence/smoke-run-2026-07-26.log: a redacted Level-3
run captured on the 4x gfx1100/W7900 host (L0-L3 OK).
Co-Authored-By: Claude <noreply@anthropic.com>
scripts/check_repo.py gains six structural-drift gates (each with a clean-on-repo
+ negative test in tests/test_check_repo.py):
- check_lock_present: REPRO.yaml must exist (a missing lock silently no-ops
every lock-driven check);
- check_no_dead_lock_filename: no tracked file references the pre-rename
`reproducibility.lock.yaml` (excludes superpowers/audit/tests + defining scripts);
- check_doc_cli_commands_exist: documented `mineru-rocm <sub>` / `python -m
mineru_rocm.<mod>` use real subcommands/modules (catches fabricated CLI like
`python -m mineru_rocm runner` / `serve-vlm`);
- check_no_root_in_doc_commands: no `/root/` in bash code blocks or shell scripts
(excludes spike investigation logs + the lock's cross_check_source);
- check_version_boundary_present: README + docs/reproducibility.md state the
validated MinerU version AND a 4.x boundary;
- check_markdown_internal_links: relative markdown links resolve (catches dead
links / deleted-file refs; external + same-file anchors skipped).
pyproject.toml: pin [tool.ruff.lint] select = E4/E7/E9/F so `ruff check .` is
deterministic across ruff versions (0.16's expanded default no longer turns a
clean tree red).
.github/workflows/ci.yml: the core job now also runs validate_schemas,
generate_evidence_manifest --check, and smoke_test --check-only (no-GPU dry-run)
before check_repo. jsonschema added to [dev].
Co-Authored-By: Claude <noreply@anthropic.com>
docs/upstream-documentation-source.md: the fact source for a community docs-only contribution to opendatalab/MinerU AMD.md (issue #5288). States the support nature (community-maintained/tested, not official, not extrapolable), the validated matrix with each row linked to an immutable metric_result at tag v1.1.0, the minimal install + smoke test, the known limitations, and the immutable-reference policy (frozen evidence at v1.1.0; new canonical docs get a TODO to pin to the next release tag — no fabricated future tag). docs/audit/upstream-readiness-completion-report.md: maps every brief phase to its deliverable, lists the 42 changed files with reasons, records the key decisions, tabulates the tests actually run with their results (203 pytest passed; ruff/validate_schemas/manifest/smoke/check_repo all green; reuse + clean-venv blocked by the container env, not the repo), states the honest unfinished items with precise follow-up commands, and gives the verdict (ready with documented limitations). Co-Authored-By: Claude <noreply@anthropic.com>
…blocked by torch build Attempted the real pipeline backend smoke on 2026-07-26 (the #1 unfinished item). Env assembled into /root/venvs/mineru-pipeline: mineru 3.4.4 + torch 2.11.0+rocm7.2 + torchvision 0.26.0+rocm7.2 + transformers 4.57.6 + mineru[pipeline] deps. The pipeline RUNS (imports resolve, 48 GB GPU detected, DocAnalysis init reaches model-load; models download via hf-mirror). Blocked at model-load: the worker dies silently (no traceback even at DEBUG, clean shutdown, not OOM). Root cause is the torch build — the validated torch (REPRO.yaml 2.14.0.dev20260717+rocm7.2, an AMD nightly) is not on the public pytorch.org rocm7.2 index (py3.12 has only 2.11.0+); the public 2.11.0 (gfx110X-dgpu build) hits the known gfx1100 kernel issue (RECOVERY.md gfx110X-all migration) at model load. Corrects the earlier "mineru not installed" claim (it IS installed in the venv; mineru-rocm doctor only probes /opt/venv). The repo's runtime evidence remains the committed 1651-page bundles (produced with the validated torch on this hardware). Verdict unchanged: ready with documented limitations. Co-Authored-By: Claude <noreply@anthropic.com>
…htly misdiagnosis The user rightly challenged "why nightly, can't a stable release torch be used?" The answer (now proven): stable torch works; the nightly was never required. Re-ran the real pipeline smoke correctly on 2026-07-26: env assembled into /root/venvs/mineru-pipeline (mineru 3.4.4 + mineru[pipeline] + six/albumentations + stable torch 2.11.0+rocm7.2 + torchvision 0.26.0+rocm7.2 + transformers 4.57.6). The pipeline ran end-to-end on examples/sample.png (gfx1100/W7900): all models loaded (DocAnalysis init done), layout+OCR+formula+table ran, produced sample.md (376 bytes of CORRECT OCR) + the full result bundle. The earlier "blocked at model-load by torch build / gfx1100 kernel bug / needs nightly" (commit b0a7c5c) was a MISDIAGNOSIS. The real blockers were mundane: (1) an incomplete weight download — the layout model.safetensors was missing (only config files present); mineru's cache treated the partial snapshot as complete and did not re-fetch; (2) the FastAPI task runner swallowed the resulting OSError (empty task error, no traceback), which I then wrongly attributed to the torch build. A direct GPU sanity test (matmul + small CNN forward on torch 2.11.0+rocm7.2) and an in-process MineruPipelineModel construction both passed, confirming the torch build is fine. docs/audit/upstream-readiness-completion-report.md: the "real smoke" row flips from "blocked" to DONE/SUCCESS, with the correction noted. REPRO.yaml reproducibility.limitations: adds that the benchmark pipeline torch (2.14.0.dev20260717+rocm7.2, an AMD nightly) is NOT on the public pytorch.org rocm7.2 index, but stable torch 2.11.0+rocm7.2 is verified to run the pipeline (the recommended public-reproduction path). The nightly pin is unchanged — it is what produced the 1651-page benchmark scores (historical fact). evidence/smoke-real-pipeline-2026-07-26.log: redacted successful-smoke record. Co-Authored-By: Claude <noreply@anthropic.com>
The staged upstream PR section (docs/upstream-pr/AMD.md.section.zh.md) said "windows-hip 未验证" — but Windows-HIP VLM was since verified (Phase 2, llama.cpp HIP / vlm-llamacpp, Strix Halo gfx1151, Overall 95.46). Updated that line to state the verification and point at docs/upstream-documentation-source.md (the authoritative matrix), while keeping the section's Linux/vLLM focus and the intentional standalone 95.46 (this draft documents the standalone mineru-rocm path for the upstream contribution, per the check_repo upstream* exclusion note). The reproducibility.lock.yaml → REPRO.yaml rename and the rest of the PR package (README.md PR description, README.row.md table change, issue-5288-comment.md pre-PR comment) are current. blob/main links remain working references; immutability pin to a release tag is deferred to docs/upstream-documentation-source.md §5 TODO (REPRO.yaml postdates the v1.1.0 tag, so no existing tag points at it). Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Converges MinerU-ROCm into a self-consistent, reproducible, CI-gated evidence store backing the upstream AMD/ROCm docs contribution (#5288). 13 commits; all gates green (203 pytest, ruff, validate_schemas, evidence-manifest 44-match, smoke --check-only, check_repo).
Headline fixes
main: thereproducibility.lock.yaml → REPRO.yamlrename (6a64af5) leftcheck_reporeading the old name →AttributeError(gate crashed) + tests erroring. Fixed to readREPRO.yaml; dead filename replaced across ~13 docs/cards.reproduce.md/reproduce.pipeline.md(python -m mineru_rocm runner,serve-vlm) rewritten to the realmineru-rocm predict/scoreinterface;/roothardcodes removed from docs/scripts/vlm.py.reproducibility.level: artifact-manifested(+ limitations).community_badge/ singularplatform) unified; support-status enum enforced.New machinery
schemas/{repro,model-card}.schema.json+scripts/validate_schemas.py(jsonschema + SHA-256 format gate).scripts/generate_evidence_manifest.py+evidence/evidence-manifest.json(44-file SHA-256 manifest, drift--check).scripts/smoke_test.py(L0–L4 +--check-only) +docs/smoke-test.md.docs/reproducibility-attestation.md+REPRO.yamlreproductionblock (status enum + independence levels; all resultscommunity-tested, no third-party repro).check_repoconsistency gates (dead-filename, CLI-existence,/root-in-doc-commands, version-boundary, markdown-links, lock-present) + ruffselectpinned + CI runs the new gates.docs/audit/{upstream-readiness-audit,upstream-readiness-completion-report}.md,docs/upstream-documentation-source.md.Real pipeline smoke (verified)
Runs end-to-end on the public stable
torch 2.11.0+rocm7.2(gfx1100/W7900) — correct OCR onexamples/sample.png(evidence/smoke-real-pipeline-2026-07-26.log). The benchmark nightly torch pin is historical; stable torch is the public-reproduction path.Out of scope
Upstream PR (
opendatalab/MinerU) staged indocs/upstream-pr/; opens separately after further review.