The public evaluation bundle is a template/reference workflow for connecting Permea Core input families to reviewer-facing generated artifacts. It shows how dataset-card, benchmark-card, and evidence-card style inputs can be linked to generated surfaces without asserting dataset access, acquisition execution, or measured performance.
- Researchers who want to inspect the artifact pattern before adding new public inputs.
- Developers who want to extend generators, validation, and tests together.
- Reviewers who want to trace an evaluation packet back to bounded input families and generated surfaces.
The contract is local, deterministic, and metadata-only. The bundle references public repository inputs, generates a packet, links existing artifact families, and hands off to validation and reproducibility commands.
dataset_cards/benchmark_cards/evidence_cards/acquisition_manifests/run_manifests/output_packages/
The public artifact standards are indexed at docs/specs/README.md. The evaluation pattern should stay aligned with the dataset card, benchmark card, evidence card, run manifest, and output package specifications.
The artifact validator command checks whether current public examples satisfy required structure, non-claim, claim-boundary, and evidence-linkage expectations before they are used as evaluation-pattern examples:
python3 scripts/permea_check.pyCopyable reference packages are available in examples/. They demonstrate how evaluation handoff metadata can be packaged without expanding claims.
python3 scripts/permea_evaluate.pydocs/examples/generated/EVALUATION_PACKET.mddocs/examples/generated/EVALUATION_PACKET.json
Start with docs/examples/generated/EVALUATION_PACKET.md, then follow its lineage links to the generated evidence surface, demo packet, artifact index, evidence matrix, reproducibility report, dry-run report, and artifact families.
Add or update dataset-card metadata, keep source and limitation fields explicit, add generator and validation coverage, and keep the generated packet linked to the relevant dataset-card family.
Add benchmark task metadata with bounded labels, split policy, metrics, and limitations. Regenerate benchmark cards, update validation where needed, and keep benchmark claims tied to generated artifacts.
Add evidence-card records only when source, uncertainty, review status, and claim boundary are clear. Regenerate the evidence-card family and rerun validation before publishing changes.
python3 scripts/permea_evaluate.py
python3 scripts/permea_check.py
python3 scripts/permea_reproduce.py
python3 scripts/permea_validate.py
python3 scripts/generate_permea_artifacts.py
python3 scripts/validate_permea_artifacts.py- no dataset downloaded
- no acquisition executed
- no redistribution rights confirmed
- no wet-lab validation by Permea
- no clinical efficacy claim
- no model performance claim
- no SOTA claim
- no solved-delivery claim
- This bundle is a public template/reference workflow, not an experimental result.
- It links existing metadata and generated surfaces; it does not load source datasets.
- It does not execute acquisition, call external services, run ML, score candidates, or measure performance.
- It does not establish biological delivery, mechanism, safety, therapeutic effect, or generalization.
- Use the evaluation packet as the reviewer-facing entry point for the template pattern.
- Add new input families only with matching generator, validation, tests, and non-claim coverage.
- Run reproduction and validation commands before proposing public evaluation changes.