Skip to content

feat: warn when CUR cost data is incomplete#149

Merged
FloWuenne merged 2 commits intoremove-seqera-predicted-costfrom
cursor/cur-missing-cost-warning-8efd
May 1, 2026
Merged

feat: warn when CUR cost data is incomplete#149
FloWuenne merged 2 commits intoremove-seqera-predicted-costfrom
cursor/cur-missing-cost-warning-8efd

Conversation

@FloWuenne
Copy link
Copy Markdown
Contributor

@FloWuenne FloWuenne commented May 1, 2026

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool, parameter, or workflow path, update the relevant docs.
  • If plugin declarations changed, update CITATIONS.md, README.md, and agent/context guidance in the same PR.
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

Description

Add explicit CUR coverage reporting so the benchmark report warns when a supplied CUR does not cover all included tasks. Instead of silently presenting unmatched tasks as zero-cost totals, the report now summarizes missing-cost runs and processes in a short warning card.

What changed

  • track CUR match coverage per included task during report aggregation
  • add cost_coverage metadata to report_data.json
  • render a concise warning card in the report when a supplied CUR is incomplete
  • add tests for both the aggregation contract and rendered warning logic

Walkthrough

benchmark_report_warning_walkthrough.mp4
Incomplete CUR warning in compiled benchmark report

Testing

  • python3 -m pytest -v modules/local/aggregate_benchmark_report_data/tests/test_aggregate.py modules/local/render_benchmark_report/tests/test_render.py
  • python3 -m pytest -v modules/local/aggregate_benchmark_report_data/tests/test_aggregate.py modules/local/normalize_benchmark_jsonl/tests/test_normalize.py modules/local/render_benchmark_report/tests/test_render.py bin/test_benchmark_report_fetch.py
  • python3 bin/benchmark_report.py normalize-jsonl --data-dir /tmp/cur-warning-demo/data --costs /tmp/cur-warning-demo/partial_cur.parquet --output-dir /tmp/cur-warning-demo/jsonl_bundle
  • python3 bin/benchmark_report.py aggregate-report-data --jsonl-dir /tmp/cur-warning-demo/jsonl_bundle --output /tmp/cur-warning-demo/report_data.json
  • python3 bin/benchmark_report.py render-html --data /tmp/cur-warning-demo/report_data.json --brand assets/brand.yml --output /tmp/cur-warning-demo/report.html
  • /root/.local/bin/pre-commit run --all-files

Notes

Container-backed nf-test / full Nextflow docker verification remains blocked in this VM because Docker launches hit the known cgroup/socket environment issue, but the Python stage tests and local normalize/aggregate/render compile flow all passed.

To show artifacts inline, enable in settings.

Open in Web Open in Cursor 

cursoragent and others added 2 commits May 1, 2026 16:20
Co-authored-by: Florian Wuennemann <flowuenne@gmail.com>
Co-authored-by: Florian Wuennemann <flowuenne@gmail.com>
@FloWuenne FloWuenne marked this pull request as ready for review May 1, 2026 20:23
@FloWuenne FloWuenne merged commit ed4654d into remove-seqera-predicted-cost May 1, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants