Skip to content

Autonomous: Score lap time against a reference lap computed from the course - #233

Merged
GrayM8 merged 2 commits into
mainfrom
gray/reference-lap
Nov 8, 2026
Merged

GrayM8 merged 2 commits into
mainfrom
gray/reference-lap

Conversation

@GrayM8

@GrayM8 GrayM8 commented Nov 8, 2026

Copy link
Copy Markdown
Member

The gate now scores lap time against a reference lap computed from the course, not a truth-stack run. The reference is the fastest standing-start lap of the world's reference centerline under the run's own speed limits, so it is the same on every machine and cannot fail the way the truth stack does (pass criteria v2).

What changed

  • lhr_metrics/reference_lap.py (new). Plain Python. It samples the centerline every 0.5 m and caps speed at each sample by v_max and by a_lat_max (floored at v_min, as pursuit_node floors it). Curvature comes from the circle through the points 3 m either side. A forward pass from rest at the start/finish gate applies max_accel, and a backward pass applies max_decel.
  • gazebo_demo.launch.py forwards the speed limits to metrics_node (lookahead_dist, a_lat_max, v_min, v_max, max_accel, max_decel, typed doubles). Before this, every Gazebo metrics row logged them as 0.0, so a row could not say what the car ran under. This is the only change in lhr_gazebo. It sits in the metrics_node block that Autonomous: drive Gazebo sensors from the FMU in lockstep #221, Autonomous: Keep FMU and Gazebo steps synchronized #222, Autonomous: import real competition track layouts #223 and Autonomous: Add fixed extreme autocross scenarios #218 also touch, so expect a small conflict for whichever lands second.
  • score_gate.py: lap_time passes at no more than 1.5 times the course reference. New columns reference_lap_s and lap_reference, and a reference-lap table in the summary. Campaigns recorded before this change have zeros for the limits and fall back to their truth-stack baselines, so they score as before.
  • run_gate.sh: BASELINE defaults to 0, which takes the matrix without skidpad from 23 runs to 18. BASELINE=1 still adds the truth-stack runs for comparison.
  • Docs: the gate README (a new "reference lap" section), the lhr_metrics and lhr_gazebo READMEs, and the pass criteria.

Testing

  • 12 reference_lap unit tests:
    • closed-form circles limited by v_max and by lateral acceleration, both within 0.05%
    • the v_min floor
    • the same time from any start point and in either direction
    • each limit only ever speeds the lap up
    • braking before corners costs time
    • invalid limits are rejected
  • New launch test: the speed limits reach metrics_node as doubles, with defaults and whole-number overrides, in both perception branches, through node startup.
  • New scorer test: course_reference finds the world from a foreign path, returns nothing for rows that logged zeros, and the course reference wins over a baseline.
  • red-mesa: colcon test for lhr_metrics and lhr_gazebo ran 365 tests with 0 failures. flake8 and pep257 are clean, and mkdocs build --strict passes.
  • red-mesa gate runs: autocross seed 1 twice and rules-compliant seed 1 all pass every check at 0.99 times the reference (33.91 s and 58.11 s). Acceleration passes. The 11/4 fusion campaign rescored with this scorer gives 18 of 18, the same as before, through the baseline fallback.
  • Every lap recorded on red-mesa through 11/8:
Stack Courses Lap / reference
No-shortcuts, Gazebo plant Autocross seeds 1 to 5, rules-compliant seeds 1 to 3 0.99 to 1.00
No-shortcuts, FMU plant Autocross seed 1 1.04
Truth stack Autocross seeds 1 to 5 1.10 to 1.19
Truth stack Rules-compliant seed 1 1.70

The car can beat the reference slightly because pure pursuit cuts inside the centerline. 1.5 leaves room for honest variation and still catches a car that crawls or stalls.

Closes #232

@GrayM8
GrayM8 requested a review from a team as a code owner November 8, 2026 17:19
@coderabbitai

coderabbitai Bot commented Nov 8, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Ignore keyword(s) in the title.

⛔ Ignored keywords: 1

  • Autonomous:

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@lhre-ultron

lhre-ultron Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Safe to merge. I found no bugs in the reference-lap maths, the scorer wiring or the launch forwarding. The comment from the earlier push is outdated and I'm dropping it.

What I checked

Checked:

  • I read the whole diff and the changed files at this push, plus ReferenceTrack and world_geometry on the PR branch.
  • Lap maths: the sampling, the three-point curvature, the forward and backward passes with a flying finish, and the trapezoid time sum.
  • Matching pursuit_node: the v_min floor in reference_lap matches how pursuit_node clamps speed. The max_accel and max_decel parameter names also match the node's.
  • Launch: the six limits reach metrics_node as doubles through _float.
  • Scorer: the course reference wins over a baseline, and old rows with zero limits fall back to the baseline. A bad world is caught and recorded as reference lap: … without crashing the score.
  • Edge cases: I looked for a division by zero or a wrap-around in the lap sum and found none, and the validation covers v_min of 0.

Didn't check:

  • I did not run the tests, ROS, Gazebo or a real gate campaign.
  • The pinned /repos/lhre predates this PR, so I did not confirm that metrics_node logs the six limits and the lap_gate_* columns to metrics.csv. I relied on the PR's own tests for that.

Assumed:

  • lhr_metrics is importable wherever score_gate.py runs, since it already needs rclpy.
  • The red-mesa ratios in the PR description are accurate.

Ultron reviewed 5924b93 · it re-reviews on new pushes

@lhre-ultron lhre-ultron Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ultron's review of beb9f05, line by line.

Comment on lines +198 to +199
except (OSError, ValueError) as error: # a bad world fails lap_time, not the score
ref, row['invalid_reason'] = None, f'reference lap: {error}'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Minor | Reliability

Catch XML parse errors from the reference world, and keep the lap's own invalid reason

If the .sdf found by course_reference is malformed or truncated, world_geometry raises ET.ParseError. That is a SyntaxError subclass, not ValueError or OSError, so it is not caught here. The whole score_gate.py run aborts and no results.csv or summary.md is written for any run, which contradicts the comment on this handler. I confirmed ParseError's MRO is ParseError, SyntaxError, Exception. The handler also overwrites invalid_reason, which held the metrics node's lap_invalid_reason, so that reason is lost from the CSV.

Suggested change
except (OSError, ValueError) as error: # a bad world fails lap_time, not the score
ref, row['invalid_reason'] = None, f'reference lap: {error}'
except (OSError, ValueError, SyntaxError) as error: # a bad world fails lap_time, not the score
ref = None
row['invalid_reason'] = '; '.join(
x for x in (row['invalid_reason'], f'reference lap: {error}') if x)
🤖 Prompt for AI agents
In autonomy/testing/tools/gate/score_gate.py around lines 198-199:
Add SyntaxError (ET.ParseError's base) to the except tuple around course_reference so a malformed world file fails only that run's lap_time. Append the 'reference lap: ...' message to the existing row['invalid_reason'] instead of replacing it.

@GrayM8
GrayM8 merged commit d1cd73e into main Nov 8, 2026
12 checks passed
@GrayM8
GrayM8 deleted the gray/reference-lap branch November 8, 2026 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Autonomous: Score lap time against a reference lap computed from the course

1 participant