Skip to content

Autonomous: Add fixed extreme autocross scenarios - #218

Open
tanushchauhan wants to merge 17 commits into
mainfrom
tanush/extreme-autocross
Open

tanushchauhan wants to merge 17 commits into
mainfrom
tanush/extreme-autocross

Conversation

@tanushchauhan

@tanushchauhan tanushchauhan commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

What

Add seven fixed, rules-checked extreme autocross scenarios and address the review findings on cone spacing, staging and campaign planning. Closes #212.

The branch targets main and includes the merged competition layouts from #223. Extreme and imported layouts share one planning loop with the standard gate stack. EXTREME_REPEATS defaults to one independently of the other repeat counts. Each scored run gets a distinct deterministic Gazebo seed. Malformed repeat counts or a plan that exhausts the seed range fail before launch.

Separate dense construction geometry from physical cone placement. Sample the longer boundary arc at stations below 2 m, preserve piece/feature endpoints, and interpolate circular sections without cutting inside the exact hairpin radius. All seven saved worlds have adjacent boundary gaps above 0.5 m and no overlapping adjacent cones. The 500 m fixture targets the sampled/scored polyline and uses 548 physical cones instead of 1,080.

Stage before the entire first feature, including the full hairpin and slalom approaches. Regenerate all worlds at ten-decimal precision with normalized negative zero; whole-file freshness now covers spawn poses. Nine decimals missed the existing exact-radius tolerance on the combined fixture, so that tolerance was retained and precision chosen accordingly.

How it's tested

  • All 16 ROS packages build in an isolated Ubuntu 24.04/Jazzy development container with read-only source and separate outputs.
  • colcon test --packages-select lhr_gazebo lhr_metrics lhr_trackgen and colcon test-result --verbose: 771 tests, zero failures/errors/skips, including ROS lint. The focused extreme geometry/planning tests report 72 passes.
  • Regressions reproduce the old dense-cone and misplaced-spawn failures. Coverage includes all seven saved worlds, full-file regeneration, minimum/maximum physical gaps including closure, complete canonical spawn-footprint clearance, exact 80 m approach and 4.5 m radius checks, the 500 m scored length, both launch branches, independent repeat counts and complete mixed imported/extreme plans.
  • Test code uses the guarded runner_prefix() helper, so a missing marker cannot start a campaign or run cleanup.
  • Repository Ruff, strict MkDocs and git diff --check pass.
  • Recorded driving results: HIL extreme-autocross campaign.

Isolated test correction

Commit c1bf194f corrects the old width test separately from the generator changes. It applied a 3.01 m maximum to cone-pair separation, although the fixture's required 3 m minimum is road clearance between boundary segments. More widely spaced turn cones need chord reserve in their pair separation. The replacement pins the actual minimum road clearance to 3.00–3.01 m and caps the paired cone clearance at 3.02 m on narrow and 3.05 m on combined. Exact turn/radius/length checks were retained; no skips or lint suppressions were added.

HIL driving evidence

All seven fixtures ran on HIL with the Gazebo plant, stereo fusion and the default 0.3 s LiDAR timeout at dea7f991. The first campaign passed 4 of 7 and Gray’s campaign passed 5 of 7. The decreasing-radius case passed on repeat. Doubleback passed two later instrumented repeats, and the combined course reproduced the scored cone contact. Those driving failures are follow-ups for the final stack.

The subsequent seed fix passed CI and the 631-test local Gazebo suite. The full Sim v2 gate with the final FMU, modeled sensors and braking is pending.

Checklist

  • Behavior changes include regression tests that reproduce the old failures.
  • No test deleted or skipped, no tolerance loosened, no lint suppression added. The separate correction to the wrong width measurement is explained above.
  • Adjacent docs updated; run-specific evidence moved to autonomy/testing/.

@coderabbitai

coderabbitai Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Ignore keyword(s) in the title.

⛔ Ignored keywords: 1

  • Autonomous:

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
Pre-merge checks: Inconclusive 2

Failed Failed checks: 2 inconclusive

Check name Status Explanation Resolution
Title check Pending This assessment has not completed. The check will be retried on the next pre-merge check run.
Description check Pending This assessment has not completed. The check will be retried on the next pre-merge check run.
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@GrayM8 GrayM8 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes. Static review of 2edc1e7 (plus regenerating all seven worlds and a plan-only run of the merged runner). The courses are the right ones: 7 of 7 match the Sim v2 criteria in order, all pass check_envelope (laps 201.6-500 m, longest straight 80 m or less, clear width at least 3.0 m), they are deterministic, and six of seven regenerate byte-identically. The staleness test does fail on a stale world. Default behavior is unchanged.

Blocking

  1. Semantic conflict with #244: the merged runner silently drops runs. The extreme-scenario line in plan() (run_gate.sh:153) still uses $sensors, which #244 removed. Git merges plan() without a conflict, but on current main it dies at the first extreme row with sensors: unbound variable (set -u). Since plan runs in subshells, expected.txt and the run loop are cut short there, so the extreme, acceleration and skidpad rows vanish and the scorer cannot flag them. Fix: use $STACK for gate rows and $truth for baselines (safety:=true now lives in stack.args). The PR's own plan test catches this once it runs after the rebase.
  2. Cones are too dense, and some overlap (extreme_autocross.py:39,55,123). Cones are offset from a densely sampled centerline, so same-side spacing has a median of 0.77-0.94 m against about 1.96 m on rules_autocross, and the inside of every 4.5 m hairpin has cones 0.25 m apart. In slalom_chicane 20 adjacent pairs, and in 500m 24, are closer than the 0.228 m cone base, so those cones physically overlap. Dense inner edges read as walls, which makes side classification and pairing easier exactly where these scenarios are meant to stress them, and the Mid-360 caster sees 2-2.5x more cones (500m has 1,080), which adds CPU load on red-mesa. Fix: keep the dense centerline for geometry but place cones by boundary arc length the way _sample_boundaries does, and add a minimum-spacing test.

Should fix
3. Spawn. _compute_spawn (generate_world.py:279-299) judges straightness over a fixed number of points and the uneven sampling fools it: slalom_chicane spawns at the slalom's midpoint turned 34°, so half the slalom is driven from a standstill, and hairpin spawns 28.75 m into its 80 m straight. An explicit spawn for these courses, pinned by a test, would fix it.
4. The new test can run the whole campaign. test_extreme_courses.py:81 cuts run_gate.sh at its marker without the "marker exists" assert #272 added, so a renamed marker would run the campaign, pkill -f 'gz sim' included, on whatever machine runs the tests. Reuse test_gate_runner.runner_prefix() and check that stack.args is a subset of each row's arguments instead of the three hard-coded strings at line 97.
5. Repeats. The extreme rows reuse REPEATS (3, so 21 runs); the criteria say 1 each. An EXTREME_REPEATS=1 default would match.
6. PR body and docs. The body should say "Closes #212" and needs How it's tested / Checklist. worlds/README.md:79-88 hard-codes a test count (568, while the body says 649) and a smoke-run anecdote, which belong in the PR body. Lines 5 and 7 ("synthetic courses, not competition recordings" and the _seed<seed> naming) become false once #223 adds amz_fsg.sdf/amz_fsi.sdf there.

Minor: 12 decimals is not stable across platforms (extreme_decreasing differs in 2 lines when regenerated here, including -0.000000000000); 6-9 decimals is plenty. The staleness test also skips the spawn pose.

Merge order: this conflicts with #223 in all 7 files they share (generate_world.py, score_gate.py, the plan() loop, three READMEs, mkdocs nav). Rebasing after #223 is about an hour of mostly mechanical work; one shared loop for both sets of fixed courses would be cleaner than two.

@tanushchauhan
tanushchauhan changed the base branch from main to tanush/competition-layouts November 16, 2026 23:57

@GrayM8 GrayM8 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes, two small ones. Re-review of dea7f99 on top of #223: static, plus all seven courses on hil.

On hil, EXTREMES=all at the 0.3 s timeout: 5 of 7 pass (narrow, hairpin, decreasing, slalom_chicane and the 500 m lap: 0 cones, max CTE 0.33 to 0.75 m, EKF error up to 0.17 m on the 500 m lap). doubleback laps (0.98 of the reference) but a STEER_TRACKING trip with no fault injected ends it in EMERGENCY, failing the stop and health checks. combined hits a cone, never gets credited a lap and keeps driving; I stopped it at 561 s of sim. Both are the stack meeting what these courses are built to stress (the steering check through tight reversals, lap progress next to a return lane), not problems with this PR, and we'll track them separately.

Both old blockers are fixed:

  • Runner: one loop plans the extreme and imported courses with $STACK/$truth through checked_plan, and the test plans EXTREMES, REAL_TRACKS, acceleration and skidpad together.
  • Cone spacing, measured from the committed worlds: same-side median gap is now 1.84 to 1.91 m on every course (was 0.77 to 0.94 m), cone counts roughly halved, and no overlapping cones (hairpin, doubleback and 500m had 24 each). The smallest gaps, 0.73 to 0.97 m, are the inside of the 4.5 m turns, the same as rules_autocross produces.
  • Also fixed: spawns at the first pair with the footprint checked on all seven, the runner tests use runner_prefix() and check gate_stack(), EXTREME_REPEATS defaults to 1 and is validated, decimals are deterministic (I regenerated all seven: 0 of 4,352 coordinates differ), and the template sections are in the body.

Please fix

  1. c1bf194 loosens the width test (test_extreme_autocross.py:30-36). The reason is fair, and it's in its own commit, but the new all(pair - BASE >= minimum) can never fail: check_envelope starts its width at the narrowest pair minus the base (rules_autocross.py:90). The upper bound is gone too, so nothing checks the course stays narrow away from its narrowest point. The chord reserve also applies on straights, so narrow is a uniform 3.017 m and combined 3.049 m, just over the old 3.01. Restore an honest bound (for example pair minus base at most 3.02 m on narrow, 3.05 on combined), or apply the reserve only on turns so 3.01 holds.
  2. ros2/README.md:294 and :619, the reference lists of track_style values and pre-generated worlds, don't mention the seven extreme styles.

Minor

  • The overlap test catches overlaps but not density: the old 0.72 m extreme_narrow would pass it. A median-gap floor around 1.5 m, or a cone count, would pin the fix.
  • worlds/README.md:31 calls the chord reserve curvature-based; it's a fixed half-angle per course.
  • testing/2026-11-16-extreme-review is a log of review fixes rather than a campaign record, and the mkdocs nav has a stray blank line after Gazebo Worlds with that entry out of date order.

Base automatically changed from tanush/competition-layouts to main November 20, 2026 05:25
@github-actions github-actions Bot added the merge-conflict Conflicts with its base, so CI is not running label Nov 20, 2026
@github-actions

Copy link
Copy Markdown

This PR conflicts with its base branch, so GitHub isn't running CI on it and All checks passed will never report. Merge or rebase the base branch in to get checks back; this label comes off on its own once the conflict is gone.

@tanushchauhan
tanushchauhan marked this pull request as ready for review November 20, 2026 05:28
@tanushchauhan
tanushchauhan requested a review from a team as a code owner November 20, 2026 05:28
@lhre-ultron

lhre-ultron Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Safe to merge. The imported-course seed bug I flagged before is fixed, and I found nothing new that breaks.

The earlier 🟠 comment on autonomy/testing/tools/gate/run_gate.sh:199 is fixed, so I've dropped it. Imported runs (amz_fsg/amz_fsi) and extreme gated runs now each get their own imu_seed. They share one sequence with regular autocross, acceleration and skidpad, and the 32-bit exhaustion check covers them.

What I checked

Checked: I read the whole diff apart from the seven generated .sdf bodies, which I checked by regenerating them. I also read extreme_autocross.py and its callers: publish_cones.GENERATORS/pairing_width_m, generate_world.py, gazebo_demo.launch.py _find_world/_launch_setup, mvs_demo.launch.py, score_gate.py (RUN_ID, course_reference, judge) and stack.sh checked_plan.

  • lhr_trackgen/test/test_extreme_autocross.py: 23/23 pass.
  • I regenerated all seven worlds through generate_world.py's main() with the ROS message imports stubbed out. Every one is byte-identical to the committed SDF, spawn pose included.
  • Each committed world passes check_envelope:
    • Laps run from 201.460 m to 500.000 m.
    • The hairpin and combined courses have an 80.000 m straight and a 4.500 m minimum radius.
    • The narrow and combined courses have 3.004 m and 3.007 m minimum clear width.
    • The cones in the SDFs are within 7e-11 m of generate_extreme_track.
  • Adjacent-cone gaps on each side, closure included, run from 0.729 m to 1.947 m.
  • The committed worlds haven't changed since dea7f991, so the HIL README's results describe the current geometry.
  • I ran run_gate.sh with PLAN_ONLY=1 for three cases. Each did what it should:
    • EXTREMES=all, with a regular seed, BASELINE=1 and acceleration: gives seven baseline/gated pairs and IMU seeds 1–9 in order.
    • EXTREME_REPEATS=08: gives 56 rows.
    • EXTREME_REPEATS=0 with nothing else planned: refused with rc 2.

Didn't check: I couldn't run colcon build/colcon test for lhr_gazebo (launch parameters, ConePublisher/TrackBuilder startup, CourseEnvelope.pose_check, the scorer in the full ROS environment). There is no ROS Jazzy in my sandbox, so I traced that code path by reading it instead. I ran no Gazebo driving.

Assumed: The stubbed rclpy/geometry_msgs/visualization_msgs modules don't affect geometry or SDF output, since generate_world.py uses them only for publishing types. The PR's base is current main (e2b65f6, with #223 merged).

Ultron reviewed a5379ae · it re-reviews on new pushes

@lhre-ultron lhre-ultron Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ultron's review of 93f23ea, line by line.

Comment on lines +195 to 197
for r in $(seq 1 "$repeats"); do
echo "${style}_s1_r$r track_style:=$style seed:=1 $STACK $acknowledge"
done

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Major | Reliability

Give each extreme (and imported) gated run its own imu_seed, like every other gated row

The shared fixed-course loop prints ${style}_s1_rN ... $STACK $acknowledge with no imu_seed, so every extreme run launches with the default imu_seed:=1 (gazebo_demo.launch.py → gz sim --seed 1). That is the same draw autocross_s1_r1 uses. I ran plan() through checked_plan with EXTREMES="extreme_hairpin extreme_narrow" EXTREME_REPEATS=3. The autocross row got imu_seed:=1 and acceleration got imu_seed:=2, but all six extreme rows had none. So EXTREME_REPEATS>1 gives identical runs, not the independent repeats the criteria count on ("a stack that passes once and fails the next time has not passed"). With EXTRA="imu_noise:=true", all seven courses also share one noise sequence. args.txt and metrics record the default 1 rather than a per-run seed, against the gate README ("Each gated run increments it") and mvp-pass-criteria.md ("assigns a distinct imu_seed to each gated run"). Main's imported-layout loop already had this gap; this PR sends seven more courses through it. Truth baselines stay unseeded, as in the regular loop.

💚 Proposed fix
Suggested change
for r in $(seq 1 "$repeats"); do
echo "${style}_s1_r$r track_style:=$style seed:=1 $STACK $acknowledge"
done
for r in $(seq 1 "$repeats"); do
[ "$imu_seed" -le 4294967295 ] || return 2
echo "${style}_s1_r$r track_style:=$style seed:=1 $STACK $acknowledge imu_seed:=$imu_seed"
imu_seed=$((imu_seed + 1))
done

Also add an assertion to test_extreme_and_imported_rows_preserve_later_events that every non-baseline row has an imu_seed and that no two rows share one.

🤖 Prompt for AI agents
In autonomy/testing/tools/gate/run_gate.sh around lines 195-197:
The extreme/imported gated rows omit imu_seed, so they all reuse Gazebo seed 1 and repeats are identical. Inside the repeat loop, add the same bound check, append imu_seed:=$imu_seed to the echoed row and increment imu_seed, as the autocross/acceleration/skidpad loops do. Then extend test_extreme_and_imported_rows_preserve_later_events in lhr_gazebo/test/test_extreme_courses.py to assert every non-baseline row carries a unique imu_seed.

@github-actions github-actions Bot removed the merge-conflict Conflicts with its base, so CI is not running label Nov 20, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Autonomous: Extreme but legal autocross scenarios

2 participants