Repository navigation
Autonomous: Add fixed extreme autocross scenarios - #218
tanushchauhan wants to merge 17 commits into
Conversation
|
Important Review skippedIgnore keyword(s) in the title. ⛔ Ignored keywords: 1
Please check the settings in the CodeRabbit UI or the
|
GrayM8
left a comment
There was a problem hiding this comment.
Requesting changes. Static review of 2edc1e7 (plus regenerating all seven worlds and a plan-only run of the merged runner). The courses are the right ones: 7 of 7 match the Sim v2 criteria in order, all pass check_envelope (laps 201.6-500 m, longest straight 80 m or less, clear width at least 3.0 m), they are deterministic, and six of seven regenerate byte-identically. The staleness test does fail on a stale world. Default behavior is unchanged.
Blocking
- Semantic conflict with #244: the merged runner silently drops runs. The extreme-scenario line in
plan()(run_gate.sh:153) still uses$sensors, which #244 removed. Git mergesplan()without a conflict, but on current main it dies at the first extreme row withsensors: unbound variable(set -u). Sinceplanruns in subshells,expected.txtand the run loop are cut short there, so the extreme, acceleration and skidpad rows vanish and the scorer cannot flag them. Fix: use$STACKfor gate rows and$truthfor baselines (safety:=truenow lives instack.args). The PR's own plan test catches this once it runs after the rebase. - Cones are too dense, and some overlap (
extreme_autocross.py:39,55,123). Cones are offset from a densely sampled centerline, so same-side spacing has a median of 0.77-0.94 m against about 1.96 m onrules_autocross, and the inside of every 4.5 m hairpin has cones 0.25 m apart. Inslalom_chicane20 adjacent pairs, and in500m24, are closer than the 0.228 m cone base, so those cones physically overlap. Dense inner edges read as walls, which makes side classification and pairing easier exactly where these scenarios are meant to stress them, and the Mid-360 caster sees 2-2.5x more cones (500mhas 1,080), which adds CPU load on red-mesa. Fix: keep the dense centerline for geometry but place cones by boundary arc length the way_sample_boundariesdoes, and add a minimum-spacing test.
Should fix
3. Spawn. _compute_spawn (generate_world.py:279-299) judges straightness over a fixed number of points and the uneven sampling fools it: slalom_chicane spawns at the slalom's midpoint turned 34°, so half the slalom is driven from a standstill, and hairpin spawns 28.75 m into its 80 m straight. An explicit spawn for these courses, pinned by a test, would fix it.
4. The new test can run the whole campaign. test_extreme_courses.py:81 cuts run_gate.sh at its marker without the "marker exists" assert #272 added, so a renamed marker would run the campaign, pkill -f 'gz sim' included, on whatever machine runs the tests. Reuse test_gate_runner.runner_prefix() and check that stack.args is a subset of each row's arguments instead of the three hard-coded strings at line 97.
5. Repeats. The extreme rows reuse REPEATS (3, so 21 runs); the criteria say 1 each. An EXTREME_REPEATS=1 default would match.
6. PR body and docs. The body should say "Closes #212" and needs How it's tested / Checklist. worlds/README.md:79-88 hard-codes a test count (568, while the body says 649) and a smoke-run anecdote, which belong in the PR body. Lines 5 and 7 ("synthetic courses, not competition recordings" and the _seed<seed> naming) become false once #223 adds amz_fsg.sdf/amz_fsi.sdf there.
Minor: 12 decimals is not stable across platforms (extreme_decreasing differs in 2 lines when regenerated here, including -0.000000000000); 6-9 decimals is plenty. The staleness test also skips the spawn pose.
Merge order: this conflicts with #223 in all 7 files they share (generate_world.py, score_gate.py, the plan() loop, three READMEs, mkdocs nav). Rebasing after #223 is about an hour of mostly mechanical work; one shared loop for both sets of fixed courses would be cleaner than two.
GrayM8
left a comment
There was a problem hiding this comment.
Requesting changes, two small ones. Re-review of dea7f99 on top of #223: static, plus all seven courses on hil.
On hil, EXTREMES=all at the 0.3 s timeout: 5 of 7 pass (narrow, hairpin, decreasing, slalom_chicane and the 500 m lap: 0 cones, max CTE 0.33 to 0.75 m, EKF error up to 0.17 m on the 500 m lap). doubleback laps (0.98 of the reference) but a STEER_TRACKING trip with no fault injected ends it in EMERGENCY, failing the stop and health checks. combined hits a cone, never gets credited a lap and keeps driving; I stopped it at 561 s of sim. Both are the stack meeting what these courses are built to stress (the steering check through tight reversals, lap progress next to a return lane), not problems with this PR, and we'll track them separately.
Both old blockers are fixed:
- Runner: one loop plans the extreme and imported courses with
$STACK/$truththroughchecked_plan, and the test plans EXTREMES, REAL_TRACKS, acceleration and skidpad together. - Cone spacing, measured from the committed worlds: same-side median gap is now 1.84 to 1.91 m on every course (was 0.77 to 0.94 m), cone counts roughly halved, and no overlapping cones (hairpin, doubleback and 500m had 24 each). The smallest gaps, 0.73 to 0.97 m, are the inside of the 4.5 m turns, the same as
rules_autocrossproduces. - Also fixed: spawns at the first pair with the footprint checked on all seven, the runner tests use
runner_prefix()and checkgate_stack(),EXTREME_REPEATSdefaults to 1 and is validated, decimals are deterministic (I regenerated all seven: 0 of 4,352 coordinates differ), and the template sections are in the body.
Please fix
- c1bf194 loosens the width test (
test_extreme_autocross.py:30-36). The reason is fair, and it's in its own commit, but the newall(pair - BASE >= minimum)can never fail:check_envelopestarts its width at the narrowest pair minus the base (rules_autocross.py:90). The upper bound is gone too, so nothing checks the course stays narrow away from its narrowest point. The chord reserve also applies on straights, sonarrowis a uniform 3.017 m andcombined3.049 m, just over the old 3.01. Restore an honest bound (for example pair minus base at most 3.02 m on narrow, 3.05 on combined), or apply the reserve only on turns so 3.01 holds. ros2/README.md:294and:619, the reference lists oftrack_stylevalues and pre-generated worlds, don't mention the seven extreme styles.
Minor
- The overlap test catches overlaps but not density: the old 0.72 m
extreme_narrowwould pass it. A median-gap floor around 1.5 m, or a cone count, would pin the fix. worlds/README.md:31calls the chord reserve curvature-based; it's a fixed half-angle per course.testing/2026-11-16-extreme-reviewis a log of review fixes rather than a campaign record, and the mkdocs nav has a stray blank line after Gazebo Worlds with that entry out of date order.
|
This PR conflicts with its base branch, so GitHub isn't running CI on it and |
|
Safe to merge. The imported-course seed bug I flagged before is fixed, and I found nothing new that breaks. The earlier 🟠 comment on What I checkedChecked: I read the whole diff apart from the seven generated
Didn't check: I couldn't run Assumed: The stubbed Ultron reviewed a5379ae · it re-reviews on new pushes |
| for r in $(seq 1 "$repeats"); do | ||
| echo "${style}_s1_r$r track_style:=$style seed:=1 $STACK $acknowledge" | ||
| done |
There was a problem hiding this comment.
🟠 Major | Reliability
Give each extreme (and imported) gated run its own imu_seed, like every other gated row
The shared fixed-course loop prints ${style}_s1_rN ... $STACK $acknowledge with no imu_seed, so every extreme run launches with the default imu_seed:=1 (gazebo_demo.launch.py → gz sim --seed 1). That is the same draw autocross_s1_r1 uses. I ran plan() through checked_plan with EXTREMES="extreme_hairpin extreme_narrow" EXTREME_REPEATS=3. The autocross row got imu_seed:=1 and acceleration got imu_seed:=2, but all six extreme rows had none. So EXTREME_REPEATS>1 gives identical runs, not the independent repeats the criteria count on ("a stack that passes once and fails the next time has not passed"). With EXTRA="imu_noise:=true", all seven courses also share one noise sequence. args.txt and metrics record the default 1 rather than a per-run seed, against the gate README ("Each gated run increments it") and mvp-pass-criteria.md ("assigns a distinct imu_seed to each gated run"). Main's imported-layout loop already had this gap; this PR sends seven more courses through it. Truth baselines stay unseeded, as in the regular loop.
💚 Proposed fix
| for r in $(seq 1 "$repeats"); do | |
| echo "${style}_s1_r$r track_style:=$style seed:=1 $STACK $acknowledge" | |
| done | |
| for r in $(seq 1 "$repeats"); do | |
| [ "$imu_seed" -le 4294967295 ] || return 2 | |
| echo "${style}_s1_r$r track_style:=$style seed:=1 $STACK $acknowledge imu_seed:=$imu_seed" | |
| imu_seed=$((imu_seed + 1)) | |
| done |
Also add an assertion to test_extreme_and_imported_rows_preserve_later_events that every non-baseline row has an imu_seed and that no two rows share one.
🤖 Prompt for AI agents
In autonomy/testing/tools/gate/run_gate.sh around lines 195-197:
The extreme/imported gated rows omit imu_seed, so they all reuse Gazebo seed 1 and repeats are identical. Inside the repeat loop, add the same bound check, append imu_seed:=$imu_seed to the echoed row and increment imu_seed, as the autocross/acceleration/skidpad loops do. Then extend test_extreme_and_imported_rows_preserve_later_events in lhr_gazebo/test/test_extreme_courses.py to assert every non-baseline row carries a unique imu_seed.
What
Add seven fixed, rules-checked extreme autocross scenarios and address the review findings on cone spacing, staging and campaign planning. Closes #212.
The branch targets
mainand includes the merged competition layouts from #223. Extreme and imported layouts share one planning loop with the standard gate stack.EXTREME_REPEATSdefaults to one independently of the other repeat counts. Each scored run gets a distinct deterministic Gazebo seed. Malformed repeat counts or a plan that exhausts the seed range fail before launch.Separate dense construction geometry from physical cone placement. Sample the longer boundary arc at stations below 2 m, preserve piece/feature endpoints, and interpolate circular sections without cutting inside the exact hairpin radius. All seven saved worlds have adjacent boundary gaps above 0.5 m and no overlapping adjacent cones. The 500 m fixture targets the sampled/scored polyline and uses 548 physical cones instead of 1,080.
Stage before the entire first feature, including the full hairpin and slalom approaches. Regenerate all worlds at ten-decimal precision with normalized negative zero; whole-file freshness now covers spawn poses. Nine decimals missed the existing exact-radius tolerance on the combined fixture, so that tolerance was retained and precision chosen accordingly.
How it's tested
colcon test --packages-select lhr_gazebo lhr_metrics lhr_trackgenandcolcon test-result --verbose: 771 tests, zero failures/errors/skips, including ROS lint. The focused extreme geometry/planning tests report 72 passes.runner_prefix()helper, so a missing marker cannot start a campaign or run cleanup.git diff --checkpass.Isolated test correction
Commit
c1bf194fcorrects the old width test separately from the generator changes. It applied a 3.01 m maximum to cone-pair separation, although the fixture's required 3 m minimum is road clearance between boundary segments. More widely spaced turn cones need chord reserve in their pair separation. The replacement pins the actual minimum road clearance to 3.00–3.01 m and caps the paired cone clearance at 3.02 m on narrow and 3.05 m on combined. Exact turn/radius/length checks were retained; no skips or lint suppressions were added.HIL driving evidence
All seven fixtures ran on HIL with the Gazebo plant, stereo fusion and the default 0.3 s LiDAR timeout at
dea7f991. The first campaign passed 4 of 7 and Gray’s campaign passed 5 of 7. The decreasing-radius case passed on repeat. Doubleback passed two later instrumented repeats, and the combined course reproduced the scored cone contact. Those driving failures are follow-ups for the final stack.The subsequent seed fix passed CI and the 631-test local Gazebo suite. The full Sim v2 gate with the final FMU, modeled sensors and braking is pending.
Checklist
autonomy/testing/.