Skip to content

[FEATURE] Add Raycaster sensor (Lidar and DepthCamera) #1772

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 28 commits into from
Oct 8, 2025
Merged

[FEATURE] Add Raycaster sensor (Lidar and DepthCamera) #1772

merged 28 commits into from
Oct 8, 2025

Conversation

Milotrince
Copy link
Contributor

@Milotrince Milotrince commented Sep 25, 2025 •
edited by duburcqa
Loading

Description

  • Add Raycaster sensor, which is used for Lidar and DepthCamera
  • deprecate 'Scene.build' optional argument 'compile_kernels'
  • Fix offscreen interactive viewer.
  • Fix viewer deadlock in case of missing display.
  • Expose option to force refreshing rasterizer context.
  • Force headless window if supported and '--vis' is not specified in unit tests.

Related Issue

Resolves #1111 #419 #45

Motivation and Context

Lidar has been a long awaited feature!

How Has This Been / Can This Be Tested?

examples/sensors/lidar_teleop.py
tests/test_sensors.py

Screenshots (if appropriate):

python examples/sensors/lidar_teleop.py --pattern spherical
Screenshot 2025-09-22 at 1 34 49 PM

python examples/sensors/lidar_teleop.py --pattern depth

depth.mov

python examples/sensors/lidar_teleop.py --pattern grid
Screenshot 2025-09-22 at 1 36 11 PM

Checklist:

  • I read the CONTRIBUTING document.
  • I followed the Submitting Code Changes section of CONTRIBUTING document.
  • I tagged the title correctly (including BUG FIX/FEATURE/MISC/BREAKING)
  • I updated the documentation accordingly or no change is needed.
  • I tested my changes and added instructions on how to test it for reviewers.

@Milotrince Milotrince marked this pull request as draft September 25, 2025 01:48
@Milotrince
Copy link
Contributor Author

Note: Review after #1770 is merged

duburcqa
duburcqa reviewed Sep 30, 2025
View reviewed changes
duburcqa
duburcqa reviewed Sep 30, 2025
View reviewed changes
duburcqa
duburcqa reviewed Sep 30, 2025
View reviewed changes
duburcqa
duburcqa reviewed Sep 30, 2025
View reviewed changes
duburcqa
duburcqa reviewed Sep 30, 2025
View reviewed changes
@duburcqa duburcqa marked this pull request as ready for review September 30, 2025 15:15
@Milotrince Milotrince mentioned this pull request Oct 1, 2025
Merged
5 tasks
@Milotrince Milotrince marked this pull request as draft October 1, 2025 07:20
@Milotrince Milotrince marked this pull request as ready for review October 2, 2025 06:24
Milotrince
Milotrince commented Oct 2, 2025
View reviewed changes
Milotrince
Milotrince commented Oct 2, 2025
View reviewed changes
duburcqa
duburcqa reviewed Oct 2, 2025
View reviewed changes
@Milotrince Milotrince force-pushed the raycaster_sensor branch 3 times, most recently from 6aacca3 to a96a405 Compare October 4, 2025 06:58
Milotrince
Milotrince commented Oct 4, 2025
View reviewed changes
@Milotrince Milotrince force-pushed the raycaster_sensor branch 2 times, most recently from 5c5a5d1 to e274cfa Compare October 6, 2025 19:58
@duburcqa duburcqa enabled auto-merge (squash) October 6, 2025 21:16
auto-merge was automatically disabled October 6, 2025 21:34

Head branch was pushed to by a user without write access

@duburcqa duburcqa force-pushed the raycaster_sensor branch 2 times, most recently from 79a9bc3 to 3db04b9 Compare October 7, 2025 09:38
duburcqa
duburcqa previously approved these changes Oct 7, 2025
View reviewed changes
Milotrince and others added 27 commits October 8, 2025 01:04
@Milotrince @duburcqa
deprecate compile_kernels, move sensor build
3d14673
@Milotrince @duburcqa
update docstring
49d731d
@Milotrince @duburcqa
update example
6e03d1d
@Milotrince @duburcqa
address review 1 part 1
c43fc0c
@Milotrince @duburcqa
address review 1 part 2
d2c20fb
@Milotrince @duburcqa
address review 1 part 3: simplify raycastpattern
9d28b30
@Milotrince @duburcqa
restore is_free/is_fixed change
e35df06
@Milotrince @duburcqa
cleanup pattern
4ce6b70
@Milotrince @duburcqa
update post init sensor option validation
608e586
@Milotrince @duburcqa
update raycaster test, rename raycaster data fields
d308fd8
@Milotrince @duburcqa
fix unintended stuffs
9983f43
@Milotrince @duburcqa
rebase updates, add sensor draw debug test
2d5194a
@Milotrince @duburcqa
sensor draw debug test fixes
f916f58
@Milotrince @duburcqa
remove only_cast_fixed option
2128c71
@Milotrince @duburcqa
minor fixes
76fb95e
@Milotrince @duburcqa
rm examples from ignore list
565fc8f
@duburcqa
cleanup rendering unit tests.
f7f58a0
@duburcqa
Fix offscreen interactive viewer.
d4ec32b
@duburcqa
Fix viewer deadlock in case of missing display.
46ae857
@duburcqa
Fix example unit tests.
fa36ee9
@duburcqa
Increase tolerance unit test to avoid flakiness.
e8224d6
@duburcqa
Add hugging face Xet deps.
dbba03e
@duburcqa
Cleanup sensor draw debug unit test.
d37c5da
@duburcqa
Fix draw debug sensor update for raycaster.
1956f04
@duburcqa
Expose option to force refreshing rasterizer context.
cc25f46
@duburcqa
Force headless window if supported and '--vis' is not specified in un…
b4c20b5 
…it tests.
@duburcqa
Fix rasterizer rendering discrepency across platforms.
bf4abf7
@duburcqa duburcqa merged commit 19b92a2 into Genesis-Embodied-AI:main Oct 8, 2025
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@duburcqa duburcqa duburcqa approved these changes

@YilingQiao YilingQiao Awaiting requested review from YilingQiao YilingQiao is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Feature]: Simple lidar, raycast, or scandots sensor

3 participants

@Milotrince @duburcqa @YilingQiao