Fix acceleration axis and leg tweaks

[9il]

Fixes #1006

This PR fixes the axis for leg tweaks! However, an additional update of thresholds may be required.

@ImUrX, please test it in standard mode with leg tweaks like skating correction.

I don't have slimes for testing.

[Stermere]

This won't affect skating-correction since the magnitude of acceleration is not being changed. The logic looks good as far as I can tell

[9il]

This won't affect skating-correction since the magnitude of acceleration is not being changed.

Not really. Some code uses only the two components, like Vector3(footDif.x, 0f, footDif.z) . It uses x and z components, which correspond to trackers to original (x y) components in this PR and trackers (y z) in the current main branch. z in the tracker coordinates corresponds to the gravity direction. That is what the fix is about.

	private fun correctUnlockedFootTracker(footPosition: Vector3, previousFootPosition: Vector3, previousFootPositionCorrected: Vector3, footVelocity: Vector3, framesUnlocked: Int): Vector3 {
		var newFootPosition = footPosition
		var footDif = footPosition - previousFootPositionCorrected
		footDif = Vector3(footDif.x, 0f, footDif.z)

[Stermere]

This won't affect skating-correction since the magnitude of acceleration is not being changed.

Not really. Some code uses only the two components, like Vector3(footDif.x, 0f, footDif.z) . It uses x and z components, which correspond to trackers to original (x y) components in this PR and trackers (y z) in the current main branch. z in the tracker coordinates corresponds to the gravity direction. That is what the fix is about.

	private fun correctUnlockedFootTracker(footPosition: Vector3, previousFootPosition: Vector3, previousFootPositionCorrected: Vector3, footVelocity: Vector3, framesUnlocked: Int): Vector3 {
		var newFootPosition = footPosition
		var footDif = footPosition - previousFootPositionCorrected
		footDif = Vector3(footDif.x, 0f, footDif.z)

That is a positional correction, the acceleration is not used like that, I just checked. Mocap mode does use the acceleration direction though so this will help that!

[9il]

@ImUrX is anything else required for this PR to be merged?

[ImUrX]

You need to wait for more reviewers

[9il]

I though that two reviwers are enough. Should we ping someone to make a review?

[Erimelowo]

code itself looks good but testing with official SlimeVR trackers, looking at the values on the GUI (not the preview), it still seems like the original way was the right way (Vector3(v.y, v.x, v.z)). Maybe it'd be worth looking in the firmware if the acceleration is transformed.

However, the accel arrows preview and the local -> global conversion for accel are both broken in some way

[9il]

@Stermere id don't know if the fusion does the order right, but, at least, they in the right order during packet formation.

If you like I could take the original AXES_OFFSET and apply it both to rotation and acceleration. So, it will be defined by mostly by software, not a math.

On the other hand, if the bug is in the fustion, than the firmware has to be fixed instead of tuning the server for the firmware. At least, we could define a new version of protocol. The reason is because slime wants to be a kind of opensource standard for VR.

The current acceleratoin offsets is defenetely wrong. Some code may work better now because it was tuned for the current wrong offsets. That could work for now but it can be so for a robust code evolution.

[9il]

rebased

[9il]

@Eirenliel @sctanf Do I need to made another rebase? when it will be merged?

[sctanf]

Front may not be a good descriptor

[9il]

@sctanf Did you post a hardware tracker axis or a slime VR axis?
Because this picture corresponds to hardware tracker axis.
If slime VR axis are the same as hardware axis, then NO conversion should be performed at all.

[9il]

The PR made acceleration conversion based on existing quaternion conversions.
If you treat SlimeVR as hardware axis on the picture, that isn't correct.
We should either to get rid of conversion and use hardware axis in the Server, or merge this PR that fixes acceleration axis.

[sctanf]

@sctanf Did you post a hardware tracker axis or a slime VR axis? Because this picture corresponds to hardware tracker axis. If slime VR axis are the same as hardware axis, then NO conversion should be performed at all.

This should be what a device reports, with axes local to the device

[9il]

This should be what a device reports, with axes local to the device

Yes, that is what is expected on input at this PR

[sctanf]

The axes may need to be rotated along the device before remap

Suggested change

	fun axisOffset(v: Vector3): Vector3 = Vector3(v.x, v.z, -v.y)
	private val rotationOffset = Quaternion(INV_SQRT_TWO, -INV_SQRT_TWO, 0f, 0f)
	fun axisRemap(v: Vector3): Vector3 = Vector3(v.x, v.z, -v.y)
	private val rotationOffset = Quaternion(INV_SQRT_TWO, -INV_SQRT_TWO, 0f, 0f)
	fun axisOffset(v: Vector3): Vector3 = axisRemap(rotationOffset.sandwich(v))

[9il]

axisOffset is an optimized version for rotationOffset.sandwich(v)
It either one or another, they are the same.

BTW, I would recommend to

• store only normalized rotations
• use a raw rotation function, which needs ~15 multiplications (wiki), or at least use two quaternion multiplications. Current sandwich code uses quaternion inversion, which us slow. With current state of Koltin you will not feel the difference. However, in the future, with Project Valhalla, Quaternion can be optimized a lot, so the division will be bottleneck. I am thinking about SlimeVR server running on lightweight XR headsets directly, not PC.

[Eirenliel]

The accel axises are definitely wrong in this PR, front and top are swapped when testing it with current slimevr fw for both BNO and ICM.

[9il]

The accel axises are definitely wrong in this PR, front and top are swapped when testing it with current slimevr fw for both BNO and ICM.

i would expect they are swapped also somewhere else, if the server shows the expected axis structure before this PR

[sctanf]

rotation is corrected for the change in system, but the axes between the two systems are already aligned, the accelerometer data most likely do not need to be remapped

[Eirenliel]

I have done testing with actual trackers with the latest firmware. With the latest firmware, trackers send accelerometer axises the same way as rotations, so they should have the same transformations. See here for the changes you might need to make to your firmware: https://github.com/SlimeVR/SlimeVR-Tracker-ESP/blob/cd97d17d9a9834051b44629c711399218dcc8868/src/sensors/sensor.cpp#L34

This also changes the comment for the transformations to clarify why this transformation is required and what do we even do with it (it's not converting to OpenGL axises).

[Eirenliel]

Seems like stuff is still broken... Debugging it

[Eirenliel]

So it seems that the remap is correct and is also needed. I'm trying to understand WHY though.

[Eirenliel]

i.. se...

[Eirenliel]

@9il @sctanf i started working on it in a separate branch, testing with a gravity vector. FOR SOME REASON, it works when used like this, but I don't understand why we need to apply the axis offset twice...

https://github.com/SlimeVR/SlimeVR-Server/tree/accel-mapping-fixes

[9il]

@Eirenliel I would like to have but it is hard to understand what is going on.

Does firmware uses global coordinate or local coordinate for acceleration?
In global coordinate Z axis always follows the gravity acceleration (looks to the sky).

What transformation represent the rotation quaternion?
How you transform from global to local or vise-verse (it is the same up to conjugation)?

Does Slime expect acceleration vector with substracted gravity or with gravity present?

Suggestion to get rid of chaos:

1. Use hardware axis structure in the Server
2. Use global coordinate for acceleration (Z -> Sky) (it isn't clear what it is now)
3. Get rid of incoming transformations
4. Adjust Server algorithms such us leg tweaks to work with this input data

Less transformations along the way.

[9il]

P.S. thank you for digging in to this rabbit hole

[Eirenliel]

@9il do you do any rotations or transformations in firmware?

[9il]

@9il do you do any rotations or transformations in firmware?

@Eirenliel Our firmware uses the same axis as defined in ICM specification (tracker axis). In this case I would say "No". On the other hand, sensor fusion algorithms like VQF has a few axis they operate during fusion, in this terms, "Yes".

[Eirenliel]

Thanks. And you confirmed that it works with just your changes? I think I know where the issue is in our code...

[9il]

@Eirenliel

I can't confirm our firmware works with SlimeVR, but, it works in our test software. We will test slimeVR later this summer.

One thing I would add is that "front", "right", and "up" aren't very descriptive because they are relative to what one had in their had when they think about a specific tracker.

Please take a look at

credit: https://techarthub.com/a-practical-guide-to-unreal-engines-coordinate-system/

Transformation between Left and Right isn't possible with Quaternion. I think this may be a part of the issue. Right-left conversion should be defined with axis permutation and rotation in the same time.

What we need to define for firmware and software API is:

1. Z up or Y up?. The gravity is positive in default tracker position. My personal preference is Z up like in ICM / tracker / blender.
2. Right hand or Left hand. my personal preference is Right up like in ICM / tracker / blender).
3. Is gravity g value subtracted from the up axis? e.g. should tracker transfer adjusted acceleration or pure acceleration (that includes g acceleration up).
4. Is rotation quaternion is transformation from local to global axis or vise-versa. They are identical up to conjugation.
5. Should the acceleration be in local or global coordinates. As far as I know fusion frameworks, it should be in global coordinates (gravity vector always up).