Fix PositionScale Scale value passed to shader#28
Open
jamierobertson1 wants to merge 1 commit intovsg-dev:masterfrom
Open
Fix PositionScale Scale value passed to shader#28jamierobertson1 wants to merge 1 commit intovsg-dev:masterfrom
jamierobertson1 wants to merge 1 commit intovsg-dev:masterfrom
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I noticed there is small rendering error when points are scaled in the shader.
Currently the points positions are moved and scaled in the shader as follows:
Where the vsg_Vertex is an 0-1 float calculate from one of Vulkans UNORM integer types depending on the bits used in the VSGPoints::Settings.
Currently VSGPoints is sending the bricksize as the scaling value.
double brickSize = brickPrecision * pow(2.0, static_cast<double>(settings.bits));When a brick for a position are calculated, it uses the full pow(2.0,nbits) bit range for a brick (i.e. divides the original position by this), and the remainder makes up the position. As an example (ignoring precision), for 8 bits, each brick will be 256 units in length, with the possible remainder being in the 0-255 range as per image below:
In the shader assuming , a value of 255 (VK_FORMAT_R8G8B8_UNORM) will be interpreted as 1.0:

(from vulkan conversion docs):
The 255 value will then be scaled by the full brick size vsg_PositionScale.w (i.e. the full 256 size) This is incorrect as it should be one precision increment less (as per diagram above), so currently the points are scaled a bit too much on screen.
Simple fix is to subtract one precision increment from the positionScale, scale value in the Brick::createRendering function. The rest of the positionScale stuff calculations seem good, it's literally just the bit used by the shader which needs to be updated.
Change:
to:
I verified this by drawing a wireframe box of known size, and then adding VSGPoint points at the corners.
Currently:
After fix:
For testing I just modified the VSGPoints application, to draw the box and points if no input file is supplied:
Tested on windows