Remove allocations on all base converters, improve TokenizerHelper

[h3xds1nz]

Description

This is probably one of the impactful PRs I've posted; optimizes conversion/parsing of all the base-generated types but also ThicknessConverter (once more), CornerRadiusConverter, KeySplineConverter and VirtualizationCacheLengthConverter.

Completely removes any allocations caused by the original Tokenizer by using ReadOnlySpan<char> instead of substrings.
I have also introduced a ref struct variant that can be used in any code that doesn't compile for net472.

MilCodeGen has also been adjusted to support this.

Single Thickness parsing (4 units)

Method	Mean [ns]	Error [ns]	StdDev [ns]	Code Size [B]	Gen0	Allocated [B]
Original	237.4 ns	4.64 ns	4.34 ns	555 B	0.0105	176 B
PR_EDITREF	191.6 ns	1.62 ns	1.51 ns	761 B	-	-

Parsing Thickness 10 times (4 units)

Method	Mean [ns]	Error [ns]	StdDev [ns]	Code Size [B]	Gen0	Allocated [B]
Original	2,434.0 ns	17.02 ns	14.08 ns	608 B	0.1030	1760 B
PR_EDITREF	1,885.1 ns	5.68 ns	4.44 ns	814 B	-	-

Creation of 10x Matrix3D via Parse

Method	Mean [ns]	Error [ns]	StdDev [ns]	Gen0	Code Size [B]	Allocated [B]
Original	7,643.3 ns	26.76 ns	20.89 ns	0.3052	6,876 B	5200 B
PR_EDIT	5,885.4 ns	67.98 ns	60.27 ns	-	7,177 B	-

Customer Impact

Improved performance by around 20% on base types, zeroed out allocations.

Regression

No.

Testing

Local build, basic testing of the tokenizer with span and some of the converters.

Risk

Low, the scope of the PR is big but with a simple change, hence any mistakes shall be easy to figure out on a DRT run.

Microsoft Reviewers: Open in CodeFlow

[h3xds1nz]

Resolved merge conflicts 12, there are better ways to spent an hour and a half.

[ThomasGoulet73]

Side note: @h3xds1nz when there's a review on a PR I recommend adding new commits and doing merges instead of rebasing. It makes it much easier to track what changed since the last review.

[ThomasGoulet73]

Is the || or && at the start of the line instead of at the end of the line wrong according to the coding style ? Both seems to be used a lot and if one isn't better than the other I don't see a reason to change it.

[ThomasGoulet73]

When I wrote this I put it on its own line on purpose because I think it's cleaner when the variable is used multiple times. Is there a reason you changed it ? I know there's an IDE0018 suggestion to change it but projects like winforms or runtime keep it as a suggestion and reason might be that sometimes it's cleaner not to do it.

[h3xds1nz]

I will revert the stylistic changes to your code before merge if you think it's cleaner, as it is your code. I just auto formatted here (which does for me a tad more than pure .editorconfig).

[himgoyalmicro]

Hey @h3xds1nz,
Sorry for the delay in response. We have observed that when we do
Rect testRect = Rect.Parse(" Empty \t");
We are hitting the exception System.FormatException: 'The input string 'Empty' was not in correct format.'

[h3xds1nz]

@himgoyalmicro Right, that was silly. Fixed.

I had to include #10683 for now to be able to run MilCodeGen, that's a PR that doesn't really need tests, could be just approved.

[codecov]

Codecov Report

Attention: Patch coverage is 45.36341% with 218 lines in your changes missing coverage. Please review.

Project coverage is 11.41570%. Comparing base (3cc9d57) to head (8b733a1).
Report is 15 commits behind head on main.

Additional details and impacted files

@@                 Coverage Diff                 @@
##                main       #9364         +/-   ##
===================================================
+ Coverage   11.17950%   11.41570%   +0.23619%     
===================================================
  Files           3314        3316          +2     
  Lines         665182      665312        +130     
  Branches       74668       74690         +22     
===================================================
+ Hits           74364       75950       +1586     
+ Misses        589525      587950       -1575     
- Partials        1293        1412        +119     

Flag	Coverage Δ
Debug	11.41570% <45.36341%> (+0.23619%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.