What's the right tolerance for dropping an oncurve

[rsheeter]

fontmake, and soon fontc - googlefonts/fontations#640 - strive to only drop oncurve when the implied value is exactly the same. This means we don't drop an oncurve that would move say 1e-6 font units if it took the implicit value.

If we were willing to drop points that are very close when they take the implied value we could save some bytes. This produces two questions:

1. Do we save enough bytes to care? - if not we can abandon the idea
2. How much shift is OK? Is 1e-6 or 1e-3 font units something anyone cares about?

[rsheeter]

@behdad noted in IM that For the thinnest of fonts 1 font unit (on a 1000 upem) might be noticeable at large sizes. In other places (gvar rounding) we use a tolerance of 0.5 on a 1000upem (varLib._add_gvar has a default tolerance of .5 and it's not overridden by the caller)

[belluzj]

Hello! Some comments after discussing this with @anthrotype: in some cases, dropping on-curve points could be undesirable, even if the point is perfectly in the middle, because that would mess with hinting. See this issue/PR for an example: fonttools/skia-pathops#47 (the PR was never finished and instead we use the old booleanOperations when VTT hinting is needed).

Maybe this behaviour should have an on/off toggle, or possibly a third middle-ground option which is to drop the oncurve unless it's an extremum? That being said, my use-case is about VTT hinting, which is not super common these days, so maybe not the highest priority.

[rsheeter]

would mess with hinting

Great point.

Let's momentarily set it aside, if hinting wasn't an issue how much shift is ok? :)