Clarify use of transfer functions

[MarkCallow]

This is a rough draft for further discussion.

[MarkCallow]

If we go ahead with this, I want to change the names of the options in ktx create from *-oetf to *-transfer. *-transfer-function is too long. We could make it shorter instead: *-tf*.

[MarkCallow]

Question for @fluppeteer. In our off-line discussions you suggested using OETF for HLG and EOTF for PG. This clarification proposes using both in the same way. Why did you propose different treatment and are you okay with using them both the same way?

[lexaknyazev]

In the BT.2100 document, PQ is defined as EOTF and OOTF (with OETF being derived) while HLG is defined as OETF and OOTF (with EOTF being derived).

This difference does not seem to be important given that KTX (with this PR applied) unambiguously defines how all those enums are interpreted.

[fluppeteer]

Question for @fluppeteer. In our off-line discussions you suggested using OETF for HLG and EOTF for PG. This clarification proposes using both in the same way. Why did you propose different treatment and are you okay with using them both the same way?

In both cases (indeed, in the case of any colour space) the application is responsible for converting to a suitable linear colour space for subsequent processing (if that's what it intends to do). Since the OOTF is often non-linear, it's up to the application to decide whether processing should be performed in the space of the captured scene, or that of the output display - although in almost all cases the OOTF as specified is a relatively mild non-linear transform intended to allow for the variation of the human visual system when acting under different viewing conditions, so it doesn't matter nearly as much as compensating for the EOTF or OETF.

In the case of HLG and PQ, the OOTF is explicit in the specification, so having both transforms documented is useful. In both cases the encoded content is "HLG" or "PQ", but it may be useful for an application to be aware of the space in which the content was authored (which would typically be the scene, and thus OETF, for HLG, and the display, hence the EOTF, for PQ). This is a pretty subtle nuance, and just using the OETF for HLG and EOTF for PQ would cover most cases if you wanted to simplify, as @lexaknyazev suggests; keeping both names in the KDFS enums is intended to simplify reference to the relevant functions in the KDFS.

Does that help?

[fluppeteer]

I don't think that's true. Most of the transfer functions correspond only to either the OETF or the EOTF, with a small number of exceptions where the two are the same. BT.601/709/2020 don't comment on the reference display, for example - that's what BT.1886 does. Equally, while the sRGB description does talk about how it can be integrated into a BT.709-based authoring environment, it is of itself only documenting the EOTF - it is for displaying computer content, so there is no concept of a camera capturing a scene in sRGB itself.

[lexaknyazev]

This is a pretty subtle nuance, and just using the OETF for HLG and EOTF for PQ would cover most cases if you wanted to simplify, as @lexaknyazev suggests; keeping both names in the KDFS enums is intended to simplify reference to the relevant functions in the KDFS.

I didn't suggest to simplify them. Having both versions explicitly signalled is essential for KTX.

[fluppeteer]

This is a pretty subtle nuance, and just using the OETF for HLG and EOTF for PQ would cover most cases if you wanted to simplify, as @lexaknyazev suggests; keeping both names in the KDFS enums is intended to simplify reference to the relevant functions in the KDFS.

I didn't suggest to simplify them. Having both versions explicitly signalled is essential for KTX.

Ah, apologies for the misunderstanding. Did you just mean that discussing the derivation via the OOTF isn't necessary? Or that people being aware of the origin space isn't important? I'm confused.

[fluppeteer]

The KDFS enumerators without a disambiguating either EOTF or OETF in the name indicate that the originating specification defines only an EOTF (such as sRGB), only an OETF (such as BT.709), or there is a linear OOTF. The proposed updated enum aliases clarify this.

[lexaknyazev]

there is a linear OOTF

I think this is the only thing KTX cares about in practice.

[fluppeteer]

Nitpick: the EOTF or OETF doesn't "have" an OOTF - they are defined in specifications which define an OOTF.

[fluppeteer]

By definition, if the OOTF is linear, the EOTF is the OETF^-1 and the OETF is the EOTF^-1. If they're symmetrical, they also have the same enum. That does mean this processing path is correct, although I would find it a little confusing to state it that way.

[fluppeteer]

Not true (there is no OETF in sRGB, and no defined OOTF, unless combined with a scene-referred standard such as BT.709).

[fluppeteer]

"Has an explicit OETF" means the enum chosen corresponds to the OETF defined by the specification, yes?

[lexaknyazev]

It means that the DFD enum has the _OETF suffix.

[fluppeteer]

I think "some transfer functions have a non-linear OOTF" read strangely to me - the OOTF is one transfer function, the OETF is another, the EOTF is a third, so they don't "have" each other. Perhaps "some color spaces", or "some standards"?

[fluppeteer]

Correspondingly, this means the enum identifies the EOTF defined by a specification? I think it's sufficient to say that KTX tools will process in the display-linear space or scene-linear space, determined by the enum encoding (for any enum that identifies either an EOTF or OETF; it would require additional information to specify the corresponding function for specifications that only define one transfer function). Applications creating a KTX file should indicate the authoring space that they used to create the content, whether that is scene- or display-referred.

[lexaknyazev]

Correspondingly, this means the enum identifies the EOTF defined by a specification?

Ditto. It means that the DFD enum has the _EOTF suffix.

Applications creating a KTX file should indicate the authoring space that they used to create the content, whether that is scene- or display-referred.

The KTX header defines how to process the file, not how it was created. The intention is to use _EOTF and _OETF enum suffixes to disambiguate the scene vs display linearity for cases when the OOTF is not linear..

[fluppeteer]

The KTX header defines how to process the file, not how it was created. The intention is to use _EOTF and _OETF enum suffixes to disambiguate the scene vs display linearity for cases when the OOTF is not linear..

I'm expecting it to affect the tooling, on the assumption that any mip chain generation probably shouldn't change the colour space identifiers and that this is a useful way to pass that information to the tool chain. If this documents how user applications might wish to receive similar instructions, sure. For an application converting content to screen space for display, it should always find an EOTF to use, whatever that EOTF may be. If converting to content that is part of a LDR video stream, it should be following the OETF instead. Hopefully the application knows what it's doing, and can make that decision.

[lexaknyazev]

Hopefully the application knows what it's doing, and can make that decision.

Yes and that's out of the KTX scope. Remember that KTX is not a general-purpose image or video format related to cameras or screens.

One could say that _EOTF files would be "images to be displayed" and _OETF files would be "material textures to be sampled from".

[lexaknyazev]

Did you just mean that discussing the derivation via the OOTF isn't necessary? Or that people being aware of the origin space isn't important? I'm confused.

The BT.2100 document defines:

1. Reference PQ EOTF via explicit math
2. Reference PQ OOTF via explicit math
3. Reference PQ OETF as the combination of the corresponding OOTF and EOTF^-1
4. Reference HLG OETF via explicit math
5. Reference HLG OOTF via explicit math
6. Reference HLG EOTF as the combination of the corresponding OOTF and OETF^-1

The fact that definitions 3 and 6 are not explicit does not make them any worse than the definitions 1 and 4 for the KTX purposes, so we can ignore that spec detail and treat all four functions equally.

[fluppeteer]

Did you just mean that discussing the derivation via the OOTF isn't necessary? Or that people being aware of the origin space isn't important? I'm confused.

The BT.2100 document defines:

1. Reference PQ EOTF via explicit math
2. Reference PQ OOTF via explicit math
3. Reference PQ OETF as the combination of the corresponding OOTF and EOTF-1
4. Reference HLG OETF via explicit math
5. Reference HLG OOTF via explicit math
6. Reference HLG EOTF as the combination of the corresponding OOTF and OETF-1

The fact that definitions 3 and 6 are not explicit does not make them any worse than the definitions 1 and 4 for the KTX purposes, so we can ignore that spec detail and treat all four functions equally.

Ah, agreed. I think the distinction is that most HLG content in the wild will have been created in a scene-referred context, and most PQ content will be display-referred, so the PQ EOTF and HLG OETF enums are likely to be more common than the PQ OETF and HLG EOTF ones. But no more valid, I just thought it was useful for people to have guidance on tagging content handed to them. (Again, it'll be rare use cases that actually need to make the distinction.)

[MarkCallow]

I've rewritten the section following my best understanding of the comments, which may be flawed. See 8f18ab5. Please review again and suggest improvements.. I've listed all cases explicitly which may be overkill.

[MarkCallow]

The addition of the transfer function aliases raises the question which names should ktx create support:

1. just the decorated names;
2. just the undecorated names;
3. both decorated and undecorated?

The only non-linear TF supported until now has been "srgb". If we want to go for no. 1, it is not an issue to keep "srgb" for backward compatibility while adding "srgb-eotf".

[fluppeteer]

I've rewritten the section following my best understanding of the comments, which may be flawed. See 8f18ab5. Please review again and suggest improvements.. I've listed all cases explicitly which may be overkill.

Looks much better to me (at least by my understanding!)

Re. the command line, I wouldn't lose too much sleep unless support for all (or most) other transfer functions is added; "sRGB" is unambiguous, and the main benefit of the enums including the EOTF/OETF suffixes is to help developers keep track of which to use in which circumstances. Not that it would be harmful to future-proof an alias if you wanted to.

[MarkCallow]

Re. the command line, I wouldn't lose too much sleep unless support for all (or most) other transfer functions is added;

We are adding support for most of the other transfer functions because the spec. now allows them to be used with non-_SRGB formats, except for "srgb" if there is an _SRGB format equivalent to the _UNORM format you are trying to use. I'm leaning towards 1. while also keeping "srgb" for backwards compatibility.

[MarkCallow]

Thanks for the approval @lexaknyazev. What is your take on the following?

The addition of the transfer function aliases raises the question which names should ktx create support:

1. just the decorated names;
2. just the undecorated names;
3. both decorated and undecorated?

[lexaknyazev]

I'd say both because both variants are going to be valid DFD enums.

[fluppeteer]

FWIW, I'm still a little philosophically uncomfortable with the "applications are intended to..." statements. My feeling is that metadata in a file format should describe what's in the format, not dictate an application's use of it. For example, if someone wants to do screen-space filtering of a BT.601 image (e.g. a decoded JPEG), they should probably be doing so in a display-referred space (either sRGB or BT.1886). That said, the OOTF is typically subtle, almost all Vulkan code will probably be doing this wrong (the Y'CbCr functionality doesn't provide advice on applying an OOTF to the decoded content), and I don't know enough about users of KTX to know whether I'm thinking of this in the right way.

That said, this does make me wonder whether the introductory section on chroma reconstruction in the KDFS ought to mention a choice of transfer function. I've just spotted that there's already a typo there... (fix coming).

[lexaknyazev]

FWIW, I'm still a little philosophically uncomfortable with the "applications are intended to..." statements. My feeling is that metadata in a file format should describe what's in the format, not dictate an application's use of it.

The file format spec technically cannot dictate anything to apps - it merely describes the expected behavior. An app can do whatever it wants because it's not bound by the KTX spec anyway.

[MarkCallow]

FWIW, I'm still a little philosophically uncomfortable with the "applications are intended to..." statements.

The spec text doesn't say that. It says "applications should", i.e. recommendations.

[fluppeteer]

FWIW, I'm still a little philosophically uncomfortable with the "applications are intended to..." statements. My feeling is that metadata in a file format should describe what's in the format, not dictate an application's use of it.

The file format spec technically cannot dictate anything to apps - it merely describes the expected behavior. An app can do whatever it wants because it's not bound by the KTX spec anyway.

Yes, I guess it's just that whether it's "expected" depends on what the app is doing. I don't know that it's even the majority case to apply the BT.709 OETF^-1, for example - arguably applying the sRGB EOTF would make more sense for a lot of applications. But it doesn't matter that much.

The spec text doesn't say that. It says "applications should", i.e. recommendations.

I guess I'm just not a rebel. :-)

[MarkCallow]

I'd say both because both variants are going to be valid DFD enums.

In the CLI should we use e.g, srgb-eotf or srgb_eotf, --assign-tf srgb-eotf or --assign-tf srgb_eotf? Since multi-word option names are written with "-", I propose to do that with the transfer function names too even though it differs from the names in `khr_df.h.

When converting a transfer function enumeration to text, e.g. in ktx info, should I use the undecorated name, which is what KDFS layout indicates is the main name, or the decorated names, which KDFS describes as aliases. Some of the values have multiple aliases, e.g. ITU so I am leaning towards printing the undecorated name. The drawback, what ever we choose, is that there will be unavoidable cases where what is printed is different from what the use gave as the argument to --assign-tf.

[lexaknyazev]

• The CLI option values should be consistent across different parameters. If Vulkan format enums are accepted with underscores, so should be the DFD enums.

• The ktx info should output the most "canonical" strings, whatever that means.

[MarkCallow]

One last, hopefully, question particularly for @lexaknyazev. In the changes to ktx create that I am working on I am using the option names --assign-tf and --convert-tf. Before I get to much further into updating the tests I'd like to know if anyone has an allergic reaction to using the short tf and would prefer transfer or verbosely transfer-function. (The old names *-oetf are still supported but deprecated.)

[lexaknyazev]

Since it was *-oetf before, changing it to *-tf is okay by extension.