Limit JSON-LD optionality to enhance developer experience

[msporny]

It has been suggested that there could be limits imposed on the use of JSON-LD for Verifiable Credentials that would enhance the developer experience. These changes would be normative changes to the current specification. This issue has been raised to discuss if this is a good idea and the mechanism(s) that could be used to accomplish the task.

[msporny]

To refine the set of concrete proposals from #929, here is a cut at some reasoning with concrete proposals that might help:

At present, the developer experience with Verifiable Credentials could be improved because:

• Developers think they need to start by creating a JSON-LD Context, which they don't need to do. Can we provide the developer with @vocab to get them started, but in a way that is "safe" or "sandboxed"?
• Developers are concerned that inline contexts will be used that require them to use a JSON-LD processor (instead of it being optional). We should remove this concern for developers that don't want to do JSON-LD processing.
• JSON-LD expanded form might be used, which would require the use of a JSON-LD processor. We should remove this concern for developers that don't want to do JSON-LD processing.

Given these concerns above, perhaps we can consider the following:

• JSON-LD Compact Form is the only allowed form of JSON-LD. JSON-LD expanded form is disallowed to eliminate the requirement to always perform JSON-LD processing in processing pipelines where its not needed.
• When @context is utilized, all values SHOULD/MUST be URLs. That is, in-line JSON-LD Contexts are strongly discouraged (we might even want to go as far as forbidding them).
• The Verifiable Credentials specification will provide an "developer-mode" JSON-LD Context (https://www.w3.org/ns/credentials/developer-mode/v1, with an @vocab value set to https://www.w3.org/2018/credentials/undefined# such that developers need not define a JSON-LD Context or Vocabulary semantics as an initial step in the development process.
• We could allow usage of @vocab as the ONLY inline property that can be used, but that has a number of developer ergonomics/safety concerns.
• Implementations SHOULD reject verification of any VC that utilizes the https://www.w3.org/ns/credentials/developer-mode/v1 JSON-LD Context in a production environment.

Some concrete proposals that we could put in front of the group are:

PROPOSAL: Verifiable Credentials that utilize @context MUST be expressed in JSON-LD Compact form.

PROPOSAL: Verifiable Credentials that utilize @context SHOULD/MUST ensure that all values associated with the @context property are URLs.

PROPOSAL: Verifiable Credentials that utilize @context SHOULD NOT utilize inline JSON-LD Contexts (objects as values) for the @context.

And the proposals to help make development easier for developers coming from a JSON background:

PROPOSAL: As an initial iteration on the idea, the Verifiable Credentials specification will define an "developer-mode" JSON-LD Context (https://www.w3.org/2018/credentials/developer-mode/v1, with an @vocab value set to https://www.w3.org/2018/credentials/undefined# such that developers need not define a JSON-LD Context or Vocabulary semantics as an initial step in the development process.

PROPOSAL: A conforming processor SHOULD raise an error if a VC that utilizes @context also uses the https://www.w3.org/ns/credentials/developer-mode/v1 JSON-LD Context in a production environment. The definition of "production environment" is left as an exercise to the implementer.

[David-Chadwick]

In order to understand the implications of the above, we have 3 cases to consider

1. Developers who do not want to use @context at all
2. Developers who want to use the standard @context but not define any new ones
3. Developers who are happy to define new @context values

Are the implications as follows?

1. These developers must use full URIs everywhere in their VCs, even for standard property names and values such as type, id, issuanceDate etc.
2. These developers can use short form property names (compact form) for the standard properties, but must use full URIs for every new property they define
3. These developers can use short form property names and values everywhere.

[msporny]

@David-Chadwick wrote:

Are the implications as follows?

1. These developers must use full URIs everywhere in their VCs, even for standard property names and values such as type, id, issuanceDate etc.

That is not what this issue is about. The discussion on NOT using @context is happening in #947.

However, on the topic of using full URLs everywhere -- no, we shouldn't do that, because that leads to non-idiomatic JSON (which we're trying to avoid). We want to promote the use of idiomatic JSON as much as possible, and the following is probably not going to go over very well with JSON developers:

   {
    "id": "http://example.edu/credentials/3732",
    "type": [
      "VerifiableCredential",
      "https://example.org/examples#UniversityDegreeCredential"
    ],
    "credentialSubject": [{
        "id": "did:example:ebfeb1f712ebc6f1c276e12ec21"
        "https://example.org/examples#degree": [{
            "http://schema.org/name": [{
                "type": "http://www.w3.org/1999/02/22-rdf-syntax-ns#HTML",
                "value": "Bachelor of Science and Arts"
              }],
            "type": ["https://example.org/examples#BachelorDegree"]
          }],
      }
    ],
  }

2. These developers can use short form property names (compact form) for the standard properties, but must use full URIs for every new property they define

No, for the same reason as the example above demonstrates. During development, per these set of proposals, you can do this:

{
  "@context": [
    "https://www.w3.org/2018/credentials/v1",
    "https://www.w3.org/2018/credentials/developer-mode/v1"
  ],
  ...

That's it... you have well formed JSON /and/ JSON-LD, though the semantics are all undefined. Developers can play around w/ their credential until they feel they've got it right. At some point, when they go to production, they need to define an @context for their VC... because without that, they don't have unambiguous global semantics for what their credential means.

To put it another way, if you don't want global semantics, you don't need to use VCs... just shove your data into a plain JSON JWT and you're done.

3. These developers can use short form property names and values everywhere.

Sure, but this class of developers are not our concern. This class of developers are already happy using Verifiable Credentials as-is and we don't require any changes for them.

What's being proposed is the following (note, this is a bog standard example from the VC Data Model spec). You don't need to use full URLs everywhere, nor do you need to use a JSON-LD processor to execute business logic based on data in the VC. We should make it more clear in the spec what you have to be sure of as a JSON-only developer (as discussed in #929) -- either make sure you're using some market-vertical standard JSON-LD Contexts OR vet them yourself to make sure they use @protected everywhere and the VCs contain @context values that are exactly the two URLs listed below in the order that they appear:

{
  "@context": [
    "https://www.w3.org/2018/credentials/v1",
    "https://www.w3.org/2018/credentials/examples/v1"
  ],
  "id": "http://example.edu/credentials/3732",
  "type": ["VerifiableCredential", "UniversityDegreeCredential"],
  "issuer": "https://example.edu/issuers/565049",
  "issuanceDate": "2010-01-01T00:00:00Z",
  "credentialSubject": {
    "id": "did:example:ebfeb1f712ebc6f1c276e12ec21",
    "degree": {
      "type": "BachelorDegree",
      "name": "Bachelor of Science and Arts"
    }
  }
}

Does that help refine the idea, @David-Chadwick ?

[David-Chadwick]

@msporny I take your answer to the case 1 developers to be "if you dont want to use @context then do not use W3C VCs." I can live with that. But what I wanted to highlight in the current issue is that there appear to be a set of developers who want to use W3C VCs and not use @context. The definitive answer to these developers will be found in the resolution of issue #947

[gvelez17]

hm @msporny I'm not sure making it more restrictive answers the concerns I would have as a new developer.

I may be missing large chunks of "context" here but my take woudl be that having a few standard templates (think create-react-app) would go a long way without necessarily laying down restrictions

If these defaults become widely used you could formalize them but it seems to me the thing needed first before making rules is to try out some default templates and see what devs do with them? Am I missing something?

For me as a relatively new vc dev, this discussion so far doesn't seem clarifying but adds new confusion - oh, this thing that's documented, but I can't do it?

[OR13]

I'm a -1 to "developer contexts".... thats what community groups are for.

I'm also a -1 to requiring specific URLs, because it sets a precedent that is counter to developer expectations with respect to inlining and conversion to RDF without the need to either:

1. bundle large context files into every application that consumes them
2. make network requests to resolve contexts you have not bundled (which is something that is implied by the use of a URL, even if you tell people not to request it... 100 Billion times).

[iherman]

The issue was discussed in a meeting on 2022-10-19

• no resolutions were taken

View the transcript

3.1. followup of the TPAC discussion on JSON/JSON-LD.

Manu Sporny: original issue has come from TPAC, but it seemed as if there were two separate issues..

See github issue vc-data-model#947.

Manu Sporny: first one requests @context to be optional.
… second one it to enhance developer experience of JSON-LD by limiting JSON-LD functionality.

See github issue vc-data-model#948.

Manu Sporny: so that a JSON only developer wont need to worry about JSON-LD processing.

Kristina Yasuda: I think it was correct to break down the original TPAC tracking issue into two, but will point out that a lot of good conversation is in that original issue that is now closed is not translated into the new issues. so will encourage folks to re-comment/re-engage..

Manu Sporny: intention is that you wont need a JSON-LD processor to use @context.
… done by adding a developer @context, but this should not be used in production.

Brent Zundel: folks should read the original issue for background and good information.

[selfissued]

(Adding a comment so that I'm notified of discussion on this issue)

[nadalin]

I'm against adding anything over and above JWT processing, so a big -1 to this approach

[dlongley]

@OR13,

I'm a -1 to "developer contexts".... thats what community groups are for.

I don't think this should be a "developer context", but rather one that clearly signals that you are opting into using undefined / ambiguous terms. We can come up with some name for it that is amenable to all.

I'm in agreement that we should help developers by having a context that defines an "undefined terms" @vocab, but we should not harm developers that do not want to use it by putting it in the core context. There are developers that would like to use standard tools that will throw errors if they have forgotten to define any terms in their contexts. If the core context has @vocab in it, then new tools have to be developed to look for those terms.

So, +1 to helping developers who want to opt-in to using a standard "undefined terms" @vocab, which we should provide via some context that has a well-defined URL so it's easy for people to code against and -1 to forcing all developers to use @vocab by putting it into the core context.

[dlongley]

@nadalin,

I'm against adding anything over and above JWT processing, so a big -1 to this approach

I have to say that this doesn't make any sense to me. JWT is already a standard, so I would expect someone to just go use JWT if that's exactly what they want -- rather than trying to make VCs become the same thing with a different name.

[David-Chadwick]

A bit of history might help. X.500 defined the schema for directory entries, so that every entry was well formed and software could check that the attributes that were present in an entry were allowed to be present. But developers (and administrators) got frustrated that they had to be continually changing the schema when they wanted to add a new attribute to a directory entry. So the unregistered object class was born. (see https://sec.cs.kent.ac.uk/x500book/Chapter.3/Chapter3c.htm). It seems that from a pragmatic perspective, some folks want the equivalent unregistered @context to be specified which will allow a VC to contain any new properties without defining them in either the @context or the crdentialSchema. The downside of this is that software will no longer be able to tell if unrecognised properties in a VC were meant to be in the VC or were inserted there by a rogue issuer, or it was simply a typing error.

[dlongley]

@David-Chadwick,

The downside of this is that software will no longer be able to tell if unrecognised properties in a VC were meant to be in the VC or were inserted there by a rogue issuer, or it was simply a typing error.

Yes, and this is a good reason to separate this feature into its own context so that people can opt into using it rather than making it in core -- where everyone has to use it. I think it's a good compromise for this group to specify that context (and its URL) so that everyone who chooses that approach does it the same way and it's easily detectable.

[msporny]

@nadalin @Sakurann @selfissued, @OR13, @mprorock, @gvelez17 -- what about something like this?

   "@context": ["https://www.w3.org/credentials/v2", "https://www.w3.org/credentials/json-only/v1"]

or this (@mprorock):

   "@context": ["https://www.w3.org/credentials/v2", "https://www.w3.org/credentials/private-terms/v1"]

That's all you'd need to include for a VC-JWT, nothing else, to use JSON-only mode. That would still be compatible w/ JSON-LD, but you wouldn't have to process it. It's an opt-in feature.

Not saying it's perfect, but is it something you could live with? It would mean you wouldn't have to define any new JSON-LD Context.

[nadalin]

@manu as long as it is still OPTIONAL and only meant to be compatible w/JSON-LD

[vongohren]

Im +1 for having a special context for opting in out on the JSON-LD. But I also feel that we should really have a clear message to the verifiers that this is not in global context, so handle the VC with care.

I hope these two issues will align on something: #953

[iherman]

The issue was discussed in a meeting on 2023-01-11

• no resolutions were taken

View the transcript

3.5. Limit JSON-LD optionality to enhance developer experience (issue vc-data-model#948)

See github issue vc-data-model#948.

Brent Zundel: Next issue is 948. Limit JSON-LD to enhance developer experience..

Manu Sporny: There are other things we can do..
… Only JSON-LD compact form could be allowed..
… We are using vocab already..
… Developer mode context has been taken over by the vocab..
… Only thing to do is to raise PR to use JSON-LD compact..

Orie Steele: See the related open PR: #1001.

[msporny]

To review where we are on this issue:

PROPOSAL: Verifiable Credentials that utilize @context SHOULD/MUST ensure that all values associated with the @context property are URLs.

This effectively happens by default, now that there is a base @vocab value.

PROPOSAL: Verifiable Credentials that utilize @context SHOULD NOT utilize inline JSON-LD Contexts (objects as values) for the @context.

This effectively happens for the vast majority of use cases, now that there is a base @vocab value.

PROPOSAL: As an initial iteration on the idea, the Verifiable Credentials specification will define an "developer-mode" JSON-LD Context (https://www.w3.org/2018/credentials/developer-mode/v1, with an @vocab value set to https://www.w3.org/2018/credentials/undefined# such that developers need not define a JSON-LD Context or Vocabulary semantics as an initial step in the development process.

This effectively happens by default, now that there is a base @vocab value.

PROPOSAL: A conforming processor SHOULD raise an error if a VC that utilizes @context also uses the https://www.w3.org/ns/credentials/developer-mode/v1 JSON-LD Context in a production environment. The definition of "production environment" is left as an exercise to the implementer.

This is unlikely to happen given where the WG is currently, but JSON-LD processors are exploring the ability to raise warnings/errors when @vocab is used to expand any property in a JSON-LD document. This means that JSON-LD processors can be programmed to halt if they desire every term to be mapped without the use of @vocab.

PROPOSAL: Verifiable Credentials that utilize @context MUST be expressed in JSON-LD Compact form.

PR #1050 has been raised to address this issue.

Given everything above, and given that @context is no longer optional in the base application/credential+ld+json media type, once PR #1050 has been merged, this issue will be closed.

[msporny]

PR #1050 has been merged, closing.