forked from cloudevents/spec
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Re-Introducing Protocol Buffer Representation (cloudevents#626)
* Re-Introducing Protocol Buffer Representation Signed-off-by: Day, Jeremy(jday) <[email protected]> * required and optional attributes are explicitly represented Signed-off-by: Day, Jeremy(jday) <[email protected]> * tweaks and examples to address review comments Signed-off-by: Day, Jeremy(jday) <[email protected]> * Updated following dicussion during 6/28 weekly meeting. - Only required attributes are formally represented. - Optional & Extension attributes are carried in the map. - Documentation & Examples changed as appropiate. - Added version marker to proto & java package. Signed-off-by: Day, Jeremy(jday) <[email protected]> * tweaks Signed-off-by: Day, Jeremy(jday) <[email protected]> * Include spec reference Signed-off-by: Day, Jeremy(jday) <[email protected]> * Changed media-type designation as-per weekly call discussion Signed-off-by: Day, Jeremy(jday) <[email protected]> * Fixed links to work before merging to master Signed-off-by: Day, Jeremy(jday) <[email protected]> * Really fixing the links this time Signed-off-by: Day, Jeremy(jday) <[email protected]> * DO NOT MERGE: data representation proposal, attribute type naming change Signed-off-by: Day, Jeremy(jday) <[email protected]> * Moved back to one_of representation for Data, updated docs Signed-off-by: Day, Jeremy(jday) <[email protected]> * Minor tweaks based on review commentary Signed-off-by: Day, Jeremy(jday) <[email protected]> * - Modified proto map property to increase raadability of generated code. - Clarified usage of `dataschema` CloudEvent attribute. - Updated examples. Signed-off-by: Day, Jeremy(jday) <[email protected]> * Changes based on review comments Signed-off-by: Day, Jeremy(jday) <[email protected]> * Added language specific package definitions Signed-off-by: Day, Jeremy(jday) <[email protected]> * - Removed c# directive, can be added back in a later PR. - Added Java directive for seperate fies. - re-pluralized 'attributes' to follow proto convention. Signed-off-by: Day, Jeremy(jday) <[email protected]> * Tidied 'one-of' names Signed-off-by: Day, Jeremy(jday) <[email protected]>
- Loading branch information
Showing
4 changed files
with
291 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,5 @@ | ||
| .idea/ | ||
| gen/ | ||
| .vscode/ | ||
| gen/ | ||
| .DS_Store |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,227 @@ | ||
| # Protobuf Event Format for CloudEvents - Version 1.0 | ||
|
|
||
| ## Abstract | ||
|
|
||
| [Protocol Buffers][proto-home] is a mechanism for marshalling structured data, | ||
| this document defines how CloudEvents are represented using [version 3][proto-3] | ||
| of that specification. | ||
|
|
||
| In this document the terms *Protocol Buffers*, *protobuf*, and *proto* are used | ||
| interchangeably. | ||
|
|
||
| ## Status of this document | ||
|
|
||
| This document is a working draft. | ||
|
|
||
| ## Table of Contents | ||
|
|
||
| 1. [Introduction](#1-introduction) | ||
| 2. [Attributes](#2-attributes) | ||
| 3. [Data](#3-data) | ||
| 4. [Transport](#4-transport) | ||
| 5. [Examples](#5-examples) | ||
|
|
||
| ## 1. Introduction | ||
|
|
||
| [CloudEvents][ce] is a standardized and protocol-agnostic definition of the | ||
| structure and metadata description of events. This specification defines how the | ||
| elements defined in the CloudEvents specification are are represented using | ||
| a protobuf schema. | ||
|
|
||
| The [Attributes](#2-attributes) section describes the naming conventions and | ||
| data type mappings for CloudEvent attributes for use as protobuf message | ||
| properties. | ||
|
|
||
| The [Data](#3-data) section describes how the event payload is carried. | ||
|
|
||
| ### 1.1. Conformance | ||
|
|
||
| The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", | ||
| "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be | ||
| interpreted as described in [RFC2119][rfc2119]. | ||
|
|
||
| ### 1.2 Content-Type | ||
|
|
||
| There is no official IANA *media-type* designation for protobuf, as such this | ||
| specification uses 'application/protobuf' to identify such content. | ||
|
|
||
| ## 2. Attributes | ||
|
|
||
| This section defines how CloudEvents attributes are represented in the protobuf | ||
| [schema][proto-schema]. | ||
|
|
||
| ## 2.1 Type System | ||
|
|
||
| The CloudEvents type system is mapped to protobuf as follows : | ||
|
|
||
| | CloudEvents | protobuf | | ||
| | ------------- | ---------------------------------------------------------------------- | | ||
| | Boolean | [boolean][proto-scalars] | | ||
| | Integer | [int32][proto-scalars] | | ||
| | String | [string][proto-scalars] | | ||
| | Binary | [bytes][proto-scalars] | | ||
| | URI | [string][proto-scalars] following [RFC 3986 §4.3][rfc3986-section43]| | ||
| | URI-reference | [string][proto-scalars] following [RFC 3986 §4.1][rfc3986-section41] | | ||
| | Timestamp | [Timestamp][proto-timestamp] | | ||
|
|
||
| ## 2.3 Required Attributes | ||
|
|
||
| Required attributes are represented explicitly as protobuf fields. | ||
|
|
||
| ## 2.4 Optional Attributes & Extensions | ||
|
|
||
| Optional and extension attributes are represented using a map construct enabling | ||
| direct support of the CloudEvent [type system][ce-types]. | ||
|
|
||
| ```proto | ||
| map<string, CloudEventAttributeValue> attributes = 1; | ||
| message CloudEventAttributeValue { | ||
| oneof attr { | ||
| bool ce_boolean = 1; | ||
| int32 ce_integer = 2; | ||
| string ce_string = 3; | ||
| bytes ce_binary = 4; | ||
| string ce_uri = 5; | ||
| string ce_uri_reference = 6; | ||
| google.protobuf.Timestamp ce_timestamp = 7; | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| In this model an attribute's name is used as the map *key* and is | ||
| associated with its *value* stored in the appropriately typed property. | ||
|
|
||
| This approach allows for attributes to be represented and transported | ||
| with no loss of *type* information. | ||
|
|
||
| ## 3. Data | ||
|
|
||
| The specification allows for data payloads of the following types to be explicitly represented: | ||
|
|
||
| * string | ||
| * bytes | ||
| * protobuf object/message | ||
|
|
||
| ```proto | ||
| oneof data { | ||
| // Binary data | ||
| bytes binary_data = 2; | ||
| // String data | ||
| string text_data = 3; | ||
| // Protobuf Message data | ||
| google.protobuf.Any proto_data = 4; | ||
| } | ||
| ``` | ||
|
|
||
| * Where the data is a protobuf message it MUST be stored in the `proto_data` property. | ||
| * `datacontenttype` MAY be populated with `application/protobuf` | ||
| * `dataschema` SHOULD be populated with the type URL of the protobuf data message. | ||
|
|
||
| * When the type of the data is text, the value MUST be stored in the `text_data` property. | ||
| * `datacontenttype` SHOULD be populated with the appropriate media-type. | ||
|
|
||
| * When the type of the data is binary the value MUST be stored in the `binary_data` property. | ||
| * `datacontenttype` SHOULD be populated with the appropriate media-type. | ||
|
|
||
|
|
||
|
|
||
| ## 4. Transport | ||
|
|
||
| Transports that support content identification MUST use the following designation: | ||
|
|
||
| ```text | ||
| application/cloudevents+protobuf | ||
| ``` | ||
|
|
||
| ## 5. Examples | ||
|
|
||
| The following code-snippets shows how proto representations might be constucted asuming the availability of some convenience methods ... | ||
|
|
||
| ### 5.1 Plain Text event data | ||
|
|
||
| ```java | ||
| public static CloudEvent plainTextExample() { | ||
| CloudEvent.Builder ceBuilder = CloudEvent.newBuilder(); | ||
|
|
||
| ceBuilder | ||
| //-- Required Attributes. | ||
| .setId(UUID.randomUUID().toString()) | ||
| .setSpecVersion("1.0") | ||
| .setType("io.cloudevent.example") | ||
| .setSource("producer-1") | ||
|
|
||
| //-- Data. | ||
| .setTextData("This is a plain text message"); | ||
|
|
||
| //-- Optional Attributes | ||
| withCurrentTime(ceBuilder, "time"); | ||
| withAttribute(ceBuilder, "datacontenttype", "text/plain"); | ||
|
|
||
| // Build it. | ||
| return ceBuilder.build(); | ||
| } | ||
|
|
||
| ``` | ||
|
|
||
| ### 5.2 Proto message as event data | ||
|
|
||
| Where the event data payload is itself a protobuf message (with its own schema) | ||
| a protocol buffer idiomatic method can be used to carry the data. | ||
|
|
||
| ```java | ||
| private static Spec.CloudEvent protoExample() { | ||
|
|
||
| //-- Build an event data protobuf object. | ||
| Test.SomeData.Builder dataBuilder = Test.SomeData.newBuilder(); | ||
|
|
||
| dataBuilder | ||
| .setSomeText("this is an important message") | ||
| .setIsImportant(true); | ||
|
|
||
| //-- Build the CloudEvent. | ||
| CloudEvent.Builder ceBuilder = Spec.CloudEvent.newBuilder(); | ||
|
|
||
| ceBuilder | ||
| .setId(UUID.randomUUID().toString()) | ||
| .setSpecVersion("1.0") | ||
| .setType("io.cloudevent.example") | ||
| .setSource("producer-2") | ||
|
|
||
| // Add the proto data into the CloudEvent envelope. | ||
| .setProtoData(Any.pack(dataBuilder.build())); | ||
|
|
||
| // Add the protto type URL | ||
| withAttribute(ceBuilder, "dataschema", ceBuilder.getProtoData().getTypeUrl()); | ||
|
|
||
| // Set Content-Type (Optional) | ||
| withAttribute(ceBuilder, "datacontenttype", "application/protobuf"); | ||
|
|
||
| //-- Done. | ||
| return ceBuilder.build(); | ||
|
|
||
| } | ||
| ``` | ||
|
|
||
| ## References | ||
|
|
||
| * [Protocol Buffer 3 Specification][proto-3] | ||
| * [CloudEvents Protocol Buffers format schema][proto-schema] | ||
|
|
||
| [Proto-3]: https://developers.google.com/protocol-buffers/docs/reference/proto3-spec | ||
| [proto-home]: https://developers.google.com/protocol-buffers | ||
| [proto-scalars]: https://developers.google.com/protocol-buffers/docs/proto3#scalar | ||
| [proto-wellknown]: https://developers.google.com/protocol-buffers/docs/reference/google.protobuf | ||
| [proto-timestamp]: https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Timestamp | ||
| [proto-schema]: ./spec.proto | ||
| [json-format]: ./json-format.md | ||
| [ce]: ./spec.md | ||
| [ce-types]: ./spec.md#type-system | ||
| [rfc2119]: https://tools.ietf.org/html/rfc2119 | ||
| [rfc3986-section41]: https://tools.ietf.org/html/rfc3986#section-4.1 | ||
| [rfc3986-section43]: https://tools.ietf.org/html/rfc3986#section-4.3 | ||
| [rfc3339]: https://tools.ietf.org/html/rfc3339 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| /** | ||
| * CloudEvent Protobuf Format | ||
| * | ||
| * - Required context attributes are explicity represented. | ||
| * - Optional and Extension context attributes are carried in a map structure. | ||
| * - Data may be represented as binary, text, or protobuf messages. | ||
| */ | ||
|
|
||
| syntax = "proto3"; | ||
|
|
||
| package io.cloudevents.v1; | ||
|
|
||
| import "google/protobuf/any.proto"; | ||
| import "google/protobuf/timestamp.proto"; | ||
|
|
||
| option go_package = "cloudevents.io/genproto/v1"; | ||
| option java_package = "io.cloudevents.v1.proto"; | ||
| option java_multiple_files = true; | ||
| option php_namespace = "Io\\CloudEvents\\V1\\Proto"; | ||
| option ruby_package = "Io::CloudEvents::V1::Proto"; | ||
|
|
||
| message CloudEvent { | ||
|
|
||
| // -- CloudEvent Context Attributes | ||
|
|
||
| // Required Attributes | ||
| string id = 1; | ||
| string source = 2; // URI-reference | ||
| string spec_version = 3; | ||
| string type = 4; | ||
|
|
||
| // Optional & Extension Attributes | ||
| map<string, CloudEventAttributeValue> attributes = 5; | ||
|
|
||
| // -- CloudEvent Data (Bytes, Text, or Proto) | ||
| oneof data { | ||
| bytes binary_data = 6; | ||
| string text_data = 7; | ||
| google.protobuf.Any proto_data = 8; | ||
| } | ||
|
|
||
| /** | ||
| * The CloudEvent specification defines | ||
| * seven attribute value types... | ||
| */ | ||
|
|
||
| message CloudEventAttributeValue { | ||
|
|
||
| oneof attr { | ||
| bool ce_boolean = 1; | ||
| int32 ce_integer = 2; | ||
| string ce_string = 3; | ||
| bytes ce_bytes = 4; | ||
| string ce_uri = 5; | ||
| string ce_uri_ref = 6; | ||
| google.protobuf.Timestamp ce_timestamp = 7; | ||
| } | ||
| } | ||
| } |