diff --git a/.gitignore b/.gitignore index f2e0f431d..a3131e21d 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,5 @@ .idea/ gen/ .vscode/ +gen/ .DS_Store diff --git a/README.md b/README.md index e6b46c7e2..98c38c2dc 100644 --- a/README.md +++ b/README.md @@ -37,6 +37,7 @@ The following documents are available: | Kafka Protocol Binding | [v1.0](https://github.com/cloudevents/spec/blob/v1.0/kafka-protocol-binding.md) | [master](https://github.com/cloudevents/spec/blob/master/kafka-protocol-binding.md) | | MQTT Protocol Binding | [v1.0](https://github.com/cloudevents/spec/blob/v1.0/mqtt-protocol-binding.md) | [master](https://github.com/cloudevents/spec/blob/master/mqtt-protocol-binding.md) | | NATS Protocol Binding | [v1.0](https://github.com/cloudevents/spec/blob/v1.0/nats-protocol-binding.md) | [master](https://github.com/cloudevents/spec/blob/master/nats-protocol-binding.md) | +| Protobuf Event Format | | [master][proto-working] | | Web hook | [v1.0](https://github.com/cloudevents/spec/blob/v1.0/http-webhook.md) | [master](https://github.com/cloudevents/spec/blob/master/http-webhook.md) | | | | **Additional Documentation:** | @@ -132,3 +133,6 @@ Periodically, the group may have in-person meetings that coincide with a major conference. Please see the [meeting minutes doc](https://docs.google.com/document/d/1OVF68rpuPK5shIHILK9JOqlZBbfe91RNzQ7u_P7YCDE/edit#) for any future plans. + +[proto-working]: ./protobuf-format.md +[proto-latest]: ./protobuf-format.md \ No newline at end of file diff --git a/protobuf-format.md b/protobuf-format.md new file mode 100644 index 000000000..018d57fd7 --- /dev/null +++ b/protobuf-format.md @@ -0,0 +1,227 @@ +# Protobuf Event Format for CloudEvents - Version 1.0 + +## Abstract + +[Protocol Buffers][proto-home] is a mechanism for marshalling structured data, +this document defines how CloudEvents are represented using [version 3][proto-3] +of that specification. + +In this document the terms *Protocol Buffers*, *protobuf*, and *proto* are used +interchangeably. + +## Status of this document + +This document is a working draft. + +## Table of Contents + +1. [Introduction](#1-introduction) +2. [Attributes](#2-attributes) +3. [Data](#3-data) +4. [Transport](#4-transport) +5. [Examples](#5-examples) + +## 1. Introduction + +[CloudEvents][ce] is a standardized and protocol-agnostic definition of the +structure and metadata description of events. This specification defines how the +elements defined in the CloudEvents specification are are represented using +a protobuf schema. + +The [Attributes](#2-attributes) section describes the naming conventions and +data type mappings for CloudEvent attributes for use as protobuf message +properties. + +The [Data](#3-data) section describes how the event payload is carried. + +### 1.1. Conformance + +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", +"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be +interpreted as described in [RFC2119][rfc2119]. + +### 1.2 Content-Type + +There is no official IANA *media-type* designation for protobuf, as such this +specification uses 'application/protobuf' to identify such content. + +## 2. Attributes + +This section defines how CloudEvents attributes are represented in the protobuf +[schema][proto-schema]. + +## 2.1 Type System + +The CloudEvents type system is mapped to protobuf as follows : + +| CloudEvents | protobuf | +| ------------- | ---------------------------------------------------------------------- | +| Boolean | [boolean][proto-scalars] | +| Integer | [int32][proto-scalars] | +| String | [string][proto-scalars] | +| Binary | [bytes][proto-scalars] | +| URI | [string][proto-scalars] following [RFC 3986 §4.3][rfc3986-section43]| +| URI-reference | [string][proto-scalars] following [RFC 3986 §4.1][rfc3986-section41] | +| Timestamp | [Timestamp][proto-timestamp] | + +## 2.3 Required Attributes + +Required attributes are represented explicitly as protobuf fields. + +## 2.4 Optional Attributes & Extensions + +Optional and extension attributes are represented using a map construct enabling +direct support of the CloudEvent [type system][ce-types]. + +```proto +map attributes = 1; + +message CloudEventAttributeValue { + + oneof attr { + bool ce_boolean = 1; + int32 ce_integer = 2; + string ce_string = 3; + bytes ce_binary = 4; + string ce_uri = 5; + string ce_uri_reference = 6; + google.protobuf.Timestamp ce_timestamp = 7; + } +} +``` + +In this model an attribute's name is used as the map *key* and is +associated with its *value* stored in the appropriately typed property. + +This approach allows for attributes to be represented and transported +with no loss of *type* information. + +## 3. Data + +The specification allows for data payloads of the following types to be explicitly represented: + +* string +* bytes +* protobuf object/message + +```proto +oneof data { + + // Binary data + bytes binary_data = 2; + + // String data + string text_data = 3; + + // Protobuf Message data + google.protobuf.Any proto_data = 4; +} +``` + +* Where the data is a protobuf message it MUST be stored in the `proto_data` property. + * `datacontenttype` MAY be populated with `application/protobuf` + * `dataschema` SHOULD be populated with the type URL of the protobuf data message. + +* When the type of the data is text, the value MUST be stored in the `text_data` property. + * `datacontenttype` SHOULD be populated with the appropriate media-type. + +* When the type of the data is binary the value MUST be stored in the `binary_data` property. + * `datacontenttype` SHOULD be populated with the appropriate media-type. + + + +## 4. Transport + +Transports that support content identification MUST use the following designation: + +```text + application/cloudevents+protobuf +``` + +## 5. Examples + +The following code-snippets shows how proto representations might be constucted asuming the availability of some convenience methods ... + +### 5.1 Plain Text event data + +```java +public static CloudEvent plainTextExample() { + CloudEvent.Builder ceBuilder = CloudEvent.newBuilder(); + + ceBuilder + //-- Required Attributes. + .setId(UUID.randomUUID().toString()) + .setSpecVersion("1.0") + .setType("io.cloudevent.example") + .setSource("producer-1") + + //-- Data. + .setTextData("This is a plain text message"); + + //-- Optional Attributes + withCurrentTime(ceBuilder, "time"); + withAttribute(ceBuilder, "datacontenttype", "text/plain"); + + // Build it. + return ceBuilder.build(); +} + +``` + +### 5.2 Proto message as event data + +Where the event data payload is itself a protobuf message (with its own schema) +a protocol buffer idiomatic method can be used to carry the data. + +```java +private static Spec.CloudEvent protoExample() { + + //-- Build an event data protobuf object. + Test.SomeData.Builder dataBuilder = Test.SomeData.newBuilder(); + + dataBuilder + .setSomeText("this is an important message") + .setIsImportant(true); + + //-- Build the CloudEvent. + CloudEvent.Builder ceBuilder = Spec.CloudEvent.newBuilder(); + + ceBuilder + .setId(UUID.randomUUID().toString()) + .setSpecVersion("1.0") + .setType("io.cloudevent.example") + .setSource("producer-2") + + // Add the proto data into the CloudEvent envelope. + .setProtoData(Any.pack(dataBuilder.build())); + + // Add the protto type URL + withAttribute(ceBuilder, "dataschema", ceBuilder.getProtoData().getTypeUrl()); + + // Set Content-Type (Optional) + withAttribute(ceBuilder, "datacontenttype", "application/protobuf"); + + //-- Done. + return ceBuilder.build(); + +} +``` + +## References + +* [Protocol Buffer 3 Specification][proto-3] +* [CloudEvents Protocol Buffers format schema][proto-schema] + +[Proto-3]: https://developers.google.com/protocol-buffers/docs/reference/proto3-spec +[proto-home]: https://developers.google.com/protocol-buffers +[proto-scalars]: https://developers.google.com/protocol-buffers/docs/proto3#scalar +[proto-wellknown]: https://developers.google.com/protocol-buffers/docs/reference/google.protobuf +[proto-timestamp]: https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Timestamp +[proto-schema]: ./spec.proto +[json-format]: ./json-format.md +[ce]: ./spec.md +[ce-types]: ./spec.md#type-system +[rfc2119]: https://tools.ietf.org/html/rfc2119 +[rfc3986-section41]: https://tools.ietf.org/html/rfc3986#section-4.1 +[rfc3986-section43]: https://tools.ietf.org/html/rfc3986#section-4.3 +[rfc3339]: https://tools.ietf.org/html/rfc3339 diff --git a/spec.proto b/spec.proto new file mode 100644 index 000000000..4bf02f4cf --- /dev/null +++ b/spec.proto @@ -0,0 +1,59 @@ +/** + * CloudEvent Protobuf Format + * + * - Required context attributes are explicity represented. + * - Optional and Extension context attributes are carried in a map structure. + * - Data may be represented as binary, text, or protobuf messages. + */ + +syntax = "proto3"; + +package io.cloudevents.v1; + +import "google/protobuf/any.proto"; +import "google/protobuf/timestamp.proto"; + +option go_package = "cloudevents.io/genproto/v1"; +option java_package = "io.cloudevents.v1.proto"; +option java_multiple_files = true; +option php_namespace = "Io\\CloudEvents\\V1\\Proto"; +option ruby_package = "Io::CloudEvents::V1::Proto"; + +message CloudEvent { + + // -- CloudEvent Context Attributes + + // Required Attributes + string id = 1; + string source = 2; // URI-reference + string spec_version = 3; + string type = 4; + + // Optional & Extension Attributes + map attributes = 5; + + // -- CloudEvent Data (Bytes, Text, or Proto) + oneof data { + bytes binary_data = 6; + string text_data = 7; + google.protobuf.Any proto_data = 8; + } + + /** + * The CloudEvent specification defines + * seven attribute value types... + */ + + message CloudEventAttributeValue { + + oneof attr { + bool ce_boolean = 1; + int32 ce_integer = 2; + string ce_string = 3; + bytes ce_bytes = 4; + string ce_uri = 5; + string ce_uri_ref = 6; + google.protobuf.Timestamp ce_timestamp = 7; + } + } +} \ No newline at end of file