Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds proto3 definition #47

Merged
merged 9 commits into from
Apr 23, 2018
Merged

Adds proto3 definition #47

merged 9 commits into from
Apr 23, 2018

Conversation

codefromthecrypt
Copy link
Member

Design notes:

This is just like json, except uses bytes for IDs and IPs and enum instead of string for span kind

When finished will fix #39

zipkin2.proto Outdated
}

message Annotation {
uint64 timestamp = 5;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the ordering here some sort of optimization? would be good to comment that if so

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

heh this is a mistake!

@codefromthecrypt
Copy link
Member Author

Added some details about defaults due to gitter chat with @jorgheymans

Also switched to fixed64 for timestamps. Epoch timestamp are encoded fixed64 are varint would also be 8 bytes, and more expensive to encode and size. Duration is stored uint64, as often the numbers are quite small.

@codefromthecrypt
Copy link
Member Author

codefromthecrypt commented Apr 15, 2018

on the fixed64 for timestamps thing:

one benefit is constant time for size calculation (returning 8 vs branching on the value)
the other is the much faster encoding, for example, the below compares quickest varint vs fixed encode of an epoch micros timestamp into a byte array:

Benchmark                              Mode  Cnt  Score    Error  Units
BufferBenchmarks.writeFixed64          avgt   15  0.004 ±  0.001  us/op
BufferBenchmarks.writeVarint64_loop    avgt   15  0.010 ±  0.001  us/op

@wu-sheng
Copy link
Member

Love protobuff. 👍

codefromthecrypt pushed a commit to openzipkin/zipkin that referenced this pull request Apr 16, 2018
codefromthecrypt pushed a commit to openzipkin/zipkin that referenced this pull request Apr 16, 2018
codefromthecrypt pushed a commit to openzipkin/zipkin that referenced this pull request Apr 16, 2018
This adds `Endpoint.ipv4Bytes()` and `Endpoint.ipv6Bytes()` needed to
prevent excess overhead marshaling into byte arrays. It is efficient to
do this once, as endpoint IPs are most often shared (local endpoint) for
all spans. Even in the case of remote endpoint, these are often parsed
from `Inet6Address` objects who already have a byte array reference.

This will be used in proto3 and possibly in a future retrofitted v1
thrift encoder.

See openzipkin/zipkin-api#47
@codefromthecrypt
Copy link
Member Author

openzipkin/zipkin#1998 shows incremental progress implementing this

@codefromthecrypt
Copy link
Member Author

On port as int32, I think we should keep it.

Here's some background.

There is no uint16 type, so we can't be perfectly efficient with ports. For example, we can't encode high ports (those above 16383) in 2 bytes, rather 3. It is common that client-side ports are high ports (ex remoteEndpoint for server spans). That said, low ports like 80 take only 1 byte as varints smaller than 7 bits (127) are 1 byte to encode. Finally, there's really nothing we can do with this in proto3
https://developers.google.com/protocol-buffers/docs/proto3#scalar

Anyway, I found 3 other protos who use int32 for port, including grpc, so at least this has repeated:
https://github.com/googleapis/googleapis/blob/5813ea167e6508bdcfe62808a06de3ae440fca5f/google/monitoring/v3/uptime.proto#L68
https://github.com/googleapis/googleapis/blob/861bb36a42562131b0a367b70db79a38e5344c29/google/cloud/redis/v1beta1/cloud_redis.proto#L192
https://github.com/grpc/grpc-proto/blob/94708ad85ebfbf4f9818c34b0c29de3bc1117e9c/grpc/channelz/channelz.proto#L240

@codefromthecrypt
Copy link
Member Author

@openzipkin/core FYI I am very confident in the shape of this at the moment. There may be details and obviously will lift the docs from the openapi spec, but please do chime in if you see any problems

Copy link
Member

@basvanbeek basvanbeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jcchavezs
Copy link
Contributor

LGTM

@codefromthecrypt
Copy link
Member Author

codefromthecrypt commented Apr 17, 2018 via email

zipkin2.proto Outdated
Kind kind = 4;
string name = 5;
fixed64 timestamp = 6;
uint64 duration = 7;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the units for timestamp and duration?

codefromthecrypt pushed a commit to openzipkin/zipkin that referenced this pull request Apr 22, 2018
The zipkin-api project has no text to define what a span is. This
clarifies text so we can use it consistently.

See openzipkin/zipkin-api#47
@codefromthecrypt codefromthecrypt changed the title WIP proto3 definition Adds proto3 definition Apr 22, 2018
@codefromthecrypt
Copy link
Member Author

I've migrated docs from the yaml here. All ready to go in my opinion

@codefromthecrypt codefromthecrypt merged commit 46db0ae into master Apr 23, 2018
@codefromthecrypt codefromthecrypt deleted the proto3 branch April 23, 2018 00:36
codefromthecrypt pushed a commit to openzipkin/zipkin that referenced this pull request Apr 23, 2018
…2013)

The zipkin-api project has no text to define what a span is. This
clarifies text so we can use it consistently.

See openzipkin/zipkin-api#47
codefromthecrypt pushed a commit that referenced this pull request Apr 23, 2018
abesto pushed a commit to abesto/zipkin that referenced this pull request Sep 10, 2019
…penzipkin#2013)

The zipkin-api project has no text to define what a span is. This
clarifies text so we can use it consistently.

See openzipkin/zipkin-api#47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

proto3 encoding
7 participants