Enforce users to plan partitioning upfront

[AlexanderYastrebov]

Hello.
I have seen already several times people create event types with single partition and then suffer with no slots available during deployment.
API doc says that EventTypeStatistics MUST be provided by users on Event Type
creation but on EventType itself default_statistic field is not marked as required.

This should be fixed to enforce users to plan partitioning upfront.

[lmontrieux]

Thanks @AlexanderYastrebov for raising this issue.

As you mention, the API spec indicates that EventTypeStatistics MUST be provided by users on Event Type creation (https://github.com/zalando/nakadi/blob/master/api/nakadi-event-bus-api.yaml#L2285). It's actually optional, and the API spec contradicts itself (https://github.com/zalando/nakadi/blob/master/api/nakadi-event-bus-api.yaml#L2239).

We'll discuss what to do here - either fix the documentation and keep default_statistic optional, or make it mandatory (and fix the documentation anyway).

I think I'd be in favour of keeping it optional, for 2 reasons:
1/ making it mandatory may break some clients, who have ignored it when creating event types
2/ I actually see it as a feature, not a bug, that someone who hasn't carefully considered how many partition they need will get only one by default. Each partition in a nakadi deployment uses some resources; and a single partition has the advantage that it guarantees the order of events.

I think this discussion is related to one of our plans to allow users to change the number of partitions in their event types.

[rcillo]

@AlexanderYastrebov the problem is with the documentation, since this field has been added once the API was already there and making it mandatory would break the API compatibility, which is not possible given our guidelines on Must: Don’t Break Backward Compatibility.

What we are going to do is to focus on the problems behind it:

1. make it possible to change the number of partitions for scalability reasons (load or publish/read parallelism).
2. implement different delivery strategies (allow more than one consumer per partition).
3. introduce a backward incompatible change to the API and enforce default_statistic as a last resort.

That's how we plan to solve this, first by making the documentation consistent and then tackling the root cause of the issue.

[rcillo]

Ticket in jira for this issue https://techjira.zalando.net/browse/ARUHA-879

[BGehrels]

I would add an additional point that could also be a quick fix: Choose a more sensible default value: 1 will be not enough for most aws consumer stacks: most stacks have at least two instances to ensure good failover in case of instance failures. During deployment, there are two stacks running in parallel. so 2 instances per stack x 2 stacks in parallel = 4 partitions should be a good default value for live environment.