bucket notifications - facilitate notif conf to override connection

[alphaprinz]

Describe the Problem

We want to allow user to describe an external server in a single connection file, but allow to specify spec per notification configuration.
Specifically, this will allow to have a single connection file for a single external Kafka server, but allow each notification to have it's own Kafka topic.

Explain the Changes

Each notification can have an "override" object that will take precedence over data from connection file.
The "effective" connect file is the original connect file merged (ie, overridden) with the override object.

Issues: Fixed #xxx / Gap #xxx

Testing Instructions:

1. Create a connection file for kafka (with or without topic field).
   {
   "notification_protocol": "kafka",
   "name": "k",
   "topic": "mytopic",
   "kafka_options_object": "{"metadata.broker.list":"localhost:9092"}"
   }

2. Configure a notification with override object:
   {
   "TopicConfigurations": [
   {
   "Id": "first",
   "TopicArn": "k?{"topic":"override"}",
   "Events": []
   }
   ]
   }

3. Create an event (ie upload object to bucket with notification). The notification will be sent with topic "override".

• Doc added/updated - TBD
• Tests added

[guymguym]

I think we should follow the arn structure format, see comments posted above.

[guymguym]

I think this ad hoc arn structure is not a good idea, we should try to map it to an arn format.

For kafka I expect the kafka arn format like in aws where ClusterName is our connection name, ClusterUuid should match the kafka cluster uuid , and TopicName would be optional:

arn:aws:kafka:::cluster/${ClusterName}/${ClusterUuid}
arn:aws:kafka:::topic/${ClusterName}/${ClusterUuid}/${TopicName}

For http webhook there is no matching arn in aws, however there is api-gateway or lambda. I would go with something in the spirit of api-gateway execute-api arn where ApiId is our connection name, Stage is less relevant as we do not manage the webhook lifecycle but I would still keep it as part of the arn structure, and finally method and path are very relevant:

arn:aws:execute-api:::${ApiId}/${Stage}/${Method}/${ApiSpecificResourcePath}

[alphaprinz]

I've made the topic for kafka to optionally be "kafka:::topic".
Still haven't found any mention of cluster UUID in the client.
Note that using '/' separator in containerized is somewhat confusing because I also need secret name in addition to connection name (currently, at least).