NiFi clustering without ZooKeeper

[malthe]

Is your feature request related to a problem?

NiFi 2.0.0 introduces new capabilities leveraging Kubernetes ConfigMaps and Leases to support clustering without the need for ZooKeeper.

We should leverage this capability to simplify the deployment.

Describe the solution you'd like to see

See https://exceptionfactory.com/posts/2024/08/10/bringing-kubernetes-clustering-to-apache-nifi/ for additional background and motivation.

Describe alternatives you've considered

No response

Additional context

No response

[juldrixx]

Hello,

We had started work on implementing the feature in this PR but we were missing a feature in NiFi for which we had requested validation of the implementation (https://issues.apache.org/jira/browse/NIFI-12590) so we were waiting for the feature to be released. But since then, we haven't necessarily had the time to go back over it.

But yes, we want to implement it. And it will maybe lead to a new CRD version for the NifiCluster.

[yakovasavr]

Hello team,

It seems issue has beed resolved on Apache side. Could you please return back to the feature?
It is quite important for product future to be able to provide a solution without zookeeper.

[juldrixx]

Hello team,

It seems issue has beed resolved on Apache side. Could you please return back to the feature? It is quite important for product future to be able to provide a solution without zookeeper.

The feature is ready with its PR (#450) It will be merged tomorow if there is no more review. And a new version will be released with it.