Horizontal scalability issue

[amityahav]

since there are in-memory stores for the backends in addition to the remote stores (yaml, redis, mongo), when thinking about scalability when more than 1 instance of frontman exists and an admin modifies the service registry he will modify the remote registry and the in-memory stores(registry and prefix tree) of the addressed frontman solely, so we need to figure out a way to gossip among peers when a modification is made.
this gossip should update in-memory store and prefix tree of the peers

[hyperioxx]

one of the instances needs to act as the control-plane/director/Leader (Whatever term we use xD). the raft consensus algorithm come's to mind (it's the same algorithm that docker swarm uses). What do you guys think ? @amityahav @nonsoike

[hyperioxx]

Explanation -> https://raft.github.io/

[amityahav]

@hyperioxx raft will do for preserving state cross load balancers cluster, but is it our responsiblity to load balance between loadbalancers or its some low level implementation by the network engineers?

[hyperioxx]

@amityahav looking into it, it's not our responsibility we just need to keep the state in each instance

[amityahav]

@hyperioxx we could create an kubernetes operator but then we would have to change config manually via yank instead of using api which creates a state for an instance

[hyperioxx]

@amityahav i moved it to a discussion :)