Scalable deployment

[multun]

Goal

OSRD as a whole needs to:

• be easily deployable on a k8s cluster
• be horizontally scalable

Current state of affairs

• the backend is horizontally scalable, BUT requests need to be routed to a server which holds the cached infrastructure
• the django service is horizontally scalable
• the mapping service (chartos) are horizontally scalable
• the edition service (editoast) is not currently scalable

Issues with the current implementation

• the edition service needs to be reworked to become stateless
• the backend does not evict cached infrastructure when forced to download a new one
• the cache hit rate of the backend service is horrible with a default k8s setup (1 / RUNNING_SERVERS)

Proposed changes

• we may be able to serialize the state of the edition service's infra cache, and store it into redis. If the size of the serialized state is acceptable, and serialization times not too bad, it could enable horizontal scaling of the edition service, even though it would benefit from preferential routing
• implement cache entry size estimation and cache eviction in the backend
• implement preferential routing of queries to instances which hold a cached copy, both for the backend and the edition service
  • a reverse proxy needs to handle load balancing in a special way
  • this load balancing might be inside the k8s cluster, already handling authentication
  • we may also need to perform load balancing when running OSRD outside of k8s