fix(scheduler): Add scaling config and upgrade paths

[lc525]

Why

Motivation

Need for a smooth upgrade path from 2.9.x to 2.10.0. Previously:

• existing SeldonConfig CRs would not smoothly work after upgrading to 2.10
• changes in SeldonConfig (for example to scalability options) would not be picked-up by the rest of the install

The smooth upgrade plan is the following:

• When using an existing SeldonConfig CR, we keep everything the same as in 2.9.1 (i.e dataflow, model-gw reading number of partitions from env vars). The scheduler considers the old numPartitions setting by fetching it via a ConfigMap from SeldonConfig.config.scalingConfig.Pipelines.MaxShardCountMultiplier. An empty scaling config in SeldonConfig.config.scalingConfig is interpreted as if SeldonConfig.config.scalingConfig.Pipelines.MaxShardCountMultiplier == 1 (one processing replica per pipeline/model)
• Any other settings in scalingConfig can be set but have no effect in 2.10. Similarly, settings in SeldonConfig.config.kafkaConfig.Topics can be set but have no effect -- this is to give customers an opportunity to upgrade their CRs in preparation for 2.11 which will use those settings
• In particular, we will ask clients to upgrade their SeldonConfig to contain settings such as general autoscaling options, kafka number of partitions, etc, to be in sync with the values they have already set in other places (env vars for various pods). Again, modifing those will have no effect until 2.11
• In 2.11, we switch to activating those values, or considering defaults if they are missing. Customers that haven't upgraded their CRs incrementally in 2.10 will need to take manual steps to configure the upgrade to 2.11. Other customers that have already set the right options in SeldonConfig will be able to smoothly upgrade to 2.11
• In 2.10, customers will be able to set/modify SeldonConfig.config.scalingConfig.Pipelines.MaxShardCountMultiplier and this change will propagate automatically through our microservices (this changes the way pipelines/models are sharded over the various microservices: it controls how many replicas of pipeline-gw process a given pipeline, how many dataflow-engine replicas process a given pipeline, and how many model-gw replicas process a given model)
• In 2.11 the other settings will work the same (in terms of propagation of changes throughout the Core 2 install)

What

Summary of changes

• Introduce a ScalingConfig setting in SeldonConfig, the settings of which are read by the scheduler only
• No longer rely on SeldonConfig.Config.KafkaConfig.Topics.* settings for microservices (aim to port over to those in 2.11) - leave the config options in place so that customers can upgrade their SeldonConfig progressively
• Add configuration watchers to react to ScalingConfig changes.

Checklist

• Added/updated unit tests
• Added/updated documentation
• Checked for typos in variable names, comments, etc.
• Added licences for new files

Notes for reviewers

The crux of the changes are in (it would help to review those first):

• operator/apis/mlops/v1alpha1/seldonconfig_types.go
• scheduler/pkg/config/watcher.go
• scheduler/pkg/scaling/config/config.go
• scheduler/cmd/scheduler/main.go
• scheduler/pkg/kafka/dataflow/server.go
• scheduler/pkg/server/server.go