Pre-defined Nebari Configuration Sets

[Adam-D-Lewis]

"Pre-defined Nebari Configuration Sets" or "Config Sets" for short.

The "why" of this issue is addressed in the discussion.

This issue deals with the "how" or the implementation details.

YAML defined config sets

User Stories

Creating a new nebari config with a config set

• User runs nebari init -s=lower-priority-config-set.yaml,higher-priority-config-set.yaml
  • The new option "--config-sets, -s" expects a csv of locations <github-repo>,<local path> to specific yaml config set definitions in order of increasing priority.
  • When this option is used, the nebari config is rendered like normal except with certain fields overridden or extended by yaml config set definition
  • In the case of multiple provided config sets, the order is increasing priority so config sets provided later override earlier config sets.

User applying a config set to an existing distribution

• nebari apply -c nebari-config.yaml -s=nebari-config.yaml,another-config-set.yaml -o new-nebari-config.yaml

Nebari Plugin defined config sets

User Stories

Creating a new nebari config with an config set

• pip install my-nebari-config-set-plugin
• Either:
  • nebari init ...
    • by default nebari config-set plugins are added as the highest priority config set
  • nebari init -s my-nebari-config-set-plugin,other-plugin.yaml
    • custom config set priority can be specified with the -s argument

More User Stories (independent of yaml or plugin defined)

Creating a new nebari config using guided-init

• pip install my-nebari-config-set-plugin
• nebari init --guided-init
• The first option that comes up is "Would you like to apply a pre defined configuration set?" and then displays a list of possible config sets. The user can select multiple and the order in which the user selects them determines the priority.

Config Set Paths

A ConfigSet can be a file or a directory with folders named after cloud providers underneath (e.g. gcp, azure, aws).

• git repo
  • we'll download the git repo to a tmp dir and parse the files similar to a local path
• local path
  • when a local path path is passed in we check if it's a single config set or a group of config sets and apply each one
  • the group of config sets is only expected for the guided init option where you can interactively select which you want to apply and in what order or if you are okay applying the config sets after they've been sorted lexicographically.

Config Set Yaml Format

• I think they'll need a metadata section to describe which versions of nebari they support.

[dcmcand]

YAML defined config sets

I still feel very uncertain about the idea of stacking config sets. I feel like it is potentially adding layers of complication dealing with prioritizing and resolving conflicts. I am also not convinced that it is needed. My suggestion is to start using a single config set and then see if there is a need for multiple config sets. I think there is clear value in config sets, but the value is far less clear for adding the complexity of stackable config sets.

If everyone else is convinced that we need stackable config sets from the start, then we need to be very clear about prioritization. In that case, it also make sense to not template at all, but rather add a imports field at the to of the nebari-config.yaml.

If we did that, we could still use your suggested cli implementation

nebari init -s=lower-priority-config-set.yaml,higher-priority-config-set.yaml

would render something like

imports:
  - lower-priority-config-set.yaml
  - higher-priority-config-set.yaml

within the nebari-config.yaml and then everything within the nebari-config.yaml is highest priority.

This brings up a couple of questions:

1. Do we deal with versioning?
2. Do we vendor in those configs or fetch them each time (assuming they are remote)?

[dcmcand]

Nebari Plugin defined config sets

I don't feel like they are config sets anymore if we are using plugins (which presumably add either infrastructure or k8s services). I would suggest that this get a different name so we aren't overloading config set and we discuss implementation of it separately as I feel it will be easier to clearly distinguish between the two that way. I think that the plugin defined way should have some way to specify which plugins to install and which yaml config sets to apply.

However, maybe we should consider having a yaml config field called plugins that installs the specified plugins prior to validating the rest of the config? That way you could still just use your yaml config set to get the plugins required for your setup and we don't have two ways of doing it.

[Adam-D-Lewis]

YAML defined config sets

I still feel very uncertain about the idea of stacking config sets. I feel like it is potentially adding layers of complication dealing with prioritizing and resolving conflicts. I am also not convinced that it is needed. My suggestion is to start using a single config set and then see if there is a need for multiple config sets. I think there is clear value in config sets, but the value is far less clear for adding the complexity of stackable config sets.

If everyone else is convinced that we need stackable config sets from the start, then we need to be very clear about prioritization. In that case, it also make sense to not template at all, but rather add a imports field at the to of the nebari-config.yaml.

I agree that most of the benefit comes from a single config set, and stacking brings some added complexity that may not really be needed and I don't mind not implementing stacking initially. I think we should choose a design that doesn't preclude us from potentially stacking them in the future though.

If we did that, we could still use your suggested cli implementation

nebari init -s=lower-priority-config-set.yaml,higher-priority-config-set.yaml

would render something like

imports:
  - lower-priority-config-set.yaml
  - higher-priority-config-set.yaml

I view the CLI options around config sets as a way to interact with the nebari-config.yaml only at the time the command is run. E.g. you run nebari init -s gpu or nebari apply -s gpu (for existing nebari config files) and then the rendered nebari config file has the gpu sections in it. It seems a lot simpler than going and getting the updated config sets each time. I think it's difficult to understand what's going to be deployed when you can't see all the config in one place since your imports need to go be resolved before each deployment.

within the nebari-config.yaml and then everything within the nebari-config.yaml is highest priority.

This brings up a couple of questions:

1. Do we deal with versioning?
2. Do we vendor in those configs or fetch them each time (assuming they are remote)?