[FR][API] Allow resetting env configs in plugins

[webknjaz]

So I'm trying to use tox_add_env_config and tox_add_core_config by following the examples of tox-gh. And I'm adding custom dynamic envs via core_conf.loaders.insert(0, MemoryLoader(env_list=env_list)) in tox_add_core_config. Then, in tox_add_env_config I'm trying to manipulate env_conf. I'm able to edit some of the values that have mutable structures, like env_conf['commands'].clear(); env_conf['commands'].append(...). But that feels hacky.

Also, it doesn't with non-list settings like env_conf['package'] = 'skip' because there's no item setter. env_conf.add_constant('package', value='skip', desc='') doesn't work either but for a different reason.

Apparently, these dynamic envs inherit [testenv] as the base env and I don't see a way to work around that.

cc @gaborbernat any ideas? examples? pointers? I'd love to be able to define envs that don't inherit anything and just populate them myself, instead of having side effects coming from whatever end-users would define in their tox.ini.

[gaborbernat]

You should not be editing those configurations. Instead you should insert a memory loader for the envs too, e.g.

tox/tests/plugin/test_plugin.py

Line 189 in 5a67ae1

env_conf.loaders.insert(0, MemoryLoader(deps=[1]))

[webknjaz]

Thanks! I was try to do something like that yesterday but it didn't seem to work. That's why I've been experimenting with other ways... I'll try again with your example.

[webknjaz]

@gaborbernat env_conf['package'] still gives a KeyError

[gaborbernat]

Do you have a link to the repository for me to take a look?

[webknjaz]

That's still local. But I think I figured it out. Tox calls the hook before declaring the package config option. And so it's not in the schema and I cannot validate it in the hook, while debugging.

[webknjaz]

@gaborbernat here's roughly what I have. I've added comment markers to places that demonstrate my discovery:

from tox.config.loader.memory import MemoryLoader

@impl
def tox_add_env_config(env_conf, state):
    if env_conf.env_name not in {'pip-compile', 'pip-compile-build-lock', 'pip-compile-tox-env-lock'}:
        return

    loader = MemoryLoader(deps=['pip-tools'], package='skip')
    env_conf.loaders = [loader]
    loader.raw['package'] = 'skip'  # overrides memloader's args, not needed really; just testing what explodes and what doesn't

    env_conf['deps']  # this works

    # env_conf.add_constant('package', value='skip', desc='')  # <- This makes the following line work but then exploads within tox because it apparently does `add_config()` some time after invoking the `tox_add_env_config()` hook
    env_conf['package']  # <- 💣 THIS EXPLODES with KeyError 💣

I kinda expected that everything tox declares natively present in the docs would also be accessible at this hook point. But evidently, it doesn't. I don't really need to access env_conf['package'] — this was my attempt to double-check that “it works” during development, under a debugger. And KeyError made it seem that it didn't work. But I've found a way to verify that tox does indeed pick up that value for the package by setting it to something invalid and watching it break.

I think, this highlights an issue of possibly incomplete documentation or misleading guarantees. The docs say

Called when configuration is built for a tox environment.

and I suppose that could be interpreted as "once the env config is fully initialized" or "some time around the config init". I was reading it as "the config is fully pre-populated by tox at this point". So discovering that it's partially came as a surprise and confused me.

I don't know if tox's own config population can be moved before calling the hook but this should probably be acknowledged in the docs.

@gaborbernat another thing I'm struggling with is populating commands. I'd like to inject commands referencing the python executable of the venv. I normally do that via {envpython} in tox.ini. But doing MemoryLoader(commands=['{envpython} -Im piptools compile --help']) ends up not substituting {envpython}:

pip-compile: commands[0]> '{envpython}' -Im piptools compile --help
pip-compile: Exception running subprocess [Errno 2] No such file or directory: '{envpython}'
pip-compile: exit 2 (0.00 seconds) /full/project/path/to/cwd> '{envpython}' -Im piptools compile --help

This also suggests that {posargs} might not work either. Is there a way to make use of the substitutions?

[webknjaz]

Looks like I'll need to spread the command construction logic across tox_add_env_config() and tox_before_run_commands() because one has access to the venv interpreter path and the other can get the posargs. There doesn't seem to be a place where the entire context is available together. Is there one?

[gaborbernat]

I don't know if tox's own config population can be moved before calling the hook but this should probably be acknowledged in the docs.

Yeah, probably more accurate to say Called when configuration is being built for a tox environment. You should not access these values here, you should use tox config -e 3.13 instead to check. Cannot be moved earlier, some things are evaluated later. If you need a value to depend on another value that's why default can be a function, to be delayed reading that value at load time, not upfront.

But doing MemoryLoader(commands=['{envpython} -Im piptools compile --help']) ends up not substituting {envpython}:

Yeah, that's not going to work. The substitution is a INI config file feature, not a core tox one. On one end there's no need to use {env_python}, python is sufficient. On other end if you want something like that you would need to do config['env_python'] which in this context would not work because is not populated at this point, we only find that out once we create the environment at a later point in time.

[gaborbernat]

Looks like I'll need to spread the command construction logic across tox_add_env_config() and tox_before_run_commands() because one has access to the venv interpreter path and the other can get the posargs. There doesn't seem to be a place where the entire context is available together. Is there one?

The core config should always contain the posargs.

[webknjaz]

Do you have an example? I think I saw unparsed configs somewhere but not just the part after --.

[gaborbernat]

Not sue I understand what you're asking for.

[webknjaz]

I can access Python (replacing {envpython}) like this:

@impl
def tox_before_run_commands(tox_env: ToxEnv) -> None:
    """Inject SOURCE_DATE_EPOCH env var into build-dists."""
    print(f'tox_before_run_commands: {tox_env=} {tox_env.env_python()=} {tox_env.session.interpreter.executable=}')

And then, {posargs} is here:

@impl
def tox_add_env_config(env_conf, state):
    print(f'tox_add_env_config: {env_conf=}, {state=}, {state.args=}, {state.conf.pos_args(to_path=None)=}')

So this effectively means that I need to avoid populating commands* via memory loader in tox_add_env_config because I cannot access {envpython} there. I can only assign state.conf.pos_args(to_path=None) to some global var in the module and then, use that var in tox_before_run_commands() where I cannot get {posargs} from the context. You said that I could get posargs from the core config, but that doesn't seem to be accessible in tox_before_run_commands() AFAICS. And so I wanted to ask if there's a way to get it there w/o me having to smuggle it from one hook to another through a module-global var.

[gaborbernat]

So this effectively means that I need to avoid populating commands* via memory loader in tox_add_env_config because I cannot access {envpython} there.

You don't need {env_python} just use python.

. You said that I could get posargs from the core config, but that doesn't seem to be accessible in tox_before_run_commands() AFAICS.

Yeah, that's true but you should not be modifying configuration in tox_before_run_commands. All configuration change should be done in tox_add_env_config otherwise tox config will not be correct.

[webknjaz]

@gaborbernat another place where I'm typically using it is the description: description = Produce a PEP 517/660 build deps lock using {envpython}. And while with the commands it's possible to get away with an abstract command name, here it would be even more important.

I think, there's an argument to be made that tox should support processing substitutions post config parsing.

[gaborbernat]

I think, there's an argument to be made that tox should support processing substitutions post config parsing.

I will say no to this.

I'd be open though to enriching the memory loader so that at load time one could access the env and core config. And then you could specify the value to be a method upon which the load could access those dynamic values. I don't plan adding this, but a PR by someone else in this direction would be something I'd consider.

[webknjaz]

Why is that problematic? I'd imagine having a single place for applying substitutions in an agnostic manner would simplify things, right?

[gaborbernat]

I don't want ini-ic features in the core layer. E.g. if you look at tox.toml https://tox.wiki/en/4.25.0/config.html#toml-only you see there's plenty things that does differently that ini. Having to do these random string replacement has caused enough bugs that I do not want to go down that path. You're free to fork the project and do whatever you like, but as someone who knows well the design philosophy of tox 4, I'm not interested to go down that path here.

[webknjaz]

Is it possible to somehow convert in-memory configs into TOML-like structure and apply the same principle? So that INI would be the exceptional case, but anything else would go the same path?

[gaborbernat]

Not really, because again, memory != TOML. Obviously one could perhaps alter TOML loader to do so, but I'm not interested to do that myself.

FWIW. I'm planning to sunset (hand over/no longer maintain) this project as soon as uv's task manager becomes good enough to replace 90% of the feature set of this project.

[webknjaz]

Oh, interesting. I'll keep that in mind.