Set incompatible_strict_action_env

[hofbi]

Reproduce what we found in bazelbuild/bazel#26587 (comment)

[hofbi]

@rickeylev I am currently working on bazelbuild/bazel#26587 and @meteorcloudy helped me to trigger https://buildkite.com/bazel/bcr-bazel-compatibility-test/builds/492 to test the flip. We found an issue in the rules_python windows build here which should be in https://github.com/bazel-contrib/rules_python/tree/main/examples/multi_python_versions.

Now I am trying to reproduce this in this repo, but CI is still green. Am I setting the flag at the right spot?

[rickeylev]

am i setting the flag right?

I don't think so. I think that error is coming from the bzlmod example, not multi_python_versions. Which makes sense, this is running the tests that modules have configured for BCR to test, right? Yeah, that'd be examples/bzlmod for rules_python.

I updated the pr to set the flag in the bzlmod rc file, lets see how CI goes.

[rickeylev]

Something that may be relevant to --incompatible_strict_action_env: iirc, part of how the "windows launcher maker" thing in bazel works is it looks up python from PATH, I think as a fallback? I can't remember exactly.

[hofbi]

Thanks @rickeylev for helping out here. Now with CI being green again, do you have any suggestion how to reproduce the errors found here?

[rickeylev]

Hm. The two suggestions I have are:

• Flip the flag everywhere in rules_python. Maybe one of the other jobs will exhibit the same behavior. I'd be +1 on a PR to do this where ever we can anyways.
• Do you have a windows machine? clone rules_python, cd to the example dir, then run the builds.
• Try and identify a difference between the BCR environment and the Bazel-CI environment used by rules_python. Different OS versions? Are the env vars and the bazel command line args the same between the two?

An oddity with the error is that it reports files missing or giving permission denied (either could mean the other, though). This is weird because those exe files should always exist (they're generated by bazel after all). I would expect more an error like: those exe's run, then fail to find python.exe (because, for some reason, they get it from PATH).

My guess is its something to do with PATH, but that's just a guess. Maybe try having it print the PATH when things pass, print it when it fails, and then try to figure out what's missing, adding bits and pieces back to see the effect.

[hofbi]

Unfortunately I don't have a windows machine available, but I can make the PR and try to identify diffs in the env.

[hofbi]

@rickeylev I tried to enable incompatible_strict_action_env as much as possible and now we have 2 actual failures:

• //tests/integration:local_toolchains_test_bazel_self Windows only
• //tests/bootstrap_impls:run_binary_zip_no_test Windows only

The latter fails with Reason: (error: 2): The system cannot find the file specified. which seems related to flipping the flag. Before that, I can even see LAUNCHER ERROR: Cannot launch process: "python.exe" which might be the actual root cause.

[rickeylev]

To recap: So the basic problem is, on windows, the outer PATH value needs to be propagated down to tests. Is that about right?

Thanks for trudging through this and figuring out the cause!

I think we might be able to partially make this more seamless by having py_test return RunEnvironmentInfo.inherited_environment doc. This will, at the least, not require an explicit env_inherit setting on test targets. It also lets us keep it windows-specific (we can check if the target is windows, then only add it if so).

There's some existing logic to create RunEnvironmentInfo, and some existing logic to detect the target OS (see common.bzl, maybe_add_test_execution_info). So, something like this:

1. Add an implicit dependency on windows constraints. @platforms//os:windows (see _apple_constraints attribute for an example).
2. Use ctx.target_platform_has_constraint to check if it's active. See target_platform_has_any_constraint() utility function in common.bzl/util.bzl
3. If (2) is true add PATH to the list of inherited env vars. See create_run_environment_info in py_executable.bzl.

The --action_env flag still needs to be passed (to tell bazel to propagate it), which isn't ideal, but as a global flag, is still tenable as a solution.

[hofbi]

f47ee12 should do what you suggested, but looks like _create_providers is also producing a RunEnvironmentInfo as well.

Would you also accept the PR as we had it green to do a follow up to clean this up?