Skip to content

Conversation

@cyrush
Copy link
Member

@cyrush cyrush commented Sep 6, 2023

Updates that adjust uberenv spack env setup so that folks can develop using python modules installed to a spack view.

Addresses most challenges outlined #120.

May still need some cleanup, so comments welcome.

Details:

  • Adds a new step that patches host configs post build to use the paths in the spack view.

Uses patterns from .uberenv_config.json spack_host_config_patches` entry to drive what is patched:

Example input:

  "spack_host_config_patches":
 {
   "PYTHON_EXECUTABLE": "view/python*/bin/python3",
   "SPHINX_EXECUTABLE": "view/python*/bin/sphinx-build"
   }

Creates a -patchd.cmake file in the prefix dir (does not modify original host config)

  • adds --spack-skip-externals option

This is mostly driven by the limitations of using an external python, but is generally useful for cases where system libs undermine you.

  • symlink the default view of the spack env into {prefix}/spack_view for easy access.

- patch spack.yaml to create the view dir under spack_env, instead of a hidden file deeper in the dir hierarchy

This should only happen in cases where spack.yaml lacks view customization.

@cyrush
Copy link
Member Author

cyrush commented Sep 6, 2023

Looks like i need to fix the view dir patching logic.

# pick the first in the list.
if l.startswith(pkg_name) and len(l.split()) > 1:
return {"name": pkg_name, "path": l.split()[-1]}
pkg_path = l.split()[-1]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

part of a fix for #123

# pick the first in the list.
if l.startswith(pkg_name) and len(l.split()) > 1:
return {"name": pkg_name, "path": l.split()[-1]}
pkg_path = l.split()[-1]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

part of a fix for #123

@cyrush
Copy link
Member Author

cyrush commented Sep 7, 2023

I removed the logic that tried to present a better default view by hacking yaml, and instead opt-ed to symlink whatever the view directory is under the prefix. This should be more robust and work with any view setup even if it was customized in an input spack.yaml

@cyrush
Copy link
Member Author

cyrush commented Sep 7, 2023

I need to test why we can't inspect and symlink the view for the --install case.

==> Updating view at /home/runner/work/uberenv/uberenv/.ci/test-project/uberenv_libs/spack_env/.spack-env/view
[exe: /home/runner/work/uberenv/uberenv/.ci/test-project/uberenv_libs/spack/bin/spack -D /home/runner/work/uberenv/uberenv/.ci/test-project/uberenv_libs/spack_env python -c 'env = spack.environment.active_environment();print(env.views["default"].get_projection_for_spec(env.matching_spec("'[email protected]%gcc ^[email protected]'")))']
[failed to find spack view info]

@cyrush
Copy link
Member Author

cyrush commented Dec 6, 2023

@white238 @kennyweiss @chapman39

This branch has been tested and is now in use for both conduit and ascent CI.
I would like to find a path to get it into uberenv main

open(spack_lib_config_src,"w").write(cfg_script)

# disable user cache dir
spack_cache_path_dir = pjoin(self.dest_dir,"spack_cache")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to add this to avoid any read / write of ~/.spack, disabling the local config is not sufficient.


def patch(self):
# force spack to use only "defaults" config scope
self.disable_spack_config_scopes()
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have to disable this first and for most, b/c w/o this -- any command may read/write to~/.spack


# setup clingo (unless specified not to)
if "spack_setup_clingo" in self.project_args and self.project_args["spack_setup_clingo"] == False:
if "spack_setup_clingo" in self.project_args and self.project_args["spack_setup_clingo"].lower() == "false":
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed this statement, prior logic would never work.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this really should be called spack_boostrap_now() or something similar -- its not setup clingo any more

@cyrush
Copy link
Member Author

cyrush commented Jan 9, 2026

closing #155 should be used instead.

@cyrush cyrush closed this Jan 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

symlinking host config is broken with envs + extra spec constraints

2 participants