Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(modules): use package set for port sources #384

Merged
merged 20 commits into from
Dec 27, 2024

Conversation

getchoo
Copy link
Member

@getchoo getchoo commented Nov 25, 2024

Closes #240

The main goal here is to avoid discrepancies with the versions of Nixpkgs we have tested on and those on our users' systems -- i.e., us falling out of sync or moving too quickly with Nixpkgs will no longer cause problems in our modules

As a side effect of these changes, we also see some other improvements:

  • No more builtin fetchers
    • Builtin fetchers have a large effect on evaluation by blocking it to download files -- similar to IFD. This is why they are not allowed in Nixpkgs.
    • Using Nixpkgs' fetchers allows us to download most sources at build time, considerably improving evaluation performance for those who use many of our modules
  • Less space taken up by (most) sources
    • Done by only packaging what's needed by the module
    • This should make interacting with sources much easier
    • It also allows us to easily share modifications to ports across home-manager and NixOS
  • Fewer rebuilds
    • Some derivations brought over from Nixpkgs have been modified to build and install all variants instead of only one set. While this slightly increases storage use, it avoids rebuilds when you change your accent, flavor, etc options

@getchoo getchoo added this to the 2.0.0 milestone Nov 25, 2024
@getchoo getchoo force-pushed the package-set branch 5 times, most recently from 2c76c34 to e7f9411 Compare November 26, 2024 20:20
modules/nixos/sddm.nix Outdated Show resolved Hide resolved
this is annoying to maintain

• Added input 'catppuccin-v1_1':
    'https://api.flakehub.com/f/pinned/catppuccin/nix/1.1.1/0193bdc0-b045-7eed-bbec-95611a8ecdf5/source.tar.gz?narHash=sha256-pCWJgwo77KD7EJpwynwKrWPZ//dwypHq2TfdzZWqK68%3D' (2024-12-13)
• Added input 'catppuccin-v1_2':
    'https://api.flakehub.com/f/pinned/catppuccin/nix/1.2.0/0193e5e0-33b7-7149-a362-bfe56b20f64e/source.tar.gz?narHash=sha256-Let3uJo4YDyfqbqaw66dpZxhJB2TrDyZWSFd5rpPLJA%3D' (2024-12-20)
• Added input 'home-manager':
    'github:nix-community/home-manager/1395379a7a36e40f2a76e7b9936cc52950baa1be?narHash=sha256-OOfI0XhSJGHblfdNDhfnn8QnZxng63rWk9eeJ2tCbiI%3D' (2024-12-19)
• Added input 'home-manager/nixpkgs':
    follows 'nixpkgs'
• Added input 'home-manager-stable':
    'github:nix-community/home-manager/80b0fdf483c5d1cb75aaad909bd390d48673857f?narHash=sha256-vykpJ1xsdkv0j8WOVXrRFHUAdp9NXHpxdnn1F4pYgSw%3D' (2024-12-16)
• Added input 'home-manager-stable/nixpkgs':
    follows 'nixpkgs-stable'
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/23e89b7da85c3640bbc2173fe04f4bd114342367?narHash=sha256-y/MEyuJ5oBWrWAic/14LaIr/u5E0wRVzyYsouYY3W6w%3D' (2024-11-19)
  → 'github:NixOS/nixpkgs/d3c42f187194c26d9f0309a8ecc469d6c878ce33?narHash=sha256-cHar1vqHOOyC7f1%2BtVycPoWTfKIaqkoe1Q6TnKzuti4%3D' (2024-12-17)
• Added input 'nixpkgs-stable':
    'github:NixOS/nixpkgs/b47fd6fa00c6afca88b8ee46cfdb00e104f50bca?narHash=sha256-nbG9TijTMcfr%2Bau7ZVbKpAhMJzzE2nQBYmRvSdXUD8g%3D' (2024-12-19)
• Added input 'nuscht-search':
    'github:NuschtOS/search/3051be7f403bff1d1d380e4612f0c70675b44fc9?narHash=sha256-Y47y%2BLesOCkJaLvj%2BdI/Oa6FAKj/T9sKVKDXLNsViPw%3D' (2024-12-09)
• Added input 'nuscht-search/flake-utils':
    'github:numtide/flake-utils/11707dc2f618dd54ca8739b309ec4fc024de578b?narHash=sha256-l0KFg5HjrsfsO/JpG%2Br7fRrqm12kzFHyUHqHCVpMMbI%3D' (2024-11-13)
• Added input 'nuscht-search/flake-utils/systems':
    'github:nix-systems/default/da67096a3b9bf56a91d16901293e51ba5b49a27e?narHash=sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768%3D' (2023-04-09)
• Added input 'nuscht-search/ixx':
    'github:NuschtOS/ixx/9fd01aad037f345350eab2cd45e1946cc66da4eb?narHash=sha256-EiOq8jF4Z/zQe0QYVc3%2BqSKxRK//CFHMB84aYrYGwEs%3D' (2024-10-26)
• Added input 'nuscht-search/ixx/flake-utils':
    follows 'nuscht-search/flake-utils'
• Added input 'nuscht-search/ixx/nixpkgs':
    follows 'nuscht-search/nixpkgs'
• Added input 'nuscht-search/nixpkgs':
    follows 'nixpkgs'
@getchoo getchoo merged commit 115c3de into catppuccin:main Dec 27, 2024
8 checks passed
@getchoo getchoo deleted the package-set branch December 27, 2024 01:42
@gador
Copy link

gador commented Dec 27, 2024

I believe this broke eval on my hydra instance:

in job ‘mac’:
error:
       … while evaluating the attribute 'value'
         at /nix/store/3cbplia94qgrzhcjzvxqxcxpjj52x3nl-source/lib/modules.nix:846:9:
          845|     in warnDeprecation opt //
          846|       { value = addErrorContext "while evaluating the option `${showOption loc}':" value;
             |         ^
          847|         inherit (res.defsFinal') highestPrio;

       … while evaluating the option `system.build':

       … while evaluating the attribute 'mergedValue'
         at /nix/store/3cbplia94qgrzhcjzvxqxcxpjj52x3nl-source/lib/modules.nix:881:5:
          880|     # Type-check the remaining definitions, and merge them. Or throw if no definitions.
          881|     mergedValue =
             |     ^
          882|       if isDefined then

       … while evaluating definitions from `/nix/store/h7ky5a89kq9n96hwcnk42ir977bf4z5j-source/modules/system':

       … while evaluating the option `home-manager.users.gador.home.file':

       … while evaluating definitions from `/nix/store/xm5jfczg8vk87z70x0rcrjr04j1m6gbc-source/modules/misc/xdg.nix':

       … while evaluating the option `home-manager.users.gador.xdg.configFile':

       … while evaluating definitions from `/nix/store/xm5jfczg8vk87z70x0rcrjr04j1m6gbc-source/modules/programs/alacritty.nix':

       … while evaluating the option `home-manager.users.gador.programs.alacritty.settings':

       … while evaluating definitions from `/nix/store/rgwcvkpzdhy6rqpn6509a1qwqjn6rg5f-source/modules/home-manager/alacritty.nix':

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: a 'aarch64-darwin' with features {} is required to build '/nix/store/l8l4fqdz4gygj8j8f8ikllnhan13r8ix-catppuccin-install-hook.drv', but I am a 'x86_64-linux' with features {benchmark, big-parallel, ca-derivations, kvm, nixos-test}

(and yes, it has been working before and the only major change was this PR)

@getchoo
Copy link
Member Author

getchoo commented Dec 28, 2024

This is going to be an expected breakage with this change. Moving to nixpkgs fetchers over their builtin equivalents reveals some previously hidden occurrences of IFD, with the Alacritty module being an example of this

I'm assuming what's been happening here is that you have a Hydra instance with no aarch64-darwin builder, but that evaluates (not builds) an aarch64-darwin configuration. Previously this worked as the builtin fetchers are system agnostic, which allowed you to download the alacritty source even though it was technically for a different system; the Nixpkgs fetchers are system-specific though, so this is no longer possible

The only workaround here would be to disable the offending module on this configuration, or don't evaluate this configuration in your Hydra instance

@gador
Copy link

gador commented Dec 28, 2024

I'm assuming what's been happening here is that you have a Hydra instance with no aarch64-darwin builder,

Actually, the hydra instance is an x86_64-linux but has access to an aarch64-darwin builder. But I guess the evaluation is happening on the linux box.

The only workaround here would be to disable the offending module on this configuration, or don't evaluate this configuration in your Hydra instance

Well, this sucks. I like to pre-build my configurations and save them to an S3 instance. Is there no way to fix alacritty or have a workaround? Right now I pinned catppuccin-nix to the revision before this PR

@getchoo
Copy link
Member Author

getchoo commented Dec 28, 2024

Is there no way to fix alacritty or have a workaround?

If your Hydra instance does have access to an aarch64-darwin builder, I would recommend setting up distributed builds between that and the evaluation machine, as it would allow latter to perform builds on the former and then continue evaluating. Alternatively, you could override the source used via catppuccin.sources to use the builtin fetcher once again -- but I would only recommend this if all else fails

@gador
Copy link

gador commented Dec 28, 2024

If your Hydra instance does have access to an aarch64-darwin builder, I would recommend setting up distributed builds between that and the evaluation machine, as it would allow latter to perform builds on the former and then continue evaluating

Well, this already exists and I could evaluate and build my whole aarch64-darwin config before..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Maintain our own derivations of ports where necessary
3 participants