Skip to content

Commit

Permalink
chore(nixery): Housekeeping for depot compatibility
Browse files Browse the repository at this point in the history
Cleans up a whole bunch of things I wanted to get out of the door
right away:

* depot internal references to //third_party/nixery have been replaced
  with //tools/nixery
* cleaned up files from Github
* fixed SPDX & Copyright headers
* code formatting and inclusion in //tools/depotfmt checks

Change-Id: Iea79f0fdf3aa04f71741d4f4032f88605ae415bb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5486
Tested-by: BuildkiteCI
Reviewed-by: tazjin <[email protected]>
Autosubmit: tazjin <[email protected]>
  • Loading branch information
tazjin authored and clbot committed Apr 20, 2022
1 parent 322589f commit 73f0087
Show file tree
Hide file tree
Showing 27 changed files with 185 additions and 455 deletions.
2 changes: 0 additions & 2 deletions .gitattributes

This file was deleted.

27 changes: 0 additions & 27 deletions .github/workflows/build-and-test.yaml

This file was deleted.

35 changes: 0 additions & 35 deletions CONTRIBUTING.md

This file was deleted.

14 changes: 12 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

-----------------

[![Build Status](https://github.com/tazjin/nixery/actions/workflows/build-and-test.yaml/badge.svg)](https://github.com/tazjin/nixery/actions/workflows/build-and-test.yaml)
[![Build status](https://badge.buildkite.com/016bff4b8ae2704a3bbbb0a250784e6692007c582983b6dea7.svg?branch=refs/heads/canon)](https://buildkite.com/tvl/depot)

**Nixery** is a Docker-compatible container registry that is capable of
transparently building and serving container images using [Nix][].
Expand All @@ -24,6 +24,15 @@ You can watch the NixCon 2019 [talk about
Nixery](https://www.youtube.com/watch?v=pOI9H4oeXqA) for more information about
the project and its use-cases.

The canonical location of the Nixery source code is
[`//tools/nixery`][depot-link] in the [TVL](https://tvl.fyi)
monorepository. If cloning the entire repository is not desirable, the
Nixery subtree can be cloned like this:

git clone https://code.tvl.fyi/depot.git:/tools/nixery.git

The subtree is infrequently mirrored to `tazjin/nixery` on Github.

## Demo

Click the image to see an example in which an image containing an interactive
Expand Down Expand Up @@ -139,8 +148,9 @@ separate Nix function, which will make it possible to build images directly in
Nix builds.

[Nix]: https://nixos.org/
[layering strategy]: https://storage.googleapis.com/nixdoc/nixery-layers.html
[layering strategy]: https://tazj.in/blog/nixery-layers
[gist]: https://gist.github.com/tazjin/08f3d37073b3590aacac424303e6f745
[buildLayeredImage]: https://grahamc.com/blog/nix-and-layered-docker-images
[public]: https://nixery.dev
[depot-link]: https://cs.tvl.fyi/depot/-/tree/tools/nixery
[gcs]: https://cloud.google.com/storage/
15 changes: 2 additions & 13 deletions builder/archive.go
Original file line number Diff line number Diff line change
@@ -1,16 +1,5 @@
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License"); you may not
// use this file except in compliance with the License. You may obtain a copy of
// the License at
//
// https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
// WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
// License for the specific language governing permissions and limitations under
// the License.
// Copyright 2022 The TVL Contributors
// SPDX-License-Identifier: Apache-2.0
package builder

// This file implements logic for walking through a directory and creating a
Expand Down
15 changes: 2 additions & 13 deletions builder/builder.go
Original file line number Diff line number Diff line change
@@ -1,16 +1,5 @@
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License"); you may not
// use this file except in compliance with the License. You may obtain a copy of
// the License at
//
// https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
// WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
// License for the specific language governing permissions and limitations under
// the License.
// Copyright 2022 The TVL Contributors
// SPDX-License-Identifier: Apache-2.0

// Package builder implements the logic for assembling container
// images. It shells out to Nix to retrieve all required Nix-packages
Expand Down
15 changes: 2 additions & 13 deletions builder/builder_test.go
Original file line number Diff line number Diff line change
@@ -1,16 +1,5 @@
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License"); you may not
// use this file except in compliance with the License. You may obtain a copy of
// the License at
//
// https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
// WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
// License for the specific language governing permissions and limitations under
// the License.
// Copyright 2022 The TVL Contributors
// SPDX-License-Identifier: Apache-2.0
package builder

import (
Expand Down
15 changes: 2 additions & 13 deletions builder/cache.go
Original file line number Diff line number Diff line change
@@ -1,16 +1,5 @@
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License"); you may not
// use this file except in compliance with the License. You may obtain a copy of
// the License at
//
// https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
// WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
// License for the specific language governing permissions and limitations under
// the License.
// Copyright 2022 The TVL Contributors
// SPDX-License-Identifier: Apache-2.0
package builder

import (
Expand Down
15 changes: 2 additions & 13 deletions builder/layers.go
Original file line number Diff line number Diff line change
@@ -1,16 +1,5 @@
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License"); you may not
// use this file except in compliance with the License. You may obtain a copy of
// the License at
//
// https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
// WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
// License for the specific language governing permissions and limitations under
// the License.
// Copyright 2022 The TVL Contributors
// SPDX-License-Identifier: Apache-2.0

// This package reads an export reference graph (i.e. a graph representing the
// runtime dependencies of a set of derivations) created by Nix and groups it in
Expand Down
15 changes: 2 additions & 13 deletions config/config.go
Original file line number Diff line number Diff line change
@@ -1,16 +1,5 @@
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License"); you may not
// use this file except in compliance with the License. You may obtain a copy of
// the License at
//
// https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
// WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
// License for the specific language governing permissions and limitations under
// the License.
// Copyright 2022 The TVL Contributors
// SPDX-License-Identifier: Apache-2.0

// Package config implements structures to store Nixery's configuration at
// runtime as well as the logic for instantiating this configuration from the
Expand Down
15 changes: 2 additions & 13 deletions config/pkgsource.go
Original file line number Diff line number Diff line change
@@ -1,16 +1,5 @@
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License"); you may not
// use this file except in compliance with the License. You may obtain a copy of
// the License at
//
// https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
// WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
// License for the specific language governing permissions and limitations under
// the License.
// Copyright 2022 The TVL Contributors
// SPDX-License-Identifier: Apache-2.0
package config

import (
Expand Down
127 changes: 60 additions & 67 deletions default.nix
Original file line number Diff line number Diff line change
@@ -1,16 +1,5 @@
# Copyright 2019-2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Copyright 2022 The TVL Contributors
# SPDX-License-Identifier: Apache-2.0

# This function header aims to provide compatibility between builds of
# Nixery taking place inside/outside of the TVL depot.
Expand All @@ -19,12 +8,13 @@
# build system and this will need some major adaptations to support
# that.
{ depot ? { nix.readTree.drvTargets = x: x; }
, pkgs ? import <nixpkgs> {}
, pkgs ? import <nixpkgs> { }
, preLaunch ? ""
, extraPackages ? []
, extraPackages ? [ ]
, maxLayers ? 20
, commitHash ? null
, ... }@args:
, ...
}@args:

with pkgs;

Expand Down Expand Up @@ -54,7 +44,8 @@ let
"-ldflags=-s -w -X main.version=${nixery-commit-hash}"
];
};
in depot.nix.readTree.drvTargets rec {
in
depot.nix.readTree.drvTargets rec {
# Implementation of the Nix image building logic
nixery-prepare-image = import ./prepare-image { inherit pkgs; };

Expand All @@ -79,55 +70,57 @@ in depot.nix.readTree.drvTargets rec {
# Container image containing Nixery and Nix itself. This image can
# be run on Kubernetes, published on AppEngine or whatever else is
# desired.
nixery-image = let
# Wrapper script for the wrapper script (meta!) which configures
# the container environment appropriately.
#
# Most importantly, sandboxing is disabled to avoid privilege
# issues in containers.
nixery-launch-script = writeShellScriptBin "nixery" ''
set -e
export PATH=${coreutils}/bin:$PATH
export NIX_SSL_CERT_FILE=/etc/ssl/certs/ca-bundle.crt
mkdir -p /tmp
# Create the build user/group required by Nix
echo 'nixbld:x:30000:nixbld' >> /etc/group
echo 'nixbld:x:30000:30000:nixbld:/tmp:/bin/bash' >> /etc/passwd
echo 'root:x:0:0:root:/root:/bin/bash' >> /etc/passwd
echo 'root:x:0:' >> /etc/group
# Disable sandboxing to avoid running into privilege issues
mkdir -p /etc/nix
echo 'sandbox = false' >> /etc/nix/nix.conf
# In some cases users building their own image might want to
# customise something on the inside (e.g. set up an environment
# for keys or whatever).
nixery-image =
let
# Wrapper script for the wrapper script (meta!) which configures
# the container environment appropriately.
#
# This can be achieved by setting a 'preLaunch' script.
${preLaunch}
exec ${nixery-bin}/bin/nixery
'';
in dockerTools.buildLayeredImage {
name = "nixery";
config.Cmd = [ "${nixery-launch-script}/bin/nixery" ];

inherit maxLayers;
contents = [
bashInteractive
cacert
coreutils
git
gnutar
gzip
iana-etc
nix
nixery-prepare-image
nixery-launch-script
openssh
zlib
] ++ extraPackages;
};
# Most importantly, sandboxing is disabled to avoid privilege
# issues in containers.
nixery-launch-script = writeShellScriptBin "nixery" ''
set -e
export PATH=${coreutils}/bin:$PATH
export NIX_SSL_CERT_FILE=/etc/ssl/certs/ca-bundle.crt
mkdir -p /tmp
# Create the build user/group required by Nix
echo 'nixbld:x:30000:nixbld' >> /etc/group
echo 'nixbld:x:30000:30000:nixbld:/tmp:/bin/bash' >> /etc/passwd
echo 'root:x:0:0:root:/root:/bin/bash' >> /etc/passwd
echo 'root:x:0:' >> /etc/group
# Disable sandboxing to avoid running into privilege issues
mkdir -p /etc/nix
echo 'sandbox = false' >> /etc/nix/nix.conf
# In some cases users building their own image might want to
# customise something on the inside (e.g. set up an environment
# for keys or whatever).
#
# This can be achieved by setting a 'preLaunch' script.
${preLaunch}
exec ${nixery-bin}/bin/nixery
'';
in
dockerTools.buildLayeredImage {
name = "nixery";
config.Cmd = [ "${nixery-launch-script}/bin/nixery" ];

inherit maxLayers;
contents = [
bashInteractive
cacert
coreutils
git
gnutar
gzip
iana-etc
nix
nixery-prepare-image
nixery-launch-script
openssh
zlib
] ++ extraPackages;
};
}
Loading

0 comments on commit 73f0087

Please sign in to comment.