From 2916956c164f83c50e853fd684408bf6d6663aca Mon Sep 17 00:00:00 2001 From: Joe Kimmel Date: Mon, 26 Jun 2023 10:17:27 -0700 Subject: [PATCH 01/11] first draft of builder spec Signed-off-by: Joe Kimmel --- distribution.md | 212 ++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 198 insertions(+), 14 deletions(-) diff --git a/distribution.md b/distribution.md index 6b217a13..c5a37718 100644 --- a/distribution.md +++ b/distribution.md @@ -15,11 +15,12 @@ This document specifies the artifact format, delivery mechanism, and order resol ## Distribution API Version -This document specifies Distribution API version `0.3`. +This document specifies Distribution API version `0.4`. Distribution API versions: - MUST be in form `.` or ``, where `` is equivalent to `.0` - When `` is greater than `0` increments to `` SHALL exclusively indicate additive changes + - Each Distributable artifact MUST contain the label io.buildpacks.distribution.api denoting the distribution API ## Artifact Format @@ -43,29 +44,212 @@ Each buildpack layer blob MUST contain a single buildpack at the following file /cnb/buildpacks/// ``` +If the buildpack ID contains a `/`, it MUST be replaced with `_` in the directory name. + +A buildpack MUST contain a `buildpack.toml` file at its root directory. + + A buildpack ID, buildpack version, and at least one stack MUST be provided in the OCI image config as a Label. -Label: `io.buildpacks.buildpackage.metadata` +#### Labels + +The following labels MUST be set in the buildpack image(through the image config's `Labels` field): + +| Label | Description | +| ------------------------------------ | ------------------------------------------------------------------------------------------------- | +| `io.buildpacks.buildpack.metadata` | A JSON object representing information about the packaged entrypoint buildpack | +| `io.buildpacks.buildpack.layers` | A JSON object representing information about all packaged buildpacks and their associated layers. | + +##### `io.buildpacks.buildpack.metadata` (JSON) + ```json { - "id": "", - "version": "", - "stacks": [ - { - "id": "", - "mixins": [""] + "id": "", + "name": "", + "version": "", + "homepage": "", +} +``` + +##### `io.buildpacks.buildpack.layers` (JSON) + +```json +{ + # schema of a meta-buildpack + "": { + "": { + "api": "", + "order": [ + { + "group": [ + { + "id": "", + "version": "" + } + ] + } + ], + "layerDiffID": "", + "homepage": "", + "name": "" + } + }, + # schema of a regular buildpack + "": { + "": { + "api": "", + "layerDiffID": "", + "homepage": "", + "name": "" } - ] + } } ``` +For each buildpack layer, the buildpack ID and the buildpack version MUST be provided in `io.buildpacks.buildpack.layers`. + The buildpack ID and version MUST match a buildpack provided by a layer blob. -For a buildpackage to be valid, each `buildpack.toml` describing a buildpack implementation MUST have all listed stacks. +For a buildpackage to be valid, each `buildpack.toml` describing a buildpack implementation MUST have all listed targets. + +### Builder + +The following specifies the artifact format for a buildpacks builder. + +A builder is an OCI Image that provides a distributable build environment. + +A platform supporting builders SHOULD allow users to configure the build environment for a provided builder. + +#### General Requirements + +The builder image MUST contain an implementation of the [lifecycle](#lifecycle), and a [build-time](#build-image) environment, and MAY contain [buildpacks](#buildpackage). Platforms SHOULD use builders to ease the build process. + +#### Filesystem + +A builder MUST have the following directories/files: + +* `/cnb/order.toml` → As defined in the [platform specification](https://github.com/buildpacks/spec/blob/main/platform.md#ordertoml-toml). + +#### Environment Variables + +A builder MUST be an extension of a Build Image, and MUST retain all the specified environment variables set on the original build image, as specified in the Build Image specifications. + +#### Labels -For each listed stack, all associated buildpacks MUST be a candidate for detection when the entrypoint buildpack ID and version are selected. +A builder MUST be an extension of a Build Image, and MUST retain all the specified Labels set on the original build image, as specified in the Build Image specifications. + +A builder image MUST contain an implementation of the [lifecycle](#lifecycle), and MUST retain all the specified Labels set on the original Lifecycle image, as specified in the lifecycle distribution specifications. + +A builder image MAY contain buildpacks, and MAY retain all the specified Labels set on the original buildpackage, as specified in the [buildpackage](#buildpackage) specification with the following exceptions: + +- `io.buildpacks.buildpack.metadata` MUST not be set. +- `io.buildpacks.buildpack.layers` on the builder SHOULD be a merged version based on all buildpackages combined and thereby have of all packaged buildpacks represented. + +In addition to all inherited labels above, the following labels MUST be set in the builder environment (through the image config's `Labels` field): + +| Label | Description | +| -------------------------------- | ------------------------------------------------------------- | +| `io.buildpacks.builder.metadata` | A JSON object representing builder metadata. | +| `io.buildpacks.buildpack.order` | A JSON object representation of the `/cnb/order.toml` file. | + +##### `io.buildpacks.builder.metadata` (JSON) + +```json +{ + "description": "", + "createdBy": { + "name": "", + "version": "", + } +} +``` + +Where: + +- `description` (optional) is a human readable description of the builder. +- `createdBy` (optional) is information about the tool that created the builder. + - `name` (optional) is the name of the tool that created the builder. + - `version` (optional) is the version of the tool that created the builder. + +## Data Format + +### Files + +#### `builder.toml` (TOML) +```toml +description = "a human readable description output when you run `pack inspect-builder`" + +[[buildpacks]] +id = "" +version = "" +uri = "" + +[[order]] +[[order.group]] +id = "" +version = "" +optional = false + +[stack] +id = "" +build-image = "build image" +run-image = "run image" +run-image-mirrors = "mirrors" + +[build] +image = "build image" + +[run] +[[run.images]] +image = "run image" +mirrors = "mirrors" + +``` + +Where: +- buildpacks list MAY be provided. + - buildpacks.id and .version are optional but if provided MUST match an available buildpack in its buildpack.toml file. + - buildpacks.uri MUST be provided. +- order list MUST be provided and MUST have at least one group list + - order.group.id + - MUST be provided + - SHOULD match an available buildpack in the buildpacks list + - MUST be unique within a group + - MAY be repeated within the order + - order.group.version (optional) MAY be omitted if the ID alone is sufficient to identify a unique buildpack. + - order.group.optional (optional) MUST be a boolean value specifying whether or not this buildpack is optional during detection. +- stack MAY be provided to platforms >= 0.12, but is deprecated in favor of `build` and `run` (see below). MUST be provided to platforms <= 0.11. + - build-image MUST be included in a stack + - run-image MUST be included + - run-image-mirrors MAY be included +- build SHOULD be provided to platforms >= 0.12, and MUST include an image +- run SHOULD be provided to platforms >= 0.12 + - run MUST have at least one `images` entry + - each `run.images` MUST include an image + - each `run.images` MAY include a list of mirrors (in the format `["mirror url", "mirror, url", ...]`) +- either stack or build and run images MUST be provided. + + +#### `package.toml` (TOML) + +```(toml) +[buildpack] +uri = "uri" + +[[buildpacks]] +uri = "uri" + +[platform] +os = "operating system" + +[[dependencies]] +uri = "uri" +image = "image uri" + +``` -Each stack ID MUST only be present once. -For a given stack, the `mixins` list MUST enumerate mixins such that no included buildpacks are missing a mixin for the stack. +Where: +- buildpack or buildpacks MUST be present, and MUST include a URI, which MAY be a relative or absolute path, or URL. +- platform MAY be provided to specify the operating system compatibility. If provided, `os` SHOULD be either `linux` (default) or `windows` +- dependencies list MAY be provided to specify URIs of order groups within a meta-buildpack. -Fewer stack entries as well as additional mixins for a stack entry MAY be specified. From 4d38d11e4526116e5f88787b9a3abe1835adbea2 Mon Sep 17 00:00:00 2001 From: Joe Kimmel Date: Mon, 26 Jun 2023 15:22:16 -0700 Subject: [PATCH 02/11] adds lifecycle to builder.toml Signed-off-by: Joe Kimmel --- distribution.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/distribution.md b/distribution.md index c5a37718..e7eab8d6 100644 --- a/distribution.md +++ b/distribution.md @@ -204,6 +204,10 @@ image = "build image" image = "run image" mirrors = "mirrors" +[lifecycle] +uri = "uri" +version = "ve.rs.ion" + ``` Where: @@ -228,6 +232,9 @@ Where: - each `run.images` MUST include an image - each `run.images` MAY include a list of mirrors (in the format `["mirror url", "mirror, url", ...]`) - either stack or build and run images MUST be provided. +- lifecyle MAY be provided to specify a custom lifecycle - either uri or version (but not both) MUST be provided + - uri MUST be a locator for a valid lifecycle image (mutually exclusive with version) + - version MUST be a valid semver string (mutually exclusive with uri) #### `package.toml` (TOML) From 93bd1cfec5f2e9f403f635ff922d81f57d9d0cb6 Mon Sep 17 00:00:00 2001 From: Joe Kimmel <86852107+joe-kimmel-vmw@users.noreply.github.com> Date: Thu, 13 Jul 2023 11:58:27 -0700 Subject: [PATCH 03/11] Update distribution.md Co-authored-by: Joe Kutner Signed-off-by: Joe Kimmel <86852107+joe-kimmel-vmw@users.noreply.github.com> --- distribution.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/distribution.md b/distribution.md index e7eab8d6..78cecc00 100644 --- a/distribution.md +++ b/distribution.md @@ -20,7 +20,7 @@ This document specifies Distribution API version `0.4`. Distribution API versions: - MUST be in form `.` or ``, where `` is equivalent to `.0` - When `` is greater than `0` increments to `` SHALL exclusively indicate additive changes - - Each Distributable artifact MUST contain the label io.buildpacks.distribution.api denoting the distribution API + - Each Distributable artifact MUST contain the label `io.buildpacks.distribution.api` denoting the distribution API ## Artifact Format From 38c0eb3e9633b64a27f8c9176723ef1b6f9affa5 Mon Sep 17 00:00:00 2001 From: Joe Kimmel <86852107+joe-kimmel-vmw@users.noreply.github.com> Date: Fri, 14 Jul 2023 10:11:42 -0700 Subject: [PATCH 04/11] Update distribution.md Co-authored-by: Joe Kutner Signed-off-by: Joe Kimmel <86852107+joe-kimmel-vmw@users.noreply.github.com> --- distribution.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/distribution.md b/distribution.md index 78cecc00..10d08938 100644 --- a/distribution.md +++ b/distribution.md @@ -143,7 +143,7 @@ A builder image MUST contain an implementation of the [lifecycle](#lifecycle), a A builder image MAY contain buildpacks, and MAY retain all the specified Labels set on the original buildpackage, as specified in the [buildpackage](#buildpackage) specification with the following exceptions: - `io.buildpacks.buildpack.metadata` MUST not be set. -- `io.buildpacks.buildpack.layers` on the builder SHOULD be a merged version based on all buildpackages combined and thereby have of all packaged buildpacks represented. +- `io.buildpacks.buildpack.layers` on the builder MUST be a merged version based on all buildpackages combined and thereby have of all packaged buildpacks represented. In addition to all inherited labels above, the following labels MUST be set in the builder environment (through the image config's `Labels` field): From e5bb1f2df6e533ac8641734e0d2f70df6b8a0343 Mon Sep 17 00:00:00 2001 From: Joe Kimmel <86852107+joe-kimmel-vmw@users.noreply.github.com> Date: Fri, 14 Jul 2023 10:12:21 -0700 Subject: [PATCH 05/11] Update distribution.md Co-authored-by: Joe Kutner Signed-off-by: Joe Kimmel <86852107+joe-kimmel-vmw@users.noreply.github.com> --- distribution.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/distribution.md b/distribution.md index 10d08938..26607aac 100644 --- a/distribution.md +++ b/distribution.md @@ -122,7 +122,7 @@ A platform supporting builders SHOULD allow users to configure the build environ #### General Requirements -The builder image MUST contain an implementation of the [lifecycle](#lifecycle), and a [build-time](#build-image) environment, and MAY contain [buildpacks](#buildpackage). Platforms SHOULD use builders to ease the build process. +The builder image MUST contain an implementation of the [lifecycle](#lifecycle), and a [build-time](#build-image) environment, and MAY contain [buildpacks](#buildpackage). Platforms SHOULD use builders to encapsulate the build process. #### Filesystem From 7d2facbd6ab85be6aa1b0edb468de97fbd2403a0 Mon Sep 17 00:00:00 2001 From: Joe Kimmel <86852107+joe-kimmel-vmw@users.noreply.github.com> Date: Wed, 19 Jul 2023 11:57:39 -0700 Subject: [PATCH 06/11] Update distribution.md Co-authored-by: Joe Kutner Signed-off-by: Joe Kimmel <86852107+joe-kimmel-vmw@users.noreply.github.com> --- distribution.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/distribution.md b/distribution.md index 26607aac..cc3828cd 100644 --- a/distribution.md +++ b/distribution.md @@ -138,7 +138,7 @@ A builder MUST be an extension of a Build Image, and MUST retain all the specifi A builder MUST be an extension of a Build Image, and MUST retain all the specified Labels set on the original build image, as specified in the Build Image specifications. -A builder image MUST contain an implementation of the [lifecycle](#lifecycle), and MUST retain all the specified Labels set on the original Lifecycle image, as specified in the lifecycle distribution specifications. +A builder MUST contain an implementation of the [lifecycle](#lifecycle), and MUST retain all the specified Labels set on the original Lifecycle image, as specified in the lifecycle distribution specifications. A builder image MAY contain buildpacks, and MAY retain all the specified Labels set on the original buildpackage, as specified in the [buildpackage](#buildpackage) specification with the following exceptions: From 064aab5cf59c61be4f5cf1cea0df2941970915d4 Mon Sep 17 00:00:00 2001 From: Joe Kimmel Date: Wed, 19 Jul 2023 11:58:32 -0700 Subject: [PATCH 07/11] remove other occurances of builder image so that we always call it builder Signed-off-by: Joe Kimmel --- distribution.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/distribution.md b/distribution.md index cc3828cd..d50ed050 100644 --- a/distribution.md +++ b/distribution.md @@ -122,7 +122,7 @@ A platform supporting builders SHOULD allow users to configure the build environ #### General Requirements -The builder image MUST contain an implementation of the [lifecycle](#lifecycle), and a [build-time](#build-image) environment, and MAY contain [buildpacks](#buildpackage). Platforms SHOULD use builders to encapsulate the build process. +The builder MUST contain an implementation of the [lifecycle](#lifecycle), and a [build-time](#build-image) environment, and MAY contain [buildpacks](#buildpackage). Platforms SHOULD use builders to encapsulate the build process. #### Filesystem @@ -140,7 +140,7 @@ A builder MUST be an extension of a Build Image, and MUST retain all the specifi A builder MUST contain an implementation of the [lifecycle](#lifecycle), and MUST retain all the specified Labels set on the original Lifecycle image, as specified in the lifecycle distribution specifications. -A builder image MAY contain buildpacks, and MAY retain all the specified Labels set on the original buildpackage, as specified in the [buildpackage](#buildpackage) specification with the following exceptions: +A builder MAY contain buildpacks, and MAY retain all the specified Labels set on the original buildpackage, as specified in the [buildpackage](#buildpackage) specification with the following exceptions: - `io.buildpacks.buildpack.metadata` MUST not be set. - `io.buildpacks.buildpack.layers` on the builder MUST be a merged version based on all buildpackages combined and thereby have of all packaged buildpacks represented. From 3d5c8a1db38b6e810116286b40ca26936605b18e Mon Sep 17 00:00:00 2001 From: Joe Kimmel Date: Wed, 19 Jul 2023 12:47:08 -0700 Subject: [PATCH 08/11] move stack example to the section that marks it as deprecated Signed-off-by: Joe Kimmel --- distribution.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/distribution.md b/distribution.md index d50ed050..ce79aa72 100644 --- a/distribution.md +++ b/distribution.md @@ -190,12 +190,6 @@ id = "" version = "" optional = false -[stack] -id = "" -build-image = "build image" -run-image = "run image" -run-image-mirrors = "mirrors" - [build] image = "build image" @@ -226,6 +220,14 @@ Where: - build-image MUST be included in a stack - run-image MUST be included - run-image-mirrors MAY be included + - Deprecated stack example: +``` +[stack] +id = "" +build-image = "build image" +run-image = "run image" +run-image-mirrors = "mirrors" +``` - build SHOULD be provided to platforms >= 0.12, and MUST include an image - run SHOULD be provided to platforms >= 0.12 - run MUST have at least one `images` entry From e05337397b421f3207713cf35be56c6d417fa24c Mon Sep 17 00:00:00 2001 From: Joe Kimmel Date: Wed, 19 Jul 2023 12:58:32 -0700 Subject: [PATCH 09/11] now the builder spec mentions the builder.toml file. Signed-off-by: Joe Kimmel --- distribution.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/distribution.md b/distribution.md index ce79aa72..850f83e1 100644 --- a/distribution.md +++ b/distribution.md @@ -120,6 +120,8 @@ A builder is an OCI Image that provides a distributable build environment. A platform supporting builders SHOULD allow users to configure the build environment for a provided builder. +A platform supporting builder creation MUST be able to create a valid builder from a valid builder.toml file. + #### General Requirements The builder MUST contain an implementation of the [lifecycle](#lifecycle), and a [build-time](#build-image) environment, and MAY contain [buildpacks](#buildpackage). Platforms SHOULD use builders to encapsulate the build process. From 2face0841fafee078329234cae3eb56ba9dce4b2 Mon Sep 17 00:00:00 2001 From: Joe Kimmel Date: Wed, 19 Jul 2023 15:36:59 -0700 Subject: [PATCH 10/11] links and names for builder.toml Signed-off-by: Joe Kimmel --- distribution.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/distribution.md b/distribution.md index 850f83e1..c80be6fd 100644 --- a/distribution.md +++ b/distribution.md @@ -120,7 +120,7 @@ A builder is an OCI Image that provides a distributable build environment. A platform supporting builders SHOULD allow users to configure the build environment for a provided builder. -A platform supporting builder creation MUST be able to create a valid builder from a valid builder.toml file. +A platform supporting builder creation MUST be able to create a valid builder from a valid [builder.toml](#builder.toml) file. (NOTE: much like (roses by any other name)[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC543212], the platform MUST parse the contents of a builder-specifying toml file regardless of how it's called.) #### General Requirements From ac752ecb405d6e4b34e346087abd24e143da5a01 Mon Sep 17 00:00:00 2001 From: Joe Kimmel <86852107+joe-kimmel-vmw@users.noreply.github.com> Date: Wed, 19 Jul 2023 15:42:22 -0700 Subject: [PATCH 11/11] fix links in builder section Signed-off-by: Joe Kimmel <86852107+joe-kimmel-vmw@users.noreply.github.com> --- distribution.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/distribution.md b/distribution.md index c80be6fd..312dcb91 100644 --- a/distribution.md +++ b/distribution.md @@ -120,7 +120,7 @@ A builder is an OCI Image that provides a distributable build environment. A platform supporting builders SHOULD allow users to configure the build environment for a provided builder. -A platform supporting builder creation MUST be able to create a valid builder from a valid [builder.toml](#builder.toml) file. (NOTE: much like (roses by any other name)[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC543212], the platform MUST parse the contents of a builder-specifying toml file regardless of how it's called.) +A platform supporting builder creation MUST be able to create a valid builder from a valid [builder.toml](#files) file. (NOTE: much like [roses by any other name](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC543212), the platform MUST parse the contents of a builder-specifying toml file regardless of how it's called.) #### General Requirements