From 45e3cb538d6e5af2b8dab83a271b88b520ea0fb3 Mon Sep 17 00:00:00 2001 From: Halil Date: Mon, 7 Jun 2021 17:36:39 +0300 Subject: [PATCH 1/5] DRAFT prepare-phase from old commit. Signed-off-by: Halil --- text/0000-prepare-phase.md | 105 +++++++++++++++++++++++++++++++++++++ 1 file changed, 105 insertions(+) create mode 100644 text/0000-prepare-phase.md diff --git a/text/0000-prepare-phase.md b/text/0000-prepare-phase.md new file mode 100644 index 000000000..1b1b4bfd9 --- /dev/null +++ b/text/0000-prepare-phase.md @@ -0,0 +1,105 @@ +# Meta +[meta]: #meta +- Name: Prepare Phase +- Start Date: 2021-01-13 +- Author(s): [@jkutner](github.com/jkutner/) +- RFC Pull Request: (leave blank) +- CNB Pull Request: [buildpacks/spec#176](https://github.com/buildpacks/spec/pull/176) +- CNB Issue: (leave blank) +- Supersedes: N/A + +# Summary +[summary]: #summary + +This is a proposal for a new Lifecycle phase, called "prepare", that would run before all other phases. + +# Definitions +[definitions]: #definitions + +* __stack descriptor__ - an enhanced version of the [`stack.toml`](https://github.com/buildpacks/spec/blob/main/platform.md#stacktoml-toml) +* __project descriptor__ - the [`project.toml`](https://github.com/buildpacks/spec/blob/main/extensions/project-descriptor.md) extension specification + +# Motivation +[motivation]: #motivation + +The prepare phase would support the following features and capabilities: +* [Stack buildpacks](https://github.com/buildpacks/rfcs/pull/111), which require a phase to read run-image mixins validation prior to detection +* [Inline buildpacks](https://github.com/buildpacks/rfcs/blob/main/text/0048-inline-buildpack.md), which require parsing of the `project.toml` in the lifecycle +* [Lifecycle configuration](https://github.com/buildpacks/rfcs/pull/128) + +# What it is +[what-it-is]: #what-it-is + +The prepare phase will run before all other phases, and prepare the execution environment for a buildpack build. This phase will have access to secrets and credentials used to access registries and other services. + +## Responsibilities + +* Stack validation, to ensure that a new run-image is campatible with the previous app image +* Retrive run-image mixins, which will be use dby subsequent phases +* Validation of registry credentials, to avoid a long build that fails during export phase +* Parsing the project descriptor and performance various operations based on its contents, include: + - downloading buildpacks + - creating ephemeral buildpacks + - applying include and exclude rules + - modifying environment variables + +## Inputs + +* Log level +* Run-image +* Stack ID +* Project descriptor (optional) +* App source code + +## Output + +* Exit status +* Info-level logs to `stdout` +* Error-level logs to `stderr` +* Stack descriptor +* Analysis metadata +* Buildpacks (derived from inline buildpacks in project descriptor, or buildpacks in project descriptor that are not present in the builder) +* Order definition (derived from buildpacks configuration in project descriptor) +* Lifecycle configuration (derived from configuration in project descriptor) +* Mutated app source code (applying include and exclude rules in project descriptor) + +# How it Works +[how-it-works]: #how-it-works + +The prepare phase would be implemented as the `/cnb/lifecycle/preparer` binary. A platform MUST execute this phases either by invoking the `/cnb/lifecycle/preparer` binary or by executing `/cnb/lifecycle/creator`. + +The `preparer` binary will have access to the [`Keychain`](https://github.com/buildpacks/lifecycle/blob/main/auth/env_keychain.go), and MUST NOT execute arbitrary code provided by either the buildpack user or buildpack author. + +The [logic in the `analyzer` phase that reads image metadata and outputs an `analyzed.toml`](https://github.com/buildpacks/lifecycle/blob/main/analyzer.go#L34-L40) would be moved into the `preparer`. + +The [logic in `pack` that parses a `project.toml`](https://github.com/buildpacks/pack/blob/main/project/project.go) would be copied or moved into the `preparer`. + +The app source code (which may be provided to the prepare either as a directory, volume, or tarball) would be mutated (either by copying it to a new location, or making changes directory against it). The `preparer` may delete files to apply the include and exclude rules from `project.toml`. + +# Drawbacks +[drawbacks]: #drawbacks + +* Yet another lifecycle phase + +# Alternatives +[alternatives]: #alternatives + +- [Reverse the order of analyze and detect phases](https://github.com/buildpacks/spec/pull/172) +- Split analyze into two parts (one to write `analyzed.toml` and one to analyze layers). The first (`analyzed.toml`) would be moved before the detect phase, and the second part (analyzing layers) would be merged into the restore phase. + +# Prior Art +[prior-art]: #prior-art + +- [Tekton prepare step](https://github.com/tektoncd/catalog/blob/11a17cfe87779099b0b61be3f1e496dfa79646b3/task/buildpacks-phases/0.1/buildpacks-phases.yaml#L61-L78) + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +- Do we still need `analyzer`, or can the remaining parts of analyze phase be rolled into restore phase? +- Does `pack` still need to parse `project.toml`, or is there value in reading it early on (before lifecycle runs)? +- Should we create a shared library for `project.toml` parsing? + +# Spec. Changes +[spec-changes]: #spec-changes + +See [buildpacks/spec PR #176](https://github.com/buildpacks/spec/pull/176) From 7c6f5727d55adc694a05b29ac9d832c55e2b6023 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Halil=20=C4=B0brahim=20ceylan?= Date: Mon, 7 Jun 2021 21:23:18 +0000 Subject: [PATCH 2/5] Update 0000-prepare-phase.md --- text/0000-prepare-phase.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/text/0000-prepare-phase.md b/text/0000-prepare-phase.md index 1b1b4bfd9..169bc3e73 100644 --- a/text/0000-prepare-phase.md +++ b/text/0000-prepare-phase.md @@ -1,8 +1,8 @@ # Meta [meta]: #meta - Name: Prepare Phase -- Start Date: 2021-01-13 -- Author(s): [@jkutner](github.com/jkutner/) +- Start Date: 2021-06-08 +- Author(s): [@jkutner](github.com/jkutner/) & [@natalieparellano](github.com/natalieparellano/) & [@haliliceylan](github.com/haliliceylan/) - RFC Pull Request: (leave blank) - CNB Pull Request: [buildpacks/spec#176](https://github.com/buildpacks/spec/pull/176) - CNB Issue: (leave blank) @@ -22,6 +22,8 @@ This is a proposal for a new Lifecycle phase, called "prepare", that would run b # Motivation [motivation]: #motivation +Parity with Pack: A Lifecycle Prepare phase should make it easier for Platform Implementers to achieve parity with features of Pack. Today, features like project.toml are only supported by Pack, and a new platform would need to write it’s own parser. + The prepare phase would support the following features and capabilities: * [Stack buildpacks](https://github.com/buildpacks/rfcs/pull/111), which require a phase to read run-image mixins validation prior to detection * [Inline buildpacks](https://github.com/buildpacks/rfcs/blob/main/text/0048-inline-buildpack.md), which require parsing of the `project.toml` in the lifecycle @@ -29,8 +31,7 @@ The prepare phase would support the following features and capabilities: # What it is [what-it-is]: #what-it-is - -The prepare phase will run before all other phases, and prepare the execution environment for a buildpack build. This phase will have access to secrets and credentials used to access registries and other services. +A new Lifecycle phase and associated binary should be available to Platform Implementers, and should be executed by Pack. The prepare phase will run before all other phases, and prepare the execution environment for a buildpack build. This phase will have access to secrets and credentials used to access registries and other services. ## Responsibilities From 637212d7b19f563b80ffbeab2b127de13a89b80b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Halil=20=C4=B0brahim=20ceylan?= Date: Tue, 8 Jun 2021 00:26:10 +0300 Subject: [PATCH 3/5] delete stack buildpacks Co-authored-by: Natalie Arellano --- text/0000-prepare-phase.md | 1 - 1 file changed, 1 deletion(-) diff --git a/text/0000-prepare-phase.md b/text/0000-prepare-phase.md index 169bc3e73..7ead8d100 100644 --- a/text/0000-prepare-phase.md +++ b/text/0000-prepare-phase.md @@ -25,7 +25,6 @@ This is a proposal for a new Lifecycle phase, called "prepare", that would run b Parity with Pack: A Lifecycle Prepare phase should make it easier for Platform Implementers to achieve parity with features of Pack. Today, features like project.toml are only supported by Pack, and a new platform would need to write it’s own parser. The prepare phase would support the following features and capabilities: -* [Stack buildpacks](https://github.com/buildpacks/rfcs/pull/111), which require a phase to read run-image mixins validation prior to detection * [Inline buildpacks](https://github.com/buildpacks/rfcs/blob/main/text/0048-inline-buildpack.md), which require parsing of the `project.toml` in the lifecycle * [Lifecycle configuration](https://github.com/buildpacks/rfcs/pull/128) From 01478e2b8401e3cd5bab0863b1b6196faa8b990c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Halil=20=C4=B0brahim=20ceylan?= Date: Tue, 8 Jun 2021 00:36:35 +0300 Subject: [PATCH 4/5] delete old information which is not valid anymore Co-authored-by: Natalie Arellano --- text/0000-prepare-phase.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/text/0000-prepare-phase.md b/text/0000-prepare-phase.md index 7ead8d100..776b71e6f 100644 --- a/text/0000-prepare-phase.md +++ b/text/0000-prepare-phase.md @@ -34,9 +34,6 @@ A new Lifecycle phase and associated binary should be available to Platform Impl ## Responsibilities -* Stack validation, to ensure that a new run-image is campatible with the previous app image -* Retrive run-image mixins, which will be use dby subsequent phases -* Validation of registry credentials, to avoid a long build that fails during export phase * Parsing the project descriptor and performance various operations based on its contents, include: - downloading buildpacks - creating ephemeral buildpacks From 2152fc5c817d971b6ead2069d82c459f432a7acc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Halil=20=C4=B0brahim=20ceylan?= Date: Mon, 14 Jun 2021 23:59:27 +0300 Subject: [PATCH 5/5] Apply suggestions from code review Co-authored-by: Natalie Arellano --- text/0000-prepare-phase.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/text/0000-prepare-phase.md b/text/0000-prepare-phase.md index 776b71e6f..40d813961 100644 --- a/text/0000-prepare-phase.md +++ b/text/0000-prepare-phase.md @@ -22,7 +22,7 @@ This is a proposal for a new Lifecycle phase, called "prepare", that would run b # Motivation [motivation]: #motivation -Parity with Pack: A Lifecycle Prepare phase should make it easier for Platform Implementers to achieve parity with features of Pack. Today, features like project.toml are only supported by Pack, and a new platform would need to write it’s own parser. +Parity with Pack: A Lifecycle Prepare phase should make it easier for Platform Implementers to achieve parity with features of Pack. Today, features like project.toml are only supported by Pack, and a new platform would need to write its own parser. The prepare phase would support the following features and capabilities: * [Inline buildpacks](https://github.com/buildpacks/rfcs/blob/main/text/0048-inline-buildpack.md), which require parsing of the `project.toml` in the lifecycle @@ -63,12 +63,10 @@ A new Lifecycle phase and associated binary should be available to Platform Impl # How it Works [how-it-works]: #how-it-works -The prepare phase would be implemented as the `/cnb/lifecycle/preparer` binary. A platform MUST execute this phases either by invoking the `/cnb/lifecycle/preparer` binary or by executing `/cnb/lifecycle/creator`. +The prepare phase would be implemented as the `/cnb/lifecycle/preparer` binary. A platform MAY execute this phases either by invoking the `/cnb/lifecycle/preparer` binary or by executing `/cnb/lifecycle/creator`. The `preparer` binary will have access to the [`Keychain`](https://github.com/buildpacks/lifecycle/blob/main/auth/env_keychain.go), and MUST NOT execute arbitrary code provided by either the buildpack user or buildpack author. -The [logic in the `analyzer` phase that reads image metadata and outputs an `analyzed.toml`](https://github.com/buildpacks/lifecycle/blob/main/analyzer.go#L34-L40) would be moved into the `preparer`. - The [logic in `pack` that parses a `project.toml`](https://github.com/buildpacks/pack/blob/main/project/project.go) would be copied or moved into the `preparer`. The app source code (which may be provided to the prepare either as a directory, volume, or tarball) would be mutated (either by copying it to a new location, or making changes directory against it). The `preparer` may delete files to apply the include and exclude rules from `project.toml`.