Support --from=$var syntax #2717

jorgemoralespou · 2022-02-21T12:01:32Z

Description
Would want to have the image used in --from in commands like COPY to support variable replacement to support something like this:

FROM base-image as tools-0.10.0
....
....

#---------------
ARG VERSION
FROM base-image

ENV VERSION=${VERSION:-0.10.0}

COPY --from=tools-${VERSION} --chown=1001:0 /home/tools /home/tools

Tried with all variations of Buildkit syntax currently publicly available

Follow up to this issue: moby/moby#18119

Steps to reproduce the issue:

Create this Dockerfile

# syntax=docker/dockerfile:1.3.1
FROM ubuntu:22.04 as tools-0.0.1
RUN echo "Hello" > /tmp/hello.msg
 
#---------------
ARG VERSION
FROM base-image
 
ENV VERSION=${VERSION:-0.0.1}
 
COPY --from=tools-${VERSION} --chown=1001:0 /tmp/hello.msg /tmp/hello.msg

Build with: docker build -t "test-bug" . or docker buildx build -t "test" .

Describe the results you received:
Currently fails with:

failed to solve with frontend dockerfile.v0: failed to create LLB definition: failed to parse stage name "tools-${VERSION}": invalid reference format: repository name must be lowercase

Describe the results you expected:
Succesful Build

Output of docker version:

docker version
Client:
 Cloud integration: v1.0.22
 Version:           20.10.12
 API version:       1.41
 Go version:        go1.16.12
 Git commit:        e91ed57
 Built:             Mon Dec 13 11:46:56 2021
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: Docker Desktop 4.5.0 (74289)
 Engine:
  Version:          20.10.12
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.12
  Git commit:       459d0df
  Built:            Mon Dec 13 11:43:56 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.12
  GitCommit:        7b11cfaabd73bb80907dd23182b9347b4245eb5d
 runc:
  Version:          1.0.2
  GitCommit:        v1.0.2-0-g52b36a2
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker info:

docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.7.1)
  compose: Docker Compose (Docker Inc., v2.2.3)
  extension: Manages Docker extensions (Docker Inc., v0.0.1-alpha.6)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 6
  Running: 4
  Paused: 0
  Stopped: 2
 Images: 24
 Server Version: 20.10.12
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7b11cfaabd73bb80907dd23182b9347b4245eb5d
 runc version: v1.0.2-0-g52b36a2
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.10.76-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 6
 Total Memory: 9.73GiB
 Name: docker-desktop
 ID: XHTR:5GC7:DH7Y:DOKS:THT4:XTLO:CDWK:RLSI:3BXB:OOHD:FJMY:G2JS
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.):
Docker for Mac

The text was updated successfully, but these errors were encountered:

cpuguy83 · 2022-02-21T15:58:08Z

As mentioned in the other issues, this should already be supported but you need the latest buildkit.
"docker buildx create --use", then do your build "docker buildx build".

jorgemoralespou · 2022-02-21T19:13:50Z

@cpuguy83 I might be missing something:

❯ docker buildx create --use
agitated_gauss
❯ docker buildx build -t "test" .
WARN[0000] No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
[+] Building 12.1s (6/6) FINISHED
 => [internal] booting buildkit                                                                                                        8.0s
 => => pulling image moby/buildkit:buildx-stable-1                                                                                     6.6s
 => => creating container buildx_buildkit_agitated_gauss0                                                                              1.4s
 => [internal] load build definition from Dockerfile                                                                                   0.1s
 => => transferring dockerfile: 304B                                                                                                   0.0s
 => [internal] load .dockerignore                                                                                                      0.0s
 => => transferring context: 2B                                                                                                        0.0s
 => resolve image config for docker.io/docker/dockerfile:1.3.1                                                                         2.0s
 => [auth] docker/dockerfile:pull token for registry-1.docker.io                                                                       0.0s
 => docker-image://docker.io/docker/dockerfile:1.3.1@sha256:42399d4635eddd7a9b8a24be879d2f9a930d0ed040a61324cfdf59ef1357b3b2           0.7s
 => => resolve docker.io/docker/dockerfile:1.3.1@sha256:42399d4635eddd7a9b8a24be879d2f9a930d0ed040a61324cfdf59ef1357b3b2               0.0s
 => => sha256:24a639a53085eb680e1d11618ac62f3977a3926fedf5b8471ace519b8c778030 9.67MB / 9.67MB                                         0.4s
 => => extracting sha256:24a639a53085eb680e1d11618ac62f3977a3926fedf5b8471ace519b8c778030                                              0.3s
Dockerfile:11
--------------------
   9 |     ENV VERSION=${VERSION:-0.0.1}
  10 |
  11 | >>> COPY --from=tools-${VERSION} --chown=1001:0 /tmp/hello.msg /tmp/hello.msg
  12 |
  13 |
--------------------
error: failed to solve: failed to parse stage name "tools-${VERSION}": invalid reference format: repository name must be lowercase

And AFAIK using latest buildkit:

 docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.7.1)
  compose: Docker Compose (Docker Inc., v2.2.3)
  extension: Manages Docker extensions (Docker Inc., v0.0.1-alpha.6)
  scan: Docker Scan (Docker Inc., v0.17.0)

jorgemoralespou · 2022-03-01T14:02:34Z

Just found this note on this Docker blog

Values for [RUN --mount flag](https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/syntax.md#build-mounts-run---mount) now support variable expansion, except for the from field:

I'm assuming this might be the case for the COPY command as well. This is with 1.3.0 syntax.

ndeloof · 2022-03-08T14:20:58Z

@thaJeztah can you please transfert this issue to https://github.com/moby/buildkit ?

thaJeztah added area/builder labels Mar 9, 2022

thaJeztah transferred this issue from moby/moby Mar 9, 2022

AkihiroSuda added kind/enhancement and removed kind/enhancement labels Jun 16, 2022

crazy-max removed area/builder labels Sep 13, 2022

jedevc linked a pull request Jan 30, 2023 that will close this issue

Allow ARGs in COPY --from and RUN --mount=from #3551

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support --from=$var syntax #2717

Support --from=$var syntax #2717

jorgemoralespou commented Feb 21, 2022 •

edited

Loading

cpuguy83 commented Feb 21, 2022

jorgemoralespou commented Feb 21, 2022

jorgemoralespou commented Mar 1, 2022

ndeloof commented Mar 8, 2022

Support --from=$var syntax #2717

Support --from=$var syntax #2717

Comments

jorgemoralespou commented Feb 21, 2022 • edited Loading

cpuguy83 commented Feb 21, 2022

jorgemoralespou commented Feb 21, 2022

jorgemoralespou commented Mar 1, 2022

ndeloof commented Mar 8, 2022

jorgemoralespou commented Feb 21, 2022 •

edited

Loading