Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace Docker-in-Docker setup with multi-stage build. #86

Merged
merged 9 commits into from
Dec 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 14 additions & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,3 +1,11 @@
FROM node:16-buster

WORKDIR /workspace
COPY . /workspace

RUN npm install
RUN npm run test

FROM nginx:stable

COPY nginx.conf /etc/nginx/nginx.conf
Expand All @@ -8,14 +16,14 @@ COPY resources /usr/share/nginx/html/resources

# Copy third party javascript from npm modules
ENV THIRDPARTY_JS_PATH /usr/share/nginx/html/resources/js/third_party/
COPY node_modules/pako/dist/pako.min.js $THIRDPARTY_JS_PATH
COPY node_modules/vue/dist/vue.min.js $THIRDPARTY_JS_PATH
COPY node_modules/jwt-decode/build/jwt-decode.min.js $THIRDPARTY_JS_PATH
COPY node_modules/jquery/dist/jquery.min.js $THIRDPARTY_JS_PATH
COPY --from=0 /workspace/node_modules/pako/dist/pako.min.js $THIRDPARTY_JS_PATH
COPY --from=0 /workspace/node_modules/vue/dist/vue.min.js $THIRDPARTY_JS_PATH
COPY --from=0 /workspace/node_modules/jwt-decode/build/jwt-decode.min.js $THIRDPARTY_JS_PATH
COPY --from=0 /workspace/node_modules/jquery/dist/jquery.min.js $THIRDPARTY_JS_PATH

# Copy third party css from npm modules
COPY node_modules/bootstrap/dist/css/bootstrap.min.css /usr/share/nginx/html/resources/css/third_party/
COPY node_modules/bootstrap/dist/css/bootstrap.min.css.map /usr/share/nginx/html/resources/css/third_party/
COPY --from=0 /workspace/node_modules/bootstrap/dist/css/bootstrap.min.css /usr/share/nginx/html/resources/css/third_party/
COPY --from=0 /workspace/node_modules/bootstrap/dist/css/bootstrap.min.css.map /usr/share/nginx/html/resources/css/third_party/

RUN rm /etc/nginx/conf.d/default.conf

Expand Down
10 changes: 4 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,20 +34,18 @@ Run unit tests with `npm run test`. Jest will pick up test in files with the `.t

To run integration tests:

1. First install chromedriver `sudo ./scripts/install-chromedriver.sh`
1. Make sure you also have a compatible version of chrome (110)
1. Make sure you have Chrome (or Chromium) installed. Depending on the platform you may also need to install chromedriver.
- On Ubuntu, `sudo apt install chromium-browser chromium-chromedriver` will install both.
1. Run the proxy and dependencies with `./scripts/dev.sh`
1. Then run tests with `npm run integration-test`

Jest will pick up tests in files with the `.itest.js` extension.

## Buildkite
1. `./scripts/make-shared-build-env.sh`: makes a shared base image containing all npm dependencies
1. `./scripts/make-integration-test-image.sh` makes the integration tests image which contains all selenium test
dependencies
1. `./scripts/run-build.sh`: creates and runs the main build env image which dockerises and pushes the app image
to docker hub
1. `./scripts/build-image.sh`: builds and pushes the main app image to docker hub
1. `./scripts/run-integration-tests.sh`: runs the app image created in the previous step along with all dependencies and
then runs the integration tests image created in step 2.
1. `./dev/run-build-minimal.sh`: builds an image `montagu-reverse-proxy-minimal` that just provides login functionality.
1. `./dev/build-minimal-image.sh`: builds an image `montagu-reverse-proxy-minimal` that just provides login functionality.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will it? build-minimal-image.sh references the . Dockerfile, which is provided in the dev folder along with the script, but if you're running that from the project root folder, won't it use the Dockerfile at the root? Might be better to give the dev (minimal) dockerfile a more explicit name like buildMinimal.dockerfile like it used to have.

Copy link
Contributor Author

@plietar plietar Dec 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like I just naively copy pasted this. Renamed to dev/minimal.dockerfile, and added the appropriate -f flag.

This is used for testing OrderlyWeb login integration without having to run an entire Montagu deployment.
7 changes: 0 additions & 7 deletions build.dockerfile

This file was deleted.

9 changes: 2 additions & 7 deletions buildkite/pipeline.yml
Original file line number Diff line number Diff line change
@@ -1,19 +1,14 @@
steps:
- label: ":construction_worker: Build shared build env image"
command: scripts/make-shared-build-env.sh

- wait

- label: ":construction_worker: Build integration test image"
command: scripts/make-integration-test-image.sh

- label: ":shipit: Build and push main image"
command: scripts/run-build.sh
command: scripts/build-image.sh

- wait

- label: ":mag: Run integration tests"
command: scripts/run-integration-tests.sh

- label: ":shipit: Build minimal test image"
command: dev/run-build-minimal.sh
command: dev/build-minimal-image.sh
22 changes: 0 additions & 22 deletions dev/Dockerfile

This file was deleted.

26 changes: 13 additions & 13 deletions dev/build-minimal-image.sh
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
#!/usr/bin/env bash
set -ex

org=vimc
name=montagu-reverse-proxy-minimal
HERE=$(dirname $0)
. $HERE/../scripts/common

# Deal with dependabot tags which look like
#
# dependabot/npm_and_yarn/app/lodash-4.17.19
#
# But docker does not like
MONTAGU_GIT_BRANCH=$(echo $MONTAGU_GIT_BRANCH | sed 's;/;-;g')
MINIMAL_BRANCH_TAG=$ORG/montagu-reverse-proxy-minimal:$GIT_BRANCH
MINIMAL_SHA_TAG=$ORG/montagu-reverse-proxy-minimal:$GIT_SHA

commit_tag=$org/$name:$MONTAGU_GIT_ID
branch_tag=$org/$name:$MONTAGU_GIT_BRANCH
docker build \
-t $MINIMAL_BRANCH_TAG \
-t $MINIMAL_SHA_TAG \
-f dev/minimal.dockerfile \
.

docker build -t $commit_tag -t $branch_tag .
docker push $commit_tag
docker push $branch_tag
if [[ "$BUILDKITE" = "true" ]]; then
docker push $MINIMAL_SHA_TAG
docker push $MINIMAL_BRANCH_TAG
fi
13 changes: 0 additions & 13 deletions dev/buildMinimal.dockerfile

This file was deleted.

30 changes: 30 additions & 0 deletions dev/minimal.dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
FROM node:16-buster

WORKDIR /workspace
COPY . /workspace

RUN npm install
RUN npm run test

FROM nginx:stable

COPY nginx.conf /etc/nginx/nginx.conf
COPY dev/nginx.minimal.conf /etc/nginx/conf.d/montagu.conf
COPY index.html /usr/share/nginx/html/index.html
COPY 404.html /usr/share/nginx/html/404.html
COPY resources /usr/share/nginx/html/resources

# Copy third party javascript from npm modules
ENV THIRDPARTY_JS_PATH /usr/share/nginx/html/resources/js/third_party/
COPY --from=0 /workspace/node_modules/pako/dist/pako.min.js $THIRDPARTY_JS_PATH
COPY --from=0 /workspace/node_modules/vue/dist/vue.min.js $THIRDPARTY_JS_PATH
COPY --from=0 /workspace/node_modules/jwt-decode/build/jwt-decode.min.js $THIRDPARTY_JS_PATH
COPY --from=0 /workspace/node_modules/jquery/dist/jquery.min.js $THIRDPARTY_JS_PATH

# Copy third party css from npm modules
COPY --from=0 /workspace/node_modules/bootstrap/dist/css/bootstrap.min.css /usr/share/nginx/html/resources/css/third_party/
COPY --from=0 /workspace/node_modules/bootstrap/dist/css/bootstrap.min.css.map /usr/share/nginx/html/resources/css/third_party/

RUN rm /etc/nginx/conf.d/default.conf

CMD exec nginx -g "daemon off;"
20 changes: 0 additions & 20 deletions dev/run-build-minimal.sh

This file was deleted.

4 changes: 2 additions & 2 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ x-logging: &log-journald

services:
api:
image: ${ORG}/montagu-api:master
image: ${ORG}/montagu-api:abf39a2
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is currently pointed at vimc/montagu-api#479, I'll update once that PR is merged.

ports:
- "8080:8080"
networks:
Expand Down Expand Up @@ -69,4 +69,4 @@ volumes:
static_volume:
token_key_volume:
networks:
proxy:
proxy:
61 changes: 22 additions & 39 deletions integration-tests.dockerfile
Original file line number Diff line number Diff line change
@@ -1,55 +1,38 @@
ARG MONTAGU_GIT_ID="UNKNOWN"
FROM vimc/montagu-reverse-proxy-shared-build-env:$MONTAGU_GIT_ID
FROM node:16-buster

RUN apt-get update && apt-get install -yq \
default-jre \
gconf-service \
fonts-liberation \
libasound2 \
libatk1.0-0 \
libatk-bridge2.0-0 \
libc6 \
libcairo2 \
libatk1.0-0 \
libatspi2.0-0 \
libcups2 \
libdbus-1-3 \
libdrm2 \
libexpat1 \
libfontconfig1 \
libgcc1 \
libgconf-2-4 \
libgdk-pixbuf2.0-0 \
libglib2.0-0 \
libgtk-3-0 \
libgbm1 \
libgtk-3-0 \
libnspr4 \
libpango-1.0-0 \
libpangocairo-1.0-0 \
libstdc++6 \
libx11-6 \
libx11-xcb1 \
libxcb1 \
libnss3 \
libvulkan1 \
libxcomposite1 \
libxcursor1 \
libxdamage1 \
libxext6 \
libxfixes3 \
libxi6 \
libxkbcommon0 \
libxrandr2 \
libxrender1 \
libxss1 \
libxtst6 \
libu2f-udev \
libvulkan1 \
ca-certificates \
fonts-liberation \
libappindicator1 \
libnss3 \
lsb-release \
xdg-utils \
wget
xdg-utils

# Pick the version from https://googlechromelabs.github.io/chrome-for-testing/
ARG CHROME_VERSION="131.0.6778.85"

RUN wget https://storage.googleapis.com/chrome-for-testing-public/$CHROME_VERSION/linux64/chrome-linux64.zip
RUN wget https://storage.googleapis.com/chrome-for-testing-public/$CHROME_VERSION/linux64/chromedriver-linux64.zip

RUN unzip -d /opt chrome-linux64.zip
RUN unzip -d /opt chromedriver-linux64.zip

RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
RUN dpkg -i google-chrome-stable_current_amd64.deb; apt-get -fy install
ENV PATH="/opt/chrome-linux64:/opt/chromedriver-linux64:$PATH"

RUN ./scripts/install-chromedriver.sh
WORKDIR /workspace
COPY . /workspace

RUN npm install
CMD npm run integration-test
Loading