Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Notes on building #5

Open
cyounkins opened this issue May 8, 2024 · 0 comments
Open

Notes on building #5

cyounkins opened this issue May 8, 2024 · 0 comments

Comments

@cyounkins
Copy link

Sticking this here to document trying to get this to build and garner sympathy as I slowly lose my mind

Because there is a build dir and a BUILD file, the project assumes a case sensitive filesystem. This is not true on macos by default. We should consider fixing this, I'm not sure if the build/ dir is needed anymore.

The specific version of bazel is required. Bazelisk is a wrapper to get the right version.

I've been trying to build using bazelisk build --config=docker-sandbox -c opt //pagespeed/automatic:automatic - hopefully this uses a docker sandbox to reduce system differences

Repository rule http_archive defined at:
  /private/var/tmp/_bazel_craig/b0ef3c47fd120d26e254b6d1cae997bb/external/bazel_tools/tools/build_defs/repo/http.bzl:372:31: in <toplevel>
ERROR: /Users/craig/Documents/Work/SpaceCowMedia/mod_pagespeed/pagespeed/kernel/image/BUILD:16:11: //pagespeed/kernel/image:image depends on @libjpeg_turbo//:libjpeg in repository @libjpeg_turbo which failed to fetch. no such package '@libjpeg_turbo//': java.io.IOException: Error downloading [https://chromium.googlesource.com/chromium/deps/libjpeg_turbo/+archive/ab7cd970a83609f98e8542cea8b81e8d92ddab83.tar.gz] to /private/var/tmp/_bazel_craig/b0ef3c47fd120d26e254b6d1cae997bb/external/libjpeg_turbo/temp8776347438360174284/ab7cd970a83609f98e8542cea8b81e8d92ddab83.tar.gz: Checksum was bc5e0169685a67acd6b398ab48d1f679fb851635a6bd75184a658bdc777b2ff3 but wanted 3a6b383a957d87b4d60b67e2e1a950c695ee3016e817d04a13af05b9a98c6aea

The tarball appears to change every time:

~ $ curl 'https://chromium.googlesource.com/chromium/deps/libjpeg_turbo/+archive/ab7cd970a83609f98e8542cea8b81e8d92ddab83.tar.gz' | shasum
b5b9ae68525b056cf7b5883dc32bb62b4b45f172  -
~ $ curl 'https://chromium.googlesource.com/chromium/deps/libjpeg_turbo/+archive/ab7cd970a83609f98e8542cea8b81e8d92ddab83.tar.gz' | shasum
2c03e325e9d058dc352f84c9a612b870418abe2f  -
~ $ curl 'https://chromium.googlesource.com/chromium/deps/libjpeg_turbo/+archive/ab7cd970a83609f98e8542cea8b81e8d92ddab83.tar.gz' | shasum
6a0f73cade396b3b0b5df1aa2bda8ec8286f34c3  -

Wild. That sha doesn't appear in the project history. There is a commit on the date in the comment - 9120a247436e84c0b4eea828cb11e8f665fcde30 and the matching SHA is 06895f4c3278157012adb7756d0d6c6c5ad63c3009edd5cc124119493041e9ad . I also changed it to pull from github because that's where it's hosted now.

Now bazel gets to pulling the sandbox image, but it doesn't appear to have python2. Note that trying to run it outside the sandbox, then in the sandbox, appears to break something in bazel cache. Run bazelisk clean. I'm worried that trying to build this on an M1 mac will cause additional issues, so switching to x86.

bazelisk doesn't work on NixOS. And we apparently need exactly 6.1.0... so here's the right nixpkgs rev for bazel 6.1.0: nix-shell -p bazel -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/be9859e8994910156989cb842a373541fde19bb1.tar.gz How does it work when dependencies have different .bazelversion? Nevermind I don't want to know.

Now I seem to get a different error each time I run bazel:

$ bazel build -c opt //pagespeed/automatic:automatic
ERROR: /home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/external/com_github_gflags_gflags/bazel/expanded_template/BUILD:1:10: Compiling bazel/expanded_template/expand_template.cc [for tool] failed: (Exit 1): gcc failed: error executing command (from target @com_github_gflags_gflags//bazel/expanded_template:expand_template) /nix/store/bfbp3ypd9nm3fapz634gvvs738blrl0y-gcc-wrapper-12.2.0/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 ... (remaining 30 arguments skipped)
...
external/com_github_gflags_gflags/bazel/expanded_template/expand_template.cc:37:25: note: 'strchr' is defined in header '<cstring>'; did you forget to '#include <cstring>'?

$ bazel build -c opt //pagespeed/automatic:automatic
INFO: Analyzed target //pagespeed/automatic:automatic (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/external/drp/BUILD.bazel:2:8: Executing genrule @drp//:registry_tables_generator failed: (Exit 127): bash failed: error executing command (from target @drp//:registry_tables_generator) /nix/store/5ynbf6wszmggr0abwifdagrixgnya5vy-bash-5.2-p15/bin/bash -c ... (remaining 1 argument skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
/nix/store/5ynbf6wszmggr0abwifdagrixgnya5vy-bash-5.2-p15/bin/bash: line 1: python2: command not found
Target //pagespeed/automatic:automatic failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 8.600s, Critical Path: 7.26s
INFO: 269 processes: 46 internal, 223 linux-sandbox.
FAILED: Build did NOT complete successfully

$ bazel build -c opt //pagespeed/automatic:automatic --verbose_failures
INFO: Analyzed target //pagespeed/automatic:automatic (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/external/libjpeg_turbo/BUILD.bazel:2:11: Middleman _middlemen/@libjpeg_Uturbo_S_S_Clibjpeg_Uheaders-cc_library-compile failed: missing input file '@libjpeg_turbo//:jconfig.h'
ERROR: /home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/external/libjpeg_turbo/BUILD.bazel:2:11: Middleman _middlemen/@libjpeg_Uturbo_S_S_Clibjpeg_Uheaders-cc_library-compile failed: missing input file '@libjpeg_turbo//:jdct.h'
ERROR: /home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/external/libjpeg_turbo/BUILD.bazel:2:11: Middleman _middlemen/@libjpeg_Uturbo_S_S_Clibjpeg_Uheaders-cc_library-compile failed: missing input file '@libjpeg_turbo//:jinclude.h'
ERROR: /home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/external/libjpeg_turbo/BUILD.bazel:2:11: Middleman _middlemen/@libjpeg_Uturbo_S_S_Clibjpeg_Uheaders-cc_library-compile failed: missing input file '@libjpeg_turbo//:jmorecfg.h'
ERROR: /home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/external/libjpeg_turbo/BUILD.bazel:2:11: Middleman _middlemen/@libjpeg_Uturbo_S_S_Clibjpeg_Uheaders-cc_library-compile failed: missing input file '@libjpeg_turbo//:jpeglib.h'
ERROR: /home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/external/libjpeg_turbo/BUILD.bazel:2:11: Middleman _middlemen/@libjpeg_Uturbo_S_S_Clibjpeg_Uheaders-cc_library-compile failed: missing input file '@libjpeg_turbo//:jpeglibmangler.h'
ERROR: /home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/external/libjpeg_turbo/BUILD.bazel:2:11: Middleman _middlemen/@libjpeg_Uturbo_S_S_Clibjpeg_Uheaders-cc_library-compile failed: 6 input file(s) do not exist
Target //pagespeed/automatic:automatic failed to build
ERROR: /home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/external/libjpeg_turbo/BUILD.bazel:2:11 Middleman _middlemen/@libjpeg_Uturbo_S_S_Clibjpeg_Uheaders-cc_library-compile failed: 6 input file(s) do not exist
INFO: Elapsed time: 1.133s, Critical Path: 0.85s
INFO: 7 processes: 7 internal.
FAILED: Build did NOT complete successfully

Maybe that libjpeg-turbo ref is not what we want? And there's the python2 error again.

I tried running it in the sandbox (though this is less important here than on M1 mac)

$ bazel build --config=docker-sandbox -c opt //pagespeed/automatic:automatic
INFO: Analyzed target //pagespeed/automatic:automatic (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: Docker sandboxing is disabled, because running 'docker info' failed: com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1
	at com.google.devtools.build.lib.shell.FutureCommandResult.get(FutureCommandResult.java:97)
	at com.google.devtools.build.lib.shell.Command.execute(Command.java:263)
	at com.google.devtools.build.lib.sandbox.DockerSandboxedSpawnRunner.isSupported(DockerSandboxedSpawnRunner.java:121)
	at com.google.devtools.build.lib.sandbox.SandboxModule.setup(SandboxModule.java:314)
	at com.google.devtools.build.lib.sandbox.SandboxModule.registerSpawnStrategies(SandboxModule.java:152)
	at com.google.devtools.build.lib.buildtool.ExecutionTool.<init>(ExecutionTool.java:175)
	at com.google.devtools.build.lib.buildtool.BuildTool.buildTargets(BuildTool.java:186)
	at com.google.devtools.build.lib.buildtool.BuildTool.processRequest(BuildTool.java:494)
	at com.google.devtools.build.lib.buildtool.BuildTool.processRequest(BuildTool.java:462)
	at com.google.devtools.build.lib.runtime.commands.BuildCommand.exec(BuildCommand.java:103)
	at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.execExclusively(BlazeCommandDispatcher.java:610)
	at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:234)
	at com.google.devtools.build.lib.server.GrpcServerImpl.executeCommand(GrpcServerImpl.java:550)
	at com.google.devtools.build.lib.server.GrpcServerImpl.lambda$run$1(GrpcServerImpl.java:614)
	at io.grpc.Context$1.run(Context.java:566)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
ERROR: 'docker' was requested for mnemonic Javac but no strategy with that identifier was registered. Valid values are: [processwrapper-sandbox, standalone, remote, worker, sandboxed, local, linux-sandbox]
INFO: Elapsed time: 0.227s, Critical Path: 0.00s
INFO: 0 processes.
FAILED: Build did NOT complete successfully

$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.9.1)
  compose: Docker Compose (Docker Inc., 2.16.0)
...

We're on bazel 6.1.0. At the Envoy commit (7fd09ae21dacb0c36324dbbb158cb905c7ac8d9c), it's using bazel 6.3.2, so let's try that. nix-shell -p bazel -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/9957cd48326fe8dbd52fdc50dd2502307f188b0d.tar.gz Aaaand --config=docker-sandbox appears to be working! But it doesn't build...

$ bazel build --config=docker-sandbox -c opt //pagespeed/automatic:automatic
DEBUG: /home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/external/rules_python/python/pip.bzl:49:10: pip_install is deprecated. Please switch to pip_parse. pip_install will be removed in a future release.
INFO: Analyzed target //pagespeed/automatic:automatic (284 packages loaded, 15834 targets configured).
INFO: Found 1 target...
INFO: Docker sandboxing is supported
INFO: Preparing Docker image envoyproxy/envoy-build-ubuntu:41c5a05d708972d703661b702a63ef5060125c33 for use...
INFO: Preparing Docker image envoyproxy/envoy-build-ubuntu:41c5a05d708972d703661b702a63ef5060125c33 for use...
ERROR: /home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/external/drp/BUILD.bazel:2:8: Executing genrule @drp//:registry_tables_generator failed: Running command Executing (without brackets): [/run/current-system/sw/bin/docker] [build] [-q] [-]; environment: {AR=ar, AS=as, CC=gcc, CONFIG_SHELL=/nix/store/xdqlrixlspkks50m9b0mpvag65m3pf2w-bash-5.2-p15/bin/bash, CXX=g++, DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus, DISPLAY=localhost:10.0, EDITOR=nano, GTK_A11Y=none, GTK_PATH=/home/craig/.nix-profile/lib/gtk-2.0:/home/craig/.nix-profile/lib/gtk-3.0:/home/craig/.nix-profile/lib/gtk-4.0:/nix/profile/lib/gtk-2.0:/nix/profile/lib/gtk-3.0:/nix/profile/lib/gtk-4.0:/home/craig/.local/state/nix/profile/lib/gtk-2.0:/home/craig/.local/state/nix/profile/lib/gtk-3.0:/home/craig/.local/state/nix/profile/lib/gtk-4.0:/etc/profiles/per-user/craig/lib/gtk-2.0:/etc/profiles/per-user/craig/lib/gtk-3.0:/etc/profiles/per-user/craig/lib/gtk-4.0:/nix/var/nix/profiles/default/lib/gtk-2.0:/nix/var/nix/profiles/default/lib/gtk-3.0:/nix/var/nix/profiles/default/lib/gtk-4.0:/run/current-system/sw/lib/gtk-2.0:/run/current-system/sw/lib/gtk-3.0:/run/current-system/sw/lib/gtk-4.0, HOME=/home/craig, HOST_PATH=/nix/store/cmx3ikvl09ycd4f8vnw6r5zqd2qnliyj-bazel-6.3.2/bin:/nix/store/y9gr7abwxvzcpg5g73vhnx1fpssr5frr-coreutils-9.3/bin:/nix/store/b6izr8wh0p7dyvh3cyg14wq2rn8d31ik-findutils-4.9.0/bin:/nix/store/q56n7lhjw724i7b33qaqra61p7m7c0cd-diffutils-3.10/bin:/nix/store/x23by79p38ll0js1alifmf3y56vqfs49-gnused-4.9/bin:/nix/store/xafzciap7acqhfx84dvqkp18bg4lrai3-gnugrep-3.11/bin:/nix/store/8kkn44iwdbgqkrj661nr4cjcpmrqqmx8-gawk-5.2.2/bin:/nix/store/89s3w7b4g78989kpzc7sy4phv0nqfira-gnutar-1.35/bin:/nix/store/2a9na7bp4r3290yqqzg503325dwglxyq-gzip-1.13/bin:/nix/store/gxknjk51s7q86llkbzpaqv43kflj9d8j-bzip2-1.0.8-bin/bin:/nix/store/2jp6cv2q4cgh91f5lp57p945rq98ldhr-gnumake-4.4.1/bin:/nix/store/xdqlrixlspkks50m9b0mpvag65m3pf2w-bash-5.2-p15/bin:/nix/store/c15ama0p8jr4mn0943yjk4rpa2hxk7ml-patch-2.7.6/bin:/nix/store/sb3sxnp4g40gfw758a0m4sjm7slvmax9-xz-5.4.4-bin/bin:/nix/store/xfjqspcc9442hi0lm0szv3sw75zswvml-file-5.45/bin, INFOPATH=/home/craig/.nix-profile/info:/home/craig/.nix-profile/share/info:/nix/profile/info:/nix/profile/share/info:/home/craig/.local/state/nix/profile/info:/home/craig/.local/state/nix/profile/share/info:/etc/profiles/per-user/craig/info:/etc/profiles/per-user/craig/share/info:/nix/var/nix/profiles/default/info:/nix/var/nix/profiles/default/share/info:/run/current-system/sw/info:/run/current-system/sw/share/info, IN_NIX_SHELL=impure, LANG=en_US.UTF-8, LD=ld, LESSKEYIN_SYSTEM=/nix/store/lrwil01abqwa2r9g2clkjkzmpirgk749-lessconfig, LESSOPEN=|/nix/store/20qpjwy2hh1yrskwcma9k9cjkv2g0l28-lesspipe-2.10/bin/lesspipe.sh %s, LIBEXEC_PATH=/home/craig/.nix-profile/lib/libexec:/nix/profile/lib/libexec:/home/craig/.local/state/nix/profile/lib/libexec:/etc/profiles/per-user/craig/lib/libexec:/nix/var/nix/profiles/default/lib/libexec:/run/current-system/sw/lib/libexec, LOCALE_ARCHIVE=/run/current-system/sw/lib/locale/locale-archive, LOCALE_ARCHIVE_2_27=/nix/store/rfvxv3hjd9k42sahbgkirkd81fzblc86-glibc-locales-2.38-66/lib/locale/locale-archive, LOGNAME=craig, LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=00:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.avif=01;35:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:*~=00;90:*#=00;90:*.bak=00;90:*.old=00;90:*.orig=00;90:*.part=00;90:*.rej=00;90:*.swp=00;90:*.tmp=00;90:*.dpkg-dist=00;90:*.dpkg-old=00;90:*.ucf-dist=00;90:*.ucf-new=00;90:*.ucf-old=00;90:*.rpmnew=00;90:*.rpmorig=00;90:*.rpmsave=00;90:, NIXPKGS_CONFIG=/etc/nix/nixpkgs-config.nix, NIX_BINTOOLS=/nix/store/mc6q3cdz5s0p1aj4y586bglsfsnsf2k8-binutils-wrapper-2.40, NIX_BINTOOLS_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu=1, NIX_BUILD_CORES=32, NIX_BUILD_TOP=/run/user/1000, NIX_CC=/nix/store/18bs92p6yf6w2wwxhbplgx02y6anq092-gcc-wrapper-12.3.0, NIX_CC_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu=1, NIX_CFLAGS_COMPILE= -frandom-seed=sc4di7wxvg, NIX_ENFORCE_NO_NATIVE=1, NIX_HARDENING_ENABLE=fortify fortify3 stackprotector pic strictoverflow format relro bindnow, NIX_LDFLAGS=-rpath /nix/store/sc4di7wxvgkhp2ncl4ml8wbli93x6qpx-shell/lib , NIX_PATH=/home/craig/.nix-defexpr/channels:nixpkgs=/nix/var/nix/profiles/per-user/root/channels/nixos:nixos-config=/etc/nixos/configuration.nix:/nix/var/nix/profiles/per-user/root/channels, NIX_PROFILES=/run/current-system/sw /nix/var/nix/profiles/default /etc/profiles/per-user/craig /home/craig/.local/state/nix/profile /nix/profile /home/craig/.nix-profile, NIX_STORE=/nix/store, NIX_USER_PROFILE_DIR=/nix/var/nix/profiles/per-user/craig, NM=nm, NO_AT_BRIDGE=1, OBJCOPY=objcopy, OBJDUMP=objdump, PAGER=less, PATH=/nix/store/yi5aswh4a0mb82kbi0afr26j5ilsghjl-bash-interactive-5.2-p15/bin:/nix/store/ay0p9mbw1w3zkvwzx3c94xq7x8jrn9wq-patchelf-0.15.0/bin:/nix/store/18bs92p6yf6w2wwxhbplgx02y6anq092-gcc-wrapper-12.3.0/bin:/nix/store/h5kvfrjmpw792v8jg7nrzfkffmn0iyy8-gcc-12.3.0/bin:/nix/store/f6in5kb2y5v06zinz1a6xy6cyg67q026-glibc-2.37-8-bin/bin:/nix/store/y9gr7abwxvzcpg5g73vhnx1fpssr5frr-coreutils-9.3/bin:/nix/store/mc6q3cdz5s0p1aj4y586bglsfsnsf2k8-binutils-wrapper-2.40/bin:/nix/store/74y3751gsixaz9797ib0hp7c658sp1y5-binutils-2.40/bin:/nix/store/cmx3ikvl09ycd4f8vnw6r5zqd2qnliyj-bazel-6.3.2/bin:/nix/store/y9gr7abwxvzcpg5g73vhnx1fpssr5frr-coreutils-9.3/bin:/nix/store/b6izr8wh0p7dyvh3cyg14wq2rn8d31ik-findutils-4.9.0/bin:/nix/store/q56n7lhjw724i7b33qaqra61p7m7c0cd-diffutils-3.10/bin:/nix/store/x23by79p38ll0js1alifmf3y56vqfs49-gnused-4.9/bin:/nix/store/xafzciap7acqhfx84dvqkp18bg4lrai3-gnugrep-3.11/bin:/nix/store/8kkn44iwdbgqkrj661nr4cjcpmrqqmx8-gawk-5.2.2/bin:/nix/store/89s3w7b4g78989kpzc7sy4phv0nqfira-gnutar-1.35/bin:/nix/store/2a9na7bp4r3290yqqzg503325dwglxyq-gzip-1.13/bin:/nix/store/gxknjk51s7q86llkbzpaqv43kflj9d8j-bzip2-1.0.8-bin/bin:/nix/store/2jp6cv2q4cgh91f5lp57p945rq98ldhr-gnumake-4.4.1/bin:/nix/store/xdqlrixlspkks50m9b0mpvag65m3pf2w-bash-5.2-p15/bin:/nix/store/c15ama0p8jr4mn0943yjk4rpa2hxk7ml-patch-2.7.6/bin:/nix/store/sb3sxnp4g40gfw758a0m4sjm7slvmax9-xz-5.4.4-bin/bin:/nix/store/xfjqspcc9442hi0lm0szv3sw75zswvml-file-5.45/bin:/run/wrappers/bin:/home/craig/.nix-profile/bin:/nix/profile/bin:/home/craig/.local/state/nix/profile/bin:/etc/profiles/per-user/craig/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin:/nix/store/3ssn79pr531nfyh578r9kwvinp0mvy72-file-5.45/bin:/nix/store/pzf6dnxg8gf04xazzjdwarm7s03cbrgz-python3-3.10.12/bin:/nix/store/hjspq68ljkw2pxlki8mh6shi32s67m89-unzip-6.0/bin:/nix/store/9n0384r446blhgla21fpvyr0qnjgjwaw-which-2.21/bin:/nix/store/r77zgzm8a4086678daxvwja7ivcz1d7l-zip-3.0/bin, PWD=/home/craig/mod_pagespeed, QTWEBKIT_PLUGIN_PATH=/home/craig/.nix-profile/lib/mozilla/plugins/:/nix/profile/lib/mozilla/plugins/:/home/craig/.local/state/nix/profile/lib/mozilla/plugins/:/etc/profiles/per-user/craig/lib/mozilla/plugins/:/nix/var/nix/profiles/default/lib/mozilla/plugins/:/run/current-system/sw/lib/mozilla/plugins/, RANLIB=ranlib, READELF=readelf, SHELL=/nix/store/yi5aswh4a0mb82kbi0afr26j5ilsghjl-bash-interactive-5.2-p15/bin/bash, SHLVL=2, SIZE=size, SOURCE_DATE_EPOCH=315532800, SSH_ASKPASS=, SSH_AUTH_SOCK=/tmp/ssh-XXXXHYBHst/agent.118863, SSH_CLIENT=192.168.1.198 54146 22, SSH_CONNECTION=192.168.1.198 54146 192.168.1.10 22, SSH_TTY=/dev/pts/2, STRINGS=strings, STRIP=strip, TEMP=/run/user/1000, TEMPDIR=/run/user/1000, TERM=tmux-256color, TERMINFO_DIRS=/home/craig/.nix-profile/share/terminfo:/nix/profile/share/terminfo:/home/craig/.local/state/nix/profile/share/terminfo:/etc/profiles/per-user/craig/share/terminfo:/nix/var/nix/profiles/default/share/terminfo:/run/current-system/sw/share/terminfo, TERM_PROGRAM=tmux, TERM_PROGRAM_VERSION=3.3a, TMP=/run/user/1000, TMPDIR=/run/user/1000, TMUX=/tmp/tmux-1000/default,118875,0, TMUX_PANE=%0, TZDIR=/etc/zoneinfo, USER=craig, XCURSOR_PATH=/home/craig/.icons:/home/craig/.local/share/icons:/home/craig/.nix-profile/share/icons:/home/craig/.nix-profile/share/pixmaps:/nix/profile/share/icons:/nix/profile/share/pixmaps:/home/craig/.local/state/nix/profile/share/icons:/home/craig/.local/state/nix/profile/share/pixmaps:/etc/profiles/per-user/craig/share/icons:/etc/profiles/per-user/craig/share/pixmaps:/nix/var/nix/profiles/default/share/icons:/nix/var/nix/profiles/default/share/pixmaps:/run/current-system/sw/share/icons:/run/current-system/sw/share/pixmaps, XDG_CONFIG_DIRS=/etc/xdg:/home/craig/.nix-profile/etc/xdg:/nix/profile/etc/xdg:/home/craig/.local/state/nix/profile/etc/xdg:/etc/profiles/per-user/craig/etc/xdg:/nix/var/nix/profiles/default/etc/xdg:/run/current-system/sw/etc/xdg, XDG_DATA_DIRS=/nix/store/ay0p9mbw1w3zkvwzx3c94xq7x8jrn9wq-patchelf-0.15.0/share:/nix/store/pj0jc9va0f5540phsd6vkjgc63k7vndl-desktops/share:/home/craig/.nix-profile/share:/nix/profile/share:/home/craig/.local/state/nix/profile/share:/etc/profiles/per-user/craig/share:/nix/var/nix/profiles/default/share:/run/current-system/sw/share, XDG_RUNTIME_DIR=/run/user/1000, XDG_SESSION_CLASS=user, XDG_SESSION_ID=3, XDG_SESSION_TYPE=tty, __ETC_PROFILE_DONE=1, __HM_SESS_VARS_SOURCED=1, __NIXOS_SET_ENVIRONMENT_DONE=1, __fish_nixos_env_preinit_sourced=1, __structuredAttrs=, buildCommandPath=/tmp/nix-shell-131350-0/.attr-0, buildInputs=/nix/store/cmx3ikvl09ycd4f8vnw6r5zqd2qnliyj-bazel-6.3.2, builder=/nix/store/xdqlrixlspkks50m9b0mpvag65m3pf2w-bash-5.2-p15/bin/bash, cmakeFlags=, configureFlags=, depsBuildBuild=, depsBuildBuildPropagated=, depsBuildTarget=, depsBuildTargetPropagated=, depsHostHost=, depsHostHostPropagated=, depsTargetTarget=, depsTargetTargetPropagated=, doCheck=, doInstallCheck=, enableParallelBuilding=1, enableParallelChecking=1, enableParallelInstalling=1, mesonFlags=, name=shell, nativeBuildInputs=, out=/nix/store/sc4di7wxvgkhp2ncl4ml8wbli93x6qpx-shell, outputs=out, passAsFile=buildCommand, patches=, propagatedBuildInputs=, propagatedNativeBuildInputs=, shell=/nix/store/xdqlrixlspkks50m9b0mpvag65m3pf2w-bash-5.2-p15/bin/bash, stdenv=/nix/store/3pfjacvdg9f491lfqc9qb2d0nknx73fb-stdenv-linux, strictDeps=, system=x86_64-linux}; working dir: /home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/execroot/mod_pagespeed failed: Dockerfile:3
--------------------
   1 |     FROM envoyproxy/envoy-build-ubuntu:41c5a05d708972d703661b702a63ef5060125c33
   2 |     RUN ["mkdir", "-p", "/execroot/mod_pagespeed"]
   3 | >>> RUN ["groupadd", "-g", "100", "bazelbuild"]
   4 |     RUN ["useradd", "-l", "-m", "-g", "100", "-d", "/execroot/mod_pagespeed", "-N", "-u", "1000", "bazelbuild"]
   5 |     RUN ["chown", "-R", "1000:100", "/execroot/mod_pagespeed"]
--------------------
ERROR: failed to solve: process "groupadd -g 100 bazelbuild" did not complete successfully: exit code: 4
: Process exited with status 1
INFO: Preparing Docker image envoyproxy/envoy-build-ubuntu:41c5a05d708972d703661b702a63ef5060125c33 for use...
Target //pagespeed/automatic:automatic failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 145.514s, Critical Path: 127.82s
INFO: 62 processes: 62 internal.
FAILED: Build did NOT complete successfully

Hmmm let's try doing that ourselves:

craig@pluto ~ [SIGINT]> docker run -it envoyproxy/envoy-build-ubuntu:41c5a05d708972d703661b702a63ef5060125c33 /bin/bash
Unable to find image 'envoyproxy/envoy-build-ubuntu:41c5a05d708972d703661b702a63ef5060125c33' locally
41c5a05d708972d703661b702a63ef5060125c33: Pulling from envoyproxy/envoy-build-ubuntu
Digest: sha256:50337314a150ed12447c87c1622eac6f611a069888722fb9a426e21ed161cc26
Status: Downloaded newer image for envoyproxy/envoy-build-ubuntu:41c5a05d708972d703661b702a63ef5060125c33
root@c4b383fe3b7f:/# groupadd -g 100 bazelbuild
groupadd: GID '100' already exists
root@c4b383fe3b7f:/# grep '100' /etc/group
users:x:100:

bazelbuild/bazel#17535 , the fix appears in 7.0.0. Ugh fine we'll try 7.0.2 - nix-shell -p bazel_7 -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/e89cf1c932006531f454de7d652163a9a5c86668.tar.gz

$ bazel build --config=docker-sandbox -c opt //pagespeed/automatic:automatic --verbose_failures
WARNING: --enable_bzlmod is set, but no MODULE.bazel file was found at the workspace root. Bazel will create an empty MODULE.bazel file. Please consider migrating your external dependencies from WORKSPACE to MODULE.bazel. For more details, please refer to https://github.com/bazelbuild/bazel/issues/18958.
...
INFO: Docker sandboxing is supported
ERROR: /home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/external/com_google_protobuf/src/google/protobuf/compiler/BUILD.bazel:140:13: @@com_google_protobuf//src/google/protobuf/compiler:protoc_aarch64_test_genrule: no such attribute 'exec_tools' in 'genrule' rule (did you mean 'executable'?)
ERROR: /home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/external/com_google_protobuf/src/google/protobuf/compiler/BUILD.bazel:145:12: @@com_google_protobuf//src/google/protobuf/compiler:protoc_x86_64_test_genrule: no such attribute 'exec_tools' in 'genrule' rule (did you mean 'executable'?)

Yeah I thought 7.0.2 was a bad idea. Back to 6.1.0 and trying to build again without sandbox

[nix-shell:~/mod_pagespeed]$ bazel clean --expunge
INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.

[nix-shell:~/mod_pagespeed]$ bazel version
Starting local Bazel server and connecting to it...
Build label: 6.1.0- (@non-git)
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Jan 1 00:00:00 1980 (315532800)
Build timestamp: 315532800
Build timestamp as int: 315532800

[nix-shell:~/mod_pagespeed]$ bazel build -c opt //pagespeed/automatic:automatic
DEBUG: /home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/external/rules_python/python/pip.bzl:49:10: pip_install is deprecated. Please switch to pip_parse. pip_install will be removed in a future release.
INFO: Analyzed target //pagespeed/automatic:automatic (284 packages loaded, 15834 targets configured).
INFO: Found 1 target...
ERROR: /home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/external/envoy/bazel/foreign_cc/BUILD:527:12: Foreign Cc - CMake: Building zlib [for tool] failed: (Exit 127): bash failed: error executing command (from target @envoy//bazel/foreign_cc:zlib) /nix/store/5ynbf6wszmggr0abwifdagrixgnya5vy-bash-5.2-p15/bin/bash -c bazel-out/k8-opt-exec-2B5CBBC6/bin/external/envoy/bazel/foreign_cc/zlib_foreign_cc/wrapper_build_script.sh

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
rules_foreign_cc: Build failed!
rules_foreign_cc: Keeping temp build directory and dependencies directory for debug.
rules_foreign_cc: Please note that the directories inside a sandbox are still cleaned unless you specify --sandbox_debug Bazel command line flag.
rules_foreign_cc: Printing build logs:
_____ BEGIN BUILD LOGS _____

Bazel external C/C++ Rules. Building library zlib

Environment:______________
BUILD_SCRIPT=bazel-out/k8-opt-exec-2B5CBBC6/bin/external/envoy/bazel/foreign_cc/zlib_foreign_cc/build_script.sh
EXT_BUILD_ROOT=/home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/sandbox/linux-sandbox/75/execroot/mod_pagespeed
BUILD_LOG=bazel-out/k8-opt-exec-2B5CBBC6/bin/external/envoy/bazel/foreign_cc/zlib_foreign_cc/CMake.log
PWD=/home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/sandbox/linux-sandbox/75/execroot/mod_pagespeed
CXX=g++
BUILD_WRAPPER_SCRIPT=bazel-out/k8-opt-exec-2B5CBBC6/bin/external/envoy/bazel/foreign_cc/zlib_foreign_cc/wrapper_build_script.sh
TMPDIR=/run/user/1000
EXT_BUILD_DEPS=/home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/sandbox/linux-sandbox/75/execroot/mod_pagespeed/bazel-out/k8-opt-exec-2B5CBBC6/bin/external/envoy/bazel/foreign_cc/zlib.ext_build_deps
BUILD_TMPDIR=/home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/sandbox/linux-sandbox/75/execroot/mod_pagespeed/bazel-out/k8-opt-exec-2B5CBBC6/bin/external/envoy/bazel/foreign_cc/zlib.build_tmpdir
SHLVL=2
INSTALLDIR=/home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/sandbox/linux-sandbox/75/execroot/mod_pagespeed/bazel-out/k8-opt-exec-2B5CBBC6/bin/external/envoy/bazel/foreign_cc/zlib
PATH=/home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/sandbox/linux-sandbox/75/execroot/mod_pagespeed:/nix/store/7xf4f4d9jip5rjkzwvxwxqgmyhzzvyqk-bash-interactive-5.2-p15/bin:/nix/store/mv5gpi07yzb8m9b6829l8hkw6rpay49c-patchelf-0.15.0/bin:/nix/store/bfbp3ypd9nm3fapz634gvvs738blrl0y-gcc-wrapper-12.2.0/bin:/nix/store/wx4nsqbssywgx9zkabwgl9sxm5l61ni3-gcc-12.2.0/bin:/nix/store/lp8qrhb6hs42jwbapzq20l05jf4kyicq-glibc-2.35-224-bin/bin:/nix/store/bg8f47vihykgqcgblxkfk9sbvc4dnksa-coreutils-9.1/bin:/nix/store/i2pdyabq6nrrnisbkma71h42fw6ha0y6-binutils-wrapper-2.40/bin:/nix/store/fhzz4yrdy17czwc9i4swhlpcp445inzb-binutils-2.40/bin:/nix/store/if24c985xidcwpnq9f7afqqgr4rjwl1d-bazel-6.1.0/bin:/nix/store/bg8f47vihykgqcgblxkfk9sbvc4dnksa-coreutils-9.1/bin:/nix/store/8w3xxhbldw1i40a6fdcs17y6vyq3l3dd-findutils-4.9.0/bin:/nix/store/vraimqnjzzvzabh9wvw5a5s32cv5k6lc-diffutils-3.8/bin:/nix/store/4f5jnd0q8b5kj934wm202km3i8ssrl6p-gnused-4.9/bin:/nix/store/4vkv3rzky44hp2b8r13d8hr4ykvqhvwh-gnugrep-3.7/bin:/nix/store/v4dhg1im8yrl1gq1697jgwh2g85mqwr6-gawk-5.2.1/bin:/nix/store/61mzw223sdk26lxawnx64b72xgjrhaj8-gnutar-1.34/bin:/nix/store/jmx6hd6qykl0a8vxrf6pv6m8q2c0kpxn-gzip-1.12/bin:/nix/store/i4ic0p0l83i37dvx8cb1l3cicq0bny5z-bzip2-1.0.8-bin/bin:/nix/store/i36xm3qb20x9mwbigq4gh5626b0vhg9p-gnumake-4.4/bin:/nix/store/5ynbf6wszmggr0abwifdagrixgnya5vy-bash-5.2-p15/bin:/nix/store/xj8vr2sgbz3rhy26xzmg44vkzr5dww1y-patch-2.7.6/bin:/nix/store/c2kkvw6klpyf9p289si6cqv8gw4q2wb9-xz-5.4.1-bin/bin:/nix/store/s29xjzid62937vc17jx6zi785nhk0plk-file-5.44/bin:/run/wrappers/bin:/home/craig/.nix-profile/bin:/nix/profile/bin:/home/craig/.local/state/nix/profile/bin:/etc/profiles/per-user/craig/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin:/nix/store/va9r2bdg56jc6qizpm6dyb97lnxfrhsy-file-5.44/bin:/nix/store/iw1vmh509hcbby8dbpsaanbri4zsq7dj-python3-3.10.10/bin:/nix/store/2nmqzvgf4rnjvpjsdkycf41lf63br42a-unzip-6.0/bin:/nix/store/v0g0r8khhdxn8gwcx3yg57wmndzfdgz5-which-2.21/bin:/nix/store/13agvq2md2q11irp9s6nibrhqmiyq319-zip-3.0/bin
CC=gcc
_=/nix/store/bg8f47vihykgqcgblxkfk9sbvc4dnksa-coreutils-9.1/bin/env
__________________________
+ /home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/sandbox/linux-sandbox/75/execroot/mod_pagespeed/external/cmake-3.23.2-linux-x86_64/bin/cmake -DCMAKE_AR=/nix/store/bfbp3ypd9nm3fapz634gvvs738blrl0y-gcc-wrapper-12.2.0/bin/ar '-DCMAKE_SHARED_LINKER_FLAGS=-shared -fuse-ld=gold -Wl,-no-as-needed -Wl,-z,relro,-z,now -B/nix/store/bfbp3ypd9nm3fapz634gvvs738blrl0y-gcc-wrapper-12.2.0/bin -pass-exit-codes -lm -Wl,--gc-sections -l:libstdc++.a' '-DCMAKE_EXE_LINKER_FLAGS=-fuse-ld=gold -Wl,-no-as-needed -Wl,-z,relro,-z,now -B/nix/store/bfbp3ypd9nm3fapz634gvvs738blrl0y-gcc-wrapper-12.2.0/bin -pass-exit-codes -lm -Wl,--gc-sections -l:libstdc++.a' -DCMAKE_CXX_COMPILER_FORCED=on -DCMAKE_C_COMPILER_FORCED=on -DSKIP_BUILD_EXAMPLES=on -DBUILD_SHARED_LIBS=off -DZLIB_COMPAT=on -DZLIB_ENABLE_TESTS=off -DWITH_OPTIM=on -DWITH_SSE4=off -DWITH_NEW_STRATEGIES=off -DUNALIGNED_OK=off -DCMAKE_BUILD_TYPE=Bazel -DCMAKE_INSTALL_PREFIX=/home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/sandbox/linux-sandbox/75/execroot/mod_pagespeed/bazel-out/k8-opt-exec-2B5CBBC6/bin/external/envoy/bazel/foreign_cc/zlib -DCMAKE_PREFIX_PATH=/home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/sandbox/linux-sandbox/75/execroot/mod_pagespeed/bazel-out/k8-opt-exec-2B5CBBC6/bin/external/envoy/bazel/foreign_cc/zlib.ext_build_deps -DCMAKE_RANLIB= -DCMAKE_MAKE_PROGRAM=/home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/sandbox/linux-sandbox/75/execroot/mod_pagespeed/external/ninja_1.11.0_linux/ninja -G Ninja /home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/sandbox/linux-sandbox/75/execroot/mod_pagespeed/external/net_zlib
bazel-out/k8-opt-exec-2B5CBBC6/bin/external/envoy/bazel/foreign_cc/zlib_foreign_cc/build_script.sh: line 101: /home/craig/.cache/bazel/_bazel_craig/4b3e6f4ab99bfbc114fb71e3556cd3d4/sandbox/linux-sandbox/75/execroot/mod_pagespeed/external/cmake-3.23.2-linux-x86_64/bin/cmake: cannot execute: required file not found
_____ END BUILD LOGS _____
rules_foreign_cc: Build wrapper script location: bazel-out/k8-opt-exec-2B5CBBC6/bin/external/envoy/bazel/foreign_cc/zlib_foreign_cc/wrapper_build_script.sh
rules_foreign_cc: Build script location: bazel-out/k8-opt-exec-2B5CBBC6/bin/external/envoy/bazel/foreign_cc/zlib_foreign_cc/build_script.sh
rules_foreign_cc: Build log location: bazel-out/k8-opt-exec-2B5CBBC6/bin/external/envoy/bazel/foreign_cc/zlib_foreign_cc/CMake.log

Target //pagespeed/automatic:automatic failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /home/craig/mod_pagespeed/pagespeed/controller/BUILD:17:16 Middleman _middlemen/_S_Spagespeed_Scontroller_Ccontroller_Uproto_Ugrpc-cc_library-compile failed: (Exit 127): bash failed: error executing command (from target @envoy//bazel/foreign_cc:zlib) /nix/store/5ynbf6wszmggr0abwifdagrixgnya5vy-bash-5.2-p15/bin/bash -c bazel-out/k8-opt-exec-2B5CBBC6/bin/external/envoy/bazel/foreign_cc/zlib_foreign_cc/wrapper_build_script.sh

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
INFO: Elapsed time: 18.655s, Critical Path: 3.25s
INFO: 368 processes: 298 internal, 70 linux-sandbox.
FAILED: Build did NOT complete successfully

The python2 error might actually be from the host, so I added a shell.nix that includes python2.

I can spend no more time on this today... Any pointers or hints would be appreciated. Should I be trying 6.1.0 or 6.3.2? Is that the right build target? Check the libjpeg-turbo commit and sha?

Here's what I have so far: https://github.com/cyounkins/mod_pagespeed/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant