Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Examples do not build #1419

Open
TomMD opened this issue Sep 13, 2020 · 10 comments · May be fixed by #2298
Open

Examples do not build #1419

TomMD opened this issue Sep 13, 2020 · 10 comments · May be fixed by #2298
Assignees

Comments

@TomMD
Copy link

TomMD commented Sep 13, 2020

Describe the bug
The examples in rules_haskell do not build.

To Reproduce
On my OS-X machine

git clone https://github.com/tweag/rules_haskell
cd rules_haskell/examples
bazel build //cat_hs/exec/cat_hs

And observe

ERROR: /private/tmp/rules_haskell/examples/cat_hs/exec/cat_hs/BUILD.bazel:6:15 HaskellCabalLibrary @stackage//:unliftio-core failed (Exit 1) cabal_wrapper failed: error executing command bazel-out/host/bin/external/rules_haskell/haskell/cabal_wrapper lib:unliftio-core unliftio-core-0.1.2.0 true external/stackage/unliftio-core-0.1.2.0/Setup.hs external/stackage/unliftio-core-0.1.2.0 ... (remaining 7 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
INFO: Elapsed time: 3.009s, Critical Path: 2.85s
INFO: 20 processes: 20 darwin-sandbox.
FAILED: Build did NOT complete successfully

Expected behavior
I thought this would build a binary that could be run.

Environment

  • OS name + version: OS-X with bazel 3.14 from brew.
  • Bazel version: 3.14
  • Version of the rules: This was on git rev aabeedc

Additional context
Add any other context about the problem here.

@TomMD
Copy link
Author

TomMD commented Sep 13, 2020

I also tried on linux:

docker run --rm -it ubuntu:20.04 bash
# follow bazel instructions here https://docs.bazel.build/versions/master/install-ubuntu.html
apt update && apt install -y git
git clone https://github.com/tweag/rules_haskell
cd rules_haskell/examples
bazel build //cat_hs/exec/cat_hs

Yielding:

# bazel build //cat_hs/exec/cat_hs
Starting local Bazel server and connecting to it...
DEBUG: /root/.cache/bazel/_bazel_root/0d15a96fd12ed9dca003553c71536f42/external/rules_haskell/haskell/private/versions.bzl:60:10: WARNING: bazel version is too recent. Supported versions range from 2.1.0 to 3.3.1, but found: 3.5.0
INFO: Repository rules_haskell_python_local instantiated at:
  no stack (--record_rule_instantiation_callstack not enabled)
Repository rule _config_python3_toolchain defined at:
  /root/.cache/bazel/_bazel_root/0d15a96fd12ed9dca003553c71536f42/external/rules_haskell/haskell/ghc_bindist.bzl:598:44: in <toplevel>
ERROR: An error occurred during the fetch of repository 'rules_haskell_python_local':
   Traceback (most recent call last):
        File "/root/.cache/bazel/_bazel_root/0d15a96fd12ed9dca003553c71536f42/external/rules_haskell/haskell/ghc_bindist.bzl", line 562, column 32, in _configure_python3_toolchain_impl
                python3_path = _find_python(repository_ctx)
        File "/root/.cache/bazel/_bazel_root/0d15a96fd12ed9dca003553c71536f42/external/rules_haskell/haskell/ghc_bindist.bzl", line 555, column 40, in _find_python
                result = repository_ctx.execute([python, "--version"])
Error in execute: Argument 0 of execute is neither a path, label, nor string.
ERROR: Analysis of target '//cat_hs/exec/cat_hs:cat_hs' failed; build aborted: Argument 0 of execute is neither a path, label, nor string.
INFO: Elapsed time: 6.960s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (14 packages loaded, 8 targets configured)
    currently loading: @bazel_tools//tools/jdk ... (2 packages)
    Fetching @rules_haskell_sh_posix_local; fetching
    Fetching @local_config_cc; fetching

@mboes
Copy link
Member

mboes commented Sep 20, 2020

The latter looks like an instance of #1416. Regarding macOS, could you run again with --verbose_failures?

@TomMD
Copy link
Author

TomMD commented Sep 20, 2020

On linux with python 3 the problem now looks much more manageable (mismatched hashes):

# bazel build --verbose_failures //cat_hs/exec/cat_hs
DEBUG: /root/.cache/bazel/_bazel_root/0d15a96fd12ed9dca003553c71536f42/external/rules_haskell/haskell/private/versions.bzl:60:10: WARNING: bazel version is too recent. Supported versions range from 2.1.0
to 3.3.1, but found: 3.5.0
INFO: Repository rules_haskell_ghc_linux_amd64 instantiated at:
  no stack (--record_rule_instantiation_callstack not enabled)
Repository rule _ghc_bindist defined at:
  /root/.cache/bazel/_bazel_root/0d15a96fd12ed9dca003553c71536f42/external/rules_haskell/haskell/ghc_bindist.bzl:362:31: in <toplevel>
INFO: Repository 'rules_haskell_ghc_linux_amd64' used the following cache hits instead of downloading the corresponding file.
 * Hash 'c419fd0aa9065fe4d2eb9a248e323860c696ddf3859749ca96a84938aee49107' for https://downloads.haskell.org/~ghc/8.6.5/ghc-8.6.5-x86_64-deb8-linux.tar.xz
If the definition of 'rules_haskell_ghc_linux_amd64' was updated, verify that the hashes were also updated.
ERROR: An error occurred during the fetch of repository 'rules_haskell_ghc_linux_amd64':
   Traceback (most recent call last):
        File "/root/.cache/bazel/_bazel_root/0d15a96fd12ed9dca003553c71536f42/external/rules_haskell/haskell/ghc_bindist.bzl", line 282, column 31, in _ghc_bindist_impl
                execute_or_fail_loudly(ctx, ["make", "install"])
        File "/root/.cache/bazel/_bazel_root/0d15a96fd12ed9dca003553c71536f42/external/rules_haskell/haskell/private/workspace_utils.bzl", line 18, column 13, in execute_or_fail_loudly
                fail("\n".join(["Command failed: " + " ".join(arguments), exec_result.stderr]))
Error in fail: Command failed: make install
src/main/tools/process-wrapper-legacy.cc:80: "execvp(make, ...)": No such file or directory
ERROR: /rules_haskell/examples/cat_hs/exec/cat_hs/BUILD.bazel:6:15: //cat_hs/exec/cat_hs:cat_hs depends on @rules_haskell_ghc_linux_amd64//:toolchain-impl in repository @rules_haskell_ghc_linux_amd64 which failed to fetch. no such package '@rules_haskell_ghc_linux_amd64//': Command failed: make install
src/main/tools/process-wrapper-legacy.cc:80: "execvp(make, ...)": No such file or directory
ERROR: Analysis of target '//cat_hs/exec/cat_hs:cat_hs' failed; build aborted: Analysis failed
INFO: Elapsed time: 31.728s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)
    Fetching @stackage; Restarting. 31s
    Fetching @rules_haskell_stack_update; fetching 31s

But on OS-X the problem is still pretty hairy looking:

Loading: 
Loading: 0 packages loaded
DEBUG: /private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/external/rules_haskell/haskell/private/versions.bzl:60:10: WARNING: bazel version is too recent. Supported versions range from 2.1.0 to 3.3.1, but found: 3.5.0
Analyzing: target //cat_hs/exec/cat_hs:cat_hs (0 packages loaded, 0 targets configured)
DEBUG: /private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/external/rules_haskell/haskell/cabal.bzl:2023:18: Stack version not recent enough. Downloading a newer version...
WARNING: /private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/external/zlib.hs/BUILD.bazel:3:11: in linkstatic attribute of cc_library rule @zlib.hs//:zlib.hs: setting 'linkstatic=1' is recommended if there are no object files. Since this rule was created by the macro 'cc_library', the error might have been caused by the macro implementation
INFO: Analyzed target //cat_hs/exec/cat_hs:cat_hs (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
[0 / 2] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[1 / 106] HaskellCabalLibrary @stackage//:colour; 1s darwin-sandbox ... (8 actions running)
ERROR: /private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/external/stackage/BUILD.bazel:266:22: HaskellCabalLibrary @stackage//:colour failed (Exit 1): cabal_wrapper failed: error executing command 
  (cd /private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples && \
  exec env - \
    LANG=en_US.UTF-8 \
    PATH=/usr/bin:/usr/local/bin:/bin:/usr/sbin:/Users/tommd/.nix-profile/bin \
    SDKROOT=macosx \
  bazel-out/host/bin/external/rules_haskell/haskell/cabal_wrapper lib:colour colour-2.3.5 true external/stackage/colour-2.3.5/Setup.lhs external/stackage/colour-2.3.5 bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/colour-2.3.5.conf.d '' '--flags=' '--ghc-option=-w' '--ghc-option=-optF=-w' '--ghc-option=-optl-Wl,-rpath,@loader_path/../../../../../_solib_darwin_x86_64/external_Srules_Uhaskell_Ughc_Udarwin_Uamd64' --)
Execution platform: @local_config_platform//:host

Use --sandbox_debug to see verbose messages from the sandbox cabal_wrapper failed: error executing command 
  (cd /private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples && \
  exec env - \
    LANG=en_US.UTF-8 \
    PATH=/usr/bin:/usr/local/bin:/bin:/usr/sbin:/Users/tommd/.nix-profile/bin \
    SDKROOT=macosx \
  bazel-out/host/bin/external/rules_haskell/haskell/cabal_wrapper lib:colour colour-2.3.5 true external/stackage/colour-2.3.5/Setup.lhs external/stackage/colour-2.3.5 bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/colour-2.3.5.conf.d '' '--flags=' '--ghc-option=-w' '--ghc-option=-optF=-w' '--ghc-option=-optl-Wl,-rpath,@loader_path/../../../../../_solib_darwin_x86_64/external_Srules_Uhaskell_Ughc_Udarwin_Uamd64' --)
Execution platform: @local_config_platform//:host

Use --sandbox_debug to see verbose messages from the sandbox
Error: DEVELOPER_DIR not set.
`cc_wrapper-python' failed in phase `C Compiler'. (Exit code: 250)
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/host/bin/external/rules_haskell/haskell/cabal_wrapper.runfiles/rules_haskell/haskell/cabal_wrapper.py", line 156, in <module>
    run([runghc] + runghc_args + [setup, "configure", \
  File "/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/host/bin/external/rules_haskell/haskell/cabal_wrapper.runfiles/rules_haskell/haskell/cabal_wrapper.py", line 42, in run
    subprocess.run(cmd, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, *args, **kwargs)
  File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/external/rules_haskell_ghc_darwin_amd64/bin/runghc', '/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/external/stackage/colour-2.3.5/Setup.lhs', 'configure', 'lib:colour', '--verbose=0', '--user', '--with-compiler=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/external/rules_haskell_ghc_darwin_amd64/bin/ghc', '--with-hc-pkg=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/external/rules_haskell_ghc_darwin_amd64/bin/ghc-pkg', '--with-ar=/usr/bin/ar', '--with-gcc=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python', '--with-strip=/usr/bin/strip', '--enable-deterministic', '--ghc-option=-pgma', '--ghc-option=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python', '--ghc-option=-pgmc', '--ghc-option=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python', '--ghc-option=-pgml', '--ghc-option=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python', '--ghc-option=-pgmP', '--ghc-option=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python -E -undef -traditional', '--ghc-option=-optc-fno-stack-protector', '--enable-relocatable', '--builddir=../../../bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/tmpmh_tuk86', '--prefix=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install', '--libdir=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/colour-2.3.5_iface', '--dynlibdir=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/lib', '--libsubdir=', '--bindir=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/bin', '--datadir=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/colour-2.3.5_data', '--datasubdir=', '--haddockdir=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/colour-2.3.5_haddock', '--htmldir=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/colour-2.3.5_haddock_html', '--package-db=clear', '--package-db=global', '--flags=', '--ghc-option=-w', '--ghc-option=-optF=-w', '--ghc-option=-optl-Wl,-rpath,@loader_path/../../../../../_solib_darwin_x86_64/external_Srules_Uhaskell_Ughc_Udarwin_Uamd64', '--package-db=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/colour-2.3.5.conf.d']' returned non-zero exit status 1.
Target //cat_hs/exec/cat_hs:cat_hs failed to build
ERROR: /private/tmp/rh/examples/cat_hs/exec/cat_hs/BUILD.bazel:6:15 HaskellCabalLibrary @stackage//:colour failed (Exit 1): cabal_wrapper failed: error executing command 
  (cd /private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples && \
  exec env - \
    LANG=en_US.UTF-8 \
    PATH=/usr/bin:/usr/local/bin:/bin:/usr/sbin:/Users/tommd/.nix-profile/bin \
    SDKROOT=macosx \
  bazel-out/host/bin/external/rules_haskell/haskell/cabal_wrapper lib:colour colour-2.3.5 true external/stackage/colour-2.3.5/Setup.lhs external/stackage/colour-2.3.5 bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/colour-2.3.5.conf.d '' '--flags=' '--ghc-option=-w' '--ghc-option=-optF=-w' '--ghc-option=-optl-Wl,-rpath,@loader_path/../../../../../_solib_darwin_x86_64/external_Srules_Uhaskell_Ughc_Udarwin_Uamd64' --)
Execution platform: @local_config_platform//:host

Use --sandbox_debug to see verbose messages from the sandbox cabal_wrapper failed: error executing command 
  (cd /private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples && \
  exec env - \
    LANG=en_US.UTF-8 \
    PATH=/usr/bin:/usr/local/bin:/bin:/usr/sbin:/Users/tommd/.nix-profile/bin \
    SDKROOT=macosx \
  bazel-out/host/bin/external/rules_haskell/haskell/cabal_wrapper lib:colour colour-2.3.5 true external/stackage/colour-2.3.5/Setup.lhs external/stackage/colour-2.3.5 bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/colour-2.3.5.conf.d '' '--flags=' '--ghc-option=-w' '--ghc-option=-optF=-w' '--ghc-option=-optl-Wl,-rpath,@loader_path/../../../../../_solib_darwin_x86_64/external_Srules_Uhaskell_Ughc_Udarwin_Uamd64' --)
Execution platform: @local_config_platform//:host

Use --sandbox_debug to see verbose messages from the sandbox
INFO: Elapsed time: 2.955s, Critical Path: 2.82s
INFO: 0 processes.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

N.B. I can run with DEVELOPER_DIR=<some directory> bazel build ... and get the same output.

@TomMD
Copy link
Author

TomMD commented Sep 21, 2020

If/when the bug is fixed at least for Linux, would you accept a PR that adds building these examples to the CI via a gitlab action or similar?

@Profpatsch
Copy link
Contributor

would you accept a PR that adds building these examples to the CI via a gitlab action or similar?

We are always open to PRs (especially improving examples/documentation/testsuites!).
Our CI is built run with buildkite (https://buildkite.com/tweag-1/rules-haskell) and CircleCI (for MacOS https://app.circleci.com/pipelines/github/tweag/rules_haskell).

@aherrmann
Copy link
Member

It's worth pointing out that the examples and tutorial are tested on CI in nixpkgs mode. However, they are not currently tested in bindist mode.

@timoffex
Copy link

timoffex commented Feb 7, 2021

Are there any workarounds for the DEVELOPER_DIR issue on OS-X? I tried setting BAZEL_USER_CPP_ONLY_TOOLCHAIN=1, made sure DEVELOPER_DIR wasn't empty (but it was empty for me before... what is it even?), and did a bazel clean before retrying. None of that seemed to change anything.

$ echo $DEVELOPER_DIR
/Applications/Xcode.app/Contents/Developer
$ echo $BAZEL_USER_CPP_ONLY_TOOLCHAIN
1
$ bazel --version
bazel 3.3.1-homebrew
$ bazel clean && bazel build //cat_hs/exec/cat_hs
<some text and eventually the "DEVELOPER_DIR not set" error>

I'm on macOS Catalina (Version 10.15.4).

@TomMD
Copy link
Author

TomMD commented Feb 13, 2021

@timoffex I think we need to consider bazel Haskell unmaintained for the non-nix case, there aren't CI tests or interested parties at this point.

The developer dir is actually a clang concept, observe:

$ DEVELOPER_DIR=foo clang
xcrun: error: missing DEVELOPER_DIR path: foo

I've tried adding export DEVELOPER_DIR=... to my .profile and rc but haven't had any impact so whatever is setting the environment isn't playing nice.

@aherrmann
Copy link
Member

We do run tests on CI for the macOS bindist use case. However, they currently do not include the example or tutorial. The main test suite and the start script on the other hand are tested on CI in the macOS bindist configuration and are expected to work.

As the example is currently only tested in a Nix configuration, it is possible that the some of the Nix provided toolchains collide with the standard toolchains in the non-Nix use-case. To test this, you could remove all the nixpkgs_* calls in the example's WORKSPACE and see if this helps.

We are working on unifying the CI setup, we've recently ported Linux and macOS to both use GH actions and are also working on covering more tests in all configurations. Unfortunately, we gotten to the examples, yet.

@tanyabouman
Copy link
Contributor

tanyabouman commented Apr 9, 2021

I reproduced the part of the problem that occurs on Ubuntu:

docker run --rm -it ubuntu:20.04 bash
# follow bazel instructions here https://docs.bazel.build/versions/master/install-ubuntu.html
apt update && apt install -y git
apt update && apt install -y python3
git clone https://github.com/tweag/rules_haskell
cd rules_haskell/examples
bazel build --verbose_failures //cat_hs/exec/cat_hs

and fixed it with

apt update && apt install -y build-essential libffi-dev libgmp-dev libtinfo5 libtinfo-dev python python3

since those are required dependencies.

However, it would be really nice if the error was a little more friendly than the current mess, so I will investigate that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants