v2.1.0-rc* clean slate builds fail due to unresolvable commit ca8daa2870f3

[kiriakos]

Checklist

• This is not a question or a support request. If you have any boost related questions, please ask in the discussion forum.
• This is not a new feature request. If it is, please file a feature request instead.
• This is not an enhancement request. If it is, please file a improvement suggestion instead.
• I have searched on the issue tracker and the discussion forum, and there is no existing related issue or discussion.
• I am running the Latest release, or the most recent RC(release canadiate) for the upcoming release or the dev branch(master), or have an issue updating to any of these.
• I did not make any code changes to boost.

Boost component

• boost daemon - storage providers
• boost client
• boost UI
• boost data-transfer
• boost index-provider
• Other

Boost Version

v2.1.0-rc3
v2.1.0-rc2
and probably others

Describe the Bug

On the morning of 2023-11-09 building boost, with the steps outlined in the installation instructions, started failing. Troubleshooting the issue pointed to a go module dependency github.com/filecoin-project/boost/extern/[email protected] that wasn't resolvable any more, neither by go's hosted module cache nor on Github itself. Checking in the repo and on github the reason for this turned out to be that commit ca8daa2870f3 wasn't referenced by the repo any more.

Thankfully after contacting @masih the commit could be reintroduced to a branch of the repo resolving the issue.

Slack discussion:
first
resolution

Logging Information

The build errors look like this:

> cmd/boost/wallet_cmd.go:18:2: github.com/filecoin-project/boost/extern/[email protected]: invalid version: unknown revision ca8daa2870f3
> cmd/boost/retrieve_cmd.go:24:2: github.com/filecoin-project/boost/extern/[email protected]: invalid version: unknown revision ca8daa2870f3

Repo Steps

To isolate the issue I created a Dockerfile to make static, clean slate builds of boost.

NOTE: as the commit in question has been made resolvable again on the filecoin-project/boost repo builds are working again so the error described here will not appear. If one where to make a similar situation though, this Dockerfile could be used to illustrate the issue.

FROM ubuntu:22.04

# Deps & Build toolchain ######################################################
RUN apt-get update \
    && apt-get install -y mesa-opencl-icd \
                          ocl-icd-opencl-dev \
                          gcc \
                          git \
                          bzr \
                          jq \
                          pkg-config \
                          curl \
                          clang \
                          build-essential \
                          hwloc \
                          libhwloc-dev \
                          wget \
                          ca-certificates \
                          gnupg \
    && rm -rf /var/lib/apt/lists/*

# Node16
# https://github.com/nodesource/distributions#installation-instructions
ENV NODE_MAJOR=16
RUN mkdir -p /etc/apt/keyrings \
    && curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg \
    && echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list \
    && apt-get update -y \
    && apt-get install -y nodejs \
    && rm -rf /var/lib/apt/lists/*

# Go
ARG BOOST_GO_VERSION=1.21.4
RUN curl -L -o go.tar.gz \
            https://go.dev/dl/go${BOOST_GO_VERSION}.linux-amd64.tar.gz \
         && rm -rf /usr/local/go \
         && tar -C /usr/local -xzf go.tar.gz

ENV PATH=$PATH:/usr/local/go/bin:/root/go/bin
RUN go version

# Rust
ENV CARGO_HOME=/usr/local/cargo
ENV RUSTUP_HOME=/usr/local/rustup
RUN curl -o /usr/local/bin/rustup \
        --proto '=https' \
        --tlsv1.2 \
        -sSf https://sh.rustup.rs \
    && chmod +x /usr/local/bin/rustup \
    && rustup -y

ENV PATH=$PATH:/usr/local/cargo/bin

ENV RUSTFLAGS="-C target-cpu=native -g"
ENV FFI_BUILD_FROM_SOURCE=1

###############################################################################

# Boost
ARG BOOST_BUILD_TARGET
ARG BOOST_VERSION=v2.1.0-rc3
WORKDIR /root
RUN git clone https://github.com/filecoin-project/boost \
    && cd boost \
    && git checkout $BOOST_VERSION \
    && go mod tidy \
    && make clean calibnet \
    && make install

EXPOSE 8044

1. Run 'docker build -f Dockerfile.static -t boost-static .'
2. See error github.com/filecoin-project/boost/extern/[email protected]: invalid version: unknown revision ca8daa2870f3
   ...

[masih]

Resolved by restoring the branch that contains the commit.

[masih]

Changing the dependency to a commit from main.