Skip to content

GH-47081: [Release] Verify reproducible source build explicitly #47082

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kou merged 2 commits into from
Jul 13, 2025
Merged

GH-47081: [Release] Verify reproducible source build explicitly #47082

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a802313
GH-47081: [Release] Fix reproducible build check on macOS
kou Jul 11, 2025
d8c6cc5
Add TEST_SOURCE_REPRODUCIBLE to verify reproducible explicitly
kou Jul 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 22 additions & 7 deletions dev/release/verify-release-candidate.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -789,14 +789,29 @@ ensure_source_directory() {
if [ ! -d "${ARROW_SOURCE_DIR}" ]; then
pushd $ARROW_TMPDIR
fetch_archive ${dist_name}
git clone https://github.com/${GITHUB_REPOSITORY}.git arrow
pushd arrow
dev/release/utils-create-release-tarball.sh ${VERSION} ${RC_NUMBER}
if ! cmp ${dist_name}.tar.gz ../${dist_name}.tar.gz; then
echo "Source archive isn't reproducible"
return 1
verify_reproducible_build=true
if [ "${GITHUB_ACTIONS}" != "true" ]; then
verify_reproducible_build=false
fi
# We need GNU tar to verify reproducible build
if ! tar --version | grep --quiet --fixed GNU || \
! gtar --version | grep --quiet --fixed GNU; then
verify_reproducible_build=false
fi
# We need GNU gzip to verify reproducible build
if ! gzip --version | grep --quiet --fixed GNU; then
verify_reproducible_build=false
fi
if [ "${verify_reproducible_build}" = "true" ]; then
git clone https://github.com/${GITHUB_REPOSITORY}.git arrow
Comment thread
amoeba marked this conversation as resolved.
Outdated
pushd arrow
dev/release/utils-create-release-tarball.sh ${VERSION} ${RC_NUMBER}
if ! cmp ${dist_name}.tar.gz ../${dist_name}.tar.gz; then
echo "Source archive isn't reproducible"
return 1
fi
popd
fi
popd
tar xf ${dist_name}.tar.gz
popd
fi
Expand Down
1 change: 1 addition & 0 deletions dev/tasks/verify-rc/github.linux.amd64.docker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ jobs:
-e PYTHON_VERSION="3.12" \
{% endif %}
-e VERIFY_RC="{{ rc|default("") }}" \
-e VERBOSE=1 \

@kou kou Jul 11, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll remove this before we merge this. It's for just checking CI results.

-e TEST_DEFAULT=0 \
-e TEST_{{ target|upper }}=1 \
{{ distro }}-verify-rc
Expand Down
15 changes: 14 additions & 1 deletion dev/tasks/verify-rc/github.macos.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,9 @@ jobs:
if [ -x "${pkgconf}" ]; then
echo "PKG_CONFIG=${pkgconf}" >> $GITHUB_ENV
fi

# For reproducible source archive verification
brew install gzip
{% endif %}

- uses: actions/setup-java@v2
Expand Down Expand Up @@ -92,5 +95,15 @@ jobs:
{% else %}
GTest_SOURCE: SYSTEM
{% endif %}
VERBOSE: 1

@kou kou Jul 11, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll remove this before we merge this. It's for just checking CI results.

run: |
arrow/dev/release/verify-release-candidate.sh {{ release|default("") }} {{ rc|default("") }}
version={{ release|default("") }}
rc={{ rc|default("") }}
if [ -n "${version}" ] && [ -n "${rc}" ]; then
args=("${version}" "${rc}")
GITHUB_REPOSITORY=apache/arrow
else
args=()
GITHUB_REPOSITORY={{ arrow.github_repo }}
fi
arrow/dev/release/verify-release-candidate.sh "${args[@]}"
Loading