[VL] Add S3 integration gluten tests#11516
Merged
Merged
Conversation
|
Run Gluten Clickhouse CI on x86 |
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
3d833d2 to
83216fa
Compare
|
Run Gluten Clickhouse CI on x86 |
Member
|
related: #9184 |
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
83216fa to
a42b41c
Compare
|
Run Gluten Clickhouse CI on x86 |
FelixYBW
changed the title
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
a42b41c to
c5b2ef3
Compare
|
Run Gluten Clickhouse CI on x86 |
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
c5b2ef3 to
d981b9b
Compare
|
Run Gluten Clickhouse CI on x86 |
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
d981b9b to
4e6db1b
Compare
|
Run Gluten Clickhouse CI on x86 |
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
4e6db1b to
c6337c3
Compare
|
Run Gluten Clickhouse CI on x86 |
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
c6337c3 to
93bb447
Compare
|
Run Gluten Clickhouse CI on x86 |
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
93bb447 to
83ba467
Compare
|
Run Gluten Clickhouse CI on x86 |
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
83ba467 to
58b43b0
Compare
|
Run Gluten Clickhouse CI on x86 |
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
58b43b0 to
29791d9
Compare
|
Run Gluten Clickhouse CI on x86 |
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
29791d9 to
80ce211
Compare
|
Run Gluten Clickhouse CI on x86 |
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
80ce211 to
c2582b5
Compare
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
a8e242e to
7fd348f
Compare
|
Run Gluten Clickhouse CI on x86 |
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
7fd348f to
94caf9d
Compare
|
Run Gluten Clickhouse CI on x86 |
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
94caf9d to
3d3e991
Compare
|
Run Gluten Clickhouse CI on x86 |
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
3d3e991 to
88b5842
Compare
|
Run Gluten Clickhouse CI on x86 |
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
2 times, most recently
from
ba0d377 to
bfd8b1f
Compare
|
Run Gluten Clickhouse CI on x86 |
1 similar comment
|
Run Gluten Clickhouse CI on x86 |
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
bfd8b1f to
3f316a8
Compare
|
Run Gluten Clickhouse CI on x86 |
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
3f316a8 to
cb47d58
Compare
|
Run Gluten Clickhouse CI on x86 |
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
cb47d58 to
e6b8855
Compare
|
Run Gluten Clickhouse CI on x86 |
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
e6b8855 to
09a6496
Compare
|
Run Gluten Clickhouse CI on x86 |
zhztheplayer
approved these changes
Feb 12, 2026
Comment on lines
+209
to
+216
| # Setup S3 JARs after gluten-it build | ||
| if [ "${{ matrix.os }}" = "ubuntu:22.04" ] && \ | ||
| [ "${{ matrix.spark }}" = "spark-3.5" ] && \ | ||
| [ "${{ matrix.java }}" = "java-8" ]; then | ||
| source $GITHUB_WORKSPACE/.github/workflows/util/install-resources.sh | ||
| SPARK_VERSION=$(echo "${{ matrix.spark }}" | sed 's/spark-//') | ||
| setup_minio "$SPARK_VERSION" | ||
| fi |
Member
There was a problem hiding this comment.
Hi @Mariamalmesfer, have you modified or added gluten-it job to write / read data actually to S3? I didn't find the relevant changes in this PR
Contributor
Author
There was a problem hiding this comment.
Hi, those changes were removed in a later commit
I’ve added them back now.
Part of apache#9184 Fixes apache#9618
Mariamalmesfer
force-pushed
the
integration-test-s3
branch
from
09a6496 to
b3da8d6
Compare
|
Run Gluten Clickhouse CI on x86 |
ReemaAlzaid
pushed a commit
to ReemaAlzaid/gluten
that referenced
this pull request
Feb 14, 2026
Co-authored-by: Mariam-Almesfer <mariam.almesfer@ibm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes are proposed in this pull request?
This patch enables S3 integration testing in the gluten-it framework by extending support for s3a:// paths and adding a new CI job.
Fixes #9618