GH-46338: [C++] Add compile step for Meson in cpp_build.sh #46339

WillAyd · 2025-05-06T20:13:04Z

Rationale for this change

This makes it possible to control the number of jobs used with Meson in the cpp_build.sh script

What changes are included in this PR?

Updates to cpp_build.sh

Are these changes tested?

Yes

Are there any user-facing changes?

No

GitHub Issue: [C++] Add Compile Step to Meson Configuration in cpp build script #46338

github-actions · 2025-05-06T20:13:29Z

⚠️ GitHub issue #46338 has been automatically assigned in GitHub to PR creator.

WillAyd · 2025-05-06T20:16:27Z

@raulcd if I understand correctly, the CMake configuration can read the CMAKE_BUILD_PARALLEL_LEVEL environment variable. Are you looking to recreate that here? Should we use a different name?

WillAyd · 2025-05-06T20:29:09Z

@github-actions crossbow submit *meson

github-actions · 2025-05-06T20:31:24Z

Revision: 6b788ef

Submitted crossbow builds: ursacomputing/crossbow @ actions-a671c6cb33

Task	Status
test-conda-cpp-meson

WillAyd · 2025-05-06T20:36:56Z

@github-actions crossbow submit *meson

github-actions · 2025-05-06T20:39:11Z

Revision: ef09ed6

Submitted crossbow builds: ursacomputing/crossbow @ actions-098f7bf390

Task	Status
test-conda-cpp-meson

kou · 2025-05-07T01:02:30Z

ci/scripts/cpp_build.sh

+    time {
+      meson compile -j $[${n_jobs} + 1];
+      meson install;
+  }


We need only compile time because install time will not be slow:

Suggested change

time {

meson compile -j $[${n_jobs} + 1];

meson install;

}

time meson compile -j $[${n_jobs} + 1]

meson install

kou · 2025-05-07T01:04:52Z

ci/scripts/cpp_build.sh

+    time {
+      meson compile -j $[${n_jobs} + 1];
+      meson install;
+  }
 else
  export CMAKE_BUILD_PARALLEL_LEVEL=${CMAKE_BUILD_PARALLEL_LEVEL:-$[${n_jobs} + 1]}


How about defining ARROW_BUILD_PARALLEL or something and use it as the default for Meson and CMake?

: ${ARROW_BUILD_PARALLEL:=$[${n_jobs} + 1]} if [ "${ARROW_USE_MESON:-OFF}" = "ON" ]; then meson compile -j ${ARROW_BUILD_PARALLEL} else export CMAKE_BUILD_PARALLEL_LEVEL=${ARROW_BUILD_PARALLEL} fi

WillAyd · 2025-05-07T06:27:27Z

@github-actions crossbow submit *meson

github-actions · 2025-05-07T06:29:54Z

Revision: f471fe1

Submitted crossbow builds: ursacomputing/crossbow @ actions-db63e10f79

Task	Status
test-conda-cpp-meson

raulcd

Thanks for working on this @WillAyd

raulcd · 2025-05-07T08:28:10Z

ci/scripts/cpp_build.sh

 else
-  export CMAKE_BUILD_PARALLEL_LEVEL=${CMAKE_BUILD_PARALLEL_LEVEL:-$[${n_jobs} + 1]}
+  export ARROW_BUILD_PARALLEL=${ARROW_BUILD_PARALLEL:-$[${n_jobs} + 1]}


CMake doesn't know what ARROW_BUILD_PARALLEL is, we should still use CMAKE_BUILD_PARALLEL_LEVEL which we can define outside of the if : ${ARROW_BUILD_PARALLEL:=$[${n_jobs} + 1]}

Suggested change

export ARROW_BUILD_PARALLEL=${ARROW_BUILD_PARALLEL:-$[${n_jobs} + 1]}

export CMAKE_BUILD_PARALLEL_LEVEL=${ARROW_BUILD_PARALLEL}

Nice catch - done!

WillAyd · 2025-05-07T14:27:45Z

@github-actions crossbow submit *meson

github-actions · 2025-05-07T14:30:03Z

Revision: 180ad7a

Submitted crossbow builds: ursacomputing/crossbow @ actions-d3f9e22713

Task	Status
test-conda-cpp-meson

kou · 2025-05-08T06:17:39Z

ci/scripts/cpp_build.sh

@@ -283,9 +283,10 @@ else
 fi

 if [ "${ARROW_USE_MESON:-OFF}" = "ON" ]; then
-  time meson install
+  time meson compile -j ${ARROW_BUILD_PARALLEL:-$[${n_jobs} + 1]}


Could you define ARROW_BUILD_PARALLEL out of this if to avoid the default value duplication?

kou · 2025-05-08T06:19:16Z

dev/release/verify-release-candidate.sh

-  export CMAKE_BUILD_PARALLEL_LEVEL=${CMAKE_BUILD_PARALLEL_LEVEL:-${NPROC}}
+  export CMAKE_BUILD_PARALLEL_LEVEL=${ARROW_BUILD_PARALLEL:-${NPROC}}


Hmm. I think that we don't need this because we always use CMake in this script.
If we want this, could you use ARROW_BUILD_PARALLEL only if CMAKE_BUILD_PARALLEL isn't defined?

Good point - for now I have just reverted this

kou · 2025-05-08T06:20:12Z

ci/scripts/cpp_build.sh

 else
-  export CMAKE_BUILD_PARALLEL_LEVEL=${CMAKE_BUILD_PARALLEL_LEVEL:-$[${n_jobs} + 1]}
+  export CMAKE_BUILD_PARALLEL_LEVEL=${ARROW_BUILD_PARALLEL:-$[${n_jobs} + 1]}


Could you use ARROW_BUILD_PARALLEL only if CMAKE_BUILD_PARALLEL_LEVEL? (If both of CMAKE_BUILD_PARALLEL_LEVEL and ARROW_BUILD_PARALLEL, CMAKE_BUILD_PARALLEL_LEVEL should be used.)

It looks a bit more complicated but let me know if the current iteration is now inline with your expectations or not

raulcd

👍 LGTM now

kou · 2025-05-09T00:46:48Z

ci/scripts/cpp_build.sh

@@ -282,10 +282,12 @@ else
    ${source_dir}
 fi

+ARROW_BUILD_PARALLEL=${CMAKE_BUILD_PARALLEL_LEVEL:-${ARROW_BUILD_PARALLEL:-$[${n_jobs} + 1]}}


Suggested change

ARROW_BUILD_PARALLEL=${CMAKE_BUILD_PARALLEL_LEVEL:-${ARROW_BUILD_PARALLEL:-$[${n_jobs} + 1]}}

: ${ARROW_BUILD_PARALLEL:=$[${n_jobs} + 1]}

kou · 2025-05-09T00:47:34Z

ci/scripts/cpp_build.sh

 else
-  export CMAKE_BUILD_PARALLEL_LEVEL=${CMAKE_BUILD_PARALLEL_LEVEL:-$[${n_jobs} + 1]}
+  export CMAKE_BUILD_PARALLEL_LEVEL=${ARROW_BUILD_PARALLEL}


Suggested change

export CMAKE_BUILD_PARALLEL_LEVEL=${ARROW_BUILD_PARALLEL}

: ${CMAKE_BUILD_PARALLEL_LEVEL:=${ARROW_BUILD_PARALLEL}}

export CMAKE_BUILD_PARALLEL_LEVEL

WillAyd · 2025-05-09T01:17:20Z

@github-actions crossbow submit *meson

github-actions · 2025-05-09T01:19:34Z

Revision: fb1d52b

Submitted crossbow builds: ursacomputing/crossbow @ actions-770b1544a4

Task	Status
test-conda-cpp-meson

WillAyd requested review from assignUser, jonkeane, kou and raulcd as code owners May 6, 2025 20:13

github-actions bot added the awaiting review Awaiting review label May 6, 2025

apacheGH-46338: [C++] Add compile step for Meson in cpp_build.sh

ef09ed6

WillAyd force-pushed the meson-add-compile-step branch from 6b788ef to ef09ed6 Compare May 6, 2025 20:36

kou reviewed May 7, 2025

View reviewed changes

github-actions bot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels May 7, 2025

github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels May 7, 2025

raulcd reviewed May 7, 2025

View reviewed changes

github-actions bot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels May 7, 2025

WillAyd force-pushed the meson-add-compile-step branch from f471fe1 to 180ad7a Compare May 7, 2025 14:12

github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels May 7, 2025

kou reviewed May 8, 2025

View reviewed changes

github-actions bot removed the awaiting change review Awaiting change review label May 8, 2025

github-actions bot added the awaiting changes Awaiting changes label May 8, 2025

WillAyd force-pushed the meson-add-compile-step branch from 180ad7a to 80d8d0c Compare May 8, 2025 13:48

github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels May 8, 2025

raulcd approved these changes May 8, 2025

View reviewed changes

github-actions bot added awaiting merge Awaiting merge and removed awaiting change review Awaiting change review labels May 8, 2025

kou reviewed May 9, 2025

View reviewed changes

github-actions bot added awaiting changes Awaiting changes and removed awaiting merge Awaiting merge labels May 9, 2025

kou feedback

fb1d52b

WillAyd force-pushed the meson-add-compile-step branch from 80d8d0c to fb1d52b Compare May 9, 2025 01:17

github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels May 9, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GH-46338: [C++] Add compile step for Meson in cpp_build.sh #46339

GH-46338: [C++] Add compile step for Meson in cpp_build.sh #46339

WillAyd commented May 6, 2025 •

edited by github-actions bot

Loading

github-actions bot commented May 6, 2025

WillAyd commented May 6, 2025

WillAyd commented May 6, 2025

github-actions bot commented May 6, 2025

WillAyd commented May 6, 2025

github-actions bot commented May 6, 2025

kou May 7, 2025

kou May 7, 2025

WillAyd commented May 7, 2025

github-actions bot commented May 7, 2025

raulcd left a comment

raulcd May 7, 2025

WillAyd May 7, 2025

WillAyd commented May 7, 2025

github-actions bot commented May 7, 2025

kou May 8, 2025

kou May 8, 2025 •

edited

Loading

WillAyd May 8, 2025

kou May 8, 2025

WillAyd May 8, 2025

raulcd left a comment

kou May 9, 2025

kou May 9, 2025

WillAyd commented May 9, 2025

github-actions bot commented May 9, 2025

	export ARROW_BUILD_PARALLEL=${ARROW_BUILD_PARALLEL:-$[${n_jobs} + 1]}
	export CMAKE_BUILD_PARALLEL_LEVEL=${ARROW_BUILD_PARALLEL}

		export CMAKE_BUILD_PARALLEL_LEVEL=${CMAKE_BUILD_PARALLEL_LEVEL:-${NPROC}}
		export CMAKE_BUILD_PARALLEL_LEVEL=${ARROW_BUILD_PARALLEL:-${NPROC}}

	ARROW_BUILD_PARALLEL=${CMAKE_BUILD_PARALLEL_LEVEL:-${ARROW_BUILD_PARALLEL:-$[${n_jobs} + 1]}}
	: ${ARROW_BUILD_PARALLEL:=$[${n_jobs} + 1]}

	export CMAKE_BUILD_PARALLEL_LEVEL=${ARROW_BUILD_PARALLEL}
	: ${CMAKE_BUILD_PARALLEL_LEVEL:=${ARROW_BUILD_PARALLEL}}
	export CMAKE_BUILD_PARALLEL_LEVEL

GH-46338: [C++] Add compile step for Meson in cpp_build.sh #46339

Are you sure you want to change the base?

GH-46338: [C++] Add compile step for Meson in cpp_build.sh #46339

Conversation

WillAyd commented May 6, 2025 • edited by github-actions bot Loading

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

github-actions bot commented May 6, 2025

WillAyd commented May 6, 2025

WillAyd commented May 6, 2025

github-actions bot commented May 6, 2025

WillAyd commented May 6, 2025

github-actions bot commented May 6, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WillAyd commented May 7, 2025

github-actions bot commented May 7, 2025

raulcd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WillAyd commented May 7, 2025

github-actions bot commented May 7, 2025

Choose a reason for hiding this comment

kou May 8, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

raulcd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WillAyd commented May 9, 2025

github-actions bot commented May 9, 2025

WillAyd commented May 6, 2025 •

edited by github-actions bot

Loading

kou May 8, 2025 •

edited

Loading