Framework: Match CPU parallelism with AT1 args #13602

sebrowne · 2024-11-14T18:33:06Z

@trilinos/framework

Motivation

Will allow us to better compare AT1 vs. AT2 performance and evaluate if we're ready to move builds onto the new system.

Will allow us to better compare AT1 vs. AT2 performance and evaluate if we're ready to move builds onto the new system. Signed-off-by: Samuel E. Browne <[email protected]>

trilinos-autotester · 2024-11-14T18:43:59Z

Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection is Not Necessary for this Pull Request.

trilinos-autotester · 2024-11-14T18:49:34Z

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: PR_gcc-openmpi-openmp

Build Num: 771
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel8_sems-gnu-8.5.0-openmpi-4.1.6-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE;AT2-SpecialApprove
PULLREQUESTNUM	13602
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	rhel8
TRILINOS_SOURCE_REPO	https://github.com/sebrowne/Trilinos
TRILINOS_SOURCE_SHA	`9ff4cb2`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`cfbf63c`

Build Information

Test Name: PR_gcc

Build Num: 821
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel8_sems-gnu-8.5.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE;AT2-SpecialApprove
PULLREQUESTNUM	13602
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	rhel8
TRILINOS_SOURCE_REPO	https://github.com/sebrowne/Trilinos
TRILINOS_SOURCE_SHA	`9ff4cb2`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`cfbf63c`

Build Information

Test Name: PR_gcc-openmpi_debug

Build Num: 822
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel8_sems-gnu-8.5.0-openmpi-4.1.6-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE;AT2-SpecialApprove
PULLREQUESTNUM	13602
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	rhel8
TRILINOS_SOURCE_REPO	https://github.com/sebrowne/Trilinos
TRILINOS_SOURCE_SHA	`9ff4cb2`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`cfbf63c`

Build Information

Test Name: PR_clang

Build Num: 820
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel8_sems-clang-11.0.1-openmpi-4.0.5-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE;AT2-SpecialApprove
PULLREQUESTNUM	13602
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	rhel8
TRILINOS_SOURCE_REPO	https://github.com/sebrowne/Trilinos
TRILINOS_SOURCE_SHA	`9ff4cb2`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`cfbf63c`

Build Information

Test Name: Trilinos_PR_python3

Build Num: 4779
Status: STARTED

Jenkins Parameters

Parameter Name	Value
BLOCKING_BUILD	false
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel8_aue-gnu-12.1.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS	AT: AUTOMERGE;AT2-SpecialApprove
PULLREQUESTNUM	13602
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	rhel8
TRILINOS_SOURCE_REPO	https://github.com/sebrowne/Trilinos
TRILINOS_SOURCE_SHA	`9ff4cb2`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`cfbf63c`

Build Information

Test Name: PR_cuda

Build Num: 819
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel8_sems-cuda-11.4.2-gnu-10.1.0-openmpi-4.1.6_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE;AT2-SpecialApprove
PULLREQUESTNUM	13602
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	rhel8-gpu
TRILINOS_SOURCE_REPO	https://github.com/sebrowne/Trilinos
TRILINOS_SOURCE_SHA	`9ff4cb2`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`cfbf63c`

Build Information

Test Name: PR_intel

Build Num: 740
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel8_sems-intel-2021.3-sems-openmpi-4.1.6_release-debug_shared_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE;AT2-SpecialApprove
PULLREQUESTNUM	13602
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	rhel8
TRILINOS_SOURCE_REPO	https://github.com/sebrowne/Trilinos
TRILINOS_SOURCE_SHA	`9ff4cb2`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`cfbf63c`

Build Information

Test Name: PR_cuda-uvm

Build Num: 819
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel8_sems-cuda-11.4.2-gnu-10.1.0-openmpi-4.1.6_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE;AT2-SpecialApprove
PULLREQUESTNUM	13602
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	rhel8
TRILINOS_SOURCE_REPO	https://github.com/sebrowne/Trilinos
TRILINOS_SOURCE_SHA	`9ff4cb2`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`cfbf63c`

Using Repos:

Repo: TRILINOS (sebrowne/Trilinos)

Pull Request Author: sebrowne

sebrowne · 2024-11-14T20:27:31Z

First pass (this PR itself) shows excellent timing comparisons:

GCC         | AT1 | AT2 |     GCC+OpenMPI | AT1 | AT2 | 
=========================     =========================
Build times | 9m  | 3m  |     Build times | 20m | 14m |
Test times  | 8m  | 5m  |     Test times  | 49m | 43m |

trilinos-autotester · 2024-11-14T20:29:02Z

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: PR_gcc-openmpi-openmp

Build Num: 771
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel8_sems-gnu-8.5.0-openmpi-4.1.6-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE;AT2-SpecialApprove
PULLREQUESTNUM	13602
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	rhel8
TRILINOS_SOURCE_REPO	https://github.com/sebrowne/Trilinos
TRILINOS_SOURCE_SHA	`9ff4cb2`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`cfbf63c`

Build Information

Test Name: PR_gcc

Build Num: 821
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel8_sems-gnu-8.5.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE;AT2-SpecialApprove
PULLREQUESTNUM	13602
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	rhel8
TRILINOS_SOURCE_REPO	https://github.com/sebrowne/Trilinos
TRILINOS_SOURCE_SHA	`9ff4cb2`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`cfbf63c`

Build Information

Test Name: PR_gcc-openmpi_debug

Build Num: 822
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel8_sems-gnu-8.5.0-openmpi-4.1.6-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE;AT2-SpecialApprove
PULLREQUESTNUM	13602
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	rhel8
TRILINOS_SOURCE_REPO	https://github.com/sebrowne/Trilinos
TRILINOS_SOURCE_SHA	`9ff4cb2`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`cfbf63c`

Build Information

Test Name: PR_clang

Build Num: 820
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel8_sems-clang-11.0.1-openmpi-4.0.5-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE;AT2-SpecialApprove
PULLREQUESTNUM	13602
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	rhel8
TRILINOS_SOURCE_REPO	https://github.com/sebrowne/Trilinos
TRILINOS_SOURCE_SHA	`9ff4cb2`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`cfbf63c`

Build Information

Test Name: Trilinos_PR_python3

Build Num: 4779
Status: PASSED

Jenkins Parameters

Parameter Name	Value
BLOCKING_BUILD	false
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel8_aue-gnu-12.1.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS	AT: AUTOMERGE;AT2-SpecialApprove
PULLREQUESTNUM	13602
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	rhel8
TRILINOS_SOURCE_REPO	https://github.com/sebrowne/Trilinos
TRILINOS_SOURCE_SHA	`9ff4cb2`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`cfbf63c`

Build Information

Test Name: PR_cuda

Build Num: 819
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel8_sems-cuda-11.4.2-gnu-10.1.0-openmpi-4.1.6_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE;AT2-SpecialApprove
PULLREQUESTNUM	13602
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	rhel8-gpu
TRILINOS_SOURCE_REPO	https://github.com/sebrowne/Trilinos
TRILINOS_SOURCE_SHA	`9ff4cb2`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`cfbf63c`

Build Information

Test Name: PR_intel

Build Num: 740
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel8_sems-intel-2021.3-sems-openmpi-4.1.6_release-debug_shared_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE;AT2-SpecialApprove
PULLREQUESTNUM	13602
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	rhel8
TRILINOS_SOURCE_REPO	https://github.com/sebrowne/Trilinos
TRILINOS_SOURCE_SHA	`9ff4cb2`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`cfbf63c`

Build Information

Test Name: PR_cuda-uvm

Build Num: 819
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel8_sems-cuda-11.4.2-gnu-10.1.0-openmpi-4.1.6_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE;AT2-SpecialApprove
PULLREQUESTNUM	13602
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	rhel8
TRILINOS_SOURCE_REPO	https://github.com/sebrowne/Trilinos
TRILINOS_SOURCE_SHA	`9ff4cb2`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`cfbf63c`

CDash Test Results for PR# 13602.

trilinos-autotester · 2024-11-14T20:29:20Z

Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ achauphan ]!

trilinos-autotester · 2024-11-14T20:29:27Z

Status Flag 'Pull Request AutoTester' - Pull Request will be Automerged

trilinos-autotester · 2024-11-14T20:29:39Z

Merge on Pull Request# 13602: IS A SUCCESS - Pull Request successfully merged

ccober6 · 2024-11-14T20:36:41Z

First pass (this PR itself) shows excellent timing comparisons:

GCC         | AT1 | AT2 |     GCC+OpenMPI | AT1 | AT2 | 
=========================     =========================
Build times | 9m  | 3m  |     Build times | 20m | 49m |
Test times  | 8m  | 5m  |     Test times  | 14m | 43m |

@sebrowne, am I reading this correctly that GCC+OpenMPI is 2-3x slower with AT2?

sebrowne · 2024-11-14T20:38:13Z

@sebrowne, am I reading this correctly that GCC+OpenMPI is 2-3x slower with AT2?

Doh, no, I flipped the axes on accident. I fixed it in my original comment.

ccober6 · 2024-11-14T20:39:40Z

Ah, very good!

Match CPU parallelism with AT1 args

9ff4cb2

Will allow us to better compare AT1 vs. AT2 performance and evaluate if we're ready to move builds onto the new system. Signed-off-by: Samuel E. Browne <[email protected]>

sebrowne added AT: AUTOMERGE Causes the PR autotester to automatically merge the PR branch once approvals are completed AT2-SpecialApprove (Beta) Special approval label for AT2. labels Nov 14, 2024

sebrowne requested review from fryeguy52, achauphan and jmlapre November 14, 2024 18:33

achauphan approved these changes Nov 14, 2024

View reviewed changes

trilinos-autotester merged commit 0dd826f into trilinos:develop Nov 14, 2024
17 checks passed

trilinos-autotester deleted the increase-at2-parallelism branch November 14, 2024 20:29

trilinos-autotester removed the AT: AUTOMERGE Causes the PR autotester to automatically merge the PR branch once approvals are completed label Nov 14, 2024

Framework: Match CPU parallelism with AT1 args #13602

Framework: Match CPU parallelism with AT1 args #13602

Conversation

sebrowne commented Nov 14, 2024

Motivation

trilinos-autotester commented Nov 14, 2024

trilinos-autotester commented Nov 14, 2024

Build Information

Test Name: PR_gcc-openmpi-openmp

Jenkins Parameters

Build Information

Test Name: PR_gcc

Jenkins Parameters

Build Information

Test Name: PR_gcc-openmpi_debug

Jenkins Parameters

Build Information

Test Name: PR_clang

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_python3

Jenkins Parameters

Build Information

Test Name: PR_cuda

Jenkins Parameters

Build Information

Test Name: PR_intel

Jenkins Parameters

Build Information

Test Name: PR_cuda-uvm

Jenkins Parameters

Using Repos:

sebrowne commented Nov 14, 2024 • edited Loading

trilinos-autotester commented Nov 14, 2024

Build Information

Test Name: PR_gcc-openmpi-openmp

Jenkins Parameters

Build Information

Test Name: PR_gcc

Jenkins Parameters

Build Information

Test Name: PR_gcc-openmpi_debug

Jenkins Parameters

Build Information

Test Name: PR_clang

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_python3

Jenkins Parameters

Build Information

Test Name: PR_cuda

Jenkins Parameters

Build Information

Test Name: PR_intel

Jenkins Parameters

Build Information

Test Name: PR_cuda-uvm

Jenkins Parameters

trilinos-autotester commented Nov 14, 2024

trilinos-autotester commented Nov 14, 2024

trilinos-autotester commented Nov 14, 2024

ccober6 commented Nov 14, 2024

sebrowne commented Nov 14, 2024

ccober6 commented Nov 14, 2024

sebrowne commented Nov 14, 2024 •

edited

Loading