+
+
diff --git a/pages/training_resources.md b/pages/training_resources.md
new file mode 100644
index 00000000..22e1645a
--- /dev/null
+++ b/pages/training_resources.md
@@ -0,0 +1,84 @@
+---
+title: Training Resources
+---
+
+
+
+| Resource Name | Description | Domain | Public | Contact |
+|--------------|-------------|---------|---------|----------|
+| [Software Carpentries](https://software-carpentry.org) | Foundational computational skills - shell, Git, Python/R | Domain agnostic | Yes | Aleksandra Nenadic |
+| [Data Carpentry](https://datacarpentry.org/) | Foundational data skills | Domain agnostic | Yes | Aleksandra Nenadic |
+| [TeSS Elixir](https://tess.elixir-europe.org/) | Registry of life sciences training materials | | yes | Aleksandra Nenadic |
+| [Tools and practices for FAIR research software](https://carpentries-incubator.github.io/fair-research-software/) | FAIR research software training course by Software sustainabilioty Institute | | Yes, Release in June '24 | Aleksandra Nenadic |
+| [ASTERICS programming school](https://github.com/Asterics2020-Obelics/School2019) | | Mostly agnostic+some domain specific course | | Thomas Vuillaume |
+| [ESCAPE data science school](https://escape2020.github.io/school2021/) | | Mostly agnostic+some domain specific course | | Thomas Vuillaume |
+| [Gray Scott battle school](https://indico.in2p3.fr/event/29755/) | Programming heterogeneous hardware | Domain agnostic | | Thomas Vuillaume |
+| [HIFIS RSE Training](https://hifis.net/services/software/training.html) | | | | Guido Juckeland |
+| [FAIR Principles for Research Software](https://www.rd-alliance.org/group/fair-research-software-fair4rs-wg/outcomes/fair-principles-research-software-fair4rs) | | | yes | Neil Chue Hong, Daniel Garijo |
+| [D5.2 - Metrics for automated FAIR software assessment in a disciplinary context](https://pure.knaw.nl/portal/en/publications/d52-metrics-for-automated-fair-software-assessment-in-a-disciplin) | | | yes | Neil Chue Hong, Daniel Garijo |
+| [EOSC Task Force for infrastructure for code quality](https://zenodo.org/records/10723608) | | | yes | Daniel Garijo |
+| [Codemeta terms](https://codemeta.github.io/terms/) | | | yes | Daniel Garijo |
+| [FAIR for Machine Learning IG](https://www.rd-alliance.org/groups/fair-machine-learning-fair4ml-ig) | | | | Fotis, Daniel Garijo |
+| [eScience center training](https://www.esciencecenter.nl/digital-skills/) | | | | Mateusz Kuzak |
+| [NL RS Training](https://researchsoftwaretraining.nl/) | | | | Mateusz Kuzak |
+| [Software Mgmt Plans Guide](https://www.esciencecenter.nl/national-guidelines-for-software-management-plans/) | | | | Mateusz Kuzak |
+| [ELIXIR-STEERS](https://elixir-europe.org/about-us/how-funded/eu-projects/steers) | | | | Silvio Tosatto / Damiano Piovesan |
+| [ELIXIR Data Platform](https://elixir-europe.org/platforms/data) | | | | Silvio Tosatto / Damiano Piovesan |
+| [ELIXIR IDP Community](https://elixir-europe.org/communities/intrinsically-disordered-proteins) | | | | Damiano Piovesan |
+| [ENVRI Knowledge base]() | | | | Zhiming Zhao |
+| [LifeWatch ERIC Training Platform](https://training.lifewatch.eu/) | | | | Alberto Basset / Eleonora Parisi |
+| [LifeWatch ERIC summer schools](https://training.lifewatch.eu/biodiversity-ecampus/resources/?category=14) | | | | Alberto Basset / Eleonora Parisi |
+| [e-Biodiversity and ecosystem sciences training](https://training.lifewatch.eu:9001/pluginfile.php/1047/mod_resource/content/1/Masters%20EBES%20Brochure%2022.pdf) | | | | Alberto Basset / Eleonora Parisi |
+| [HEP Software Foundation Training WG](https://hepsoftwarefoundation.org/workinggroups/training.html) | Self-taught and instructor based set of course on tools and languages in software engineering | agnostic | yes | Stefan Roiser |
+| [EIROforum WG Training and Career Development](https://www.eiroforum.org/about-eiroforum/governance/) | Exchange of information about training activities and best practices | agnostic | probably (currenty being setup) | Stefan Roiser |
+| [CERN schools on computing](https://csc.web.cern.ch/) | various thematic and general computing schools | partialy specific | yes | Stefan Roiser |
+| [HEP C++ Course and Hands-on Training](https://github.com/hsf-training/cpluspluscourse) | hybrid 3 day courses on different levels on C++ programming | agnostic | planned for the future | Stefan Roiser |
+| [NextGen Triggers training](https://nextgentriggers.web.cern.ch/) | aim to create a curriculum for software development | TBD | started in Jan '24 | Stefan Roiser |
+| [Good Enough Practices in Scientific Computing](https://carpentries-lab.github.io/good-enough-practices/) | This lesson is targeted at a broad audience of researchers who want to learn how to be more efficient and effective in their data analysis and computing, whatever their career stage. | agnostic | yes | Aleks Nenadic |
+| [UNIVERSE-HPC](https://www.universe-hpc.ac.uk/) | Developing curriculum and course for RSEs specialising in large scale computing (simulation & modelling and data science) | agnostic | yes | Neii Chue Hong |
+| [Bioconductor training material](https://bioconductor.org/help/) | | specific | yes | Fotis Psomopoulos |
+| [Git repositories with bioinformatics training material](https://glittr.org/) | | specific | yes | Fotis Psomopoulos |
+| [Code Refinery](https://coderefinery.org/lessons/from-coderefinery/) | Workshop / self-study training materials in basic computational science skills (version control, notebooks, social coding, etc) | agnositc | yes | |
+| [Ed-DaSH](https://edcarp.github.io/Ed-DaSH/) | | specific | | Giacomo Peru |
+| [Posit (Data science resources - R, Python)](https://posit.co/resources/) | | agnositc | yes | Fotis Psomopoulos |
+| [A Beginner’s Guide to Conducting Reproducible Research, Jesse M. Alston, Jessica A. Rick, Bulletin of The Ecological Society of America 102 (2) (2021), https://doi.org/10.1002/bes2.1801](https://esajournals.onlinelibrary.wiley.com/doi/10.1002/bes2.1801) | | agnostic | yes | Aleks Nenadic |
+| [The Turing Way Community. (2022). The Turing Way: A handbook for reproducible, ethical and collaborative research. Zenodo. https://doi.org/10.5281/zenodo.3233853](https://the-turing-way.netlify.app/index.html) | | agnostic | yes | Aleks Nenadic |
+| [The Turing Way’s “Guide for Reproducible Research”, online book - part of the The Turing Way: A handbook for reproducible, ethical and collaborative research](https://the-turing-way.netlify.app/reproducible-research/reproducible-research) | | agnostic | yes | Aleks Nenadic |
+| [“Ten reproducible research things” tutorial](https://guereslib.github.io/ten-reproducible-research-things/) | | agnostic | yes | Aleks Nenadic |
+| [Reproducibility for Everyone’s (R4E) resources, community-led education initiative to increase adoption of open research practices at scale](https://www.repro4everyone.org/) | | agnostic | yes | Aleks Nenadic |
+| [Training materials on different aspects of research software engineering (including open source, reproducibility, research software testing, engineering, design, continuous integration, collaboration, version control, packaging, etc.), compiled by the INTERSECT project](https://intersect-training.org/training-links/) | | agnostic | yes | Aleks Nenadic |
+| [“Good Enough Practices in Scientific Computing” course](https://intersect-training.org/training-links/) | | agnostic | yes | Aleks Nenadic |
+| [Curated resources by the Framework for Open and Reproducible Research Training (FORRT)](https://forrt.org/resources/) | | agnostic | yes | Aleks Nenadic |
+| [Barker, M., Chue Hong, N.P., Katz, D.S. et al. Introducing the FAIR Principles for research software. Sci Data 9, 622 (2022). https://doi.org/10.1038/s41597-022-01710-x](https://www.nature.com/articles/s41597-022-01710-x) | | agnostic | yes | Aleks Nenadic |
+| [Wilkinson, M., Dumontier, M., Aalbersberg, I. et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data 3, 160018 (2016). https://doi.org/10.1038/sdata.2016.18.](https://www.nature.com/articles/sdata201618) | | agnostic | yes | Aleks Nenadic |
+| [The FAIR Cookbook, online recipes for life scientists that help make and keep data FAIR](https://faircookbook.elixir-europe.org/content/home.html) | | agnostic | yes | Aleks Nenadic |
+| [Top 10 FAIR Data & Software Things - brief guides for different disciplines that can be used by the research community to understand how they can make their research (data and software) more FAIR](https://librarycarpentry.org/Top-10-FAIR/) | | agnostic | yes | Aleks Nenadic |
+| [10 easy things to make your research software FAIR, poster, part of Top 10 FAIR Data & Software Things](https://librarycarpentry.org/Top-10-FAIR/files/poster_10things_FAIRsoftware.pdf) | | agnostic | yes | Aleks Nenadic |
+| [Five recommendations for FAIR software, by the Netherlands eScience Center and DANS](https://fair-software.eu/) | | agnostic | yes | Aleks Nenadic |
+| [Short online courses on various aspects of research software (including FAIR), by the NESC Research Software Support](https://esciencecenter-digital-skills.github.io/software-support-essentials/) | | agnostic | yes | Aleks Nenadic |
+| [Awesome Research Software Registries, a list of research software registries (by country, organisation, domain and programming language) where research software can be registered to help promote its discovery](https://github.com/NLeSC/awesome-research-software-registries) | | agnostic | yes | Aleks Nenadic |
+| [Software Carpentry’s Git Novice lesson](https://swcarpentry.github.io/git-novice) | | agnostic | yes | Aleks Nenadic |
+| [“How Git Works” course on Pluralsight](https://www.pluralsight.com/courses/how-git-works) | | agnostic | yes | Aleks Nenadic |
+| [A cookiecutter software project template to kickstart a modern best-practice Python project with FAIR metadata](https://github.com/Materials-Data-Science-and-Informatics/fair-python-cookiecutter) | | agnostic | yes | Aleks Nenadic |
+| [A self-assessment checklist for FAIR research software, by the Netherlands eScience Center and Australian Research Data Commons](https://fairsoftwarechecklist.net/) | | agnostic | yes | Aleks Nenadic |
+| [Open source definition, by the Open Source Initiative](https://opensource.org/osd/) | | agnostic | yes | Aleks Nenadic |
+| [CODECHECK, an approach for independent execution of computations underlying research articles](https://codecheck.org.uk/project/) | | agnostic | yes | Aleks Nenadic |
+| [Introduction to R](https://carpentries-lab.github.io/metagenomics-R/) | Super Basic Introdution to R programming | Agnostic | Yes | Jacky Bossey |
+| [Introduction to the Command Line for Metagenomics](https://carpentries-lab.github.io/metagenomics-shell/) | Introduction into using command lines to navigate and work inside the file system | Agnostic | Yes | Jacky Bossey |
+| [Tutorial: Workflows - Combining Tools for Data Analysis](https://www.ebi.ac.uk/training/online/courses/workflows-combining-tools-for-data-analysis/) | Understand the use of a Workflow and the tools to do it | Agnostic | Yes | Jacky Bossey |
+| [Course: ResOps - Cloud-native Tools and Technology for Researchers](https://tsi-ccdoc.readthedocs.io/en/master/ResOps/2020/Agenda-2020.html#introduction) | Gaining practical experience in building, deploying and running applications in cloud platforms | Agnostic | Yes | Jacky Bossey |
+| [BSC Training Course: introduction to CUDA Programming @ BSC](https://events.prace-ri.eu/event/1534/) | Introduction to developing applications software for processors with massively parallel computing resources. | Agnostic | | Jacky Bossey |
+| [PUMPS+I Summer School](https://pumps.bsc.es/2024/about-pumps-ai) | aimed at enriching the skills of researchers, graduate students and instructors with cutting-edge technique and hands-on experience in developing applications for many-core processors with massively parallel computing resources like GPU accelerators, with a special focus on artificial intelligence | Agnostic | | Jacky Bossey |
+| [Ensembl REST API](https://www.ebi.ac.uk/training/online/courses/ensembl-rest-api/) | Introduction to using Esembl REST API using Jupyter Notebooks | Specific | Yes | Jacky Bossey |
+| [Introduction to User Experience Design](https://www.ebi.ac.uk/training/online/courses/user-experience-design/what-is-user-experience-design/) | An Introduction to UX and it's benefits as well as tools and resources | Agnostic | Yes | Jacky Bossey |
+| [BSC Training Course: introdution to OpenACC](https://www.bsc.es/education/training/bsc-training/bsc-training-course-introduction-openacc) | Introduction to using OpenACC in GPU computing | Agnostic | | Jacky Bossey |
+| [The CodeMeta Project](https://codemeta.github.io/) | Tools to create standardized MetaData | Agnostic | Yes | Jacky Bossey |
+| [RSE Toolkit](https://github.com/RSEToolkit/rse-competencies-toolkit) | Support RSEs in tracking and managing their development | Agnostic | Yes | Jacky Bossey |
+| [Intermediate Research Software Development](https://carpentries-incubator.github.io/python-intermediate-development/) | Intermediate course focused on the development of research software | Agnostic | Yes | Jacky Bossey |
+| [Basic Lab Skills for Research Computing](https://software-carpentry.org/lessons/) | | Agnostic | Yes | Jacky Bossey |
+| [SSI training hub](https://www.software.ac.uk/training/training-hub) | Multiple courses and tutorial on various topics | Agnostic | Yes | Jacky Bossey |
+| [eScience Center - Research Software Support](https://esciencecenter-digital-skills.github.io/software-support-essentials/) | FAIR4RS Pinciples and Guidance for RSE | Agnostic | Yes | Jacky Bossey |
+| [US-RSE - RSE Training Resources](https://us-rse.org/wg/education_training/training/) | | Agnostic | Yes | Jacky Bossey |
+| [The Alan Turing Institute - Research Software Engineering with Python](https://alan-turing-institute.github.io/rse-course/html/index.html) | Complete 0 to advanced course in python for RSE | Agnostic | Yes | Jacky Bossey |
+| [Interesect Research Software Engineering Training](https://intersect-training.org/curriculum/) | Complete curriculum segmented on various mmodules | Agnostic | Yes | Jacky Bossey |
+| [VersionControl with Git](https://swcarpentry.github.io/git-novice/) | | Agnostic | Yes | Jacky Bossey |
diff --git a/pages/your_tasks/ci_testing_matrix_for_research_softwareV4.md b/pages/your_tasks/ci_testing_matrix_for_research_softwareV4.md
new file mode 100644
index 00000000..69028d90
--- /dev/null
+++ b/pages/your_tasks/ci_testing_matrix_for_research_softwareV4.md
@@ -0,0 +1,309 @@
+---
+title: "Managing complex CI testing matrices for research software"
+search_exclude: true
+description: "Efficiently test research software across multiple compilers, libraries, architectures, and platforms while managing CI resource constraints and maintaining fast feedback loops"
+contributors: []
+page_id: complex_ci_testing_matrix
+related_pages:
+ your_tasks: [ci_cd, task_automation_github_actions, task_automation_gitlab_ci_cd]
+training:
+ - name: "Pairwise Testing with allpairspy"
+ registry: "PyPI"
+ url: "https://pypi.org/project/allpairspy/"
+ - name: "GitLab Dynamic Child Pipelines"
+ registry: "GitLab"
+ url: "https://docs.gitlab.com/ee/ci/pipelines/downstream_pipelines.html#dynamic-child-pipelines"
+ - name: "Docker Multi-stage Builds"
+ registry: "Docker"
+ url: "https://docs.docker.com/build/building/multi-stage/"
+---
+
+## How can I efficiently test my research software across multiple compiler versions, library dependencies, and target platforms?
+
+### Description
+
+Research software, particularly performance-portable libraries and simulation codes, often needs to support extensive combinations of compilers, library versions, target architectures, and runtime environments. For example, accelerator abstraction libraries like [Alpaka](https://github.com/alpaka-group/alpaka) require testing across multiple GCC versions, Clang versions, CUDA SDK versions, CMake versions, and Boost versions. A naive approach testing all combinations can create thousands of test jobs, making CI pipelines impractically long and resource-intensive.
+
+Consider this real-world example from accelerator development:
+
+- 4 GCC compiler versions
+- 6 Clang compiler versions
+- 10 CUDA SDK versions
+- 4 CMake versions
+- 7 Boost library versions
+
+This results in **2,800 potential combinations**, requiring approximately **~9.3 hours** of compute time at 6 minutes per job, even with 30 parallel runners.
+
+### Considerations
+
+- Combinatorial explosion: The number of possible combinations grows exponentially with each additional parameter (compilers × architectures × libraries × versions)
+- Resource constraints: CI runners have limited capacity, and excessive parallelization can monopolize shared infrastructure across multiple projects
+- Time constraints: Full test matrices can take many hours to complete, creating bottlenecks in development workflows
+- Hardware diversity: Different combinations may require specific hardware (NVIDIA GPUs, AMD GPUs, ARM processors, PowerPC architectures)
+- Invalid combinations: Some parameter combinations may be incompatible (e.g., older CUDA versions with newer GCC compilers)
+- Coverage vs efficiency: Need adequate test coverage without redundant or meaningless test combinations
+- Maintenance overhead: Large test matrices become difficult to update and debug when new versions are released
+
+### Solutions
+
+#### Pairwise Testing Implementation
+
+- Use dynamic child pipelines: Dynamic child pipelines are essential for implementing pairwise testing as they allow programmatic generation of CI configurations at runtime. Leverage CI systems that support programmatically generated pipeline configurations based on computed test matrices, enabling runtime optimization based on available resources.
+
+- Implement pairwise testing algorithms: Use mathematical approaches that ensure every combination of two parameter values appears in at least one test job, dramatically reducing total tests while maintaining comprehensive interaction coverage.
+
+- Use specialized job generation libraries: Implement dynamic job generators using tools like `allpairspy`:
+
+ ```python
+ from allpairspy import AllPairs
+
+ parameters = {
+ "host_compiler": ["gcc-8", "gcc-9", "gcc-10", "gcc-11"],
+ "device_compiler": ["clang-10", "clang-11", "clang-12", "clang-13", "clang-14", "clang-15"],
+ "cuda_sdk": ["cuda-10.2", "cuda-11.0", "cuda-11.1", "cuda-11.2", "cuda-11.3",
+ "cuda-11.4", "cuda-11.5", "cuda-11.6", "cuda-11.7", "cuda-11.8"],
+ "cmake": ["cmake-3.18", "cmake-3.19", "cmake-3.20", "cmake-3.21"],
+ "boost": ["boost-1.68", "boost-1.70", "boost-1.72", "boost-1.74",
+ "boost-1.75", "boost-1.76", "boost-1.78"]
+ }
+
+ keys = list(parameters.keys())
+ values = [parameters[key] for key in keys]
+
+ for i, pairs in enumerate(AllPairs(values)):
+ config = dict(zip(keys, pairs))
+ print(f"Job {i+1}: {config}")
+
+ # This reduces 2800 combinations to ~60-100 jobs
+ # Each combination of compiler + CUDA version appears at least once
+ ```
+
+- Develop domain-specific combination rules: Create libraries that encode your project's specific compatibility requirements and testing priorities, such as the [Alpaka](https://github.com/alpaka-group/alpaka) approach.
+
+- Implement exclusion logic: Define rules to automatically exclude known incompatible combinations:
+
+ ```yaml
+ # Example exclusion rules for GPU computing
+ exclusions:
+ - cuda_version: "11.0"
+ gcc_version: "gcc-11" # Incompatible combination
+ - architecture: "ppc64le"
+ cuda_version: "*" # CUDA not available on PowerPC
+ ```
+
+#### Mathematical Optimization Analysis
+
+| Testing Approach | Total Jobs | Estimated Runtime (30 jobs parallel, 6 mins/job) | Coverage Type |
+|-----------------------|------------|---------------------------------------------------|---------------|
+| Full matrix (naive) | 2800 | ~9.3 hours | 100% combinations |
+| Pairwise testing | ~60–100 | ~20–30 minutes | All 2-way interactions |
+| Random sampling | ~200 | ~40 minutes | Statistical coverage |
+
+## How can I optimize CI pipeline performance while maintaining comprehensive testing?
+
+### Description
+
+Even with reduced test matrices, complex research software CI pipelines face performance challenges. Multiple optimization strategies are needed to provide fast developer feedback while maintaining thorough testing coverage across diverse computing environments.
+
+### Considerations
+
+- Build time bottlenecks: Repeatedly compiling large dependency sets (like HPC libraries, scientific computing frameworks, or large C++ template libraries) wastes significant time
+- Resource competition: Simultaneous job execution can overwhelm shared CI infrastructure, affecting other projects
+- Failure feedback delays: Critical bugs may not be detected until late in pipeline execution
+- Development vs production workflows: Full test suites may be unnecessary during iterative development
+- Storage and bandwidth: Large scientific computing containers and datasets impact transfer times
+- Platform-specific testing: Different hardware platforms may have varying performance characteristics
+
+### Solutions
+
+#### Container Optimization Strategies
+
+- Implement pre-built container strategies: Create and maintain container images with pre-compiled dependencies. Multi-stage builds allow you to separate dependency installation from application code, producing smaller, faster final images by copying only necessary artifacts from build to runtime:
+
+ ```dockerfile
+ # Multi-stage build for scientific computing dependencies
+ FROM nvidia/cuda:11.8-devel as builder
+ RUN apt-get update && apt-get install -y \
+ gcc-10 g++-10 clang-12 \
+ cmake libboost-all-dev \
+ libomp-dev libfftw3-dev
+
+ FROM builder as runtime
+ COPY --from=builder /usr/local /usr/local
+ # Application-specific layers added dynamically
+ ```
+
+- Deploy container registry optimization: Host container images in the same data center as CI runners to minimize transfer times and bandwidth costs. Use container registries that support layer caching (reusing unchanged layers between builds) and delta compression (only transferring changed parts of images). Check your registry documentation if it advertises support for these features - most modern registries like GitLab Container Registry, Harbor, and AWS ECR support them.
+
+- Optimize container layer caching: Structure container builds to maximize reuse of intermediate layers and minimize rebuild times. Group frequently changing components in separate layers from stable dependencies. For best practices, see [Docker's layer caching guide](https://docs.docker.com/build/cache/). You can also follow this tutorial for hands-on learning: [Docker Layer Caching Tutorial by Earthly](https://docs.earthly.dev/earthly-0.6/docs/guides/cache).
+
+#### Wave Scheduling Implementation
+
+Running all jobs simultaneously can overwhelm shared infrastructure and delay results. By grouping jobs into sequential stages ("waves"), critical tests can run earlier and free up resources faster. Hence, wave scheduling prevents resource monopolization by running jobs in stages, allowing other projects to use CI infrastructure between waves while providing early feedback on critical tests.
+
+- Use wave scheduling for resource management: Distribute jobs across pipeline stages to periodically release CI resources:
+
+ ```yaml
+ # GitLab CI wave scheduling example
+ stages:
+ - wave1_critical
+ - wave2_compatibility
+ - wave3_performance
+ - wave4_extended
+
+ # Critical tests run first for fast feedback
+ test_core_functionality:
+ stage: wave1_critical
+ script: [run core unit tests]
+
+ # Extended testing runs after resources freed
+ test_gpu_performance:
+ stage: wave4_extended
+ script: [run performance benchmarks]
+ ```
+
+- Implement intelligent job prioritization: Order jobs to maximize early failure detection:
+ - Place strict compiler configurations in early waves
+ - Run compatibility tests with cutting-edge tool versions first
+ - Schedule resource-intensive performance tests in later stages
+
+- Visualize wave scheduling structure:
+
+```
+Wave 1 ─────▶ Fast compile checks, style, small matrix
+ ↓
+Wave 2 ─────▶ Medium-sized combinations, functional tests
+ ↓
+Wave 3 ─────▶ Full matrix, slowest GPU/HPC tests
+```
+
+This structure helps fail early and frees resources for other users.
+
+#### Development Workflow Optimization
+
+- Enable selective testing during development: Allow developers to run targeted subsets of CI pipeline during development using commit-message-based filtering to avoid running full pipeline for iterative development work. This reduces pipeline load during focused development:
+
+ ```yaml
+ # GitLab CI example - Allow developers to run tests based on commit message tags
+ rules:
+ - if: '$CI_COMMIT_MESSAGE =~ /\[cuda-only\]/'
+ variables:
+ TEST_FILTER: "cuda"
+ - if: '$CI_COMMIT_MESSAGE =~ /\[cpu-only\]/'
+ variables:
+ TEST_FILTER: "cpu"
+ ```
+
+ ```bash
+ # Example usage - commit message to run only CUDA tests
+ git commit -m "Add CUDA kernel optimization [ci:cuda-only]"
+ ```
+
+## How can I manage the infrastructure complexity required for multi-platform research software testing?
+
+### Description
+
+Supporting comprehensive test matrices for research software requires sophisticated CI infrastructure that can handle diverse hardware requirements, manage resources efficiently across multiple projects, and provide reliable service for computationally intensive workloads.
+
+### Considerations
+
+- Hardware diversity requirements: Research software often targets HPC systems, requiring testing on multiple CPU architectures (x86, ARM, PowerPC), GPU vendors (NVIDIA, AMD), and specialized accelerators
+- Resource scheduling complexity: Balancing competing demands from multiple research projects while ensuring fair resource allocation
+- Performance benchmarking: Validating not just correctness but also performance characteristics across different hardware configurations
+- HPC system integration: Connecting CI pipelines with production HPC environments for realistic performance testing
+- Cost and sustainability: Managing infrastructure costs while supporting open-source research software development
+- Reliability at scale: Maintaining consistent performance as research groups add more complex testing requirements
+
+### Solutions
+
+#### Performance Testing Integration
+
+- Implement performance regression detection: Integrate performance benchmarking into CI pipelines to catch performance regressions early:
+
+ ```yaml
+ # Example performance testing job
+ performance_benchmark:
+ stage: performance
+ script:
+ - cmake --build build --target benchmark
+ - python benchmark_analysis.py --baseline previous_results.json
+ - python performance_regression_check.py
+ artifacts:
+ reports:
+ performance: performance_results.json
+ ```
+
+- Configure performance thresholds: Establish automated performance regression detection with configurable thresholds for different hardware configurations and algorithm implementations.
+
+#### Comprehensive Testing Strategy Implementation
+
+- Monitor and profile pipeline performance: Track job duration, resource usage, and failure patterns to continuously optimize the pipeline structure:
+
+ | Metric | Target | Monitoring Method |
+ |--------|---------|------------------|
+ | Job Duration | <10 minutes average | Pipeline analytics |
+ | Queue Time | <5 minutes | Runner utilization metrics |
+ | Failure Rate | <5% for stable configurations | Historical trend analysis |
+ | Resource Utilization | 70-90% of capacity | Real-time monitoring |
+
+These metrics can be obtained from your CI platform's monitoring dashboard analytics ([Gitlab CI/CD Analytics](https://docs.gitlab.com/ee/user/analytics/ci_cd_analytics.html), GitHub Actions insights) or third-party monitoring tools like Prometheus or Grafana with GitLab Runner exporters, or APIs.
+
+- Use matrix optimization libraries: Leverage existing tools and libraries for combinatorial testing, such as specialized job matrix libraries developed for performance-portable software testing.
+
+## How can I implement this approach for my research software project?
+
+### Description
+
+Transitioning from simple CI testing to comprehensive multi-platform testing matrices requires careful planning, tool selection, and gradual implementation to avoid disrupting existing development workflows.
+
+### Considerations
+
+- Current CI maturity: Existing testing infrastructure and team familiarity with CI/CD concepts
+- Project complexity: Size of parameter space and critical compatibility requirements
+- Resource availability: Access to diverse hardware platforms and CI infrastructure budgets
+- Team expertise: Developer familiarity with containerization, CI configuration, and testing strategies
+- Integration requirements: Compatibility with existing development tools and workflows
+
+### Solutions
+
+- Start with parameter identification: Systematically catalog all dimensions that require testing validation:
+
+ ```python
+ # Example parameter definition for scientific computing library
+ testing_parameters = {
+ 'compilers': ['gcc-9', 'gcc-10', 'gcc-11', 'clang-12', 'clang-13', 'clang-14'],
+ 'cuda_versions': ['11.0', '11.2', '11.4', '11.6', '11.8', '12.0'],
+ 'cmake_versions': ['3.18', '3.20', '3.22', '3.24'],
+ 'boost_versions': ['1.72', '1.75', '1.78', '1.80', '1.82'],
+ 'architectures': ['x86_64', 'arm64'],
+ 'build_types': ['Release', 'Debug']
+ }
+ ```
+
+- Implement gradual migration strategy:
+ 1. Begin with core compatibility testing using pairwise algorithms
+ 2. Add specialized hardware testing incrementally
+ 3. Introduce performance testing for stable configurations
+ 4. Expand to full multi-platform validation
+
+- Use established toolchains: Leverage proven solutions from successful research software projects:
+ - Job matrix generation: Implement using libraries like `allpairspy` or domain-specific tools
+ - Container strategies: Base images on established scientific computing containers
+ - CI integration: Use GitLab dynamic child pipelines or GitHub Actions matrix strategies
+
+- Document testing rationale: Maintain clear documentation explaining testing parameter choices and exclusion rules to facilitate maintenance and onboarding.
+
+- Consider resource sustainability: Even with optimized matrices, extensive testing may be technically possible, but consumes computational resources and energy. Balance testing thoroughness with environmental impact by running full matrices only when necessary (e.g., before releases) and using smaller subsets for regular development work. Consider tradeoffs between coverage and efficiency when designing your matrix and scheduling jobs.
+
+## References
+
+This approach was successfully implemented by the Helmholtz-Zentrum Dresden-Rossendorf for the Alpaka performance-portability library and PIConGPU particle-in-cell simulation code, demonstrating significant reductions in CI resource usage while maintaining comprehensive testing coverage across multiple compilers, accelerator platforms, and HPC architectures.
+
+Further resources:
+
+- [Continuous Integration in Complex Research Software - Handling Complexity](https://zenodo.org/records/14643958)
+- [PIConGPU](https://github.com/ComputationalRadiationPhysics/picongpu)
+- [Alpaka](https://github.com/alpaka-group/alpaka)
+- [Alpaka Job Matrix Library](https://github.com/alpaka-group/alpaka-job-matrix-library)
+- [Container Registry for CI Images](https://codebase.helmholtz.cloud/crp/alpaka-group-container)
+- [Dynamic CI Pipelines in GitLab](https://docs.gitlab.com/ee/ci/pipelines/downstream_pipelines.html#dynamic-child-pipelines)
\ No newline at end of file
diff --git a/pages/your_tasks/local_gitlab_ci_infra_for_github_projectV4.md b/pages/your_tasks/local_gitlab_ci_infra_for_github_projectV4.md
new file mode 100644
index 00000000..01210f44
--- /dev/null
+++ b/pages/your_tasks/local_gitlab_ci_infra_for_github_projectV4.md
@@ -0,0 +1,327 @@
+---
+title: "Using local GitLab CI infrastructure for your GitHub project"
+search_exclude: true
+description: "Leverage local GitLab CI runners and specialized hardware for GitHub-hosted projects when standard CI resources are insufficient"
+contributors: []
+page_id: github_gitlab_ci_integration
+related_pages:
+ your_tasks: [ci_cd, task_automation_github_actions, task_automation_gitlab_ci_cd]
+training:
+ - name: "GitLab CI/CD Documentation"
+ registry: "GitLab"
+ url: "https://docs.gitlab.com/ee/ci/"
+ - name: "GitHub Actions Documentation"
+ registry: "GitHub"
+ url: "https://docs.github.com/en/actions"
+ - name: "Alpaka Job Matrix Library"
+ registry: "GitHub"
+ url: "https://github.com/alpaka-group/alpaka-job-matrix-library"
+---
+
+## How can I use my organization's local GitLab CI infrastructure for a GitHub-hosted project?
+
+### Description
+
+Many research software projects are hosted on {% tool "github" %} to benefit from its large open-source community and collaboration features. However, GitHub's free CI resources may be insufficient for complex research software that requires specialized hardware (GPUs, specific CPU architectures), extensive testing matrices, or simply more computational resources than the free tier provides. Organizations often have local {% tool "gitlab" %} instances with powerful runners and specialized hardware that could address these limitations.
+
+Research projects like [PIConGPU](https://github.com/ComputationalRadiationPhysics/picongpu) and [Alpaka](https://github.com/alpaka-group/alpaka) demonstrate this challenge perfectly. These projects require testing across multiple hardware configurations and extensive parameter combinations that exceed GitHub's free tier capabilities, yet benefit from GitHub's collaborative ecosystem for open-source development.
+
+### Considerations
+
+- Resource limitations on GitHub: Free {% tool "github-actions" %} have monthly limits and lack access to specialized hardware like GPUs or alternative CPU architectures required for performance-portable research software
+- Available local infrastructure: Your organization may have {% tool "gitlab-ci-cd" %} runners with specialized hardware including multiple CPU architectures (ARM, Power, AMD, Intel), GPUs (NVIDIA, AMD), HPC systems, or different operating systems (macOS, Windows)
+- Fork contribution workflow: Contributors often work from forks, which complicates direct integration with external CI systems, especially important for open-source research projects with distributed contributors
+- GitLab Premium limitations: GitLab's native GitHub integration requires Premium edition and has limitations with fork pull requests, making it unsuitable for many research organizations
+- Security considerations: External contributors need access to CI results without compromising internal infrastructure security
+- Webhook reliability: Ensuring consistent communication between GitHub events and GitLab CI execution
+- Status reporting complexity: Maintaining clear CI status visibility on GitHub while running on external infrastructure
+
+### Solutions
+
+#### Repository Mirroring Infrastructure
+
+- Implement GitLab native project mirroring: Configure {% tool "gitlab" %} to automatically synchronize your GitHub repository to a local GitLab instance. See [GitLab repository mirroring documentation](https://docs.gitlab.com/ee/user/project/repository/mirror/) for setup instructions. Initially mirror only local branches to maintain security and avoid unnecessary synchronization of all external forks. Note that GitLab's native mirroring requires push access to the source repository, which means you need either a deploy key or personal access token with write permissions to the GitHub repository.
+
+- Configure selective branch mirroring: Set up mirroring rules that focus on main development branches and exclude temporary or experimental branches to reduce synchronization overhead. Configure branch filters using [GitLab's mirroring options](https://docs.gitlab.com/ee/user/project/repository/mirror/#mirror-only-protected-branches).
+
+- Set up mirror monitoring: Implement monitoring to ensure mirroring remains active and responsive, with automated alerts for synchronization failures.
+
+#### Fork Integration System
+
+- Deploy automated fork mirroring bot: Create a webhook-driven bot that monitors GitHub pull request events from forks and automatically creates corresponding branches in the GitLab mirror. Use [GitHub webhooks documentation](https://docs.github.com/en/developers/webhooks-and-events/webhooks/creating-webhooks) and [GitLab API for branch creation](https://docs.gitlab.com/ee/api/branches.html#create-repository-branch). Use systematic naming conventions such as `username-repo-feature-branch` to maintain organization and traceability.
+
+- Implement webhook security: Configure webhook signatures and validation to ensure only legitimate GitHub events trigger mirroring operations, preventing unauthorized access to your internal GitLab infrastructure. Follow [GitHub webhook security best practices](https://docs.github.com/en/developers/webhooks-and-events/webhooks/securing-your-webhooks).
+
+- Handle fork branch lifecycle: Implement automated cleanup of mirrored fork branches when corresponding pull requests are closed or merged, maintaining repository hygiene.
+
+#### CI Status Integration
+
+- Configure bidirectional status reporting: Implement a system that sends GitLab pipeline status back to GitHub using commit hashes for identification. Use [GitHub's commit status API](https://docs.github.com/en/rest/commits/statuses) to report build statuses from external CI systems and [GitLab CI/CD pipeline events](https://docs.gitlab.com/ee/user/project/integrations/webhook_events.html#pipeline-events). See also [GitLab's GitHub integration guide](https://docs.gitlab.com/ci/ci_cd_for_external_repos/github_integration/) for comprehensive setup instructions. This ensures pull request status checks are properly updated regardless of the execution platform. Use consistent naming patterns for external status checks, such as "gitlab-ci/pipeline-name" or "gitlab-ci/job-name", to avoid conflicts with other CI systems and maintain clear identification of status sources.
+
+- Set up detailed status descriptions: Provide clear, descriptive status messages that indicate the testing platform, job types, and specific failure reasons to help developers understand CI results.
+
+- Implement status aggregation: For complex pipelines with multiple stages, aggregate status information to provide clear pass/fail indicators while maintaining access to detailed logs.
+
+#### Access Management and Security
+
+- Establish guest access procedures: Create documented procedures for external contributors to access GitLab CI logs and results, including temporary guest access workflows that maintain security boundaries. Note that this is primarily relevant for non-public GitLab projects where CI logs are not publicly accessible. Practical implementation:
+ 1. Create a standard request template for CI log access (include GitHub username, PR number, specific job logs needed)
+ 2. Set up a dedicated GitLab group for external contributors with guest permissions
+ 3. Use GitLab's [temporary project access](https://docs.gitlab.com/ee/user/project/members/#add-users-to-a-project) with expiration dates (e.g., 7 days)
+ 4. Automate access removal using GitLab's API or scheduled cleanup scripts
+ 5. Document the process in your project's CONTRIBUTING.md file
+ Configure using [GitLab project members management](https://docs.gitlab.com/ee/user/project/members/) and [guest permissions](https://docs.gitlab.com/ee/user/permissions.html#project-members-permissions) for setting up access control.
+
+- Configure permission mapping: Establish clear mapping between GitHub repository permissions and GitLab project access levels to ensure appropriate access control. Practical implementation:
+ 1. Document permission equivalencies:
+ - GitHub repository admins → GitLab project maintainers (can manage runners, settings)
+ - GitHub write access → GitLab developer role (can trigger pipelines, view logs)
+ - GitHub read access → GitLab reporter role (can view pipeline results only)
+ 2. Create GitLab groups that mirror your GitHub team structure
+ 3. Use GitLab's [SAML/LDAP integration](https://docs.gitlab.com/ee/integration/saml.html) if available for automated synchronization
+ 4. Implement regular access reviews (monthly/quarterly) to ensure permissions stay synchronized
+ 5. Use [GitLab's audit events](https://docs.gitlab.com/administration/compliance/audit_event_reports/) to track permission changes
+ Document these mappings and implement them consistently using [GitLab's role-based permissions](https://docs.gitlab.com/ee/user/permissions.html#project-members-permissions).
+
+- Implement audit logging: Maintain comprehensive logs of all cross-platform CI activities for security monitoring and troubleshooting. Configure logging for:
+ - Webhook processing events and failures
+ - Mirror synchronization activities
+ - User access grants and revocations
+ - Pipeline trigger events from GitHub
+ Use [GitLab's audit events](https://docs.gitlab.com/administration/compliance/audit_event_reports/) and implement custom logging for webhook processing activities.
+
+#### Infrastructure Configuration
+
+- Deploy specialized GitLab runners: Configure runners with the specific hardware configurations your project requires. Follow [GitLab Runner installation guide](https://docs.gitlab.com/runner/install/) and [runner configuration documentation](https://docs.gitlab.com/runner/configuration/):
+
+The following table shows common runner configurations for research software testing. Choose hardware specifications based on your project's specific computational requirements:
+
+| Runner Type | Hardware Configuration | Use Case | Rationale |
+|-------------|----------------------|----------|-----------|
+| Standard | x86_64 CPU | Build testing, unit tests | Most common development environment |
+| GPU-NVIDIA | x86_64 + NVIDIA GPU | CUDA development, GPU computing | NVIDIA ecosystem dominance in HPC |
+| GPU-AMD | x86_64 + AMD GPU | ROCm/HIP testing, OpenCL | Alternative GPU vendor support |
+| ARM | ARM64 CPU | Cross-platform validation | Growing ARM adoption in HPC/cloud |
+| PowerPC | ppc64le CPU (little-endian PowerPC architecture commonly used in modern HPC systems)| HPC compatibility testing | Legacy HPC system support |
+
+Hardware sizing considerations:
+
+- Memory: Scale based on your largest datasets and compilation requirements (typically 8GB minimum, 32GB+ for large scientific codes)
+- Storage: Ensure sufficient space for container images, build artifacts, and test data
+- CPU cores: More cores reduce build times but increase infrastructure costs
+
+- Configure runner tagging: Implement comprehensive tagging systems that allow jobs to target specific hardware configurations while maintaining flexibility for resource allocation. Practical implementation:
+
+ ```toml
+ # Example runner configuration with tags (config.toml format)
+ [[runners]]
+ name = "gpu-nvidia-runner"
+ tags = ["gpu", "nvidia", "cuda", "high-memory"]
+
+ [[runners]]
+ name = "cpu-arm-runner"
+ tags = ["cpu", "arm64", "cross-platform"]
+ ```
+
+ ```yaml
+ # Example job targeting specific runners
+ cuda_tests:
+ tags:
+ - gpu
+ - nvidia
+ script:
+ - nvcc --version
+ - ./run_cuda_tests.sh
+
+ arm_build:
+ tags:
+ - cpu
+ - arm64
+ script:
+ - ./build_for_arm.sh
+ ```
+
+See [GitLab Runner tags documentation](https://docs.gitlab.com/ee/ci/runners/configure_runners.html#use-tags-to-control-which-jobs-a-runner-can-run).
+
+- Organize runners by capability: Structure your runners based on hardware capabilities and project requirements to ensure fair resource distribution. Use GitLab's runner types strategically:
+ - Shared runners: Available to all projects in your GitLab instance (good for standard builds)
+ - Group runners: Shared within specific groups/organizations (good for department-level resources)
+ - Project-specific runners: Dedicated to individual projects (good for specialized hardware)
+
+ Example conceptual organization strategy (conceptual hierarchy):
+
+ ```yaml
+ # Example runner configuration with types
+ Shared Runners (Institute-wide):
+ ├── standard-cpu (tags: cpu, x86_64)
+ └── basic-gpu (tags: gpu, nvidia, shared)
+
+ Group Runners (Research Group):
+ ├── high-memory-cpu (tags: cpu, high-memory, research-group)
+ └── specialized-gpu (tags: gpu, tesla, research-group)
+
+ Project Runners (Specific Projects):
+ └── hpc-powerpc (tags: powerpc, hpc, project-specific)
+ ```
+
+ See [GitLab runner types documentation](https://docs.gitlab.com/ee/ci/runners/runners_scope.html) for configuration guidance.
+
+#### Monitoring and Maintenance
+
+- Consider implementation complexity: The monitoring and maintenance requirements for this hybrid CI setup represent significant operational overhead that research software engineers should carefully consider. While the technical implementation provides powerful capabilities for complex research software testing, the ongoing maintenance requires dedicated infrastructure expertise and time investment that may exceed the capacity of smaller research teams.
+
+- Implement comprehensive monitoring: Monitor webhook processing, mirroring bot health, runner availability, and pipeline execution metrics to ensure reliable service. Use [GitLab's monitoring features](https://docs.gitlab.com/ee/administration/monitoring/) and [runner monitoring](https://docs.gitlab.com/runner/monitoring/).
+
+- Create automated monitoring scripts: Provide monitoring scripts for webhook health, GitLab API connectivity, and runner availability. These scripts can be integrated with GitLab's monitoring framework to provide practical monitoring solutions that can be deployed.
+
+ Example webhook health monitoring script:
+
+ ```bash
+ #!/bin/bash
+ # webhook_health_monitor.sh - Monitor webhook endpoint health
+
+ WEBHOOK_URL="https://gitlab.example.com/webhook"
+ ALERT_EMAIL="admin@yourorg.com"
+
+ # Test webhook endpoint
+ if ! curl -f -s -o /dev/null "$WEBHOOK_URL/health"; then
+ echo "Webhook endpoint $WEBHOOK_URL is down" | mail -s "Webhook Alert" "$ALERT_EMAIL"
+ exit 1
+ fi
+
+ # Check GitLab API connectivity
+ if ! curl -f -s -H "Authorization: Bearer $GITLAB_TOKEN" \
+ "https://gitlab.example.com/api/v4/projects" > /dev/null; then
+ echo "GitLab API connectivity failed" | mail -s "GitLab API Alert" "$ALERT_EMAIL"
+ exit 1
+ fi
+
+ echo "All systems healthy"
+ ```
+
+ Deploy this script via cron to run every 5 minutes by adding the following line to your crontab:
+
+ ```bash
+ # Add this line to your crontab (run: crontab -e)
+ */5 * * * * /usr/local/bin/webhook_health_monitor.sh >/dev/null
+ ```
+
+ This runs the script every 5 minutes and suppresses output unless there's an error.
+
+- Configure automated alerts: Set up alerting for critical failures in the integration chain, including mirroring delays, webhook processing errors, and runner unavailability.
+
+- Establish maintenance procedures: Create documented procedures for routine maintenance, including runner updates, bot deployments, and mirror configuration changes.
+
+- Monitor webhook processing metrics: Track webhook processing success rates, maintaining above 95% success rates to ensure reliable integration. Monitor GitLab pipeline trigger delays, as synchronization delays exceeding 5 minutes may indicate infrastructure issues requiring investigation. Track runner availability across different hardware types, particularly for specialized GPU or ARM runners that may have limited availability.
+
+- Configure specific alerting thresholds: Set up alerts for webhook processing failures exceeding 3 consecutive failures, mirroring synchronization delays beyond 10 minutes, and runner unavailability lasting more than 30 minutes. Implement escalation procedures that automatically notify infrastructure administrators when automated recovery attempts fail.
+
+- Establish regular maintenance schedules: Implement weekly runner health checks, monthly authentication token validation, and quarterly review of webhook configurations. Document these procedures with specific commands and expected outputs to ensure consistency across different team members managing the infrastructure.
+
+## How do I ensure reliable integration between GitHub and GitLab CI systems?
+
+### Description
+
+Maintaining a stable, reliable integration between GitHub repositories and GitLab CI infrastructure requires careful attention to the communication pathways, error handling, and monitoring systems that connect these platforms. The integration must handle various failure modes gracefully while providing clear feedback to developers.
+
+### Considerations
+
+- Network reliability: Communication between GitHub and GitLab may experience intermittent failures, requiring robust retry mechanisms
+- Authentication management: Maintaining valid authentication tokens across both platforms while ensuring security
+- Rate limiting: Both GitHub and GitLab APIs have rate limits that must be respected to avoid service interruptions
+- Error propagation: Failures in any part of the integration chain should be clearly communicated to developers
+- Synchronization delays: Mirroring introduces latency that must be managed to provide timely feedback
+- Configuration drift: Keeping webhook configurations and mirroring settings synchronized as projects evolve
+
+### Solutions
+
+- Implement robust webhook processing: Design webhook handlers that reliably process GitHub webhook events. Implement error handling for downstream operations (like GitLab API calls), queue failed operations for retry, and use dead letter queues for events that cannot be processed after multiple attempts. Note that GitHub handles webhook delivery retries automatically, but you must ensure reliable processing of received events. Practical implementation:
+ 1. Validate webhook signatures using GitHub's secret token
+ 2. Return HTTP 200 immediately to acknowledge receipt, then process asynchronously
+ 3. Use a message queue (Redis, RabbitMQ) to handle processing failures
+ 4. Implement exponential backoff for failed GitLab API calls
+ 5. Store failed events in a dead letter queue for manual review
+ Example using Python/Flask:
+
+ ```python
+ @app.route('/webhook', methods=['POST'])
+ def handle_webhook():
+ # Validate signature first
+ if not verify_signature(request):
+ return 'Unauthorized', 401
+
+ # Input Validation step
+ if not request.json or 'action' not in request.json:
+ return 'Bad Request', 400
+
+ # Queue for async processing
+ queue.enqueue(process_webhook, request.json)
+ return 'OK', 200
+ ```
+
+- Configure webhook redundancy: Set up multiple webhook endpoints with failover mechanisms to ensure continuous operation even during maintenance or unexpected outages. Practical implementation:
+ 1. Configure multiple webhook URLs in your GitHub repository settings
+ 2. Use a load balancer (nginx, HAProxy) to distribute webhook traffic
+ 3. Deploy webhook handlers on multiple servers/containers
+ 4. Implement health checks for each endpoint
+ 5. Use monitoring to detect failed endpoints and route traffic accordingly
+
+ Example nginx configuration:
+
+ ```nginx
+ upstream webhook_backends {
+ server webhook1.example.com:8080 max_fails=3 fail_timeout=30s;
+ server webhook2.example.com:8080 max_fails=3 fail_timeout=30s;
+ }
+
+ server {
+ location /webhook {
+ proxy_pass http://webhook_backends;
+ }
+ }
+ ```
+
+- Establish authentication token rotation: Implement automated token rotation for both GitHub and GitLab APIs to maintain long-term reliability without manual intervention. This includes updating webhook configurations simultaneously that use these tokens for authentication as webhook authentication will fail if tokens become desynchronized between platforms. Configure using [GitHub personal access tokens](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) and [GitLab access tokens](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html). Automate webhook updates using [GitHub's webhook API](https://docs.github.com/en/rest/webhooks) and [GitLab's webhook API](https://docs.gitlab.com/api/project_webhooks/) when tokens are rotated. Note that GitHub tokens require "repo" and "admin:repo_hook" scopes for full functionality, while GitLab tokens need "api" scope for repository and webhook management.
+
+- Create comprehensive error logging: Implement detailed logging throughout the integration pipeline to facilitate troubleshooting and identify patterns in failures.
+
+- Set up health monitoring dashboards: Create monitoring dashboards that provide real-time visibility into the integration health, including webhook processing rates, mirroring delays, and CI execution metrics.
+
+## How do I handle the complexity of managing both GitHub and GitLab workflows?
+
+### Description
+
+Running a hybrid GitHub-GitLab CI setup introduces workflow complexity that needs to be managed carefully to maintain developer productivity and project maintainability. This complexity is amplified when dealing with research software that requires specialized testing infrastructure and has contributors with varying levels of CI/CD expertise.
+
+### Considerations
+
+- Developer experience: Contributors should have a seamless experience regardless of which CI system is running their code
+- Debugging complexity: Failed CI runs may require access to GitLab logs while the discussion happens on GitHub
+- Permission management: Different access controls between GitHub and GitLab may create permission mismatches
+- Documentation maintenance: Hybrid workflows require comprehensive documentation that stays current with both platforms
+- Onboarding complexity: New contributors need to understand both the GitHub collaboration model and GitLab CI execution environment
+
+### Solutions
+
+- Provide comprehensive documentation: Create detailed documentation explaining the hybrid CI setup, including troubleshooting guides and examples specific to your research domain. Include step-by-step guides for common developer workflows.
+
+- Implement automated documentation updates: Set up systems to automatically update documentation when CI configurations change, ensuring documentation remains current.
+
+- Create developer onboarding guides: Develop specific onboarding materials that explain how to work with the hybrid system, including how to access logs, interpret status messages, and request runner access.
+
+- Establish clear escalation procedures: Document clear procedures for developers to follow when they encounter CI issues that require GitLab access or infrastructure support.
+
+- Implement automated issue routing: Create systems that automatically route CI-related issues to appropriate support channels based on whether they originate from GitHub or GitLab components.
+
+## References
+
+This approach was successfully implemented by the Helmholtz-Zentrum Dresden-Rossendorf for the Alpaka project, demonstrating how research organizations can effectively combine GitHub's collaborative features with local specialized CI infrastructure. The implementation showcases the infrastructure integration techniques that make complex research software CI pipelines practical and maintainable.
+
+Further resources:
+
+- [Continuous Integration in Complex Research Software - Handling Complexity](https://zenodo.org/records/14643958)
+- [PIConGPU](https://github.com/ComputationalRadiationPhysics/picongpu)
+- [Alpaka](https://github.com/alpaka-group/alpaka)
+- [Alpaka Job Matrix Library](https://github.com/alpaka-group/alpaka-job-matrix-library)
+- [Container Registry for CI Images](https://codebase.helmholtz.cloud/crp/alpaka-group-container)
+- [Dynamic CI Pipelines in GitLab](https://docs.gitlab.com/ee/ci/pipelines/downstream_pipelines.html#dynamic-child-pipelines)
\ No newline at end of file