Add latest machine learning and openmp projects for Clad (#1691)

Author: aaronj0

_gsocproposals/2025/proposal_Clad-LLM.md

@@ -0,0 +1,44 @@
+---
+title: Enhancing LLM Training with Clad for efficient differentiation
+
+layout: gsoc_proposal
+project: Clad
+year: 2025
+difficulty: medium
+duration: 350
+mentor_avail: June-October
+organization:
+  - CompRes
+---
+
+## Description
+
+This project aims to leverage Clad, an automatic differentiation (AD) plugin for Clang, to optimize large language model (LLM) training primarily in C++. Automatic differentiation is a crucial component of deep learning training, enabling efficient computation of gradients for optimization algorithms such as stochastic gradient descent (SGD). While most modern LLM frameworks rely on Python-based ecosystems, their heavy reliance on interpreted code and dynamic computation graphs can introduce performance bottlenecks. By integrating Clad into C++-based deep learning pipelines, we can enable high-performance differentiation at the compiler level, reducing computational overhead and improving memory efficiency. This will allow developers to build more optimized training workflows without sacrificing flexibility or precision.
+
+Beyond performance improvements, integrating Clad with LLM training in C++ opens new possibilities for deploying AI models in resource-constrained environments, such as embedded systems and HPC clusters, where minimizing memory footprint and maximizing computational efficiency are critical. Additionally, this work will bridge the gap between modern deep learning research and traditional scientific computing by providing a more robust and scalable AD solution for physics-informed machine learning models. By optimizing the differentiation process at the compiler level, this project has the potential to enhance both research and production-level AI applications, aligning with compiler-research.org’s broader goal of advancing computational techniques for scientific discovery.
+
+
+## Expected Results
+
+* Develop a simplified LLM setup in C++
+* Apply Clad to compute gradients for selected layers and loss functions
+* Enhance clad to support it if necessary, and prepare performance benchmarks
+* Enhance the LLM complexity to cover larger projects such as llama
+* Repeat bugfixing and benchmarks
+* Develop tests to ensure correctness, numerical stability, and efficiency
+* Document the approach, implementation details, and performance gains
+* Present progress and findings at relevant meetings and conferences
+
+## Requirements
+
+* Automatic differentiation
+* Parallel programming
+* Reasonable expertise in C++ programming
+* Background in LLM  is preferred but not required
+
+## Mentors
+* **[Vassil Vassilev](mailto:[email protected])**
+* [David Lange](mailto:[email protected])
+
+## Links
+* [Repo](https://github.com/vgvassilev/clad)

_gsocproposals/2025/proposal_Clad-ONNX.md

@@ -0,0 +1,43 @@
+---
+title: Enable automatic differentiation of OpenMP programs with Clad
+layout: gsoc_proposal
+project: Clad
+year: 2025
+difficulty: medium
+duration: 350
+mentor_avail: June-October
+organization:
+  - CompRes
+---
+
+## Description
+
+Clad is an automatic differentiation (AD) clang plugin for C++. Given a C++ source code of a mathematical function, it can automatically generate C++ code for computing derivatives of the function. Clad is useful in powering statistical analysis and uncertainty assessment applications.
+ONNX (Open Neural Network Exchange) provides a standardized format for machine learning models, widely used for interoperability between frameworks like PyTorch and TensorFlow
+
+This project aims to integrate Clad, an automatic differentiation (AD) plugin for Clang, with ONNX-based machine learning models. Clad can generate derivative computations for C++ functions, making it useful for sensitivity analysis, optimization, and uncertainty quantification. By extending Clad’s capabilities to ONNX models, this project will enable efficient differentiation of neural network operations within an ONNX execution environment.
+
+## Expected Results
+
+* Enumerate ONNX modules with increasing complexity and analyze their differentiation requirements.
+* Develop a structured plan for differentiating the identified ONNX operations.
+* Implement forward mode differentiation for selected ONNX operations.
+* Extend support to reverse mode differentiation for more complex cases.
+* Create comprehensive tests to validate correctness and efficiency.
+* Write clear documentation to ensure ease of use and future maintenance.
+* Present results at relevant meetings and conferences.
+
+
+## Requirements
+
+* Automatic differentiation
+* Parallel programming
+* Reasonable expertise in C++ programming
+* Basic knowledge of Clang is preferred but not mandatory
+
+## Mentors
+* **[Vassil Vassilev](mailto:[email protected])**
+* [David Lange](mailto:[email protected])
+
+## Links
+* [Repo](https://github.com/vgvassilev/clad)

_gsocproposals/2025/proposal_Clad-OpenMP.md

@@ -0,0 +1,39 @@
+---
+title: Enable automatic differentiation of OpenMP programs with Clad
+layout: gsoc_proposal
+project: Clad
+year: 2025
+difficulty: medium
+duration: 350
+mentor_avail: June-October
+organization:
+  - CompRes
+---
+
+## Description
+
+Clad is an automatic differentiation (AD) clang plugin for C++. Given a C++ source code of a mathematical function, it can automatically generate C++ code for computing derivatives of the function. Clad is useful in powering statistical analysis and uncertainty assessment applications.
+OpenMP (Open Multi-Processing) is an application programming interface (API) that supports multi-platform shared-memory multiprocessing programming in C, C++, and other computing platforms. 
+
+This project aims to develop infrastructure in Clad to support the differentiation of programs that contain OpenMP primitives.
+
+## Expected Results
+
+* Extend the pragma handling support 
+* List the most commonly used OpenMP concurrency primitives and prepare a plan for how they should be handled in both forward and reverse accumulation in Clad
+* Add support for concurrency primitives in Clad’s forward and reverse mode automatic differentiation.
+* Add proper tests and documentation.
+* Present the work at the relevant meetings and conferences.
+
+## Requirements
+
+* Automatic differentiation
+* C++ programming
+* Parallel Programming
+
+## Mentors
+* **[Vassil Vassilev](mailto:[email protected])**
+* [David Lange](mailto:[email protected])
+
+## Links
+* [Repo](https://github.com/vgvassilev/clad)