From cb7665fc4852ae5173cdd3ac13014a06ad00cc11 Mon Sep 17 00:00:00 2001 From: kdesnos Date: Fri, 12 Dec 2025 14:58:50 +0100 Subject: [PATCH 01/32] (Tuto) Create tutorial for parallelizing training of the pendulum. --- docs/_pages/parallel_training.md | 134 +++++++++++++++++++++++++++++++ 1 file changed, 134 insertions(+) create mode 100644 docs/_pages/parallel_training.md diff --git a/docs/_pages/parallel_training.md b/docs/_pages/parallel_training.md new file mode 100644 index 0000000..c7f3a1f --- /dev/null +++ b/docs/_pages/parallel_training.md @@ -0,0 +1,134 @@ +--- +title: Parallel Training of Tangled Program Graphs +permalink: /tutos/parallel-training +toc: true +toc_sticky: true +--- + +The objective of this tutorial is to activate parallel training of Tangled Program Graphs (TPGs) with Gegelati by: +- instantiating a `ParallelLearningAgent`, and +- making the `PendulumWrapper` safely copyable so worker threads receive independent environments. + +The starting point of this tutorial is the C++ project obtained at the end of the _[GEGELATI introductory tutorial](/gegelati-tutorial)_. While completing the introductory tutorial is strongly advised, a copy of the project resulting from this tutorial can be downloaded at the following link: [pendulum_wrapper_solution.zip](/gegelati-tutorial/data/gegelati-tutorial-solution.zip). + +## Why make the environment copyable? + +The learning process of TPGs involves two main time-consuming steps per generation: +- Evaluation of the fitness of each individual TPG root within the `PendulumWrapper` learning environment. This step takes time `T_eval` seconds at each generation in the printed log. +- Mutation of the TPG population. This step takes time `T_mutat` seconds at each generation in the printed log. + +When using a `LearningAgent`, both steps are performed sequentially on a single thread. To accelerate training, it is possible to parallelize these steps across multiple threads/cores by using `ParallelLearningAgent`. + +To better take not of the benefits of parallel training, keep a copy of the logs produced by the sequential training for comparison. + +An important feature of Gegelati is that the parallelization of training is fully deterministic, which means that running the same training with the same random seed will always produce the same results, regardless of the number of threads used. This is achieved by ensuring that each worker thread operates on its own independent copy of the learning environment. + +## 0. Parallelize mutations + +To enable parallel mutations, the sequential `LearningAgent` must be replaced with `ParallelLearningAgent`. By default, the number of threads is set to the number of available hardware threads on the machine. + +#### TODO 1: +Edit the `/gegelati-tutorial/src/training/main-training.cpp` by replacing the line that instantiates the `LearningAgent` with a line that instantiates a `ParallelLearningAgent`: + +{% details Solution to #1 (Click to expand) %} +```cpp +/* main-training.cpp */ +// Instantiate and initialize the Learning Agent (LA) +Learn::ParallelLearningAgent la(pendulumLE, instructionSet, params); +``` + +{% enddetails %} + +Build and run the `main-training` target of the project. You should observe that `T_mutat` times have slightly decreased compared to the sequential training log. Other columns relative to the trained TPG characteristics (`NbVert`, `NbActR`, `NbTeamR`) and the fitness of agents (`Min`, `Avg`, `Max`) should remain identical to the sequential training. + +## 1. Parallelize evaluations + +To enable parallel evaluations, the `PendulumWrapper` must be made safely copyable. This is done first by implementing the copy constructor of the `PendulumWrapper` class, and then by overriding the `clone()` method inherited from the `LearningEnvironment` base class. + +#### TODO 2: +Edit the `/gegelati-tutorial/src/environments/pendulum_wrapper.h` and `/gegelati-tutorial/src/environments/pendulum_wrapper.cpp` to add a copy constructor `PendulumWrapper(const PendulumWrapper& other)` to the class. + +It is important to note that the default copy constructor generated by the compiler would perform a shallow copy of the member variables, which is not suitable in this case. Therefore, a custom copy constructor must be implemented to ensure that all member variables are properly duplicated. + +Special care should be taken to handle the `std::vector> data` attribute, this attribute must be initialized as a copy-constructed copy of the `other.data` attribute. Then the pointers contained in the vector must be updated to point to the attributes of the `this->pendulum`, and not to `other.pendulum` as is the case after copy-constructing the `data` attribute. + + +{% details Solution to #2 (Click to expand) %} +```cpp +/* pendulum_wrapper.h */ +// Copy constructor +PendulumWrapper(const PendulumWrapper& other); +``` + +```cpp +/* pendulum_wrapper.cpp */ +// Copy constructor implementation +PendulumWrapper::PendulumWrapper(const PendulumWrapper& other) + : LearningEnvironment(other), // Call base class copy constructor + pendulum(other.pendulum), // Copy-construct the pendulum + data(other.data) // Copy-construct the data vector +{ + // Update pointers in data to point to this->pendulum's attributes + data.at(0).setPointer(&this->pendulum.getAngle()); + data.at(1).setPointer(&this->pendulum.getVelocity()); +} +``` + +{% enddetails %} + +#### TODO 3: +Next, override the `clone()` method in the `PendulumWrapper` class to return a new instance of `PendulumWrapper` created using the copy constructor. + +{% details Solution to #3 (Click to expand) %} +```cpp +/* pendulum_wrapper.h */ +// Override clone method +Data::LearningEnvironment* clone() const override; +``` + +```cpp +/* pendulum_wrapper.cpp */ +// Override clone method implementation +Data::LearningEnvironment* PendulumWrapper::clone() const { + return new PendulumWrapper(*this); // Use copy constructor +} +``` + +{% enddetails %} + + +#### TODO 4: +To signal to Gegelati that the `PendulumWrapper` can be safely copied for parallel evaluation, the `LearningEnvironment::isCopyable()` method must be overridden to return `true`. + +{% details Solution to #4 (Click to expand) %} +```cpp +/* pendulum_wrapper.h */ +// Override isCopyable method +bool isCopyable() const override; +``` + +```cpp +/* pendulum_wrapper.cpp */ +// Override isCopyable method implementation +bool PendulumWrapper::isCopyable() const { + return true; // Indicate that this environment is copyable +} +``` + +{% enddetails %} + +#### Test parallel evaluations +Build and run the `main-training` target of the project. You should observe that `T_eval` times have significantly decreased compared to the sequential training log. Other columns relative to the trained TPG characteristics (`NbVert`, `NbActR`, `NbTeamR`) and the fitness of agents (`Min`, `Avg`, `Max`) should remain identical to the sequential training. + +It it possible to control the number of threads used by the `ParallelLearningAgent` by setting the `nbThreads` parameter in the `/gegelati-tutorial/params.json` file as follows: + +```json +"nbThreads": 4, +``` + +## Conclusion +In this tutorial, you have successfully enabled parallel training of Tangled Program Graphs (TPGs) in Gegelati by replacing the sequential `LearningAgent` with `ParallelLearningAgent` and making the `PendulumWrapper` safely copyable. + +More information about parallel training with Gegelati can be found in the following publication: + +[_K. Desnos, N. Sourbier, P.-Y. Raumer, O. Gesny and M. Pelcat. GEGELATI: Lightweight Artificial Intelligence through Generic and Evolvable Tangled Program Graphs. In Workshop on Design and Architectures for Signal and Image Processing (DASIP), ACM, 2021_](https://arxiv.org/pdf/2012.08296) \ No newline at end of file From 0c6d35534a556d4e59c7db1ee7018855627bca07 Mon Sep 17 00:00:00 2001 From: kdesnos Date: Fri, 12 Dec 2025 16:42:32 +0100 Subject: [PATCH 02/32] (Tuto) Prepare code for parallel tuto solution. --- src/training/main-training.cpp | 4 ++++ src/training/pendulum_wrapper.cpp | 22 ++++++++++++++++++++++ src/training/pendulum_wrapper.h | 27 +++++++++++++++++++++++++++ 3 files changed, 53 insertions(+) diff --git a/src/training/main-training.cpp b/src/training/main-training.cpp index 3ae134c..3ac1d5c 100644 --- a/src/training/main-training.cpp +++ b/src/training/main-training.cpp @@ -48,7 +48,11 @@ void train_main(std::atomic& exitProgram, std::atomic& doDisplay, st #endif // Instantiate and initialize the Learning Agent (LA) + #ifdef SOLUTION_PARALLEL + Learn::ParallelLearningAgent la(pendulumLE, instructionSet, params); + #else Learn::LearningAgent la(pendulumLE, instructionSet, params); + #endif // SOLUTION_PARALLEL la.init(); // Basic logger for the training process diff --git a/src/training/pendulum_wrapper.cpp b/src/training/pendulum_wrapper.cpp index cd7e435..fb979ca 100644 --- a/src/training/pendulum_wrapper.cpp +++ b/src/training/pendulum_wrapper.cpp @@ -18,6 +18,21 @@ PendulumWrapper::PendulumWrapper() : LearningEnvironment(actions.size()) } #endif // SOLUTION +#ifdef SOLUTION_PARALLEL +PendulumWrapper::PendulumWrapper(const PendulumWrapper& other) : LearningEnvironment(other), pendulum(), data(other.data) +{ + // Set pointers of the copy to its own pendulum. + data.at(0).setPointer(&this->pendulum.getAngle()); + data.at(1).setPointer(&this->pendulum.getVelocity()); +} +#endif // SOLUTION_PARALLEL + +#ifdef SOLUTION_PARALLEL +Learn::LearningEnvironment* PendulumWrapper::clone(void) const{ + return new PendulumWrapper(*this); +} +#endif // SOLUTION_PARALLEL + std::vector> PendulumWrapper::getDataSources() { #ifdef SOLUTION @@ -77,3 +92,10 @@ bool PendulumWrapper::isTerminal(void) const { return false; } + +#ifdef SOLUTION_PARALLEL +bool PendulumWrapper::isCopyable(void) const +{ + return true; +} +#endif // SOLUTION_PARALLEL diff --git a/src/training/pendulum_wrapper.h b/src/training/pendulum_wrapper.h index 2c149f2..113a4fe 100644 --- a/src/training/pendulum_wrapper.h +++ b/src/training/pendulum_wrapper.h @@ -44,6 +44,22 @@ class PendulumWrapper : public Learn::LearningEnvironment { /// Default constructor for the PendulumWrapper PendulumWrapper(); +#ifdef SOLUTION_PARALLEL + /// Copy constructor for the PendulumWrapper + PendulumWrapper(const PendulumWrapper& other); +#endif // SOLUTION_PARALLEL + +#ifdef SOLUTION_PARALLEL + /** + * \brief Get a copy of the LearningEnvironment. + * + * This method should return a deep copy of the LearningEnvironment. + * + * \return a copy of the LearningEnvironment. + */ + virtual Learn::LearningEnvironment* clone(void) const override; +#endif // SOLUTION_PARALLEL + /** * \brief Get the data sources for this LearningEnvironment. * @@ -127,6 +143,17 @@ class PendulumWrapper : public Learn::LearningEnvironment { * \return a boolean indicating termination. */ virtual bool isTerminal(void) const override; + +#ifdef SOLUTION_PARALLEL + /** + * \brief Can the LearningEnvironment be copy constructed to evaluate + * several LearningAgent in parallel. + * + * \return true if the LearningEnvironment can be copied and run in + * parallel. + */ + virtual bool isCopyable() const override; +#endif // SOLUTION_PARALLEL }; #endif // !PENDULUM_WRAPPER_H From 0d0de07e35098deb27e0f6bc1ac25519b87080cb Mon Sep 17 00:00:00 2001 From: kdesnos Date: Fri, 12 Dec 2025 17:14:43 +0100 Subject: [PATCH 03/32] (RelEng) Prepare evolution of python script for preparing template. --- scripts/prepare_template.py | 80 ++++++++++++++++--------------------- 1 file changed, 35 insertions(+), 45 deletions(-) diff --git a/scripts/prepare_template.py b/scripts/prepare_template.py index 33f4811..07abcd6 100644 --- a/scripts/prepare_template.py +++ b/scripts/prepare_template.py @@ -8,71 +8,53 @@ # Function filtering the solution out of the input file. # also filters double empty lines that may result from filtering. -def filterSolution(inputFile, outputEmptyFile, outputSolutionFile): - +def filterSolution(inputFile, outputFile, keepSolution, pattern): + # rewind input file + inputFile.seek(0) + # Scan lines isSolution = False isTemplate = False - emptyEmptyLine = False - emptySolutionLine = False + emptyLine = False for line in inputFile: # Check if the line is the #define - if(re.match(r'.*#define SOLUTION.*\n', line)): + if(re.match(rf'.*#define SOLUTION.*\n', line)): continue # skip the line # Check if the line starts a solution block - if(re.match(r'.*#ifdef SOLUTION.*\n', line)): + if(re.match(rf'.*#ifdef {pattern}\s*\n', line)): isSolution = True continue # skip the line - + # Check if the line start a template block if(isSolution and re.match(r'.*#else.*\n', line)): isSolution = False isTemplate = True continue # skip the line - # Check if the line start a template block - if((isSolution or isTemplate ) and re.match(r'.*#endif // SOLUTION.*\n', line)): + # Check if the line ends a block + if((isSolution or isTemplate ) and re.match(rf'.*#endif // {pattern}\s*\n', line)): isSolution = False isTemplate = False continue # skip the line - printEmptyLine = False - printSolutionLine = False - - if(isTemplate): - printEmptyLine=True - - if(isSolution): - printSolutionLine = True + printLine = False + if keepSolution: + if isSolution or (not isSolution and not isTemplate): + printLine = True + else: + if isTemplate or (not isSolution and not isTemplate): + printLine = True - if(not isSolution and not isTemplate): - printEmptyLine=True - printSolutionLine = True - - # Print line in empty file - if(printEmptyLine): + if printLine: if(re.match(r'\s*\n', line)): - if(emptyEmptyLine): + if emptyLine: continue # skipLine else: - emptyEmptyLine=True + emptyLine = True else: - emptyEmptyLine=False - - outputEmptyFile.write(line) - - # Print line in solution file - if(printSolutionLine): - if(re.match(r'\s*\n', line)): - if(emptySolutionLine): - continue # skipLine - else: - emptySolutionLine=True - else: - emptySolutionLine=False - - outputSolutionFile.write(line) + emptyLine = False + outputFile.write(line) # Open the files @@ -87,12 +69,20 @@ def filterSolution(inputFile, outputEmptyFile, outputSolutionFile): txtSolutionCMakeListsFile = open("./CMakeLists_solution.txt", "w") if(not cppInputFile or not cppEmptyOutputFile or not hInputFile or not hEmptyOutputFile or not txtEmptyCMakeListsFile or not txtInputCMakeListsFile or not txtSolutionCMakeListsFile): - exit + exit + + +## Filter header file +filterSolution(hInputFile, hEmptyOutputFile, False, "SOLUTION.*") +filterSolution(hInputFile, hSolutionOutputFile, True, "SOLUTION.*") + +## Filter cpp file +filterSolution(cppInputFile, cppEmptyOutputFile, False, "SOLUTION.*") +filterSolution(cppInputFile, cppSolutionOutputFile, True, "SOLUTION.*") -## Filter cpp files -filterSolution(hInputFile, hEmptyOutputFile, hSolutionOutputFile) -filterSolution(cppInputFile, cppEmptyOutputFile, cppSolutionOutputFile) -filterSolution(txtInputCMakeListsFile, txtEmptyCMakeListsFile, txtSolutionCMakeListsFile) +## Filter CMakeLists file +filterSolution(txtInputCMakeListsFile, txtEmptyCMakeListsFile, False, "SOLUTION.*") +filterSolution(txtInputCMakeListsFile, txtSolutionCMakeListsFile, True, "SOLUTION.*") # Close files cppInputFile.close() From e273f2b0b35a1d46908133c82b0206667baef32a Mon Sep 17 00:00:00 2001 From: kdesnos Date: Mon, 15 Dec 2025 14:27:47 +0100 Subject: [PATCH 04/32] (Releng) Factorize code. --- scripts/prepare_template.py | 67 +++++++++++++++++-------------------- 1 file changed, 30 insertions(+), 37 deletions(-) diff --git a/scripts/prepare_template.py b/scripts/prepare_template.py index 07abcd6..9b3f47b 100644 --- a/scripts/prepare_template.py +++ b/scripts/prepare_template.py @@ -57,40 +57,33 @@ def filterSolution(inputFile, outputFile, keepSolution, pattern): outputFile.write(line) -# Open the files -cppInputFile = open("./src/training/pendulum_wrapper.cpp","r") -cppEmptyOutputFile = open("./src/training/pendulum_wrapper_empty.cpp", "w") -cppSolutionOutputFile = open("./src/training/pendulum_wrapper_solution.cpp", "w") -hInputFile = open("./src/training/pendulum_wrapper.h","r") -hEmptyOutputFile = open("./src/training/pendulum_wrapper_empty.h", "w") -hSolutionOutputFile = open("./src/training/pendulum_wrapper_solution.h", "w") -txtInputCMakeListsFile = open("./CMakeLists.txt", "r") -txtEmptyCMakeListsFile = open("./CMakeLists_empty.txt", "w") -txtSolutionCMakeListsFile = open("./CMakeLists_solution.txt", "w") - -if(not cppInputFile or not cppEmptyOutputFile or not hInputFile or not hEmptyOutputFile or not txtEmptyCMakeListsFile or not txtInputCMakeListsFile or not txtSolutionCMakeListsFile): - exit - - -## Filter header file -filterSolution(hInputFile, hEmptyOutputFile, False, "SOLUTION.*") -filterSolution(hInputFile, hSolutionOutputFile, True, "SOLUTION.*") - -## Filter cpp file -filterSolution(cppInputFile, cppEmptyOutputFile, False, "SOLUTION.*") -filterSolution(cppInputFile, cppSolutionOutputFile, True, "SOLUTION.*") - -## Filter CMakeLists file -filterSolution(txtInputCMakeListsFile, txtEmptyCMakeListsFile, False, "SOLUTION.*") -filterSolution(txtInputCMakeListsFile, txtSolutionCMakeListsFile, True, "SOLUTION.*") - -# Close files -cppInputFile.close() -cppEmptyOutputFile.close() -cppSolutionOutputFile.close() -hInputFile.close() -hEmptyOutputFile.close() -hSolutionOutputFile.close() -txtSolutionCMakeListsFile.close() -txtInputCMakeListsFile.close() -txtEmptyCMakeListsFile.close() +# Files to filter +files = [ + ["./src/training/pendulum_wrapper.cpp", "./src/training/pendulum_wrapper_empty.cpp", "./src/training/pendulum_wrapper_solution.cpp"], + ["./src/training/pendulum_wrapper.h", "./src/training/pendulum_wrapper_empty.h", "./src/training/pendulum_wrapper_solution.h"], + ["./CMakeLists.txt", "./CMakeLists_empty.txt", "./CMakeLists_solution.txt"] +] + +# Prepare files +for fileSet in files: + inputFilePath = fileSet[0] + emptyOutputFilePath = fileSet[1] + solutionOutputFilePath = fileSet[2] + + # Open the files + inputFile = open(inputFilePath, "r") + emptyOutputFile = open(emptyOutputFilePath, "w") + solutionOutputFile = open(solutionOutputFilePath, "w") + + if(not inputFile or not emptyOutputFile or not solutionOutputFile): + exit + + # Filter empty version + filterSolution(inputFile, emptyOutputFile, False, "SOLUTION.*") + # Filter solution version + filterSolution(inputFile, solutionOutputFile, True, "SOLUTION.*") + + # Close files + inputFile.close() + emptyOutputFile.close() + solutionOutputFile.close() From 1c4522d457e2ff712b6e7eb21a41df627bcdf8b4 Mon Sep 17 00:00:00 2001 From: kdesnos Date: Mon, 15 Dec 2025 15:23:23 +0100 Subject: [PATCH 05/32] (RelEng) Improve multi-tuto templating. --- scripts/prepare_template.py | 62 +++++++++++++++++++++++++------------ 1 file changed, 43 insertions(+), 19 deletions(-) diff --git a/scripts/prepare_template.py b/scripts/prepare_template.py index 9b3f47b..bae5185 100644 --- a/scripts/prepare_template.py +++ b/scripts/prepare_template.py @@ -4,6 +4,7 @@ # License: CeCILL-C import re +import io # Function filtering the solution out of the input file. @@ -58,32 +59,55 @@ def filterSolution(inputFile, outputFile, keepSolution, pattern): # Files to filter +# Each entry: [inputPath, [(outputPath, patternToRemove, patternToKeep), ...]] files = [ - ["./src/training/pendulum_wrapper.cpp", "./src/training/pendulum_wrapper_empty.cpp", "./src/training/pendulum_wrapper_solution.cpp"], - ["./src/training/pendulum_wrapper.h", "./src/training/pendulum_wrapper_empty.h", "./src/training/pendulum_wrapper_solution.h"], - ["./CMakeLists.txt", "./CMakeLists_empty.txt", "./CMakeLists_solution.txt"] + ["./src/training/pendulum_wrapper.cpp", [ + ["./src/training/pendulum_wrapper_empty.cpp", "SOLUTION.*", ""], + ["./src/training/pendulum_wrapper_solution.cpp", "SOLUTION_PARALLEL", "SOLUTION.*"], + ["./src/training/pendulum_wrapper_parallel.cpp", "", "SOLUTION.*"], + ]], + ["./src/training/pendulum_wrapper.h", [ + ["./src/training/pendulum_wrapper_empty.h", "SOLUTION.*", ""], + ["./src/training/pendulum_wrapper_solution.h", "SOLUTION_PARALLEL", "SOLUTION.*"], + ["./src/training/pendulum_wrapper_parallel.h", "", "SOLUTION.*"], + ]], + ["./CMakeLists.txt", [ + ["./CMakeLists_empty.txt", "SOLUTION.*", ""], + ["./CMakeLists_solution.txt", "", "SOLUTION.*"], + ]], + ["./src/training/main-training.cpp", [ + ["./src/training/main-training_empty.cpp", "SOLUTION.*", ""], + ["./src/training/main-training_parallel.cpp", "", "SOLUTION.*"], + ]], ] # Prepare files for fileSet in files: inputFilePath = fileSet[0] - emptyOutputFilePath = fileSet[1] - solutionOutputFilePath = fileSet[2] + outputs = fileSet[1] - # Open the files inputFile = open(inputFilePath, "r") - emptyOutputFile = open(emptyOutputFilePath, "w") - solutionOutputFile = open(solutionOutputFilePath, "w") - - if(not inputFile or not emptyOutputFile or not solutionOutputFile): - exit - - # Filter empty version - filterSolution(inputFile, emptyOutputFile, False, "SOLUTION.*") - # Filter solution version - filterSolution(inputFile, solutionOutputFile, True, "SOLUTION.*") + if not inputFile: + continue + + for out in outputs: + outputPath, patternRemove, patternKeep = out + outputFile = open(outputPath, "w") + if patternRemove and not patternKeep: + filterSolution(inputFile, outputFile, False, patternRemove) + elif patternKeep and not patternRemove: + filterSolution(inputFile, outputFile, True, patternKeep) + elif patternKeep and patternRemove: + # Remove first into an in-memory buffer, then keep from that buffer + temp = io.StringIO() + filterSolution(inputFile, temp, False, patternRemove) + filterSolution(temp, outputFile, True, patternKeep) + temp.close() + else: + # No pattern provided: copy file as-is + inputFile.seek(0) + for line in inputFile: + outputFile.write(line) + outputFile.close() - # Close files inputFile.close() - emptyOutputFile.close() - solutionOutputFile.close() From 85fef7cccf5e213291b1ba28ad6819facc507e1e Mon Sep 17 00:00:00 2001 From: kdesnos Date: Mon, 15 Dec 2025 16:09:43 +0100 Subject: [PATCH 06/32] (RelEng) Create result of parallel tutorial. --- scripts/prepare_archives.py | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/scripts/prepare_archives.py b/scripts/prepare_archives.py index 9ceeee9..0b45b70 100644 --- a/scripts/prepare_archives.py +++ b/scripts/prepare_archives.py @@ -76,5 +76,16 @@ def zipFilesInDir(dirName, zipObj, regex, parentName="", withSubdirectories = Tr tutorialSolutionArchive.write("CMakeLists_empty.txt", mainFolder + "CMakeLists.txt") # overwrite empty_file tutorialSolutionArchive.close() -# Make the main-inference.cpp file available +# Create the gegelati-tutorial-parallel-solution archive by copying the solution archive +mainFolder = "gegelati-tutorial/" +# copy the solution archive as a base +shutil.copy2("./docs/data/gegelati-tutorial-solution.zip", "./docs/data/gegelati-tutorial-parallel-solution.zip") +# open the copied archive in append mode and overwrite only the differing entries +tutorialParallelSolutionArchive = ZipFile("./docs/data/gegelati-tutorial-parallel-solution.zip", "a") +tutorialParallelSolutionArchive.write("src/training/pendulum_wrapper_parallel.cpp", mainFolder + "src/training/pendulum_wrapper.cpp") +tutorialParallelSolutionArchive.write("src/training/pendulum_wrapper_parallel.h", mainFolder + "src/training/pendulum_wrapper.h") +tutorialParallelSolutionArchive.write("src/training/main-training_parallel.cpp", mainFolder + "src/training/main-training.cpp") +tutorialParallelSolutionArchive.close() + +# Make the main-inference.cpp file available for download shutil.copy2("./src/inference/main-inference.cpp", "./docs/data/") \ No newline at end of file From 147aab249cb3f8dff8009ec4ccbad3eb35220d30 Mon Sep 17 00:00:00 2001 From: kdesnos Date: Tue, 16 Dec 2025 14:39:03 +0100 Subject: [PATCH 07/32] (RelEng) Clean zip file generation. --- scripts/prepare_archives.py | 51 +++++++++++++++++++++---------------- 1 file changed, 29 insertions(+), 22 deletions(-) diff --git a/scripts/prepare_archives.py b/scripts/prepare_archives.py index 0b45b70..9af328e 100644 --- a/scripts/prepare_archives.py +++ b/scripts/prepare_archives.py @@ -38,7 +38,24 @@ def zipFilesInDir(dirName, zipObj, regex, parentName="", withSubdirectories = Tr # Add file to zip zipObj.write(filePath, parentName + filePath) +def replace_file_in_zip(zip_path, file_to_add, arcname): + """ + Replace a file in a zip archive by first removing the existing file (if present), + then adding the new file. + """ + import tempfile + # Create a temporary zip file + tmpfd, tmpname = tempfile.mkstemp(suffix='.zip') + os.close(tmpfd) + with ZipFile(zip_path, 'r') as zin, ZipFile(tmpname, 'w') as zout: + for item in zin.infolist(): + if item.filename != arcname: + zout.writestr(item, zin.read(item.filename)) + # Now add the new file + zout.write(file_to_add, arcname) + # Replace the original zip with the modified one + shutil.move(tmpname, zip_path) # Create the tutorialTemplate archive mainFolder = "gegelati-tutorial/" @@ -49,9 +66,10 @@ def zipFilesInDir(dirName, zipObj, regex, parentName="", withSubdirectories = Tr zipFilesInDir("./lib/",tutorialTemplateArchive, r'.*', mainFolder) zipFilesInDir("src/",tutorialTemplateArchive, r'.*', mainFolder, False) zipFilesInDir("src/manual/",tutorialTemplateArchive, r'.*', mainFolder) -zipFilesInDir("src/training",tutorialTemplateArchive, r'^(?!.*(pendulum_wrapper))', mainFolder, False) # all files except pendulum_wrapper +zipFilesInDir("src/training",tutorialTemplateArchive, r'^(?!.*(pendulum_wrapper|main-training))', mainFolder, False) # all files except pendulum_wrapper or main-training tutorialTemplateArchive.write("src/training/pendulum_wrapper_empty.cpp", mainFolder + "src/training/pendulum_wrapper.cpp" ) # overwrite empty_file tutorialTemplateArchive.write("src/training/pendulum_wrapper_empty.h", mainFolder + "src/training/pendulum_wrapper.h") # overwrite empty_file +tutorialTemplateArchive.write("src/training/main-training_empty.cpp", mainFolder + "src/training/main-training.cpp") # overwrite empty_file tutorialTemplateArchive.write("CMakeLists_empty.txt", mainFolder + "CMakeLists.txt") # overwrite empty_file tutorialTemplateArchive.close() @@ -61,31 +79,20 @@ def zipFilesInDir(dirName, zipObj, regex, parentName="", withSubdirectories = Tr pendulumWrapperSolutionArchive.write("src/training/pendulum_wrapper_solution.h", "pendulum_wrapper.h") # overwrite empty_file pendulumWrapperSolutionArchive.close() -# Create the gegelati-tutorial-solution archive + +# Create the gegelati-tutorial-solution archive by copying the template and patching needed files mainFolder = "gegelati-tutorial/" -tutorialSolutionArchive = ZipFile("./docs/data/gegelati-tutorial-solution.zip", "w") -zipFileAdd(tutorialSolutionArchive,"bin/", mainFolder) -zipFilesInDir("./",tutorialSolutionArchive, r'^(?!.*(CMakeLists))[^\.]+.*', mainFolder, False) # exclude .gitgnore and CMakeLists files -zipFilesInDir("./dat/",tutorialSolutionArchive, r'.*', mainFolder) -zipFilesInDir("./lib/",tutorialSolutionArchive, r'.*', mainFolder) -zipFilesInDir("src/",tutorialSolutionArchive, r'.*', mainFolder, False) -zipFilesInDir("src/manual/",tutorialSolutionArchive, r'.*', mainFolder) -zipFilesInDir("src/training",tutorialSolutionArchive, r'^(?!.*(pendulum_wrapper))', mainFolder, False) # all files except pendulum_wrapper -tutorialSolutionArchive.write("src/training/pendulum_wrapper_solution.cpp", mainFolder + "src/training/pendulum_wrapper.cpp" ) # overwrite empty_file -tutorialSolutionArchive.write("src/training/pendulum_wrapper_solution.h", mainFolder + "src/training/pendulum_wrapper.h") # overwrite empty_file -tutorialSolutionArchive.write("CMakeLists_empty.txt", mainFolder + "CMakeLists.txt") # overwrite empty_file -tutorialSolutionArchive.close() +shutil.copy2("./docs/data/gegelati-tutorial.zip", "./docs/data/gegelati-tutorial-solution.zip") +replace_file_in_zip("./docs/data/gegelati-tutorial-solution.zip", "src/training/pendulum_wrapper_solution.cpp", mainFolder + "src/training/pendulum_wrapper.cpp") +replace_file_in_zip("./docs/data/gegelati-tutorial-solution.zip", "src/training/pendulum_wrapper_solution.h", mainFolder + "src/training/pendulum_wrapper.h") # Create the gegelati-tutorial-parallel-solution archive by copying the solution archive mainFolder = "gegelati-tutorial/" -# copy the solution archive as a base shutil.copy2("./docs/data/gegelati-tutorial-solution.zip", "./docs/data/gegelati-tutorial-parallel-solution.zip") -# open the copied archive in append mode and overwrite only the differing entries -tutorialParallelSolutionArchive = ZipFile("./docs/data/gegelati-tutorial-parallel-solution.zip", "a") -tutorialParallelSolutionArchive.write("src/training/pendulum_wrapper_parallel.cpp", mainFolder + "src/training/pendulum_wrapper.cpp") -tutorialParallelSolutionArchive.write("src/training/pendulum_wrapper_parallel.h", mainFolder + "src/training/pendulum_wrapper.h") -tutorialParallelSolutionArchive.write("src/training/main-training_parallel.cpp", mainFolder + "src/training/main-training.cpp") -tutorialParallelSolutionArchive.close() +replace_file_in_zip("./docs/data/gegelati-tutorial-parallel-solution.zip", "src/training/pendulum_wrapper_parallel.cpp", mainFolder + "src/training/pendulum_wrapper.cpp") +replace_file_in_zip("./docs/data/gegelati-tutorial-parallel-solution.zip", "src/training/pendulum_wrapper_parallel.h", mainFolder + "src/training/pendulum_wrapper.h") +replace_file_in_zip("./docs/data/gegelati-tutorial-parallel-solution.zip", "src/training/main-training_parallel.cpp", mainFolder + "src/training/main-training.cpp") # Make the main-inference.cpp file available for download -shutil.copy2("./src/inference/main-inference.cpp", "./docs/data/") \ No newline at end of file +shutil.copy2("./src/inference/main-inference.cpp", "./docs/data/") + From 5c1b3fa9a3d31bcd882bcfb3f91a892437bd56eb Mon Sep 17 00:00:00 2001 From: kdesnos Date: Tue, 16 Dec 2025 14:43:30 +0100 Subject: [PATCH 08/32] (RelEng) Start building CI for testing tutorial build. --- .github/workflows/test-tuto.yml | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 .github/workflows/test-tuto.yml diff --git a/.github/workflows/test-tuto.yml b/.github/workflows/test-tuto.yml new file mode 100644 index 0000000..75a1aab --- /dev/null +++ b/.github/workflows/test-tuto.yml @@ -0,0 +1,20 @@ +name: Build Archives (CI Only) + +on: + push: + branches: + - '**' + +jobs: + build_archives: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + + - name: Prepare template files + run: | + python ./scripts/prepare_template.py + + - name: Prepare archives + run: | + python ./scripts/prepare_archives.py \ No newline at end of file From 76f5128acb88256b6fe4183a26bdad014b3479b3 Mon Sep 17 00:00:00 2001 From: kdesnos Date: Tue, 16 Dec 2025 14:51:50 +0100 Subject: [PATCH 09/32] (RelEng) Build tuto (ubuntu only for now) --- .github/workflows/test-tuto.yml | 81 ++++++++++++++++++++++++++++++++- 1 file changed, 80 insertions(+), 1 deletion(-) diff --git a/.github/workflows/test-tuto.yml b/.github/workflows/test-tuto.yml index 75a1aab..d356e06 100644 --- a/.github/workflows/test-tuto.yml +++ b/.github/workflows/test-tuto.yml @@ -17,4 +17,83 @@ jobs: - name: Prepare archives run: | - python ./scripts/prepare_archives.py \ No newline at end of file + python ./scripts/prepare_archives.py + + - name: Upload archives as artifacts + uses: actions/upload-artifact@v4 + with: + name: tutorial-archives + path: | + docs/data/gegelati-tutorial.zip + docs/data/gegelati-tutorial-solution.zip + docs/data/gegelati-tutorial-parallel-solution.zip + + test_archives: + needs: build_archives + runs-on: ${{ matrix.os }} + strategy: + fail-fast: false + matrix: + archive: + - gegelati-tutorial.zip + - gegelati-tutorial-solution.zip + - gegelati-tutorial-parallel-solution.zip + os: [ubuntu-latest] ##, windows-latest, macos-latest] + compiler: [gcc] ##, clang, msvc] + ## exclude: + # Exclude MSVC on non-Windows + ## - os: ubuntu-latest + ## compiler: msvc + ## - os: macos-latest + ## compiler: msvc + # Exclude GCC and Clang on Windows (unless you want to setup MinGW/LLVM) + ## - os: windows-latest + ## compiler: gcc + ## - os: windows-latest + ## compiler: clang + + name: Test ${{ matrix.archive }} on ${{ matrix.os }} with ${{ matrix.compiler }} + steps: + - uses: actions/checkout@v3 + + - name: Download archives artifact + uses: actions/download-artifact@v4 + with: + name: tutorial-archives + path: archives + + - name: Unzip archive + run: | + unzip -q archives/${{ matrix.archive }} -d tutorial + shell: bash + + - name: Set up compiler + if: matrix.compiler == 'gcc' + uses: egor-tensin/setup-gcc@v1 + with: + version: latest + # GCC is default on Linux, but this ensures it's available + + - name: Set up Clang + if: matrix.compiler == 'clang' + uses: egor-tensin/setup-clang@v1 + with: + version: latest + + - name: Set up MSVC + if: matrix.compiler == 'msvc' && runner.os == 'Windows' + uses: ilammy/msvc-dev-cmd@v1 + + - name: Configure CMake project + run: | + mkdir -p tutorial/build + cmake -S tutorial/gegelati-tutorial -B build + env: + CC: ${{ matrix.compiler == 'gcc' && 'gcc' || matrix.compiler == 'clang' && 'clang' || '' }} + CXX: ${{ matrix.compiler == 'gcc' && 'g++' || matrix.compiler == 'clang' && 'clang++' || '' }} + shell: bash + + - name: Build project + run: | + cmake --build build --config Release + shell: bash \ No newline at end of file From 2739bd876bb5444fe428762f33ff8d9d2a7d61e0 Mon Sep 17 00:00:00 2001 From: kdesnos Date: Tue, 16 Dec 2025 15:12:03 +0100 Subject: [PATCH 10/32] (RelEng) Install gegelati for build. --- .github/workflows/test-tuto.yml | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/.github/workflows/test-tuto.yml b/.github/workflows/test-tuto.yml index d356e06..26ba29b 100644 --- a/.github/workflows/test-tuto.yml +++ b/.github/workflows/test-tuto.yml @@ -84,6 +84,20 @@ jobs: if: matrix.compiler == 'msvc' && runner.os == 'Windows' uses: ilammy/msvc-dev-cmd@v1 + - name: Install Libraries (Linux) + if: matrix.os == 'ubuntu-latest' + run: | + sudo apt install -y libsdl2-dev libsdl2-image-dev libsdl2-ttf-dev + + - name: Build Gegelati (Linux/MacOS) + if: matrix.os == 'ubuntu-latest' || matrix.os == 'macos-latest' + run: | + git clone -b master https://github.com/gegelati/gegelati.git lib/gegelati + cd lib/gegelati/bin + cmake .. -DBUILD_TESTING=OFF -DSKIP_DOXYGEN_BUILD=ON -DCMAKE_BUILD_TYPE=Release + sudo cmake --build . --target install --parallel $(nproc) + shell: bash + - name: Configure CMake project run: | mkdir -p tutorial/build From b9522fcf5a92ea4be1fc1a42c619a7c37a0e0fa6 Mon Sep 17 00:00:00 2001 From: kdesnos Date: Tue, 16 Dec 2025 15:28:52 +0100 Subject: [PATCH 11/32] (RelEng) Add build targets. --- .github/workflows/test-tuto.yml | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/.github/workflows/test-tuto.yml b/.github/workflows/test-tuto.yml index 26ba29b..456fbd4 100644 --- a/.github/workflows/test-tuto.yml +++ b/.github/workflows/test-tuto.yml @@ -107,7 +107,13 @@ jobs: CXX: ${{ matrix.compiler == 'gcc' && 'g++' || matrix.compiler == 'clang' && 'clang++' || '' }} shell: bash - - name: Build project + - name: Build manual-control target run: | - cmake --build build --config Release + cmake --build build --config Release --target manual-control -- -j$(nproc) + shell: bash + + - name: Build tpg-training target + if: matrix.archive != 'gegelati-tutorial.zip' + run: | + cmake --build build --config Release --target tpg-training -- -j$(nproc) shell: bash \ No newline at end of file From faf7e30b124497f478f59c796bc1b9b06474f45e Mon Sep 17 00:00:00 2001 From: kdesnos Date: Tue, 16 Dec 2025 15:44:03 +0100 Subject: [PATCH 12/32] (Releng) Run the training. --- .github/workflows/test-tuto.yml | 4 +++- CMakeLists.txt | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/.github/workflows/test-tuto.yml b/.github/workflows/test-tuto.yml index 456fbd4..4c3f0c6 100644 --- a/.github/workflows/test-tuto.yml +++ b/.github/workflows/test-tuto.yml @@ -112,8 +112,10 @@ jobs: cmake --build build --config Release --target manual-control -- -j$(nproc) shell: bash - - name: Build tpg-training target + - name: Build and run tpg-training target if: matrix.archive != 'gegelati-tutorial.zip' run: | + sed -i 's/"nbGenerations": [0-9]*/"nbGenerations": 4/' tutorial/gegelati-tutorial/params.json cmake --build build --config Release --target tpg-training -- -j$(nproc) + ./build/Release/tpg-training shell: bash \ No newline at end of file diff --git a/CMakeLists.txt b/CMakeLists.txt index 9d8b53a..aca2ee2 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -52,7 +52,7 @@ endif() # Add definitions for testing purposes if(${TESTING}) MESSAGE("Testing mode") - add_definitions(-DNO_CONSOLE_CONTROL -DNB_GENERATIONS=2) + add_definitions(-DNO_CONSOLE_CONTROL -DNB_GENERATIONS=2 -DDEACTIVATE_DISPLAY=1) endif() # ******************************************* From d1a295cde647efa1e624f553f25ac4060d0faa81 Mon Sep 17 00:00:00 2001 From: kdesnos Date: Tue, 16 Dec 2025 16:21:19 +0100 Subject: [PATCH 13/32] (Tuto) Deactivate rendering code when relevant. --- src/training/main-training.cpp | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/src/training/main-training.cpp b/src/training/main-training.cpp index 3ac1d5c..6902251 100644 --- a/src/training/main-training.cpp +++ b/src/training/main-training.cpp @@ -91,15 +91,14 @@ int main(int argc, char** argv) { #if ( DEACTIVATE_DISPLAY == 0 ) // Start training in secondary thread std::thread threadTraining(train_main, std::ref(exitProgram), std::ref(doDisplay), std::ref(generation), std::ref(time_delta), std::ref(replay)); + // Replay code + Renderer::replayThread(exitProgram, doDisplay, generation, time_delta, replay); #else std::cout << "No display version, send interrupt signal to process to exit." << std::endl; // Start training in main thread train_main(exitProgram, doDisplay, generation, time_delta, replay); #endif - // Replay code - Renderer::replayThread(exitProgram, doDisplay, generation, time_delta, replay); - #if ( DEACTIVATE_DISPLAY == 0 ) // Exit the display thread threadTraining.join(); From b17b74efb90047e22851a54c6df227bb04afe9b0 Mon Sep 17 00:00:00 2001 From: kdesnos Date: Tue, 16 Dec 2025 16:27:38 +0100 Subject: [PATCH 14/32] (RelEng) Compile for test --- .github/workflows/test-tuto.yml | 2 +- src/training/main-training.cpp | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/workflows/test-tuto.yml b/.github/workflows/test-tuto.yml index 4c3f0c6..6b5fb99 100644 --- a/.github/workflows/test-tuto.yml +++ b/.github/workflows/test-tuto.yml @@ -101,7 +101,7 @@ jobs: - name: Configure CMake project run: | mkdir -p tutorial/build - cmake -S tutorial/gegelati-tutorial -B build + cmake -S tutorial/gegelati-tutorial -B build -DTESTING=ON env: CC: ${{ matrix.compiler == 'gcc' && 'gcc' || matrix.compiler == 'clang' && 'clang' || '' }} CXX: ${{ matrix.compiler == 'gcc' && 'g++' || matrix.compiler == 'clang' && 'clang++' || '' }} diff --git a/src/training/main-training.cpp b/src/training/main-training.cpp index 6902251..543ef6e 100644 --- a/src/training/main-training.cpp +++ b/src/training/main-training.cpp @@ -16,7 +16,9 @@ #include "pendulum_wrapper.h" +#ifndef DEACTIVATE_DISPLAY #define DEACTIVATE_DISPLAY 0 +#endif /** From f2c1a362dc3cad316af759297570aa1d1396a63e Mon Sep 17 00:00:00 2001 From: kdesnos Date: Wed, 17 Dec 2025 11:44:34 +0100 Subject: [PATCH 15/32] (Tuto) Start developping code for multi-episode evaluation. --- scripts/prepare_template.py | 143 +++++++++++++++++------------- src/training/pendulum_wrapper.cpp | 11 +++ src/training/pendulum_wrapper.h | 5 ++ 3 files changed, 95 insertions(+), 64 deletions(-) diff --git a/scripts/prepare_template.py b/scripts/prepare_template.py index bae5185..0b01e2f 100644 --- a/scripts/prepare_template.py +++ b/scripts/prepare_template.py @@ -9,75 +9,90 @@ # Function filtering the solution out of the input file. # also filters double empty lines that may result from filtering. -def filterSolution(inputFile, outputFile, keepSolution, pattern): - # rewind input file - inputFile.seek(0) - - # Scan lines - isSolution = False - isTemplate = False - emptyLine = False - for line in inputFile: - # Check if the line is the #define - if(re.match(rf'.*#define SOLUTION.*\n', line)): - continue # skip the line - - # Check if the line starts a solution block - if(re.match(rf'.*#ifdef {pattern}\s*\n', line)): - isSolution = True - continue # skip the line - - # Check if the line start a template block - if(isSolution and re.match(r'.*#else.*\n', line)): - isSolution = False - isTemplate = True - continue # skip the line - - # Check if the line ends a block - if((isSolution or isTemplate ) and re.match(rf'.*#endif // {pattern}\s*\n', line)): - isSolution = False - isTemplate = False - continue # skip the line - - printLine = False - if keepSolution: - if isSolution or (not isSolution and not isTemplate): - printLine = True - else: - if isTemplate or (not isSolution and not isTemplate): - printLine = True +def filterSolution(inputFile, outputFile, keepSolution, patterns): + # Start with the original input file + currentInput = inputFile - if printLine: - if(re.match(r'\s*\n', line)): - if emptyLine: - continue # skipLine - else: - emptyLine = True + # Apply each pattern sequentially + for pattern in patterns: + # Use an in-memory buffer to store intermediate results + tempOutput = io.StringIO() + currentInput.seek(0) + + # Scan lines + isSolution = False + isTemplate = False + emptyLine = False + for line in currentInput: + # Check if the line is the #define + if re.match(rf'.*#define SOLUTION.*\n', line): + continue # skip the line + + # Check if the line starts a solution block + if re.match(rf'.*#ifdef {pattern}\s*\n', line): + isSolution = True + continue # skip the line + + # Check if the line starts a template block + if isSolution and re.match(r'.*#else.*\n', line): + isSolution = False + isTemplate = True + continue # skip the line + + # Check if the line ends a block + if (isSolution or isTemplate) and re.match(rf'.*#endif // {pattern}\s*\n', line): + isSolution = False + isTemplate = False + continue # skip the line + + printLine = False + if keepSolution: + if isSolution or (not isSolution and not isTemplate): + printLine = True else: - emptyLine = False - outputFile.write(line) + if isTemplate or (not isSolution and not isTemplate): + printLine = True + + if printLine: + if re.match(r'\s*\n', line): + if emptyLine: + continue # skipLine + else: + emptyLine = True + else: + emptyLine = False + tempOutput.write(line) + + # Replace the current input with the output of this iteration + tempOutput.seek(0) + currentInput = tempOutput + + # Write the final result to the output file + currentInput.seek(0) + for line in currentInput: + outputFile.write(line) # Files to filter -# Each entry: [inputPath, [(outputPath, patternToRemove, patternToKeep), ...]] +# Each entry: [inputPath, [(outputPath, patternsToRemove, patternsToKeep), ...]] files = [ ["./src/training/pendulum_wrapper.cpp", [ - ["./src/training/pendulum_wrapper_empty.cpp", "SOLUTION.*", ""], - ["./src/training/pendulum_wrapper_solution.cpp", "SOLUTION_PARALLEL", "SOLUTION.*"], - ["./src/training/pendulum_wrapper_parallel.cpp", "", "SOLUTION.*"], + ["./src/training/pendulum_wrapper_empty.cpp", ["SOLUTION","SOLUTION_PARALLEL"], []], + ["./src/training/pendulum_wrapper_solution.cpp", ["SOLUTION_PARALLEL"], ["SOLUTION"]], + ["./src/training/pendulum_wrapper_parallel.cpp", [], ["SOLUTION_PARALLEL", "SOLUTION"]], ]], ["./src/training/pendulum_wrapper.h", [ - ["./src/training/pendulum_wrapper_empty.h", "SOLUTION.*", ""], - ["./src/training/pendulum_wrapper_solution.h", "SOLUTION_PARALLEL", "SOLUTION.*"], - ["./src/training/pendulum_wrapper_parallel.h", "", "SOLUTION.*"], + ["./src/training/pendulum_wrapper_empty.h", ["SOLUTION.*"], []], + ["./src/training/pendulum_wrapper_solution.h", ["SOLUTION_PARALLEL"], ["SOLUTION.*"]], + ["./src/training/pendulum_wrapper_parallel.h", [], ["SOLUTION.*"]], ]], ["./CMakeLists.txt", [ - ["./CMakeLists_empty.txt", "SOLUTION.*", ""], - ["./CMakeLists_solution.txt", "", "SOLUTION.*"], + ["./CMakeLists_empty.txt", ["SOLUTION.*"], []], + ["./CMakeLists_solution.txt", [], ["SOLUTION.*"]], ]], ["./src/training/main-training.cpp", [ - ["./src/training/main-training_empty.cpp", "SOLUTION.*", ""], - ["./src/training/main-training_parallel.cpp", "", "SOLUTION.*"], + ["./src/training/main-training_empty.cpp", ["SOLUTION.*"], []], + ["./src/training/main-training_parallel.cpp", [], ["SOLUTION.*"]], ]], ] @@ -91,17 +106,17 @@ def filterSolution(inputFile, outputFile, keepSolution, pattern): continue for out in outputs: - outputPath, patternRemove, patternKeep = out + outputPath, patternsRemove, patternsKeep = out outputFile = open(outputPath, "w") - if patternRemove and not patternKeep: - filterSolution(inputFile, outputFile, False, patternRemove) - elif patternKeep and not patternRemove: - filterSolution(inputFile, outputFile, True, patternKeep) - elif patternKeep and patternRemove: + if patternsRemove and not patternsKeep: + filterSolution(inputFile, outputFile, False, patternsRemove) + elif patternsKeep and not patternsRemove: + filterSolution(inputFile, outputFile, True, patternsKeep) + elif patternsKeep and patternsRemove: # Remove first into an in-memory buffer, then keep from that buffer temp = io.StringIO() - filterSolution(inputFile, temp, False, patternRemove) - filterSolution(temp, outputFile, True, patternKeep) + filterSolution(inputFile, temp, False, patternsRemove) + filterSolution(temp, outputFile, True, patternsKeep) temp.close() else: # No pattern provided: copy file as-is diff --git a/src/training/pendulum_wrapper.cpp b/src/training/pendulum_wrapper.cpp index fb979ca..767556d 100644 --- a/src/training/pendulum_wrapper.cpp +++ b/src/training/pendulum_wrapper.cpp @@ -47,10 +47,21 @@ std::vector> PendulumWrapper::ge void PendulumWrapper::reset(size_t seed, Learn::LearningMode mode, uint16_t iterationNumber, uint64_t generationNumber) { +#ifdef SOLUTION_PARALLEL + // Seed the RNG + this->rng.setSeed(seed); + // Randomize the initial angle between [pi - 0.5, pi + 0.5] + double initialAngle = M_PI + this->rng.getDouble(-0.5, 0.5); + this->pendulum.setAngle(initialAngle); + // Randomize the initial velocity between [-1.0, 1.0] + double initialVelocity = this->rng.getDouble(-1.0, 1.0); + this->pendulum.setVelocity(initialVelocity); +#else #ifdef SOLUTION this->pendulum.setAngle(M_PI); this->pendulum.setVelocity(0.0); #endif // SOLUTION +#endif // SOLUTION_PARALLEL #ifdef SOLUTION this->accumulatedReward = 0.0; #endif // SOLUTION diff --git a/src/training/pendulum_wrapper.h b/src/training/pendulum_wrapper.h index 113a4fe..a161a6f 100644 --- a/src/training/pendulum_wrapper.h +++ b/src/training/pendulum_wrapper.h @@ -41,6 +41,11 @@ class PendulumWrapper : public Learn::LearningEnvironment { double accumulatedReward; #endif // SOLUTION +#ifdef SOLUTION_PARALLEL + /// Random Number Generator for the environment + Mutator::RNG rng; +#endif // SOLUTION_PARALLEL + /// Default constructor for the PendulumWrapper PendulumWrapper(); From 02d1f89ef8278f3672f270b728f47daadf902530 Mon Sep 17 00:00:00 2001 From: kdesnos Date: Wed, 17 Dec 2025 12:04:08 +0100 Subject: [PATCH 16/32] (Tuto) Filter params.json for initial and parallel tuto archives. --- params.json | 4 ++++ scripts/prepare_archives.py | 2 ++ scripts/prepare_template.py | 4 ++++ 3 files changed, 10 insertions(+) diff --git a/params.json b/params.json index 182c53b..af73127 100644 --- a/params.json +++ b/params.json @@ -94,7 +94,11 @@ "nbIterationsPerJob": 1, // Number of evaluation of each root per generation. // "nbIterationsPerPolicyEvaluation" : 5, // Default value + //#ifdef SOLUTION_PARALLEL + "nbIterationsPerPolicyEvaluation": 5, + //#else "nbIterationsPerPolicyEvaluation": 1, + //#endif // SOLUTION_PARALLEL // Number of Constant available in each Program. // "nbProgramConstant" : 0, // Default value "nbProgramConstant": 0, diff --git a/scripts/prepare_archives.py b/scripts/prepare_archives.py index 9af328e..02745cb 100644 --- a/scripts/prepare_archives.py +++ b/scripts/prepare_archives.py @@ -71,6 +71,7 @@ def replace_file_in_zip(zip_path, file_to_add, arcname): tutorialTemplateArchive.write("src/training/pendulum_wrapper_empty.h", mainFolder + "src/training/pendulum_wrapper.h") # overwrite empty_file tutorialTemplateArchive.write("src/training/main-training_empty.cpp", mainFolder + "src/training/main-training.cpp") # overwrite empty_file tutorialTemplateArchive.write("CMakeLists_empty.txt", mainFolder + "CMakeLists.txt") # overwrite empty_file +tutorialTemplateArchive.write("params_empty.json", mainFolder + "params.json") # overwrite empty_file tutorialTemplateArchive.close() # Create the pendulum_wrapper_solution archive @@ -92,6 +93,7 @@ def replace_file_in_zip(zip_path, file_to_add, arcname): replace_file_in_zip("./docs/data/gegelati-tutorial-parallel-solution.zip", "src/training/pendulum_wrapper_parallel.cpp", mainFolder + "src/training/pendulum_wrapper.cpp") replace_file_in_zip("./docs/data/gegelati-tutorial-parallel-solution.zip", "src/training/pendulum_wrapper_parallel.h", mainFolder + "src/training/pendulum_wrapper.h") replace_file_in_zip("./docs/data/gegelati-tutorial-parallel-solution.zip", "src/training/main-training_parallel.cpp", mainFolder + "src/training/main-training.cpp") +replace_file_in_zip("./docs/data/gegelati-tutorial-parallel-solution.zip", "params_parallel.json", mainFolder + "params.json") # Make the main-inference.cpp file available for download shutil.copy2("./src/inference/main-inference.cpp", "./docs/data/") diff --git a/scripts/prepare_template.py b/scripts/prepare_template.py index 0b01e2f..592d92b 100644 --- a/scripts/prepare_template.py +++ b/scripts/prepare_template.py @@ -94,6 +94,10 @@ def filterSolution(inputFile, outputFile, keepSolution, patterns): ["./src/training/main-training_empty.cpp", ["SOLUTION.*"], []], ["./src/training/main-training_parallel.cpp", [], ["SOLUTION.*"]], ]], + ["./params.json", [ + ["./params_empty.json", ["SOLUTION.*"], []], + ["./params_parallel.json", [], ["SOLUTION_PARALLEL"]], + ]], ] # Prepare files From 07c6b4154c21b28e644748485d815fdd4400d6c4 Mon Sep 17 00:00:00 2001 From: kdesnos Date: Wed, 17 Dec 2025 12:11:33 +0100 Subject: [PATCH 17/32] (Tuto) Activate validation --- params.json | 4 ++++ src/training/pendulum_wrapper.cpp | 12 ++++++++++-- 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/params.json b/params.json index af73127..b734bf9 100644 --- a/params.json +++ b/params.json @@ -8,7 +8,11 @@ // Boolean used to activate an evaluation of the surviving roots in validation // mode after the training at each generation. // "doValidation" : false, // Default value + // #ifdef SOLUTION_PARALLEL + "doValidation": true, + // #else "doValidation": false, + // #endif // SOLUTION_PARALLEL // Maximum number of actions performed on the learning environment during the // each evaluation of a root. // "maxNbActionsPerEval" : 1000, // Default value diff --git a/src/training/pendulum_wrapper.cpp b/src/training/pendulum_wrapper.cpp index 767556d..3ff9e3a 100644 --- a/src/training/pendulum_wrapper.cpp +++ b/src/training/pendulum_wrapper.cpp @@ -48,8 +48,16 @@ std::vector> PendulumWrapper::ge void PendulumWrapper::reset(size_t seed, Learn::LearningMode mode, uint16_t iterationNumber, uint64_t generationNumber) { #ifdef SOLUTION_PARALLEL - // Seed the RNG - this->rng.setSeed(seed); + // In TRAINING mode, randomize the initial state + if (mode == Learn::LearningMode::TRAINING) { + // Seed the RNG differently for each iteration + this->rng.setSeed(seed + iterationNumber); + } + else { + // In VALIDATION and TESTING modes, use fixed seeds for reproducibility + this->rng.setSeed(iterationNumber); + } + // Randomize the initial angle between [pi - 0.5, pi + 0.5] double initialAngle = M_PI + this->rng.getDouble(-0.5, 0.5); this->pendulum.setAngle(initialAngle); From 4c9f4719a0d3a395f197453aab8f0068419b0505 Mon Sep 17 00:00:00 2001 From: kdesnos Date: Wed, 17 Dec 2025 12:13:44 +0100 Subject: [PATCH 18/32] (Tuto) Fix params.json double include in zip --- scripts/prepare_archives.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/prepare_archives.py b/scripts/prepare_archives.py index 02745cb..d1120ab 100644 --- a/scripts/prepare_archives.py +++ b/scripts/prepare_archives.py @@ -61,7 +61,7 @@ def replace_file_in_zip(zip_path, file_to_add, arcname): mainFolder = "gegelati-tutorial/" tutorialTemplateArchive = ZipFile("./docs/data/gegelati-tutorial.zip", "w") zipFileAdd(tutorialTemplateArchive,"bin/", mainFolder) -zipFilesInDir("./",tutorialTemplateArchive, r'^(?!.*(CMakeLists))[^\.]+.*', mainFolder, False) # exclude .gitgnore and CMakeLists files +zipFilesInDir("./",tutorialTemplateArchive, r'^(?!.*(CMakeLists|params))[^\.]+.*', mainFolder, False) # exclude .gitgnore and CMakeLists files zipFilesInDir("./dat/",tutorialTemplateArchive, r'.*', mainFolder) zipFilesInDir("./lib/",tutorialTemplateArchive, r'.*', mainFolder) zipFilesInDir("src/",tutorialTemplateArchive, r'.*', mainFolder, False) From 2b88afd7208b9619b0e8e9aa2a2f383a81b47851 Mon Sep 17 00:00:00 2001 From: kdesnos Date: Thu, 18 Dec 2025 16:15:02 +0100 Subject: [PATCH 19/32] (Tuto) Start tuto on multi-episode training (separate from parallelization) --- docs/_pages/strengthening_agents.md | 91 +++++++++++++++++++++++++++++ src/training/pendulum_wrapper.cpp | 4 +- 2 files changed, 93 insertions(+), 2 deletions(-) create mode 100644 docs/_pages/strengthening_agents.md diff --git a/docs/_pages/strengthening_agents.md b/docs/_pages/strengthening_agents.md new file mode 100644 index 0000000..9af18da --- /dev/null +++ b/docs/_pages/strengthening_agents.md @@ -0,0 +1,91 @@ +--- +title: Strengthening Reinforcement Learning Agents in with Multi-Episode Evaluation and Validation Phases +permalink: /tutos/strengthening-agents +toc: true +toc_sticky: true +--- + +The objective of this tutorial is two-fold: +1. Strengthen the built reinforcement learning agents by evaluating them over multiple episodes during training, and +2. Activate a validation phase at the end of each generation to monitor potential overfitting, and + +The starting point of this tutorial is the C++ project obtained at the end of the _[GEGELATI introductory tutorial](/gegelati-tutorial)_. While completing the introductory tutorial is strongly advised, a copy of the project resulting from this tutorial can be downloaded at the following link: [pendulum_wrapper_solution.zip](/gegelati-tutorial/data/gegelati-tutorial-solution.zip). + +## Multi-episode evaluation setup +### Why evaluate over multiple episodes? +An episode refers to a complete sequence of interactions between a reinforcement learning agent and its environment, starting from an initial state and ending when a terminal condition is met. For example, in the initial tutorial, an episode consists of the agent attempting to balance the pendulum for a fixed duration of 1500 time steps, as defined by the `maxNbActionsPerEval` parameter in `params.json`. + +In reinforcement learning, evaluating an agent's performance over multiple episodes is crucial for obtaining a reliable evaluation of its true capabilities. This is because the performance of an agent can vary significantly from one episode to another due to the inherent stochasticity of the environment and the agent's policy. By averaging the results over multiple episodes, we can mitigate the effects of randomness and strengthen the robustness of the learned policy. + +Implementing multi-episode evaluation in Gegelati involves modifying the `PendulumWrapper` class to support multiple episodes during the evaluation phase. To vary the starting conditions of each episode, the pendulum's angle and angular velocity will be randomly initialized at the beginning of each episode. + +### 0. Modify PendulumWrapper to support multi-episode evaluation +To implement multi-episode evaluation, we will first modify the `PendulumWrapper` class to support a stochastic reset of the pendulum's state at the beginning of each episode. + +To support random initialization, we will use a pseudo-random number generator to generate random values for the pendulum's angle and angular velocity within specified ranges. + +#### TODO 1: +Edit the `/gegelati-tutorial/src/environments/pendulum_wrapper.h`to add a random number generator as a member variable of the `PendulumWrapper` class. This pseudo-random number generator is provided in Gegelati with the `Mutator::RNG` class. + +{% details Solution to #1 (Click to expand) %} +```cpp +/* pendulum_wrapper.h */ +class PendulumWrapper : public Learn::LearningEnvironment { +public: + // Existing code... + + /// Random Number Generator for the environment + Mutator::RNG rng; +``` + +{% enddetails %} + +#### TODO 2: +Next, we will modify the `reset(size_t seed, Learn::LearningMode mode, uint16_t iterationNumber, uint64_t generationNumber)` method of the `PendulumWrapper` class to randomly initialize the pendulum's angle and angular velocity at the beginning of each episode. + +When calling the `reset(...)` method of the environment, Gegelati notably provides a `seed` parameter that can be used to seed the environment random number generator, using the `Mutator::RNG::seed(size_t seed)` method. Using this seeding mechanism ensures deterministic reproducibility of the random initialization across different runs. + +Once the RNG is seeded, we will use the `Mutator::RNG::getDouble(double min, double max)` method to generate random values within specified ranges. For example, we can set the angle to be randomly initialized between -π and π radians, and the angular velocity to be randomly initialized between -1.0 and 1.0 radians per second. + +{% details Solution to #2 (Click to expand) %} +The reset method be modified as follows: +```cpp +/* pendulum_wrapper.cpp */ +void PendulumWrapper::reset(size_t seed, Learn::LearningMode mode, uint16_t iterationNumber, uint64_t generationNumber) { + // Seed the RNG differently for each iteration + this->rng.setSeed(seed); + + // Randomize the initial angle between [-pi, pi] + double initialAngle = this->rng.getDouble(-M_PI, M_PI); + this->pendulum.setAngle(initialAngle); + // Randomize the initial velocity between [-1.0, 1.0] + double initialVelocity = this->rng.getDouble(-1.0, 1.0); + this->pendulum.setVelocity(initialVelocity); +} +``` + +{% enddetails %} + +### 1. Configure multi-episode evaluation in params.json +To enable multi-episode evaluation during training, we need to modify the training parameters in the `params.json` file of the project. + +#### TODO 3: +Edit the `/gegelati-tutorial/params.json` file to set the `nbEpisodesPerEval` parameter to a value greater than 1. This parameter specifies the number of episodes over which each agent will be evaluated during training. For this tutorial, set it to 5. + +{% details Solution to #3 (Click to expand) %} +```json +{ + // Existing parameters... + "nbEpisodesPerEval": 5, + // Existing parameters... +} +``` + +{% enddetails %} + +## Conclusion +In this tutorial, you have successfully enabled multi-episode evaluation for reinforcement learning agents in Gegelati. By evaluating agents over multiple episodes, you have strengthened the robustness of the learned policy and mitigated the effects of randomness in the environment. + +More information about reinforcement learning with Gegelati can be found in the following publication: + +[_K. Desnos, N. Sourbier, P.-Y. Raumer, O. Gesny and M. Pelcat. GEGELATI: Lightweight Artificial Intelligence through Generic and Evolvable Tangled Program Graphs. In Workshop on Design and Architectures for Signal and Image Processing (DASIP), ACM, 2021_](https://arxiv.org/pdf/2012.08296) \ No newline at end of file diff --git a/src/training/pendulum_wrapper.cpp b/src/training/pendulum_wrapper.cpp index 3ff9e3a..67215f3 100644 --- a/src/training/pendulum_wrapper.cpp +++ b/src/training/pendulum_wrapper.cpp @@ -58,8 +58,8 @@ void PendulumWrapper::reset(size_t seed, Learn::LearningMode mode, uint16_t iter this->rng.setSeed(iterationNumber); } - // Randomize the initial angle between [pi - 0.5, pi + 0.5] - double initialAngle = M_PI + this->rng.getDouble(-0.5, 0.5); + // Randomize the initial angle between [- pi, pi] + double initialAngle = this->rng.getDouble(-M_PI, M_PI); this->pendulum.setAngle(initialAngle); // Randomize the initial velocity between [-1.0, 1.0] double initialVelocity = this->rng.getDouble(-1.0, 1.0); From 384b0edede394f7ffb592d5b28cf50bc0a2c1c5e Mon Sep 17 00:00:00 2001 From: kdesnos Date: Thu, 18 Dec 2025 16:45:13 +0100 Subject: [PATCH 20/32] (Tuto) Put solution guards for strengthening tuto (breaks CI). --- src/training/pendulum_wrapper.cpp | 4 ++-- src/training/pendulum_wrapper.h | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/training/pendulum_wrapper.cpp b/src/training/pendulum_wrapper.cpp index 67215f3..8850711 100644 --- a/src/training/pendulum_wrapper.cpp +++ b/src/training/pendulum_wrapper.cpp @@ -47,7 +47,7 @@ std::vector> PendulumWrapper::ge void PendulumWrapper::reset(size_t seed, Learn::LearningMode mode, uint16_t iterationNumber, uint64_t generationNumber) { -#ifdef SOLUTION_PARALLEL +#ifdef SOLUTION_STRENGTHENING // In TRAINING mode, randomize the initial state if (mode == Learn::LearningMode::TRAINING) { // Seed the RNG differently for each iteration @@ -69,7 +69,7 @@ void PendulumWrapper::reset(size_t seed, Learn::LearningMode mode, uint16_t iter this->pendulum.setAngle(M_PI); this->pendulum.setVelocity(0.0); #endif // SOLUTION -#endif // SOLUTION_PARALLEL +#endif // SOLUTION_STRENGTHENING #ifdef SOLUTION this->accumulatedReward = 0.0; #endif // SOLUTION diff --git a/src/training/pendulum_wrapper.h b/src/training/pendulum_wrapper.h index a161a6f..31fb594 100644 --- a/src/training/pendulum_wrapper.h +++ b/src/training/pendulum_wrapper.h @@ -41,10 +41,10 @@ class PendulumWrapper : public Learn::LearningEnvironment { double accumulatedReward; #endif // SOLUTION -#ifdef SOLUTION_PARALLEL +#ifdef SOLUTION_STRENGTHENING /// Random Number Generator for the environment Mutator::RNG rng; -#endif // SOLUTION_PARALLEL +#endif // SOLUTION_STRENGTHENING /// Default constructor for the PendulumWrapper PendulumWrapper(); From cb0dd30300882a6f8f1f777a62d72d33fd53e81d Mon Sep 17 00:00:00 2001 From: kdesnos Date: Fri, 19 Dec 2025 10:51:32 +0100 Subject: [PATCH 21/32] (RelEng) Update prepare template to support nested ifdef. --- params.json | 4 ++-- scripts/prepare_template.py | 45 ++++++++++++++++++++----------------- 2 files changed, 26 insertions(+), 23 deletions(-) diff --git a/params.json b/params.json index b734bf9..7dda43c 100644 --- a/params.json +++ b/params.json @@ -98,11 +98,11 @@ "nbIterationsPerJob": 1, // Number of evaluation of each root per generation. // "nbIterationsPerPolicyEvaluation" : 5, // Default value - //#ifdef SOLUTION_PARALLEL + //#ifdef SOLUTION_STRENGTHENING "nbIterationsPerPolicyEvaluation": 5, //#else "nbIterationsPerPolicyEvaluation": 1, - //#endif // SOLUTION_PARALLEL + //#endif // SOLUTION_STRENGTHENING // Number of Constant available in each Program. // "nbProgramConstant" : 0, // Default value "nbProgramConstant": 0, diff --git a/scripts/prepare_template.py b/scripts/prepare_template.py index 592d92b..0f0c8b5 100644 --- a/scripts/prepare_template.py +++ b/scripts/prepare_template.py @@ -19,38 +19,41 @@ def filterSolution(inputFile, outputFile, keepSolution, patterns): tempOutput = io.StringIO() currentInput.seek(0) - # Scan lines - isSolution = False - isTemplate = False + # Stack to track active blocks + blockStack = [] + inElseBlock = False # Flag to track if we are in an `#else` block emptyLine = False + for line in currentInput: # Check if the line is the #define if re.match(rf'.*#define SOLUTION.*\n', line): continue # skip the line # Check if the line starts a solution block - if re.match(rf'.*#ifdef {pattern}\s*\n', line): - isSolution = True + matchIfdef = re.match(rf'.*#ifdef ({pattern})\s*\n', line) + if matchIfdef: + blockStack.append(matchIfdef.group(1)) # Push the matched pattern onto the stack + inElseBlock = False # Reset the `else` flag continue # skip the line # Check if the line starts a template block - if isSolution and re.match(r'.*#else.*\n', line): - isSolution = False - isTemplate = True + if blockStack and re.match(r'.*#else.*\n', line): + inElseBlock = True # Mark that we are in the `else` section continue # skip the line # Check if the line ends a block - if (isSolution or isTemplate) and re.match(rf'.*#endif // {pattern}\s*\n', line): - isSolution = False - isTemplate = False + matchEndif = re.match(rf'.*#endif // ({pattern})\s*\n', line) + if blockStack and matchEndif and blockStack[-1] == matchEndif.group(1): + blockStack.pop() # Pop the matched block + inElseBlock = False # Reset the `else` flag continue # skip the line printLine = False if keepSolution: - if isSolution or (not isSolution and not isTemplate): + if not blockStack or not inElseBlock: printLine = True else: - if isTemplate or (not isSolution and not isTemplate): + if not blockStack or inElseBlock: printLine = True if printLine: @@ -77,26 +80,26 @@ def filterSolution(inputFile, outputFile, keepSolution, patterns): # Each entry: [inputPath, [(outputPath, patternsToRemove, patternsToKeep), ...]] files = [ ["./src/training/pendulum_wrapper.cpp", [ - ["./src/training/pendulum_wrapper_empty.cpp", ["SOLUTION","SOLUTION_PARALLEL"], []], - ["./src/training/pendulum_wrapper_solution.cpp", ["SOLUTION_PARALLEL"], ["SOLUTION"]], - ["./src/training/pendulum_wrapper_parallel.cpp", [], ["SOLUTION_PARALLEL", "SOLUTION"]], + ["./src/training/pendulum_wrapper_empty.cpp", ["SOLUTION","SOLUTION_.*"], []], + ["./src/training/pendulum_wrapper_solution.cpp", ["SOLUTION_.*"], ["SOLUTION"]], + ["./src/training/pendulum_wrapper_parallel.cpp", [], ["SOLUTION_(PARALLEL|STRENGTHENING)", "SOLUTION"]], ]], ["./src/training/pendulum_wrapper.h", [ ["./src/training/pendulum_wrapper_empty.h", ["SOLUTION.*"], []], - ["./src/training/pendulum_wrapper_solution.h", ["SOLUTION_PARALLEL"], ["SOLUTION.*"]], - ["./src/training/pendulum_wrapper_parallel.h", [], ["SOLUTION.*"]], + ["./src/training/pendulum_wrapper_solution.h", ["SOLUTION_.*"], ["SOLUTION.*"]], + ["./src/training/pendulum_wrapper_parallel.h", [], ["SOLUTION_(PARALLEL|STRENGTHENING)", "SOLUTION"]], ]], ["./CMakeLists.txt", [ ["./CMakeLists_empty.txt", ["SOLUTION.*"], []], - ["./CMakeLists_solution.txt", [], ["SOLUTION.*"]], + ["./CMakeLists_solution.txt", [], ["SOLUTION"]], ]], ["./src/training/main-training.cpp", [ ["./src/training/main-training_empty.cpp", ["SOLUTION.*"], []], - ["./src/training/main-training_parallel.cpp", [], ["SOLUTION.*"]], + ["./src/training/main-training_parallel.cpp", [], ["SOLUTION_PARALLEL"]], ]], ["./params.json", [ ["./params_empty.json", ["SOLUTION.*"], []], - ["./params_parallel.json", [], ["SOLUTION_PARALLEL"]], + ["./params_parallel.json", [], ["SOLUTION_(PARALLEL|STRENGTHENING)"]], ]], ] From 856da54fd92f139ab666cc976fbfcb9a959d11e4 Mon Sep 17 00:00:00 2001 From: kdesnos Date: Fri, 19 Dec 2025 10:58:37 +0100 Subject: [PATCH 22/32] (RelEng) Strengthen template prepa --- params.json | 4 ++-- scripts/prepare_template.py | 2 +- src/training/main-training.cpp | 2 +- src/training/pendulum_wrapper.cpp | 10 +++++----- 4 files changed, 9 insertions(+), 9 deletions(-) diff --git a/params.json b/params.json index 7dda43c..7846f51 100644 --- a/params.json +++ b/params.json @@ -10,7 +10,7 @@ // "doValidation" : false, // Default value // #ifdef SOLUTION_PARALLEL "doValidation": true, - // #else + // #else // SOLUTION_PARALLEL "doValidation": false, // #endif // SOLUTION_PARALLEL // Maximum number of actions performed on the learning environment during the @@ -100,7 +100,7 @@ // "nbIterationsPerPolicyEvaluation" : 5, // Default value //#ifdef SOLUTION_STRENGTHENING "nbIterationsPerPolicyEvaluation": 5, - //#else + //#else // SOLUTION_STRENGTHENING "nbIterationsPerPolicyEvaluation": 1, //#endif // SOLUTION_STRENGTHENING // Number of Constant available in each Program. diff --git a/scripts/prepare_template.py b/scripts/prepare_template.py index 0f0c8b5..015aca7 100644 --- a/scripts/prepare_template.py +++ b/scripts/prepare_template.py @@ -37,7 +37,7 @@ def filterSolution(inputFile, outputFile, keepSolution, patterns): continue # skip the line # Check if the line starts a template block - if blockStack and re.match(r'.*#else.*\n', line): + if blockStack and re.match(rf'.*#else // ({pattern})\s*\n', line): inElseBlock = True # Mark that we are in the `else` section continue # skip the line diff --git a/src/training/main-training.cpp b/src/training/main-training.cpp index 543ef6e..ab693a8 100644 --- a/src/training/main-training.cpp +++ b/src/training/main-training.cpp @@ -52,7 +52,7 @@ void train_main(std::atomic& exitProgram, std::atomic& doDisplay, st // Instantiate and initialize the Learning Agent (LA) #ifdef SOLUTION_PARALLEL Learn::ParallelLearningAgent la(pendulumLE, instructionSet, params); - #else + #else // SOLUTION_PARALLEL Learn::LearningAgent la(pendulumLE, instructionSet, params); #endif // SOLUTION_PARALLEL la.init(); diff --git a/src/training/pendulum_wrapper.cpp b/src/training/pendulum_wrapper.cpp index 8850711..52c1693 100644 --- a/src/training/pendulum_wrapper.cpp +++ b/src/training/pendulum_wrapper.cpp @@ -2,7 +2,7 @@ #ifdef SOLUTION const std::vector PendulumWrapper::actions{ -1.0, -0.66, -0.33, 0.0, 0.33, 0.66, 1.0 }; -#else +#else // SOLUTION const std::vector PendulumWrapper::actions{ 0.0 }; #endif // SOLUTION @@ -12,7 +12,7 @@ PendulumWrapper::PendulumWrapper() : LearningEnvironment(actions.size()), pendul data.at(0).setPointer(&this->pendulum.getAngle()); data.at(1).setPointer(&this->pendulum.getVelocity()); } -#else +#else // SOLUTION PendulumWrapper::PendulumWrapper() : LearningEnvironment(actions.size()) { } @@ -40,7 +40,7 @@ std::vector> PendulumWrapper::ge result.push_back(this->data.at(0)); result.push_back(this->data.at(1)); return result; -#else +#else // SOLUTION return std::vector>(); #endif // SOLUTION } @@ -64,7 +64,7 @@ void PendulumWrapper::reset(size_t seed, Learn::LearningMode mode, uint16_t iter // Randomize the initial velocity between [-1.0, 1.0] double initialVelocity = this->rng.getDouble(-1.0, 1.0); this->pendulum.setVelocity(initialVelocity); -#else +#else // SOLUTION_STRENGTHENING #ifdef SOLUTION this->pendulum.setAngle(M_PI); this->pendulum.setVelocity(0.0); @@ -102,7 +102,7 @@ double PendulumWrapper::getScore(void) const { #ifdef SOLUTION return accumulatedReward; -#else +#else // SOLUTION return 0.0; #endif // SOLUTION } From 8baff4ca99ea068f81340d809ccb49473538c906 Mon Sep 17 00:00:00 2001 From: kdesnos Date: Fri, 19 Dec 2025 11:11:54 +0100 Subject: [PATCH 23/32] (RelEng) Test Strenghtening tutorial archive. --- .github/workflows/test-tuto.yml | 2 ++ CMakeLists.txt | 4 ++-- params.json | 6 +++--- scripts/prepare_archives.py | 7 +++++++ scripts/prepare_template.py | 5 ++++- 5 files changed, 18 insertions(+), 6 deletions(-) diff --git a/.github/workflows/test-tuto.yml b/.github/workflows/test-tuto.yml index 6b5fb99..6c453f8 100644 --- a/.github/workflows/test-tuto.yml +++ b/.github/workflows/test-tuto.yml @@ -26,6 +26,7 @@ jobs: path: | docs/data/gegelati-tutorial.zip docs/data/gegelati-tutorial-solution.zip + docs/data/gegelati-tutorial-strengthening-solution.zip docs/data/gegelati-tutorial-parallel-solution.zip test_archives: @@ -37,6 +38,7 @@ jobs: archive: - gegelati-tutorial.zip - gegelati-tutorial-solution.zip + - gegelati-tutorial-strengthening-solution.zip - gegelati-tutorial-parallel-solution.zip os: [ubuntu-latest] ##, windows-latest, macos-latest] compiler: [gcc] ##, clang, msvc] diff --git a/CMakeLists.txt b/CMakeLists.txt index aca2ee2..53c9ec7 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -97,7 +97,7 @@ add_executable(tpg-training ${pendulum_files} ${training_files}) target_link_libraries(tpg-training ${GEGELATI_LIBRARIES} ${SDL2_LIBRARY} ${SDL2_IMAGE_LIBRARY} ${SDL2TTF_LIBRARY}) target_compile_definitions(tpg-training PRIVATE ROOT_DIR="${CMAKE_SOURCE_DIR}") -#ifdef SOLUTION +#ifdef SOLUTION_INFERENCE # Sub project for inference file(GLOB inference_files @@ -111,4 +111,4 @@ include_directories(${GEGELATI_INCLUDE_DIRS} ${SDL2_INCLUDE_DIR} ${SDL2_IMAGE_IN add_executable(tpg-inference ${pendulum_files} ${inference_files}) target_link_libraries(tpg-inference ${GEGELATI_LIBRARIES} ${SDL2_LIBRARY} ${SDL2_IMAGE_LIBRARY} ${SDL2TTF_LIBRARY}) target_compile_definitions(tpg-inference PRIVATE ROOT_DIR="${CMAKE_SOURCE_DIR}") -#endif // SOLUTION \ No newline at end of file +#endif // SOLUTION_INFERENCE \ No newline at end of file diff --git a/params.json b/params.json index 7846f51..5fcae3a 100644 --- a/params.json +++ b/params.json @@ -8,11 +8,11 @@ // Boolean used to activate an evaluation of the surviving roots in validation // mode after the training at each generation. // "doValidation" : false, // Default value - // #ifdef SOLUTION_PARALLEL + // #ifdef SOLUTION_STRENGTHENING "doValidation": true, - // #else // SOLUTION_PARALLEL + // #else // SOLUTION_STRENGTHENING "doValidation": false, - // #endif // SOLUTION_PARALLEL + // #endif // SOLUTION_STRENGTHENING // Maximum number of actions performed on the learning environment during the // each evaluation of a root. // "maxNbActionsPerEval" : 1000, // Default value diff --git a/scripts/prepare_archives.py b/scripts/prepare_archives.py index d1120ab..046b57e 100644 --- a/scripts/prepare_archives.py +++ b/scripts/prepare_archives.py @@ -87,6 +87,13 @@ def replace_file_in_zip(zip_path, file_to_add, arcname): replace_file_in_zip("./docs/data/gegelati-tutorial-solution.zip", "src/training/pendulum_wrapper_solution.cpp", mainFolder + "src/training/pendulum_wrapper.cpp") replace_file_in_zip("./docs/data/gegelati-tutorial-solution.zip", "src/training/pendulum_wrapper_solution.h", mainFolder + "src/training/pendulum_wrapper.h") +# Create the gegelati-tutorial-strengthening-solution archive by copying the solution archive +mainFolder = "gegelati-tutorial/" +shutil.copy2("./docs/data/gegelati-tutorial-solution.zip", "./docs/data/gegelati-tutorial-strengthening-solution.zip") +replace_file_in_zip("./docs/data/gegelati-tutorial-strengthening-solution.zip", "src/training/pendulum_wrapper_strengthening.cpp", mainFolder + "src/training/pendulum_wrapper.cpp") +replace_file_in_zip("./docs/data/gegelati-tutorial-strengthening-solution.zip", "src/training/pendulum_wrapper_strengthening.h", mainFolder + "src/training/pendulum_wrapper.h") +replace_file_in_zip("./docs/data/gegelati-tutorial-strengthening-solution.zip", "params_strengthening.json", mainFolder + "params.json") + # Create the gegelati-tutorial-parallel-solution archive by copying the solution archive mainFolder = "gegelati-tutorial/" shutil.copy2("./docs/data/gegelati-tutorial-solution.zip", "./docs/data/gegelati-tutorial-parallel-solution.zip") diff --git a/scripts/prepare_template.py b/scripts/prepare_template.py index 015aca7..4454f5d 100644 --- a/scripts/prepare_template.py +++ b/scripts/prepare_template.py @@ -82,16 +82,18 @@ def filterSolution(inputFile, outputFile, keepSolution, patterns): ["./src/training/pendulum_wrapper.cpp", [ ["./src/training/pendulum_wrapper_empty.cpp", ["SOLUTION","SOLUTION_.*"], []], ["./src/training/pendulum_wrapper_solution.cpp", ["SOLUTION_.*"], ["SOLUTION"]], + ["./src/training/pendulum_wrapper_strengthening.cpp", ["SOLUTION_PARALLEL"], ["SOLUTION(_STRENGTHENING)*"]], ["./src/training/pendulum_wrapper_parallel.cpp", [], ["SOLUTION_(PARALLEL|STRENGTHENING)", "SOLUTION"]], ]], ["./src/training/pendulum_wrapper.h", [ ["./src/training/pendulum_wrapper_empty.h", ["SOLUTION.*"], []], ["./src/training/pendulum_wrapper_solution.h", ["SOLUTION_.*"], ["SOLUTION.*"]], + ["./src/training/pendulum_wrapper_strengthening.h", ["SOLUTION_PARALLEL"], ["SOLUTION(_STRENGTHENING)*"]], ["./src/training/pendulum_wrapper_parallel.h", [], ["SOLUTION_(PARALLEL|STRENGTHENING)", "SOLUTION"]], ]], ["./CMakeLists.txt", [ ["./CMakeLists_empty.txt", ["SOLUTION.*"], []], - ["./CMakeLists_solution.txt", [], ["SOLUTION"]], + ["./CMakeLists_inference.txt", [], ["SOLUTION_INFERENCE"]], ]], ["./src/training/main-training.cpp", [ ["./src/training/main-training_empty.cpp", ["SOLUTION.*"], []], @@ -99,6 +101,7 @@ def filterSolution(inputFile, outputFile, keepSolution, patterns): ]], ["./params.json", [ ["./params_empty.json", ["SOLUTION.*"], []], + ["./params_strengthening.json", ["SOLUTION_PARALLEL"], ["SOLUTION_STRENGTHENING"]], ["./params_parallel.json", [], ["SOLUTION_(PARALLEL|STRENGTHENING)"]], ]], ] From d87f43d91c4087607ba80dfd6a916b8966855b9b Mon Sep 17 00:00:00 2001 From: kdesnos Date: Fri, 19 Dec 2025 11:19:48 +0100 Subject: [PATCH 24/32] (RelEng) Parallel archive now based on strengthening. --- scripts/prepare_archives.py | 3 +-- scripts/prepare_template.py | 1 - 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/scripts/prepare_archives.py b/scripts/prepare_archives.py index 046b57e..b129576 100644 --- a/scripts/prepare_archives.py +++ b/scripts/prepare_archives.py @@ -96,11 +96,10 @@ def replace_file_in_zip(zip_path, file_to_add, arcname): # Create the gegelati-tutorial-parallel-solution archive by copying the solution archive mainFolder = "gegelati-tutorial/" -shutil.copy2("./docs/data/gegelati-tutorial-solution.zip", "./docs/data/gegelati-tutorial-parallel-solution.zip") +shutil.copy2("./docs/data/gegelati-tutorial-strengthening-solution.zip", "./docs/data/gegelati-tutorial-parallel-solution.zip") replace_file_in_zip("./docs/data/gegelati-tutorial-parallel-solution.zip", "src/training/pendulum_wrapper_parallel.cpp", mainFolder + "src/training/pendulum_wrapper.cpp") replace_file_in_zip("./docs/data/gegelati-tutorial-parallel-solution.zip", "src/training/pendulum_wrapper_parallel.h", mainFolder + "src/training/pendulum_wrapper.h") replace_file_in_zip("./docs/data/gegelati-tutorial-parallel-solution.zip", "src/training/main-training_parallel.cpp", mainFolder + "src/training/main-training.cpp") -replace_file_in_zip("./docs/data/gegelati-tutorial-parallel-solution.zip", "params_parallel.json", mainFolder + "params.json") # Make the main-inference.cpp file available for download shutil.copy2("./src/inference/main-inference.cpp", "./docs/data/") diff --git a/scripts/prepare_template.py b/scripts/prepare_template.py index 4454f5d..d13fe88 100644 --- a/scripts/prepare_template.py +++ b/scripts/prepare_template.py @@ -102,7 +102,6 @@ def filterSolution(inputFile, outputFile, keepSolution, patterns): ["./params.json", [ ["./params_empty.json", ["SOLUTION.*"], []], ["./params_strengthening.json", ["SOLUTION_PARALLEL"], ["SOLUTION_STRENGTHENING"]], - ["./params_parallel.json", [], ["SOLUTION_(PARALLEL|STRENGTHENING)"]], ]], ] From da36fa05e8e19a5e9d0419163dc98cf4e08debbb Mon Sep 17 00:00:00 2001 From: kdesnos Date: Sat, 20 Dec 2025 13:38:34 +0100 Subject: [PATCH 25/32] (RelEng) Better filtrate else block in templates --- scripts/prepare_template.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/prepare_template.py b/scripts/prepare_template.py index d13fe88..cf232e0 100644 --- a/scripts/prepare_template.py +++ b/scripts/prepare_template.py @@ -31,7 +31,7 @@ def filterSolution(inputFile, outputFile, keepSolution, patterns): # Check if the line starts a solution block matchIfdef = re.match(rf'.*#ifdef ({pattern})\s*\n', line) - if matchIfdef: + if matchIfdef and not inElseBlock: blockStack.append(matchIfdef.group(1)) # Push the matched pattern onto the stack inElseBlock = False # Reset the `else` flag continue # skip the line From a2de253bab6f07f531b57c059b3794ba735e228f Mon Sep 17 00:00:00 2001 From: kdesnos Date: Sat, 20 Dec 2025 14:35:24 +0100 Subject: [PATCH 26/32] (Tuto) Keep only relevant params. --- params.json | 93 +++-------------------------------------------------- 1 file changed, 4 insertions(+), 89 deletions(-) diff --git a/params.json b/params.json index 5fcae3a..7fd7e34 100644 --- a/params.json +++ b/params.json @@ -1,101 +1,31 @@ { - // Number of recordings held in the Archive. - // "archiveSize" : 50, // Default value - "archiveSize": 2000, - // Probability of archiving the result of each Program execution. - // "archivingProbability" : 0.05, // Default value - "archivingProbability": 0.01, + // #ifdef SOLUTION_STRENGTHENING // Boolean used to activate an evaluation of the surviving roots in validation // mode after the training at each generation. // "doValidation" : false, // Default value - // #ifdef SOLUTION_STRENGTHENING "doValidation": true, - // #else // SOLUTION_STRENGTHENING - "doValidation": false, // #endif // SOLUTION_STRENGTHENING // Maximum number of actions performed on the learning environment during the // each evaluation of a root. // "maxNbActionsPerEval" : 1000, // Default value "maxNbActionsPerEval": 1500, + // #ifdef SOLUTION_STRENGTHENING // Maximum number of times a given root is evaluated.After this number is // reached, possibly after several generations, the score of the root will be // fixed, and no further evaluation will be done. // "maxNbEvaluationPerPolicy" : 1000, // Default value "maxNbEvaluationPerPolicy": 10, + // #endif // SOLUTION_STRENGTHENING "mutation": { - "prog": { - // Maximum constant value possible. - // "maxConstValue" : 100, // Default value - "maxConstValue": 10, - // Maximum number of Line within the Program of the TPG. - // "maxProgramSize" : 96, // Default value - "maxProgramSize": 20, - // Minimum constant value possible. - // "minConstValue" : -10, // Default value - "minConstValue": -10, - // Probability of inserting a line in the Program. - // "pAdd" : 0.5, // Default value - "pAdd": 0.5, - // Probability of each constant to be mutated. - // "pConstantMutation" : 0.5, // Default value - "pConstantMutation": 0.5, - // Probability of deleting a line of the Program. - // "pDelete" : 0.5, // Default value - "pDelete": 0.5, - // Probability of altering a line of the Program. - // "pMutate" : 1.0, // Default value - "pMutate": 1.0, - // Probability of creating a new program. - // "pNewProgram" : 0.0, // Default value - "pNewProgram": 0.0, - // Probability of swapping two lines of the Program. - // "pSwap" : 1.0, // Default value - "pSwap": 1.0 - }, "tpg": { - // When a Program is mutated, makes sure its behavior is no longer the same. - // "forceProgramBehaviorChangeOnMutation" : false, // Default value - "forceProgramBehaviorChangeOnMutation": false, - // Number of root TPGTeams at the initialisation of a TPGGraph. - // If 0, if will be init to the number of surviving roots - // "nbRoots" : 0, // Default value - "initNbRoots": 0, - // Maximum number of TPGEdge connected to each TPGTeam of the TPGGraph when - // initialized. - // "maxInitOutgoingEdges" : 3, // Default value - "maxInitOutgoingEdges": 3, - // Maximum number of outgoing edge during TPGGraph mutations. - // "maxOutgoingEdges" : 5, // Default value - "maxOutgoingEdges": 5, // Number of root TPGTeams to maintain when populating the TPGGraph // "nbRoots" : 100, // Default value - "nbRoots": 150, - // Probability of adding an outgoing Edge to a Team. - // "pEdgeAddition" : 0.7, // Default value - "pEdgeAddition": 0.7, - // Probability of deleting an outgoing Edge of a Team. - // "pEdgeDeletion" : 0.7, // Default value - "pEdgeDeletion": 0.7, - // Probability of changing the destination of an Edge. - // "pEdgeDestinationChange" : 0.1, // Default value - "pEdgeDestinationChange": 0.1, - // Probability of the new destination of an Edge to be an Action. - // "pEdgeDestinationIsAction" : 0.5, // Default value - "pEdgeDestinationIsAction": 0.5, - // Probability of mutating the Program of an outgoing Edge. - // "pProgramMutation" : 0.2, // Default value - "pProgramMutation": 0.2 + "nbRoots": 150 } }, // Number of generations of the training. // "nbGenerations" : 500, // Default value "nbGenerations": 1200, - // [Only used in AdversarialLearningAgent.] - // Number of times each job is evaluated in the learning process. - // Each root may belong to several jobs, hence this parameter should be lower - // than the nbIterationsPerPolicyEvaluation parameter. - // "nbIterationsPerJob" : 1, // Default value - "nbIterationsPerJob": 1, // Number of evaluation of each root per generation. // "nbIterationsPerPolicyEvaluation" : 5, // Default value //#ifdef SOLUTION_STRENGTHENING @@ -103,19 +33,4 @@ //#else // SOLUTION_STRENGTHENING "nbIterationsPerPolicyEvaluation": 1, //#endif // SOLUTION_STRENGTHENING - // Number of Constant available in each Program. - // "nbProgramConstant" : 0, // Default value - "nbProgramConstant": 0, - // Number of registers for the Program execution. - // "nbRegisters" : 8, // Default value - "nbRegisters": 8, - // [Only used in ParallelLearningAgent and child classes.] - // Number of threads used for the training process. - // When undefined in the json file, this parameter is automatically set to the - // number of cores of the CPU. - // /* "nbThreads" : 0,*/ // Commented by default - /* "nbThreads" : 0,*/ - // Percentage of deleted (and regenerated) root TPGVertex at each generation. - // "ratioDeletedRoots" : 0.5, // Default value - "ratioDeletedRoots": 0.85 } \ No newline at end of file From 38c60bd1a7b7b408d6daa15dd9704aef1b5c2884 Mon Sep 17 00:00:00 2001 From: kdesnos Date: Sat, 20 Dec 2025 14:37:07 +0100 Subject: [PATCH 27/32] (Tuto) Make it possible to sync replay with training, or vice-versa. --- src/renderer.cpp | 59 +++++++++++++++++++++++++++++++++++++++++++----- src/renderer.h | 4 +++- 2 files changed, 56 insertions(+), 7 deletions(-) diff --git a/src/renderer.cpp b/src/renderer.cpp index 63a9435..ca3a68d 100644 --- a/src/renderer.cpp +++ b/src/renderer.cpp @@ -1,4 +1,3 @@ - #include #include #include @@ -123,7 +122,7 @@ void Renderer::displayText(const char* text, int posX, int posY) { } -int Renderer::renderEnv(double state, double torque, uint64_t frame, uint64_t generation, double timeDelta) { +int Renderer::renderEnv(double state, double torque, uint64_t frame, uint64_t generation, double timeDelta, bool syncReset) { // Select the color for drawing. It is set to red here. SDL_SetRenderDrawColor(display.renderer, 255, 255, 255, 255); // Clear the entire screen to our selected color. @@ -161,6 +160,16 @@ int Renderer::renderEnv(double state, double torque, uint64_t frame, uint64_t ge sprintf(frameNumber, "frame: %4" PRId64, frame); Renderer::displayText(frameNumber, 0, 22); + // Print Sync status + char syncString[30]; + if (syncReset) { + sprintf(syncString, "[W] Training waits on display."); + } + else { + sprintf(syncString, "[W] Display resets on training"); + } + Renderer::displayText(syncString, DISPLAY_W - 320, 0); + // Proceed to the actual display SDL_RenderPresent(display.renderer); @@ -176,6 +185,8 @@ int Renderer::renderEnv(double state, double torque, uint64_t frame, uint64_t ge // This is needed because repeated action are not grabbed at every frame // even when the key remains pressed. static int action = 0; + // Flag to ensure 'w' is only sent once per physical press + static bool w_consumed = false; SDL_Event event; // Grab all next events off the queue. @@ -205,12 +216,24 @@ int Renderer::renderEnv(double state, double torque, uint64_t frame, uint64_t ge case SDLK_l: action = 3; break; + case SDLK_w: + // only on initial keydown (no OS repeat) + if (event.key.repeat == 0) { + action = 4; + // mark as not yet consumed for this press + w_consumed = false; + } + break; } break; case SDL_QUIT: action = INT_MIN; break; case SDL_KEYUP: + // Reset per-key consumption state when released + if (event.key.keysym.sym == SDLK_w) { + w_consumed = false; + } action = 0; break; default: @@ -222,11 +245,31 @@ int Renderer::renderEnv(double state, double torque, uint64_t frame, uint64_t ge } } - return action; + // Ensure 'w' action is only returned once per physical press. + int ret = action; + if (ret == 4) { + if (!w_consumed) { + // first time we return 4 for this press: mark consumed and clear persistent action + w_consumed = true; + action = 0; + } + else { + // already consumed: don't repeat + ret = 0; + } + } + + return ret; } void Renderer::replayThread(std::atomic& exit, std::atomic& doDisplay, std::atomic& generation, double& delta, std::deque>& replay) { + std::cout << "Pendulum training." << std::endl; + std::cout << "By default, the replay will be reset when a new generation is over." << std::endl << + "\t [W]: Toggle stalling the TPG training until the replay of previous generation is over." << std::endl << + "\t [Q]: Exit the simulator." << std::endl; + std::cout << std::endl << "Press [Enter] to start the training."; + // Init Display renderInit(); @@ -235,12 +278,13 @@ void Renderer::replayThread(std::atomic& exit, std::atomic& doDispla double angleDisplay = M_PI; double torqueDisplay = 0.0; uint64_t frame = 0; + bool waitForReplayEnd = false; std::deque> localReplay; while (!exit) { // Was a replay requested? - if (doDisplay) { + if (doDisplay && (!waitForReplayEnd || localReplay.empty())) { // copy the replay localReplay = replay; doDisplay = false; @@ -253,12 +297,15 @@ void Renderer::replayThread(std::atomic& exit, std::atomic& doDispla localReplay.pop_front(); } - int event = Renderer::renderEnv(angleDisplay, torqueDisplay, frame, generation, delta); + int event = Renderer::renderEnv(angleDisplay, torqueDisplay, frame, generation, delta, waitForReplayEnd); switch (event) { case INT_MIN: exit = true; doDisplay = false; break; + case 4: + waitForReplayEnd = !waitForReplayEnd; + break; case 0: default: // Nothing to do @@ -277,4 +324,4 @@ void Renderer::renderFinalize() SDL_DestroyTexture(display.textureArrow); SDL_DestroyRenderer(display.renderer); SDL_DestroyWindow(display.screen); -} +} \ No newline at end of file diff --git a/src/renderer.h b/src/renderer.h index 52c7926..0f4be42 100644 --- a/src/renderer.h +++ b/src/renderer.h @@ -49,13 +49,15 @@ namespace Renderer { * \param[in] torque the torque currently applied to the pendulum. * \param[in] frame the frame number to display in the top left corner. * \param[in] generation the generation number to display in the top left corner. + * \param[in] syncReset Flag to indicate whether the training is synced + * with the replay. * \return An int value is returned to the controller loop depending * on the action made by the user: * - [-3, 3]: 6 actions available to apply a torque to the pendulum. * - INT_MIN: Exit request. * \param[in] timeDelta time in second between two frames. */ - int renderEnv(double state, double torque, uint64_t frame, uint64_t generation, double timeDelta); + int renderEnv(double state, double torque, uint64_t frame, uint64_t generation, double timeDelta, bool syncReset); /** * \brief Separate control loop for displaying replays in parallel to training. From 7dc94b60134bf52f4b3985abc72cd1331f751cad Mon Sep 17 00:00:00 2001 From: kdesnos Date: Sat, 20 Dec 2025 14:46:01 +0100 Subject: [PATCH 28/32] (Tuto) Fix string size. --- src/renderer.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/renderer.cpp b/src/renderer.cpp index ca3a68d..693d476 100644 --- a/src/renderer.cpp +++ b/src/renderer.cpp @@ -161,12 +161,12 @@ int Renderer::renderEnv(double state, double torque, uint64_t frame, uint64_t ge Renderer::displayText(frameNumber, 0, 22); // Print Sync status - char syncString[30]; + char syncString[32]; if (syncReset) { sprintf(syncString, "[W] Training waits on display."); } else { - sprintf(syncString, "[W] Display resets on training"); + sprintf(syncString, "[W] Display resets on training."); } Renderer::displayText(syncString, DISPLAY_W - 320, 0); From 5918b475a0f0639c0a18eaebd3ef6788219dfdac Mon Sep 17 00:00:00 2001 From: kdesnos Date: Sat, 20 Dec 2025 14:49:27 +0100 Subject: [PATCH 29/32] (RelEng) trigger msvc build --- .github/workflows/test-tuto.yml | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/.github/workflows/test-tuto.yml b/.github/workflows/test-tuto.yml index 6c453f8..a8139fa 100644 --- a/.github/workflows/test-tuto.yml +++ b/.github/workflows/test-tuto.yml @@ -40,17 +40,17 @@ jobs: - gegelati-tutorial-solution.zip - gegelati-tutorial-strengthening-solution.zip - gegelati-tutorial-parallel-solution.zip - os: [ubuntu-latest] ##, windows-latest, macos-latest] - compiler: [gcc] ##, clang, msvc] - ## exclude: + os: [ubuntu-latest, windows-latest] ##, windows-latest, macos-latest] + compiler: [gcc, msvc] ##, clang, msvc] + exclude: # Exclude MSVC on non-Windows - ## - os: ubuntu-latest - ## compiler: msvc + - os: ubuntu-latest + compiler: msvc ## - os: macos-latest - ## compiler: msvc + ## compiler: msvc # Exclude GCC and Clang on Windows (unless you want to setup MinGW/LLVM) - ## - os: windows-latest - ## compiler: gcc + - os: windows-latest + compiler: gcc ## - os: windows-latest ## compiler: clang From 7dc94d1765677221346cbc4c9356c025e309496c Mon Sep 17 00:00:00 2001 From: kdesnos Date: Sat, 20 Dec 2025 14:52:04 +0100 Subject: [PATCH 30/32] (Tuto) Fix manual control display. --- src/manual/main-manual.cpp | 2 +- src/renderer.cpp | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/manual/main-manual.cpp b/src/manual/main-manual.cpp index 3301f52..0db2ddd 100644 --- a/src/manual/main-manual.cpp +++ b/src/manual/main-manual.cpp @@ -37,7 +37,7 @@ int main(int argc, char** argv) { while(!exit) { frame++; - int action = Renderer::renderEnv(p.getAngle(), torque, frame, 0, p.TIME_DELTA); + int action = Renderer::renderEnv(p.getAngle(), torque, frame, 0, p.TIME_DELTA, false); exit = (action == INT_MIN); if (exit) { diff --git a/src/renderer.cpp b/src/renderer.cpp index 693d476..cb006ff 100644 --- a/src/renderer.cpp +++ b/src/renderer.cpp @@ -166,7 +166,7 @@ int Renderer::renderEnv(double state, double torque, uint64_t frame, uint64_t ge sprintf(syncString, "[W] Training waits on display."); } else { - sprintf(syncString, "[W] Display resets on training."); + sprintf(syncString, ""); } Renderer::displayText(syncString, DISPLAY_W - 320, 0); From caaf2a32aa61647a212a561ffb0337fa519e8e6a Mon Sep 17 00:00:00 2001 From: kdesnos Date: Sat, 20 Dec 2025 14:55:20 +0100 Subject: [PATCH 31/32] (RelEng) Fix parallel build with cmake --- .github/workflows/test-tuto.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/test-tuto.yml b/.github/workflows/test-tuto.yml index a8139fa..12866f1 100644 --- a/.github/workflows/test-tuto.yml +++ b/.github/workflows/test-tuto.yml @@ -111,13 +111,13 @@ jobs: - name: Build manual-control target run: | - cmake --build build --config Release --target manual-control -- -j$(nproc) + cmake --build build --config Release --target manual-control --parallel $(nproc) shell: bash - name: Build and run tpg-training target if: matrix.archive != 'gegelati-tutorial.zip' run: | sed -i 's/"nbGenerations": [0-9]*/"nbGenerations": 4/' tutorial/gegelati-tutorial/params.json - cmake --build build --config Release --target tpg-training -- -j$(nproc) + cmake --build build --config Release --target tpg-training --parallel$(nproc) ./build/Release/tpg-training shell: bash \ No newline at end of file From 0f61f1a2760401dc282e03bc90c8f37d1d5da39a Mon Sep 17 00:00:00 2001 From: kdesnos Date: Sat, 20 Dec 2025 15:07:42 +0100 Subject: [PATCH 32/32] (Releng) Fix msvc run. --- .github/workflows/test-tuto.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/test-tuto.yml b/.github/workflows/test-tuto.yml index 12866f1..4cee1bc 100644 --- a/.github/workflows/test-tuto.yml +++ b/.github/workflows/test-tuto.yml @@ -118,6 +118,6 @@ jobs: if: matrix.archive != 'gegelati-tutorial.zip' run: | sed -i 's/"nbGenerations": [0-9]*/"nbGenerations": 4/' tutorial/gegelati-tutorial/params.json - cmake --build build --config Release --target tpg-training --parallel$(nproc) - ./build/Release/tpg-training + cmake --build build --config Release --target tpg-training --parallel $(nproc) + cd build && ./Release/tpg-training # cd needed for windows dll shell: bash \ No newline at end of file