ci: Use ccache to cache build objects for speeding up building #6104
+128
−38
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| find_program(CCACHE_PATH "ccache") | ||
| if (CCACHE_PATH) | ||
mathbunnyru marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| if (MSVC) | ||
| # Chocolatey uses a shim executable that we cannot use directly, in | ||
|
||
| # which case we have to find the executable it points to. If we cannot | ||
| # find the target executable then we cannot use ccache. | ||
| message(STATUS "Ccache path: ${CCACHE_PATH}") | ||
| if ("${CCACHE_PATH}" MATCHES "chocolatey") | ||
| find_program(BASH_PATH "bash") | ||
| if (NOT BASH_PATH) | ||
| message(WARNING "Could not find bash.") | ||
| return() | ||
| endif () | ||
| execute_process( | ||
| COMMAND bash -c "export LC_ALL='en_US.UTF-8'; ${CCACHE_PATH} --shimgen-noop | grep -oP 'path to executable: \\K.+' | head -c -1" | ||
| OUTPUT_VARIABLE CCACHE_PATH) | ||
| if (NOT CCACHE_PATH) | ||
| message(WARNING "Could not find ccache target.") | ||
| return() | ||
| endif () | ||
| file(TO_CMAKE_PATH "${CCACHE_PATH}" CCACHE_PATH) | ||
| endif () | ||
|
|
||
| # Tell cmake to use ccache for compiling with Visual Studio. | ||
| file(COPY_FILE | ||
| ${CCACHE_PATH} ${CMAKE_BINARY_DIR}/cl.exe | ||
| ONLY_IF_DIFFERENT) | ||
| set(CMAKE_VS_GLOBALS | ||
| "CLToolExe=cl.exe" | ||
| "CLToolPath=${CMAKE_BINARY_DIR}" | ||
| "TrackFileAccess=false" | ||
| "UseMultiToolTask=true") | ||
mathbunnyru marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| # By default Visual Studio generators will use /Zi, which is not | ||
| # compatible with ccache, so tell it to use /Z7 instead. When running in | ||
| # CI the /Zi option will have already been stripped out, in which case | ||
| # the following will be a no-op, see XrplCompiler.cmake. | ||
mathbunnyru marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| set(CMAKE_MSVC_DEBUG_INFORMATION_FORMAT "$<$<CONFIG:Debug,RelWithDebInfo>:Embedded>") | ||
| else () | ||
| # For Linux and macOS we can use the ccache binary directly. | ||
| set(CMAKE_C_COMPILER_LAUNCHER "${CCACHE_PATH}") | ||
| set(CMAKE_CXX_COMPILER_LAUNCHER "${CCACHE_PATH}") | ||
| endif () | ||
| message(STATUS "Using ccache: ${CCACHE_PATH}") | ||
| endif () | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You always download cache, add files to it (while building new code), and then upload a bigger cache.
This will eventually grow the cache size, and make it useless
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. I had hoped it would be more like Bazel/BuildBuddy where the least recently used files would be dropped automatically once a certain time/size limit was reached, but here the caches are considered as a whole unit and dropped entirely.
My original idea was to use ccache for caching across PRs, so unchanged files wouldn't need to be built again. It seems like that is not possible with the current set up. Should a separate cache be used per PR?
Per actions/cache#1071, one comment states "This functioning is expected and cache is built on the principle that "each key uniquely identifies a cache". if a cache contents have changed and you need to update it, its key should also reflect that.". So would this mean that each commit that modifies some source files needs its own cache? That entirely defeats the purpose...
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know if that would work well in rippled's case, because it depends on how active people are pushing their PRs.
I think if there are many active PRs and they override the same cache, it's rarely going to work.
Clio doesn't create caches on PRs, only on commits on develop.
This also solves the problem with fork/non-fork.
And we add a commit hash to the cache id.