[Dependency Cache] Enable Dependency Cache on CI #21584
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Release Notes: https://github.com/Automattic/ a8c-ci-toolkit-buildkite-plugin/releases/tag/3.9.1 FYI: This change points to that version of the 'A8C CI Toolkit' where both, the #135 and #138 PRs, are merged into 'trunk', and, the new dependency cache mechanism, per project, without 'GRADLE_RO_DEP_CACHE', is fully operational. A8C CI Toolkit #135 PR: [Dependency Cache] Dependency Cache on CI per Project [without GRADLE_RO_DEP_CACHE] #135 - Automattic/a8c-ci-toolkit-buildkite-plugin#135 A8C CI Toolkit #138 PR: [Dependency Cache] Fix Dependency Cache #138 - Automattic/a8c-ci-toolkit-buildkite-plugin#138
|
| App Name |  Jetpack |
|
| Flavor | Jalapeno | |
| Build Type | Debug | |
| Version | pr21584-dfbef04 | |
| Commit | dfbef04 | |
| Direct Download | jetpack-prototype-build-pr21584-dfbef04.apk |
|
| App Name |  WordPress |
|
| Flavor | Jalapeno | |
| Build Type | Debug | |
| Version | pr21584-dfbef04 | |
| Commit | dfbef04 | |
| Direct Download | wordpress-prototype-build-pr21584-dfbef04.apk |
FYI: This job will be then used by 'buildkite-ci' and configured as a 'buildkite_pipeline_schedule' with a weekly frequency. PS.1: The targeted 'pipeline' related jobs are: - Mobile App - Lint - Unit Tests - Android tests PS.2: Since the 'WordPress' app is being build from the very same codebase as the 'Jetpack' app, targeting only the 'Jetpack' related 'pipeline' jobs are enough to download and cache all the dependencies required for both apps.
Otherwise, using 'multi-cmd' will produce 'no such file or directory' type of errors.
FYI: The targeted 'pipeline' related jobs are: - Mobile App - Lint - Unit Tests - Android Tests (Commented Out) PS.1: The 'detekt' and 'checkstyle' jobs aren't targeted because they only take about 30+ seconds of network request to download all the dependencies needed for those specific jobs, which is about the same time it takes for the dependency cache to actually be restored (20+ seconds). Thus, this optimization is not really worth it for these specific jobs. For all those other jobs targeted, it take more than a minute of network requests to download all the dependencies needed, and as such worth the diff is worth it. PS.2: Although the 'Android Tests' are commented-out, I chose to include cache restoration into those as well, just in case they will be reactivated at some point in the future.
WCAndroid PR Comment: https://github.com/woocommerce/ woocommerce-android/pull/13303#discussion_r1916752265
When using 'single-cmd' and not using this double quotes structure it is is producing 'did not find expected key' type of errors.
… into ci/enable-dependency-cache-on-ci-final
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## trunk #21584 +/- ##
=======================================
Coverage 39.48% 39.48%
=======================================
Files 2117 2117
Lines 99464 99464
Branches 15296 15296
=======================================
Hits 39269 39269
Misses 56714 56714
Partials 3481 3481 ☔ View full report in Codecov by Sentry. |
… into ci/enable-dependency-cache-on-ci-final
|
This was referenced Jan 17, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Project Thread: paaHJt-7CE-p2
Related: #21550 + #21540
Depends On: a8c-ci-toolkit#135 -> a8c-ci-toolkit#138
Required By: buildkite-ci#578
Description
This PR enables dependency cache on CI for this project. 💾
The dependency cache mechanism is split between:
assembleJetpackJalapenoDebug)lintJetpackJalapenoDebug)assembleJetpackJalapenoDebugUnitTest assembleDebugUnitTest testClasses)assembleJetpackJalapenoDebugAndroidTest).shscripts:prototype-build.sh)prototype-build.sh)lint.sh)lint.sh)run-unit-tests.sh)run-unit-tests.sh)run-unit-tests.sh)run-unit-tests.sh)run-unit-tests.sh)run-instrumented-tests.sh)run-instrumented-tests.sh)PS.1: The
detektandcheckstylejob aren't targeted because it only takes about 30+ seconds of network request to download all the dependencies needed for those specific jobs (build + scan fordetekt), which is about the same time it takes for the dependency cache to actually be restored, which is more or less about 20+ seconds (build). Thus, this optimization is not really worth it for these specific jobs. For all those other jobs targeted, it takes more than a minute of network requests to download all the dependencies needed (build + scan), and as such the diff is worth it.PS.2: Although the
Android Testare commented-out, I chose to include cache restoration into those as well, just in case they will be reactivated at some point in the future.PS.3: Dependency cache isn't enabled on the release-builds.yml pipeline just yet, but just temporarily, and until we become comfortable with running this new process on the main pipeline.yml pipeline. Having said that, and due to the fact that the release-builds.yml pipeline is also utilizing the lint.sh for its Lint steps, due to the fact that the restore-cache.sh is within that
lint.shscript, restoring cache will partially work on the release-builds.yml pipeline as well.Future Improvements
Me and @wzieba has been discussing on DMs about some possible future improvement, which are including but are not limited to the below list:
assemble android testtask, which depends onassembletask anyway, and would most likely resolve most of the dependencies, for all other tasks to benefit from. However, this needs further testing to make sure that running other tasks such uslintandassemble unit testare providing little to no help.init.d). Using the init script, go through all available modules for a project, and then, using that custom Gradle task, resolve all dependencies for all tasks that are available per module. We could take some inspiration from plugins such as this.dependency cachebuild to run weekly (buildkite-ci#578), make it run more often, possibly scheduling it to even run daily. However, since dependencies are not being updated too often, and, even when they do on that specific week, it would only be for a few of them. This will indeed cause some extra network requests, but most of the dependencies cached would still benefit from the existing and weekly refreshed dependency cache.To Test:
This dependency cache solution was thoroughly tested via this #21550 test-only PR, but to help you asserting that this change is indeed making an impact, plus not causing any trouble for the project, try and follow the below:
Save Cache:dependency cachejob to complete: 14m 31s#android-core-notifs):The dependency cache has been updated.(p1736959083969609-slack-C0787KRDYDC)Failure to update the dependency cache.(not tested, but should behave similarly to ☝️)Restore Cache:- 1m 54s0s1m 13s- 2m 15s0s1m 13s- 1m 48s0s47s- 1m 44s0s59s- 1m 34s0s1m 14s- 1m 37s0s1m 4s+ 18s0s6s- 1m 4s0s29s- 1m 28s0s1m 18sNote
Without: Without Dependency CacheWith: With Dependency CacheBT: Build TimeNR: Network RequestsRC: Restore CacheWin: NR (Without) - RC (With)Warning
(*): The
BT (Diff)presented above is relative, so don't read too much into it, it might not match with theWIN, which is a much more solid number. But still, it is nice to have and compare those two number together. To to be clear about that, although I tried to pick the most representative builds, without and with the dependency cache mechanism in place, there are other factors that can make any build faster, or slower for that matter (likebuild cache,firebase test lab, etc), so again, don't read too much into it.(**): Both, the
JetpackandWordPress Android Testsare currently disabled and as such not included in the aboveRestore Cacherelated table.Caution
(❗️) The dependency cache restoration for
Image Editor Unit Testsjob took almost twice as expected 34s. This is quite interesting and I am not sure how to explain it. For a cache of more-or-less 1 GB in size, and as per all the other jobs suggest, it takes about20+seconds to download the cache entry from s3.Merge Instructions
Regression Notes
Potential unintended areas of impact
What I did to test those areas of impact (or what existing automated tests I relied on)
To Testsection above.What automated tests I added (or what prevented me from doing so)
PR Submission Checklist:
RELEASE-NOTES.txtif necessary.Testing Checklist (strike-out the not-applying and unnecessary ones):
N/A