Repository navigation
Reduce redundant native builds across parity CI jobs #871
Description
Delivered
PR875 is merged as05a68180c2dee8fb9604888ab9154a5cbe81f212. Normal trusted-main Shared40988042415 passes44/44 (all30 consumers and8 native-test cells), PHP/Sonar pass, and independent Security accepts the actual artifact identity/trust boundary. Completed acceptance, main measurements and limitations. Required tests, matrix cases and cache trust are preserved.
Observed whole workflow time is23m31s versus26m30s on the earlier same-native-tree main baseline; affected consumer+producer runner elapsed is2,323s lower. All matrix jobs start within a one-second interval in the repaired main run. Runner/load variance prevents a guaranteed speedup claim; the earlier slower candidate and cache-key-only timings remain retained. Product PR delivery resumes one qualified PR at a time; renewed delay still triggers CI-first diagnosis.
Before merged PR #872, native Cargo dependency caches hashed the entire .github/workflows/server-parity.yml file. Adding a PHP comparator test or changing a frozen reference therefore changes every native matrix cache key, even when the compiler and Rust dependencies are identical. Pull requests can restore only an exact trusted-main key, so these harness-only edits force cold dependency builds across all database cells.
Observed while qualifying #870: the shared run had33 jobs queued/six running at20:57UTC; its optimized codec build took10m42s and the matrix continued long enough to delay other required qualification. These are timing observations, not a measured cache-fix speedup.
General owns the CI repair: bind all native compiler invocations and cache keys to one immutable compiler-image setting; hash Cargo manifests/lock/vendor/configuration instead of unrelated workflow contents. Preserve every matrix cell/test/check, trusted-main-only cache writes, exact keys with no restore prefix, and removal of current workspace outputs before rebuilding and saving. No native runtime or frozen fixture changes.
Acceptance: lint and existing classifier/aggregate/cache/image-pull controls pass; unrelated workflow edits preserve dependency inputs, while compiler/lock/manifest edits invalidate them; source review confirms cache trust/rebuild boundaries remain; normal hosted qualification passes. Record actual main warm-cache hit/build timings and a later equivalent-input run before claiming performance improvement. Coordinate with the native owner and keep next product PRs local while CI is behind.
Earlier cache-key-only evidence: protected #872 is merged, trusted-main run40963572736 passes, and exact main-scoped v2 cache object9728416470 (269,870,492bytes) is verified. Seed receipt, measured restore/build comparison, compile-level refinement. Warm restoration reduced compiled units from214 to7, but observed seed/warm wall times did not improve. That correction alone has no established walltime speedup.
Activity
rmcdaniel commentedon Oct 29, 2026
Trusted-main v2 seed verified after PR #872 merged as c36ff6c. Push run https://github.com/durable-workflow/server/actions/runs/40963572736 job 123763893085 completed successfully at 22:27:21 UTC; both the owned-output cleanup and native Cargo cache save steps succeeded. The Actions cache API now returns main-scoped object 9728416470, created at 22:27:19 UTC, size 269,870,492 bytes, with the exact v2 key bound to compiler digest ba81bc3e... and dependency hash 223e7f2f.... This establishes the trusted seed only, not a performance improvement. Acceptance still requires a later equivalent-input exact restore hit and actual job timings before claiming a speedup; General owns that PR #870 follow-through.
rmcdaniel commentedon Oct 29, 2026
Full trusted-main qualification is now terminal green: push run https://github.com/durable-workflow/server/actions/runs/40963572736 completed 43/43 jobs, including the aggregate result, with zero failures. The previously recorded main-scoped v2 seed remains cache object 9728416470 from successful job 123763893085. The remaining acceptance evidence is the later equivalent-input exact restore hit plus actual warm timings; this run proves the seed and full normal qualification, not the speedup.
rmcdaniel commentedon Oct 29, 2026
PR #872 merged through the protected queue at
c36ff6ccdd3099c6ab3e4d2d574b94e94bcb272a. The normal trusted-main Shared Server fixtures run 40963572736 is terminal success, including the full matrix and aggregate.The sole cache writer, SQLite native-tests job 123763893085, successfully removed workspace outputs before saving. The cache API independently confirms main-scoped object
9728416470, created at 2026-10-29 22:27:19 UTC, size 269,870,492 bytes. Its exactnative-cargo-deps-v2key includes compiler image digestba81bc3eaa4422af576c0262515d96b0111a628a6ccc2c86557cf55c9a4bbee0and dependency-input hash223e7f2f5dc590142a52c1d74bd993741749b4d0c0fd799247b83328f4fd3e48.This proves successful trusted-main seeding. Performance improvement remains unclaimed. Already-green #870 was marked ready and entered the normal merge queue at 22:43:17 UTC, exact head
5eddc262dea97b3c99f5188ba9577eb15c692a48, behind #869. Its merge-group run will provide the later equivalent-input restore and actual build timings. All required checks, matrix cells, compiler/package rebuilds and cache trust boundaries remain enabled. Keep #871 open for that measured receipt and final task cleanup.rmcdaniel commentedon Oct 29, 2026
Later trusted-main warm-cache evidence is now terminal for the representative job.
After PR #869 merged as 8ace5c1, main run https://github.com/durable-workflow/server/actions/runs/40969012926 started with the same compiler digest and dependency-input hash as the seeded v2 key. Its execution-and-codec native-tests job https://github.com/durable-workflow/server/actions/runs/40969012926/job/123782116624 completed successfully from 22:55:04 to 23:04:24 UTC, 9 minutes 20 seconds total.
Measured steps:
The cache API independently shows the same main-scoped object 9728416470, size 269,870,492 bytes, with last_accessed_at advanced to 22:57:46 UTC. The key is still native-cargo-deps-v2 with compiler digest ba81bc3e... and dependency hash 223e7f2f....
Scope: this matrix part skips the separate Build the native server step, so this receipt does not assign a timing to that skipped step or claim a controlled overall speedup. It does establish the required later equivalent-input exact warm restore plus actual terminal job timings on trusted main. The earlier full 43/43 seed run remains the normal hosted qualification; PR #870 current-head checks and protected merge remain separate delivery work.
rmcdaniel commentedon Oct 29, 2026
The later trusted-main run 40969012926 has a terminal successful SQLite native-tests writer, job 123782116624. It restores the same main-scoped native v2 cache object/key and retains the workspace cleanup boundary. This confirms reuse after native source and unrelated workflow inputs changed, with the same compiler/dependency inputs.
The observed timing does not establish a speedup:
The warm job ran22:55:04–23:04:24 UTC; its restore step took4s and its official codec/native step ran22:56:16–23:04:17. The seed ran22:19:50–22:27:21, with its official step22:21:00–22:27:11. Native source and runners differ between these observations, so this is not a controlled throughput benchmark or proof that the cache itself made the run slower. Original job-log retrieval timed out; no Cargo-level compile/test breakdown is claimed from metadata alone.
The key-invalidation defect is corrected, but the measured CI pacing work remains open. Source inspection shows current workspace compilation is repeated across matrix jobs. General is coordinating the next bounded workflow/artifact slice with the native owner: build the exact current-head debug server and test executables once per run, then execute every existing database/profile case in its existing container. Reuse would stay within one workflow run, with immutable compiler/source/digest binding and no cross-run workspace cache or reduced coverage. This is a proposed next step, not implemented or qualified. No new product PR is being added while CI remains behind; #870 still follows its normal current-head hosted checks and merge queue.
rmcdaniel commentedon Oct 29, 2026
The full later main run is now terminal success, 43/43 jobs. Its completed logs refine the timing receipt above.
The exact warm hit materially changed what Cargo rebuilt: the warm job logged only 7 compile entries before the test targets, versus 214 in the seed cache-miss job. That proves dependency reuse rather than a nominal key hit.
It still did not improve observed wall time:
The runs differ in source and runner conditions, so these are not a controlled speed comparison. The correct conclusion is narrow: the v2 cache avoids recompiling most dependencies, but this evidence shows no wall-time improvement and supports no speedup claim. Repeated current-workspace build and test cost still dominates the matrix. Any follow-up should address that cost without restoring current workspace outputs across runs or removing matrix coverage.
[-]Keep native CI dependencies cached across parity-only workflow edits[/-][+]Reduce redundant native builds across parity CI jobs[/+]rmcdaniel commentedon Oct 30, 2026
PR #875 exact head 598ed24 is fully qualified on its normal PR run: Shared40978654861 succeeds 44/44 jobs, including all30 non-native consumers and all8 native-test cells. All30 consumers passed actual artifact verification/install. The complete PR has57 successful checks and1 conditional skip; native-owner and Security exact-head reviews accept. It entered the normal protected merge queue at00:50:56 UTC, position2 behind #870. Merge-group and trusted-main follow-through remain pending.
Actual timing comparison uses completed baseline main9670/run40977005218 and candidate598/run40978654861. Both have native tree 91bf000e870ad5080b4ed4dfeac62e227d15d54a, the same pinned compiler, all38 matrix cells and the same product workload. These are observed hosted runs, not a controlled runner-load benchmark.
The consumer elapsed sum is2,708s lower; including the additional138s producer gives2,570s (42m50s) less observed runner elapsed across this affected slice. This sum is not wall time or a billing-minute claim. The workflow itself took6m29s longer, with much larger job-start delays. Start delays include dependencies and queueing; this metadata does not isolate pure queue wait. Runner/cache/load variance remains, so no overall speedup or sole-cause attribution is supported.
The demonstrated change is one same-run, identity-checked debug executable serving30 parts instead of their repeated builds. PostgreSQL/MySQL baseline bootstrap steps combine compilation with schema/bootstrap work; their full elapsed time is not labeled pure compile time. Native source tests, bootstrap/schema/catalog stages, matrix selections, cache trust and required checks are preserved. The new artifact's normal producer tests, thin upload and all actual consumers succeed.
Next: complete the protected merge-group checks, verify the merged main producer/consumers and actual queue/run behavior, then resume additional product PR delivery when CI keeps pace. Keep the earlier cache-key-only result and failed intermediate875 attempts in their recorded scopes. No policy or check was waived.
rmcdaniel commentedon Oct 30, 2026
Completed main branch verification and timing readback at the same commit, October 30, 02:16 UTC.
Both the protected merge-group run and trusted-main run use exact commit
05a68180c2dee8fb9604888ab9154a5cbe81f212. Each completes the same 44 named jobs successfully, including the same 38 matrix cells. No cell or check was dropped. Main's PHP Feature, SonarQube and Public Boundary workflows also completed successfully; the main SonarCloud Code Analysis check succeeded at 01:55:33 UTC.The main run takes 624s less from creation to its final completed job. Summed observed job elapsed is 601s higher. These are complete observations across different events and background load, with identical source and jobs. They provide current pacing evidence; causal code-performance and billing conclusions require separate evidence. The earlier same-native-tree pre/post source comparison remains the source-efficiency measurement, including its longer whole-run result.
This completes the trusted-main result and timing evidence for the owning pacing assessment. Required coverage, exact artifact identity and the existing native/publication boundaries remain unchanged.
rmcdaniel commentedon Oct 30, 2026
CI repair is delivered on protected main 05a6818. PR875 merged at01:48:49 UTC after its complete protected merge-group qualification. Normal-main Shared40988042415 succeeds 44/44, including all30 actual artifact consumers and all8 native-test cells. PHP40988042309 and Sonar40988042335 also succeed. The actual main producer123843606305 succeeds, and every intended consumer verifies the same run-bound artifact; native-test cells retain their source compilation path. Independent Security acceptance accepts the main artifact/trust boundary.
Completed main timing comparison uses the same native source tree91bf000e870ad5080b4ed4dfeac62e227d15d54a and pinned compiler as baseline9670/run40977005218:
The affected consumer+producer slice uses2,323s (38m43s) less observed runner elapsed. Whole workflow time is2m59s lower in this main observation; all matrix jobs start within one second of each other, with no later capacity backlog visible in this run. The shared debug build takes101s once; the five baseline SQLite builds alone total463s. PostgreSQL/MySQL bootstrap time includes schema work and is not labeled pure compilation.
These are actual hosted observations, not a controlled load benchmark or guaranteed/billing-minute saving. The earlier candidate PR observation had a longer32m59s wall time and greater start delays; that evidence remains. The cache-key-only seed/warm result also remains scoped and did not independently improve wall time. No old failure is relabeled.
All original cache trust, compiler identity, current-source rebuild, matrix, native Cargo, bootstrap/schema/catalog and required-check boundaries remain. No policy waiver or larger runner was introduced. This completes871's implemented/normal-main/measured acceptance. Resume product delivery one qualified PR at a time, starting with the prepared Server/Core consumer; inspect any renewed queue/runtime bottleneck under the same CI-first rule.