Skip to content

Commit 4be1252

Browse files
committed
Squashed commit of the following:
commit e787b2d Merge: d56c3b9 0b3ec9b Author: Yi Lin <[email protected]> Date: Tue Mar 25 03:02:42 2025 +0000 Merge branch 'master' into check-fragmentation-immixspace commit 0b3ec9b Author: Kunal Sareen <[email protected]> Date: Mon Mar 24 17:11:27 2025 +1100 Make work packet buffer size configurable from one location (mmtk#1285) Closes mmtk#1281 commit 129362d Author: Kunshan Wang <[email protected]> Date: Thu Mar 20 17:16:35 2025 +0800 Fix lychee command (mmtk#1286) Removed `--base`. All relative URLs in Markdown are relative to the file itself. Increased verbosity to print the checked URLs and redirections. Excluded `dl.acm.org`. It always responses 403 when using Lychee to check links. --------- Co-authored-by: Yi Lin <[email protected]> commit c4f5a02 Author: Yi Lin <[email protected]> Date: Thu Mar 20 16:58:33 2025 +1300 Fix to bytemuck_derive 1.8.1 (mmtk#1288) `bytemuck` uses `bytemuck_derive ^1.4.1`. Recent `bytemuck_derive` versions are not compatible with our MSRV 1.74.1 (1.9.1 requires Rust 1.84+, 1.9.0 requires edition 2024 which is stablized in 1.84). So fix `bytemuck_derive` to the last version before 1.9.0. commit 1a78557 Author: Kunshan Wang <[email protected]> Date: Mon Mar 17 17:37:41 2025 +0800 Fixing MSRV-breaking dependencies (mmtk#1284) Some dependencies started to require MSRV above our current MSRV. The build dependency `built` transitively depends on some crates from the ICU4X project that recently started to depend on Rust 1.81, such as `litemap`. The dependency on ICU4X is completely unnecessary, and we removed it from our dependency tree by forcing the use of a particular version of `idna_adapter`. See: https://docs.rs/crate/idna_adapter/1.2.0 The dev dependency `criterion` depends on `ciborium` which depends on the `half` crate. Since v2.5.0, `half` started to depend on Rust 1.81. `ciborium` needs to be fixed because its MSRV is 1.58 and shouldn't depend on a crate that requires Rust 1.81. We lock the version of the `half` to 2.4.1. This kind of problem should be properly addressed with the new MSRV-aware dependency resolver introduced in Rust 1.84. See: https://doc.rust-lang.org/cargo/reference/resolver.html#rust-version commit 8dded8f Author: Kunshan Wang <[email protected]> Date: Wed Feb 26 14:51:51 2025 +0800 Fix clippy warning about operator precedence (mmtk#1280) In Rust 1.85, Clippy started to warn about the precedence of `<<` and `|` in one of our use cases, although that lint was added before Rust 1.29. https://rust-lang.github.io/rust-clippy/master/index.html#precedence commit 3832b0d Author: Kunshan Wang <[email protected]> Date: Wed Feb 26 13:38:49 2025 +0800 Remove dead trace_object methods. (mmtk#1277) Now that we are using the `#[derive(HasSpaces, PlanTraceObject)]` derive macros to generate the trace_object methods, we can remove the manually-written dead code. commit 84545cc Author: Kunshan Wang <[email protected]> Date: Thu Feb 20 16:19:48 2025 +0800 Special topic chapter for finalizers and weak references (mmtk#1265) This PR adds a special topic chapter in the Porting Guide for supporting finalizers and weak references. This topic is frequently asked and somewhat complex, and needs a dedicated chapter. We also updated the doc comments of the `Scanning::process_weak_refs` API to add code example of the intended use case, and warn the users about potential pitfalls. commit df5e0cd Author: Kunshan Wang <[email protected]> Date: Thu Feb 20 11:27:05 2025 +0800 Bump MSRV to 1.74.1 (mmtk#1276) The current latest version of the "built" crate (v0.7.7) requires MSRV 1.74. We bump the MSRV to 1.74.1. Since version 0.7.6 of the "built" crate, it generates `static` items instead of `const` items for `PKG_VERSION`, `FEATURES_STR`, etc. Our `build_info.rs` used to define `const` items that take their values. After this change, the Rust compiler now interpret those lines as taking references of `static` items, which is unstable until Rust 1.83. We instead replaced those `const` items in `build_info.rs` with `use` statements that create aliases of the items generated by "built". Bumping MSRV to 1.74.1 also allows us to bump the version of the dependency "criterion" to 0.5 which also requires MSRV 1.74. Previously, we locked the version of "criterion" to 0.4 due to its MSRV requirement. We also updated all dependencies to their latest versions. Among those changes, the "sysinfo" crate renamed several `new` methods to `nothing`. We make changes accordingly. We also use `usize::div_ceil` which was introduced in Rust 1.73. This fixes a clippy warning. commit 4ca8812 Author: Yi Lin <[email protected]> Date: Thu Feb 6 12:30:42 2025 +1300 Fix julia extended tests (mmtk#1270) This PR changes the extended testing workflow for Julia: 1. It now tests with `master` in the upstream Julia repo, and `master` in the binding repo by default. 2. In addition to specifying the binding version, we can also specify the Julia version to run with. This change is necessary, as we no longe record the Julia version in the binding (we record the binding version in Julia instead). commit 054feef Author: tianleq <[email protected]> Date: Wed Jan 29 14:07:49 2025 +1100 Clear stale line mark state (mmtk#1268) ![image](https://github.com/user-attachments/assets/5ce787c6-03c0-4845-b0a9-ade68bb0f8d6) ![image](https://github.com/user-attachments/assets/605711ba-1140-42b0-9306-849d4e15e244) ![image](https://github.com/user-attachments/assets/2f1ba1d5-9ca8-4678-b75a-45eac5789990) Overall, this fix does not incur significant overhead commit 051bc74 Author: Kunshan Wang <[email protected]> Date: Tue Jan 21 13:31:06 2025 +0800 Make GC triggering and heap resizing consistent (mmtk#1266) This PR fixes a bug where the MemBalancer did not increase the heap size enough to accommodate the amount of side metadata needed by the pending allocation. It manifested as looping infinitely between triggering GC and (not actually) resizing the heap size after a GC when the minimum heap size is too small. Now it always includes the side metadata amount when increasing heap size. This PR also refactors the calculation of "shifting right and rounding up" which is used in multiple places. We also replace `alloc_rshift` with `log_data_meta_ratio` for two reasons. (1) The previous implementation would cause unsigned overflow before converting the result to `i32`. (2) `log_data_meta_ratio` has clearer semantics. commit c61e6c8 Author: Kunshan Wang <[email protected]> Date: Thu Jan 16 20:05:08 2025 +0800 Force fixed heap size when using NoGC (mmtk#1264) The dynamic heap size trigger needs GC to adjust the heap size, but NoGC can't do GC. So it doesn't make sense to use dynamic heap size with NoGC. Currently, if the user selects the NoGC plan and the dynamic heap size trigger, it will trigger GC at the minimum heap size and then panic immediately. With this change, MMTk will give a warning and use fixed heap size trigger instead, using the maximum heap size specified in the dynamic trigger as the heap size. commit 2f6f078 Author: Kunshan Wang <[email protected]> Date: Tue Jan 14 21:03:49 2025 +0800 Fix Clippy warning in 1.84.0 (mmtk#1262) Rust 1.84.0 added a new lint "unnecessary_map_or". We use `Option::is_some_and` (introduced in Rust 1.70.0) as suggested by the lint. commit 541298f Author: Kunshan Wang <[email protected]> Date: Tue Jan 14 21:03:46 2025 +0800 Fix a subtraction overflow in get_free_pages. (mmtk#1261) The used pages can also be greater than the total pages for the same reason as those in computing `get_available_pages`, and it can also happen if the VM binding disabled GC, in which case we may over-allocate without triggering GC. When it overflows, `get_free_pages` will cause subtraction overflow, and will panic in debug build. We switch to `saturating_sub` so that it will return 0 if overflow happens. It still makes sense. 0 means there is no free pages because we are over-allocating beyond the current heap size set by the GC trigger. commit 68bf1b6 Author: Kunshan Wang <[email protected]> Date: Thu Jan 9 14:40:42 2025 +0800 Unique object enqueuing option (mmtk#1255) Added a constant `VMBinding::UNIQUE_OBJECT_ENQUEUING`. When set to true, MMTk will guarantee that each object is enqueued at most once in each GC. This can be useful for VMs that piggyback on object scanning to visit objects during GC. Implementation-wise, the mark bit is set atomically when `VMBinding::UNIQUE_OBJECT_ENQUEUING` is true. This PR only affects the native MarkSweep space. Other spaces already do this atomically. Fixes: mmtk#1254 commit ec74535 Author: Yi Lin <[email protected]> Date: Tue Jan 7 09:56:50 2025 +1300 Move to Rust 1.83 (mmtk#1253) This PR updates our pinned Rust version to 1.83. This also updates `ci-perf-kit` https://github.com/mmtk/ci-perf-kit/releases/tag/0.8.2 that includes this Rust 1.83 migration as a new epoch. commit c0f9788 Author: Yi Lin <[email protected]> Date: Fri Dec 20 15:28:58 2024 +1300 Bump version to v0.30 (mmtk#1252) commit 2e548e5 Author: Yi Lin <[email protected]> Date: Mon Dec 9 21:23:21 2024 +1300 Allow setting object metadata for VM space objects. Expose VO bit under a feature. (mmtk#1248) This PR changes a few things for vo bit: 1. Add a function to set object metadata for an object in the VM space: `MMTK::initialize_vm_space_object`. 2. Add a feature `vo_bit_access` to expose VO bit and a binding may use it at its own risk. 3. Mark VO bit side metadata base address only avilable for 64 bits. The second is needed for Julia. The Julia binding uses MMTk immortal allocation or VM space for a region of memory, and pop the regions with boot image objects with no clear way to identify each object. The easist workaround is to bulk set VO bit for the region. The problem from this is that MMTk cannot identify valid objects in those regions. However, Julia binding only uses VO bit for pinning objects. Objects in the immortal space or the VM space will not be moved so failing to pinning objects in those regions is benign. Currently the Julia binding duplicates a bunch of side metadata code to bulk set VO bit only using the VO bit side metadata base address. See mmtk/mmtk-julia#200 commit 3c1418a Author: Yi Lin <[email protected]> Date: Mon Dec 9 18:26:42 2024 +1300 Check the option before aggregating live bytes data. Panic if the option is enabled on malloc space. (mmtk#1250) We see failures [here](https://github.com/mmtk/mmtk-core/actions/runs/12193674159/job/34020377988?pr=1248) in OpenJDK tests. ``` [2024-12-06T07:09:55Z INFO mmtk::memory_manager] Initialized MMTk with MarkSweep (FixedHeapSize(54525952)) [2024-12-06T07:09:55Z WARN mmtk::memory_manager] The feature 'extreme_assertions' is enabled. MMTk will run expensive run-time checks. Slow performance should be expected. ===== DaCapo fop starting ===== [2024-12-06T07:10:07Z INFO mmtk::util::heap::gc_trigger] [POLL] MallocSpace: Triggering collection (13313/13312 pages) thread '<unnamed>' panicked at /home/runner/work/mmtk-core/mmtk-core/mmtk-openjdk/repos/mmtk-core/src/policy/marksweepspace/malloc_ms/global.rs:158:9: internal error: entered unreachable code stack backtrace: 0: rust_begin_unwind at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/panicking.rs:647:5 1: core::panicking::panic_fmt at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/panicking.rs:72:14 2: core::panicking::panic at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/panicking.rs:144:5 3: <mmtk::policy::marksweepspace::malloc_ms::global::MallocSpace<VM> as mmtk::policy::space::Space<VM>>::common at ./repos/mmtk-core/src/policy/marksweepspace/malloc_ms/global.rs:158:9 4: mmtk::policy::space::Space::get_descriptor at ./repos/mmtk-core/src/policy/space.rs:332:9 5: mmtk::mmtk::MMTK<VM>::aggregate_live_bytes_in_last_gc::{{closure}} at ./repos/mmtk-core/src/mmtk.rs:542:29 6: <mmtk::plan::marksweep::global::MarkSweep<VM> as mmtk::plan::global::HasSpaces>::for_each_space at ./repos/mmtk-core/src/plan/marksweep/global.rs:31:10 7: mmtk::mmtk::MMTK<VM>::aggregate_live_bytes_in_last_gc at ./repos/mmtk-core/src/mmtk.rs:540:9 8: <mmtk::scheduler::gc_work::Release<C> as mmtk::scheduler::work::GCWork<<C as mmtk::scheduler::work::GCWorkContext>::VM>>::do_work at ./repos/mmtk-core/src/scheduler/gc_work.rs:162:13 9: mmtk::scheduler::work::GCWork::do_work_with_stat at ./repos/mmtk-core/src/scheduler/work.rs:45:9 10: mmtk::scheduler::worker::GCWorker<VM>::run at ./repos/mmtk-core/src/scheduler/worker.rs:255:13 11: mmtk::memory_manager::start_worker at ./repos/mmtk-core/src/memory_manager.rs:491:5 12: start_worker at ./mmtk/src/api.rs:214:9 13: _ZN6Thread8call_runEv at ./repos/openjdk/src/hotspot/share/runtime/thread.cpp:402:12 14: thread_native_entry at ./repos/openjdk/src/hotspot/os/linux/os_linux.cpp:826:19 15: <unknown> 16: <unknown> note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. fatal runtime error: failed to initiate panic, error 5 ``` The issue is that `aggregate_live_bytes_in_last_gc` was not guarded by the condition that the option `count_live_bytes_in_gc` is enabled. So it was executed in our tests. The function accesses the space descriptor through `CommonSpace` and `MallocSpace` does not use `CommonSpace`, thus we see the panic. This PR adds a check before calling `aggregate_live_bytes_in_last_gc`. When the option is not enabled, we will not call the function. This PR also adds a panic for `MallocSpace`. If `count_live_bytes` is turned on, we simply panic, as we cannot provide live bytes vs total page stats for `MallocSpace`. commit e8ff7c6 Author: Yi Lin <[email protected]> Date: Thu Dec 5 15:32:12 2024 +1300 Use macos-15 for style check (mmtk#1249) mmtk#1216 updated the test runner image from `macos-12` to `macos-15`, but I forgot to update the image for style checks. This PR updates the runner for style checks as well. commit a753093 Author: Kunshan Wang <[email protected]> Date: Tue Dec 3 17:37:46 2024 +0800 Minor changes for debugging. (mmtk#1245) Added `MMTK::debug_print_vm_map` which prints the memory ranges of spaces. `NullableObjectReference` now implements `Clone`, `Copy`, `Display` and `Debug`. This allows the binding to print its value like `Address` and `ObjectReference`, and is useful for logging API functions that involve `NullableObjectReference` parameters. commit 8a398e0 Author: Yi Lin <[email protected]> Date: Tue Dec 3 17:03:34 2024 +1300 Collect live bytes per space, and report by space (mmtk#1238) The current `count_live_bytes_in_gc` feature adds the size of all live objects and compare with the used pages reported by the plan. There are two issues with the feature: 1. VM space is not included in the used pages reported by the plan, but the live objects include objects in the VM space. So the reported fragmentation/utilization is wrong when the VM space is in use. 2. Spaces/policies have very different fragmentation ratio. Reporting the fragmentation for the entire heap is not useful. This PR refactors the current `count_live_bytes_in_gc` feature so we collect live bytes per space, and report by space. commit 3d7bc11 Author: Yi Lin <[email protected]> Date: Mon Dec 2 14:55:37 2024 +1300 Fix warnings for lifetime in MmapAnnotation impl (mmtk#1244) mmtk#1242 fixed most similar issues in the repo, but mmtk#1236 introduced `MmapAnnotation` and introduced a new warning. commit cd2fe83 Author: Kunshan Wang <[email protected]> Date: Fri Nov 29 17:17:14 2024 +0800 Annotate mmap ranges using PR_SET_VMA (mmtk#1236) We demand that every invocation of `mmap` within mmtk-core to be accompanied with an "annotation" for the purpose of the mmap. On Linux, we will use `PR_SET_VMA_ANON_NAME` to set the attribute after `mmap` so that it can be seen in `/proc/pid/maps`. This will greatly improve the debugging experience. commit 5bc6ce5 Author: Kunshan Wang <[email protected]> Date: Fri Nov 29 13:16:22 2024 +0800 Fix clippy warnings for Rust 1.83 (mmtk#1242) Clippy 1.83 produces some new warnings: - `needless_lifetimes` is extended to suggest eliding `impl` lifetimes. - `empty_line_after_doc_comments` is added to the `suspicious` group. commit 8640ab8 Author: Yi Lin <[email protected]> Date: Fri Nov 8 19:36:42 2024 +1300 Bump version to v0.29 (mmtk#1232) commit 41501a5 Author: Kunal Sareen <[email protected]> Date: Thu Nov 7 13:35:00 2024 +1100 Fix nightly build and add `inline` attributes to `{un,}likely` (mmtk#1228) commit 753f71c Author: Yi Lin <[email protected]> Date: Thu Nov 7 15:17:58 2024 +1300 Fix auto merge branches (mmtk#1230) This PR changes the auto merge workflow. For each binding, the workflow now allows inputs for base repo and base ref. This change is mostly for the Julia binding which uses `dev` instead of `master` as the default branch. mmtk#1221 only changed for the correctness testing, this PR made corresponding changes for auto merge. commit 59ea62e Author: Kunshan Wang <[email protected]> Date: Thu Nov 7 05:31:13 2024 +0800 Use modern syntax for optional dependencies (mmtk#1229) Use the "dep:" prefix to specify optional dependencies in Cargo features. An optional crate dependency implicitly generates a feature of the same name, and can be leaked to the user of the current crate. But if a feature specifies a crate dependency with the "dep:" prefix, it will not implicitly generate the feature, hiding it from the users. The "dep:" prefix was introduced in Rust 1.60. commit 3575521 Author: Kunshan Wang <[email protected]> Date: Wed Nov 6 13:49:45 2024 +0800 Make env_logger an optional dependency (mmtk#1226) Now the built-in `env_logger` is guarded behind a Cargo feature "builtin_env_logger". It is a default feature, but can be disabled in Cargo.toml by setting `dependencies.mmtk.default-features = false`. In this way, VM bindings that want to implement its own logger can remove the `env_logger` crate from its dependencies. Fixes: mmtk#744 commit 3830168 Author: Yi Lin <[email protected]> Date: Fri Nov 1 15:02:03 2024 +1300 Change the default testing branch for Julia tests (mmtk#1221) We recently re-organised branches in `mmtk-julia` and `julia`. Namely the previous `master` was renamed to `dev`, and we will use `master` for the version that works with Julia upstream. This PR extracts the default testing repos and branches, and changes the default testing branch for Julia. commit 618fde4 Author: Yi Lin <[email protected]> Date: Mon Oct 21 18:48:08 2024 +1300 Document the policy about performance testing environment and epochs (mmtk#1206) Co-authored-by: Kunshan Wang <[email protected]> commit f032697 Author: Kunshan Wang <[email protected]> Date: Mon Oct 21 13:46:06 2024 +0800 Performance history canary (mmtk#1209) This PR adds a "canary" build to the performance regression CI of OpenJDK. The "canary" is a chosen revision of mmtk-core and mmtk-openjdk that is tested alongside each merged PR. The performance of the "canary" should not change unless there is an environment change or there is a noise. Spotting a change in the "canary" performance can help us identify environment changes that are unintended or otherwise unnoticed, and also identify the noise level. Currently, we choose a specific release version as the version of the "canary". Using a release version has the advantage of being easy to specify the exact revision of both the mmtk-core and the mmtk-openjdk repository. We may also switch to some methods of automatically select the canary version in the future. There are other minor changes made. - We slightly change the directory structure. We create two directory, namely `latest` and `canary`. In each of the directories, we check out `mmtk-core` and `mmtk-openjdk` of the latest and the canary versions, respectively. - We use the `ci-replace-mmtk-dep.py` script to replace the revision of the `mmtk-core` dependency in `mmtk-openjdk`. As a result, we no longer need to use `sed`, and no longer need to copy the `mmtk-core` directory into `mmtk-openjdk/repos`. - We no longer set the `RUSTUP_TOOLCHAIN` environment variable because 1. The latest and the canary version may not use the same toolchain, and, 2. the right toolchain will be chosen when running the `cargo` command according to the `rust-toolchain` file in the directory. - The scripts in https://github.com/mmtk/ci-perf-kit are changed to take the canary into consideration, too. commit 80b11a0 Author: Patrick LaFontaine <[email protected]> Date: Sun Oct 20 20:05:29 2024 -0400 Remove space for nogc link (mmtk#1217) commit 0883898 Author: Yi Lin <[email protected]> Date: Thu Oct 17 20:00:03 2024 +1300 Update CI macos image (mmtk#1216) macos-12 will no longer be supported: actions/runner-images#10721 commit 328deb6 Author: Kunshan Wang <[email protected]> Date: Fri Oct 11 16:07:53 2024 +0800 Fix a race between forwarding bits and VO bits. (mmtk#1214) The current code sets the forwarding bits before setting the VO bit when copying an object. If another GC worker is attempting to forward the same object, it may observe the forwarding bits being `FORWARDED` but the VO bit is not set. This violates the semantics of VO bits because VO bits should be set for both from-space and to-space copies. This will affect VM bindings that assert slots always refer to a valid object when scanning objects and may update the same slot multiple times for some reasons. This revision provides a mechanism to ensure that all necessary metadata are set before setting forwarding bits to `FORWARDED`. Currently it affects the VO bits and the mark bits (which are used to update the VO bits in Immix-based plans). It may be used for other metadata introduced in the future. commit 58b3b35 Author: Kunshan Wang <[email protected]> Date: Fri Oct 11 14:04:55 2024 +0800 Install cargo-msrv using stable toolchain. (mmtk#1215) When running the CI check "msrv", we install the cargo-msrv command using the stable Rust toolchain because it sometimes requires a higher Rust version than our chosen version in the file `rust-toolchain`. commit c4fdce0 Author: Yi Lin <[email protected]> Date: Fri Sep 27 22:27:47 2024 +1200 Bump version to v0.28 (mmtk#1212) Merge this PR after mmtk#1208 and mmtk#1211. commit de10fa4 Author: Yi Lin <[email protected]> Date: Fri Sep 27 16:41:37 2024 +1200 Update migration guide for mmtk#1205 (mmtk#1211) Co-authored-by: Kunshan Wang <[email protected]> commit 7cfebda Author: Yi Lin <[email protected]> Date: Fri Sep 27 15:37:20 2024 +1200 Update ci-perf-kit to plot epoch (mmtk#1208) Use mmtk/ci-perf-kit#46 to plot performance data. commit 5605237 Author: Kunal Sareen <[email protected]> Date: Fri Sep 20 16:37:13 2024 +1000 Return if a GC ran or not for `handle_user_collection_request` (mmtk#1205) Closes mmtk#1204 --------- Co-authored-by: Yi Lin <[email protected]> commit d56c3b9 Merge: 4cfac97 dd84218 Author: Eduardo Souza <[email protected]> Date: Wed Mar 27 00:26:59 2024 +0000 Merge remote-tracking branch 'mmtk/master' into feature/check-fragmentation-immixspace commit 4cfac97 Author: Eduardo Souza <[email protected]> Date: Tue Mar 26 23:27:59 2024 +0000 Refactor code; turn on logs; set block size to 16K commit aeb3aeb Author: Eduardo Souza <[email protected]> Date: Wed Mar 6 23:12:08 2024 +0000 Adding statistics about number of objects scanned and objects moved in immixspace commit 6f1c924 Author: Eduardo Souza <[email protected]> Date: Wed Mar 6 23:02:10 2024 +0000 Change printing info commit f294948 Author: Eduardo Souza <[email protected]> Date: Tue Mar 5 01:08:04 2024 +0000 Adding assertion for live lines in immixspace; properly counting live bytes in los commit 184822c Author: Eduardo Souza <[email protected]> Date: Tue Mar 5 01:07:19 2024 +0000 Removing dependency on chrono commit 1547428 Author: Eduardo Souza <[email protected]> Date: Mon Mar 4 09:52:28 2024 +0000 Print stats for sticky immix as well commit 06284e1 Author: Eduardo Souza <[email protected]> Date: Mon Mar 4 09:37:45 2024 +0000 Adding feature to dump memory stats (from los and immixspace) commit 1f39198 Author: Eduardo Souza <[email protected]> Date: Mon Mar 4 04:12:19 2024 +0000 Trying to count live blocks and live lines commit bd3305f Author: Eduardo Souza <[email protected]> Date: Mon Mar 4 03:26:42 2024 +0000 Zeroing the live bytes right before GC starts commit ee21a9c Author: Eduardo Souza <[email protected]> Date: Thu Feb 29 22:59:56 2024 +0000 Moving stats from global state to immixspace commit 530051b Author: Eduardo Souza <[email protected]> Date: Thu Feb 29 04:33:27 2024 +0000 Refactor range check commit f081e4f Author: Eduardo Souza <[email protected]> Date: Thu Feb 29 04:23:19 2024 +0000 Removing duplicated method commit c2a79ad Author: Eduardo Souza <[email protected]> Date: Thu Feb 29 04:13:55 2024 +0000 Adding feature to query the fragmentation of immixspace
1 parent 2ff0db0 commit 4be1252

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

77 files changed

+2261
-457
lines changed

.github/scripts/ci-common.sh

+3
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,9 @@ cargo_toml=$project_root/Cargo.toml
99

1010
dummyvm_toml=$project_root/docs/dummyvm/Cargo.toml
1111

12+
# Pin certain deps for our MSRV
13+
cargo update -p [email protected] --precise 0.5.5 # This can be removed once we move to Rust 1.81 or newer
14+
1215
# Repeat a command for all the features. Requires the command as one argument (with double quotes)
1316
for_all_features() {
1417
# without mutually exclusive features

.github/workflows/auto-merge-inner.yml

+5-1
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,10 @@ on:
1515
ref:
1616
required: true
1717
type: string
18+
# The upstream branch where the binding PR is targeting, such as master, dev
19+
base_ref:
20+
required: true
21+
type: string
1822
# The core commit hash that the binding should be using.
1923
core_commit:
2024
required: true
@@ -35,7 +39,7 @@ jobs:
3539
- name: Check input conditions
3640
id: check-input
3741
run: |
38-
if [[ "${{ inputs.repo }}" == ${{ inputs.base_repo }} ]] && [[ "${{ inputs.ref }}" == "master" ]]; then
42+
if [[ "${{ inputs.repo }}" == ${{ inputs.base_repo }} ]] && [[ "${{ inputs.ref }}" == "${{ inputs.base_ref }}" ]]; then
3943
echo "Conditions not met"
4044
echo "skip=true" >> $GITHUB_OUTPUT
4145
else

.github/workflows/auto-merge.yml

+14-6
Original file line numberDiff line numberDiff line change
@@ -36,8 +36,9 @@ jobs:
3636
needs: [get-merged-pr, binding-refs]
3737
with:
3838
repo: ${{ needs.binding-refs.outputs.openjdk_binding_repo }}
39-
base_repo: mmtk/mmtk-openjdk
39+
base_repo: ${{ needs.binding-refs.outputs.openjdk_binding_repo_default }}
4040
ref: ${{ needs.binding-refs.outputs.openjdk_binding_ref }}
41+
base_ref: ${{ needs.binding-refs.outputs.openjdk_binding_ref_default }}
4142
core_commit: ${{ needs.get-merged-pr.outputs.commit }}
4243
update_lockfile: cargo build
4344
secrets: inherit
@@ -47,8 +48,9 @@ jobs:
4748
needs: [get-merged-pr, binding-refs]
4849
with:
4950
repo: ${{ needs.binding-refs.outputs.jikesrvm_binding_repo }}
50-
base_repo: mmtk/mmtk-jikesrvm
51+
base_repo: ${{ needs.binding-refs.outputs.jikesrvm_binding_repo_default }}
5152
ref: ${{ needs.binding-refs.outputs.jikesrvm_binding_ref }}
53+
base_ref: ${{ needs.binding-refs.outputs.jikesrvm_binding_ref_default }}
5254
core_commit: ${{ needs.get-merged-pr.outputs.commit }}
5355
# `cargo generate-lockfile` will update other dependencies. We avoid using it for the bindings.
5456
# But we do not have a good option for JikesRVM. The Rust project in JikesRVM needs some source files
@@ -62,8 +64,9 @@ jobs:
6264
needs: [get-merged-pr, binding-refs]
6365
with:
6466
repo: ${{ needs.binding-refs.outputs.v8_binding_repo }}
65-
base_repo: mmtk/mmtk-v8
67+
base_repo: ${{ needs.binding-refs.outputs.v8_binding_repo_default }}
6668
ref: ${{ needs.binding-refs.outputs.v8_binding_ref }}
69+
base_ref: ${{ needs.binding-refs.outputs.v8_binding_ref_default }}
6770
core_commit: ${{ needs.get-merged-pr.outputs.commit }}
6871
update_lockfile: cargo build --features nogc
6972
secrets: inherit
@@ -73,19 +76,24 @@ jobs:
7376
needs: [get-merged-pr, binding-refs]
7477
with:
7578
repo: ${{ needs.binding-refs.outputs.julia_binding_repo }}
76-
base_repo: mmtk/mmtk-julia
79+
base_repo: ${{ needs.binding-refs.outputs.julia_binding_repo_default }}
7780
ref: ${{ needs.binding-refs.outputs.julia_binding_ref }}
81+
base_ref: ${{ needs.binding-refs.outputs.julia_binding_ref_default }}
7882
core_commit: ${{ needs.get-merged-pr.outputs.commit }}
79-
update_lockfile: cargo build --features immix
83+
# `cargo generate-lockfile` will update other dependencies. We avoid using it for the bindings.
84+
# mmtk-julia uses bindgen during building and requires the Julia repo. This is a similar situation
85+
# as mmtk-jikesrvm. To make thigns simpler, we just use `cargo generate-lockfile`.
86+
update_lockfile: cargo generate-lockfile
8087
secrets: inherit
8188

8289
check-merge-ruby-pr:
8390
uses: ./.github/workflows/auto-merge-inner.yml
8491
needs: [get-merged-pr, binding-refs]
8592
with:
8693
repo: ${{ needs.binding-refs.outputs.ruby_binding_repo }}
87-
base_repo: mmtk/mmtk-ruby
94+
base_repo: ${{ needs.binding-refs.outputs.ruby_binding_repo_default }}
8895
ref: ${{ needs.binding-refs.outputs.ruby_binding_ref }}
96+
base_ref: ${{ needs.binding-refs.outputs.ruby_binding_ref_default }}
8997
core_commit: ${{ needs.get-merged-pr.outputs.commit }}
9098
update_lockfile: cargo build
9199
secrets: inherit

.github/workflows/cargo-msrv.yml

+3-2
Original file line numberDiff line numberDiff line change
@@ -24,11 +24,12 @@ jobs:
2424
- run: cargo --version
2525

2626
- name: Install cargo-msrv
27-
run: cargo install cargo-msrv
27+
# The cargo-msrv tool sometimes requires a higher Rust version than our current rust-toolchain.
28+
run: cargo +stable install cargo-msrv
2829
# Verify the MSRV defined in Cargo.toml
2930
- name: Verify MSRV
3031
run: cargo msrv verify
3132
# If the previous step fails, find MSRV
3233
- name: Find MSRV
3334
if: failure()
34-
run: cargo msrv
35+
run: cargo msrv find

.github/workflows/extended-tests-bindings.yml

+6-1
Original file line numberDiff line numberDiff line change
@@ -121,10 +121,15 @@ jobs:
121121
- name: Setup
122122
run: |
123123
cd mmtk-julia
124-
./.github/scripts/ci-checkout.sh
124+
./.github/scripts/ci-checkout.sh ${{ needs.binding-refs.outputs.julia_vm_repo }} ${{ needs.binding-refs.outputs.julia_vm_ref }}
125125
./.github/scripts/ci-setup.sh
126126
sed -i 's/^mmtk[[:space:]]=/#ci:mmtk=/g' mmtk/Cargo.toml
127127
sed -i 's/^#[[:space:]]mmtk/mmtk/g' mmtk/Cargo.toml
128+
# removing these two LLVM installations as they cause a conflict within bindgen
129+
- name: Hack to make bindgen work for Github images
130+
run: |
131+
sudo rm -rf /usr/lib/llvm-14
132+
sudo rm -rf /usr/lib/llvm-13
128133
- name: Overwrite MMTk core in Julia binding
129134
run: |
130135
mkdir -p mmtk-julia/repos/mmtk-core

.github/workflows/link-check.yml

+17-2
Original file line numberDiff line numberDiff line change
@@ -17,11 +17,26 @@ jobs:
1717
key: cache-lychee-${{ github.sha }}
1818
restore-keys: cache-lychee-
1919
- name: Check links in docs/*.md
20-
uses: lycheeverse/lychee-action@v1.9.3
20+
uses: lycheeverse/lychee-action@v2
2121
with:
2222
fail: true
2323
token: ${{ secrets.GITHUB_TOKEN }}
24-
args: --base docs --accept '200,201,202,203,204,429,500' --no-progress --cache --max-cache-age 1d './docs/**/*.md' --exclude https://users.cecs.anu.edu.au/~steveb/pubs/papers/**
24+
args:
25+
# Notes:
26+
# - Do not use `--base`. All relative URLs in Markdown are based on the file itself.
27+
# - Verbosity level 2 enables debug logs and shows redirections.
28+
# - Exclude dl.acm.org because it responses 403.
29+
# According to the YAML syntax,
30+
# the following lines will be concatenated into one single YAML string,
31+
# which is intended because the `args:` argument of the lychee-action takes one single string.
32+
--accept '200..=204,429,500'
33+
--verbose --verbose
34+
--no-progress
35+
--cache
36+
--max-cache-age 1d
37+
--exclude 'https://users.cecs.anu.edu.au/~steveb/pubs/papers/**'
38+
--exclude 'https://dl.acm.org/**'
39+
'./docs/**/*.md'
2540
- name: Save lychee cache
2641
uses: actions/cache/save@v4
2742
if: always()

.github/workflows/micro-bm.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ jobs:
6666
with:
6767
repository: mmtk/ci-perf-kit
6868
token: ${{ secrets.GITHUB_TOKEN }}
69-
ref: "0.7.6"
69+
ref: "0.8.2"
7070
path: ci-perf-kit
7171
submodules: true
7272
# Use rust-toolchain in the trunk (it doesnt matter much - if the toolchains defined in the trunk and the branch are different, we cant run anyway)

.github/workflows/minimal-tests-core.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ jobs:
3636
target:
3737
- { os: ubuntu-22.04, triple: x86_64-unknown-linux-gnu }
3838
- { os: ubuntu-22.04, triple: i686-unknown-linux-gnu }
39-
- { os: macos-12, triple: x86_64-apple-darwin }
39+
- { os: macos-15, triple: x86_64-apple-darwin }
4040
rust: ${{ fromJson(needs.setup-test-matrix.outputs.rust )}}
4141

4242
name: minimal-tests-core/${{ matrix.target.triple }}/${{ matrix.rust }}
@@ -87,7 +87,7 @@ jobs:
8787
target:
8888
- { os: ubuntu-22.04, triple: x86_64-unknown-linux-gnu }
8989
- { os: ubuntu-22.04, triple: i686-unknown-linux-gnu }
90-
- { os: macos-12, triple: x86_64-apple-darwin }
90+
- { os: macos-15, triple: x86_64-apple-darwin }
9191
rust: ${{ fromJson(needs.setup-test-matrix.outputs.rust )}}
9292

9393
name: style-check/${{ matrix.target.triple }}/${{ matrix.rust }}

.github/workflows/perf-baseline.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ jobs:
4242
with:
4343
token: ${{ secrets.CI_ACCESS_TOKEN }}
4444
repository: mmtk/ci-perf-kit
45-
ref: "0.7.6"
45+
ref: "0.8.2"
4646
path: ci-perf-kit
4747
submodules: true
4848
# setup
@@ -94,7 +94,7 @@ jobs:
9494
with:
9595
token: ${{ secrets.CI_ACCESS_TOKEN }}
9696
repository: mmtk/ci-perf-kit
97-
ref: "0.7.6"
97+
ref: "0.8.2"
9898
path: ci-perf-kit
9999
submodules: true
100100
# setup

.github/workflows/perf-compare-ci.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -112,7 +112,7 @@ jobs:
112112
with:
113113
repository: mmtk/ci-perf-kit
114114
token: ${{ secrets.CI_ACCESS_TOKEN }}
115-
ref: "0.7.6"
115+
ref: "0.8.2"
116116
path: ci-perf-kit
117117
submodules: true
118118
# setup
@@ -219,7 +219,7 @@ jobs:
219219
with:
220220
repository: mmtk/ci-perf-kit
221221
token: ${{ secrets.CI_ACCESS_TOKEN }}
222-
ref: "0.7.6"
222+
ref: "0.8.2"
223223
path: ci-perf-kit
224224
submodules: true
225225
# setup

.github/workflows/perf-regression-ci.yml

+64-25
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ jobs:
4545
uses: actions/checkout@v4
4646
with:
4747
repository: mmtk/ci-perf-kit
48-
ref: "0.7.6"
48+
ref: "0.8.2"
4949
path: ci-perf-kit
5050
token: ${{ secrets.CI_ACCESS_TOKEN }}
5151
submodules: true
@@ -97,45 +97,81 @@ jobs:
9797
openjdk-perf-regression:
9898
runs-on: [self-hosted, Linux, freq-scaling-off]
9999
timeout-minutes: 1440
100+
env:
101+
# This version will be used as the canary version, and will be used to checkout both
102+
# `mmtk-core` and `mmtk-openjdk`.
103+
#
104+
# A "canary" is a chosen version that is tested alongside each merged pull request. The
105+
# performance of the canary should not change unless
106+
#
107+
# 1. There is an environment change, such as changes of the operating system, the hardware,
108+
# the firmware, or the methodology we use for testing, or
109+
# 2. There are non-deterministic factors (i.e. noises) affecting each execution.
110+
#
111+
# Running the canary alongside regular regression tests help us identify unnoticed environment
112+
# changes and the level of noise.
113+
#
114+
# Currently, we choose one release version as the canary, and will use it until we can no
115+
# longer run it for any reason, such as the toolchain for compiling that version is no longer
116+
# available. Then we may change to another release version and mark the change of canary on
117+
# the timeline, or introduce a mechanism to dynamically choose the canary version.
118+
CANARY_VERSION: "v0.28.0"
100119
steps:
101-
- name: Checkout MMTk Core
120+
# checkout latest versions
121+
- name: Checkout MMTk Core (latest)
102122
uses: actions/checkout@v4
103123
with:
104-
path: mmtk-core
105-
- name: Checkout OpenJDK Binding
124+
path: latest/mmtk-core
125+
- name: Checkout OpenJDK Binding (latest)
106126
uses: actions/checkout@v4
107127
with:
108128
repository: mmtk/mmtk-openjdk
109-
path: mmtk-openjdk
110-
- name: Checkout OpenJDK
111-
working-directory: mmtk-openjdk
129+
path: latest/mmtk-openjdk
130+
- name: Checkout OpenJDK (latest)
131+
working-directory: latest/mmtk-openjdk
112132
run: |
113133
./.github/scripts/ci-checkout.sh
114134
# checkout perf-kit
115135
- name: Checkout Perf Kit
116136
uses: actions/checkout@v4
117137
with:
118138
repository: mmtk/ci-perf-kit
119-
ref: "0.7.6"
139+
ref: "0.8.2"
120140
path: ci-perf-kit
121-
token: ${{ secrets.CI_ACCESS_TOKEN }}
122141
submodules: true
123-
# setup
124-
- name: Overwrite MMTk core in openjdk binding
125-
run: cp -r mmtk-core mmtk-openjdk/repos/
126-
- name: Setup Rust Toolchain
127-
run: echo "RUSTUP_TOOLCHAIN=`cat mmtk-core/rust-toolchain`" >> $GITHUB_ENV
128-
# cleanup previosu build
129-
- name: Cleanup previous build
142+
# checkout canary versions.
143+
- name: Checkout MMTk Core (canary)
144+
uses: actions/checkout@v4
145+
with:
146+
ref: ${{ env.CANARY_VERSION }}
147+
path: canary/mmtk-core
148+
- name: Checkout OpenJDK Binding (canary)
149+
uses: actions/checkout@v4
150+
with:
151+
ref: ${{ env.CANARY_VERSION }}
152+
repository: mmtk/mmtk-openjdk
153+
path: canary/mmtk-openjdk
154+
- name: Checkout OpenJDK (canary)
155+
working-directory: canary/mmtk-openjdk
130156
run: |
131-
rm -rf mmtk-openjdk/repos/openjdk/scratch
132-
rm -rf mmtk-openjdk/repos/openjdk/build
133-
- name: Setup
157+
./.github/scripts/ci-checkout.sh
158+
# setup
159+
- name: Setup directory structures
134160
run: |
135-
./ci-perf-kit/scripts/history-run-setup.sh
136-
sed -i 's/^mmtk[[:space:]]=/#ci:mmtk=/g' mmtk-openjdk/mmtk/Cargo.toml
137-
sed -i 's/^#[[:space:]]mmtk/mmtk/g' mmtk-openjdk/mmtk/Cargo.toml
138-
- id: branch
161+
for BASE_DIR in ./latest ./canary; do
162+
pushd $BASE_DIR
163+
# replace dependency
164+
# Note that ci-replace-mmtk-dep.sh will apply `realpath()` to the `--mmtk-core-path` option.
165+
# so we specify the relative path from the PWD to the mmtk-core repo.
166+
./mmtk-core/.github/scripts/ci-replace-mmtk-dep.sh mmtk-openjdk/mmtk/Cargo.toml \
167+
--mmtk-core-path mmtk-core
168+
# cleanup previous build
169+
rm -rf mmtk-openjdk/repos/openjdk/scratch
170+
rm -rf mmtk-openjdk/repos/openjdk/build
171+
popd
172+
done
173+
- name: Setup branch name
174+
id: branch
139175
# we cannot use env vars in action input (the deploy step). So put the env var to this step's outputs.
140176
run: echo "branch_name=$(echo ${GITHUB_REF#refs/heads/} | sed 's/\//_/g')" >> $GITHUB_OUTPUT
141177
# run
@@ -145,7 +181,10 @@ jobs:
145181
export RESULT_REPO_BRANCH=${{ env.RESULT_REPO_BRANCH }}
146182
export RESULT_REPO_ACCESS_TOKEN=${{ secrets.CI_ACCESS_TOKEN }}
147183
export FROM_DATE=2020-07-10
148-
./ci-perf-kit/scripts/openjdk-history-run.sh ./mmtk-openjdk ./reports/${{ steps.branch.outputs.branch_name }}
184+
./ci-perf-kit/scripts/openjdk-history-run.sh \
185+
./latest/mmtk-openjdk \
186+
./canary/mmtk-openjdk \
187+
./reports/${{ steps.branch.outputs.branch_name }}
149188
# deploy
150189
- name: Deploy to Github Page
151190
if: ${{ env.DEPLOY == 'true' }}
@@ -191,7 +230,7 @@ jobs:
191230
uses: actions/checkout@v4
192231
with:
193232
repository: mmtk/ci-perf-kit
194-
ref: "0.7.6"
233+
ref: "0.8.2"
195234
path: ci-perf-kit
196235
token: ${{ secrets.CI_ACCESS_TOKEN }}
197236
submodules: true

0 commit comments

Comments
 (0)