3tb+ world check doesn't provide summary

[LonelySquirrel15]

When running check mode on a 3tb+ world, it processes to 100% but never prints the optimization summary. I used backtrace after a couple failed attempts hoping to get more info on the error. Doesn't mean much to me though.

89% █████████████████░░░ 845825/954291 [05:27:24>00:52:08, 34.667/s]                                   thread '<unnamed>' panicked at src/region_loader/chunk_loader/chunk.rs:34:35:
range end index 13835781 out of range for slice of length 13832192
stack backtrace:
   0:     0x55962467c14a - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h5b6bd5631a6d1f6b
   1:     0x55962469e503 - core::fmt::write::h7550c97b06c86515
   2:     0x5596246795d3 - std::io::Write::write_fmt::h7b09c64fe0be9c84
   3:     0x55962467bf92 - std::sys::backtrace::BacktraceLock::print::h2395ccd2c84ba3aa
   4:     0x55962467d07c - std::panicking::default_hook::{{closure}}::he19d4c7230e07961
   5:     0x55962467cec2 - std::panicking::default_hook::hf614597d3c67bbdb
   6:     0x55962467d657 - std::panicking::rust_panic_with_hook::h8942133a8b252070
   7:     0x55962467d4ea - std::panicking::begin_panic_handler::{{closure}}::hb5f5963570096b29
   8:     0x55962467c629 - std::sys::backtrace::__rust_end_short_backtrace::h6208cedc1922feda
   9:     0x55962467d17c - rust_begin_unwind
  10:     0x5596245aed10 - core::panicking::panic_fmt::h0c3082644d1bf418
  11:     0x5596245af1d7 - core::slice::index::slice_end_index_len_fail::h8536b7c3e486e738
  12:     0x5596245b59ac - minecraft_world_trimmer::region_loader::chunk_loader::chunk::Chunk::from_location::h564c059b95ff33c9
  13:     0x5596245ce0d5 - minecraft_world_trimmer::region_loader::region::Region::from_file_name::h5eb762944c44c880
  14:     0x5596245cb79e - <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::next::h8e175a8ed8c4d338
  15:     0x5596245cc3b5 - rayon::iter::plumbing::Folder::consume_iter::hbcb2697a78c8a890
  16:     0x5596245bf1e9 - rayon::iter::plumbing::bridge_producer_consumer::helper::h53ffab92171f17e5
  17:     0x5596245c3fdd - rayon_core::job::StackJob<L,F,R>::run_inline::h6131e5ab634d1c16
  18:     0x5596245c356d - rayon_core::join::join_context::{{closure}}::h3b450a137027bb51
  19:     0x5596245bf30c - rayon::iter::plumbing::bridge_producer_consumer::helper::h53ffab92171f17e5
  20:     0x5596245c33cd - rayon_core::join::join_context::{{closure}}::h3b450a137027bb51
  21:     0x5596245bf30c - rayon::iter::plumbing::bridge_producer_consumer::helper::h53ffab92171f17e5
  22:     0x5596245c3fdd - rayon_core::job::StackJob<L,F,R>::run_inline::h6131e5ab634d1c16
  23:     0x5596245c356d - rayon_core::join::join_context::{{closure}}::h3b450a137027bb51
  24:     0x5596245bf30c - rayon::iter::plumbing::bridge_producer_consumer::helper::h53ffab92171f17e5
  25:     0x5596245c3fdd - rayon_core::job::StackJob<L,F,R>::run_inline::h6131e5ab634d1c16
  26:     0x5596245c356d - rayon_core::join::join_context::{{closure}}::h3b450a137027bb51
  27:     0x5596245bf30c - rayon::iter::plumbing::bridge_producer_consumer::helper::h53ffab92171f17e5
  28:     0x5596245c33cd - rayon_core::join::join_context::{{closure}}::h3b450a137027bb51
  29:     0x5596245bf30c - rayon::iter::plumbing::bridge_producer_consumer::helper::h53ffab92171f17e5
  30:     0x5596245c33cd - rayon_core::join::join_context::{{closure}}::h3b450a137027bb51
  31:     0x5596245bf30c - rayon::iter::plumbing::bridge_producer_consumer::helper::h53ffab92171f17e5
  32:     0x5596245c5299 - <rayon_core::job::StackJob<L,F,R> as rayon_core::job::Job>::execute::h612c8474f6aa8d29
  33:     0x5596245abba6 - rayon_core::registry::WorkerThread::wait_until_cold::h0023d4d7aabcaaac
  34:     0x5596245e13a2 - rayon_core::registry::ThreadBuilder::run::hc81de6c0a80d69ea
  35:     0x5596245e6b3a - std::sys::backtrace::__rust_begin_short_backtrace::h4f467c76c8a8282b
  36:     0x5596245e589c - core::ops::function::FnOnce::call_once{{vtable.shim}}::hd617626513317572
  37:     0x55962467fa1b - std::sys::pal::unix::thread::Thread::new::thread_start::hcc78f3943333fa94
89% █████████████████░░░ 845829/954291 [05:27:24>00:53:08, 34.0175/s]                                    38:     0x7ffaccf751f5 - <unknown>
  39:     0x7ffaccff589c - <unknown>
  40:                0x0 - <unknown>
100% ███████████████████░ 954270/954291 [06:06:01>00:00:00, 64.0003/s]

World trimmer continues past the panic and reaches 100%, but then just exits without printing the optimization summary. I tested on the end dimension which is the smallest dimension by copying the level.dat to DIM1 and it gave a summary. Tried it on the nether dimension which is the largest and got the same issue.

Any ideas what's wrong? Think I'll be safe just trying to write it? I have backups, but it will probably take ages to transfer.

Please help. This seems like the only option to keep my server running without buying an 8tb ssd. I don't think mca selector can handle worlds this large. Can it?

[Quozul]

The program likely trimmed your world successfully as it processes and saves each region file individually, it just soft-crashes because there are way too many chunks. I would need to have access to a super large world to replicate the issue and fix it possibly. Running the program twice in a row should ensure that the world is fully trimmed. I can try to fix any unsafe code so that it doesn't panic but instead display a pretty warning.

[LonelySquirrel15]

I'll give it a shot in write mode then. I just wasn't sure if it was safe to do so. Even though I do have backups. Thanks.

[LonelySquirrel15]

It trimmed the world form 3188GB down to 912GB. I ran it 3 times with no change after that and still got the panic errors. I'm not sure if the issue is because there are so many world files or something else. It is an old world. From what I can tell it is Zlib compression. But for all I know it may have been changed to LZ4 at some point and then back to Zlib.