mmap/sbrk/brk implementation and integration into glibc/wasmtime

[qianxichen233]

This PR finalized the implementation of mmap/munmap/sbrk/brk.

• On rawposix side, the changes are mainly

  1. Finishing the remaining program break implementation and file backed mmap feature
  2. Minor changes in vmmap to allow the pre-specified start and end address when find map space
  3. Minor adjustment on the vmmap initialization functions invoked from wasmtime.

• On wasmtime side, the changes are mainly about bypassing part of wasmtime's internal memory management logic, including:

  1. Disabled wasmtime's sys mmap function, as this is handled by rawposix right now.
  2. Have wasmtime always creating a length of 4294967296 bytes of accessible region, so that some of the wasm's instruction handled by wasmtime will not be rejected due to memory address.
  3. Restructured memory initialization code interacted with rawposix.

• On glibc side, the changes are mainly about

  1. Modified glibc's entry point of sbrk/brk syscall so that it directs to rawposix
  2. Revert pthread's memory allocation from malloc (old solution when mmap hasn't been implemented) to mmap.
  3. Modified lind_syscall.c to special handle mmap result. Since mmap is returning a 32-bit address, which might overflow int type and become negative, we need to distinguish the valid memory address from errno as errno are also returned as negative number from rawposix. This is achieved by checking the range of the negative value since mmap result is always multiple of pages (4096), while errno has a maximum value that is far less than 4096.

[qianxichen233]

There is a minor issue that I just noticed. Since mmap is now find memory space from highest address, so the first mmap will usually return an address that is close to 4294967296. In wasmtime, previously I was using i32 to hold some of the address when doing the fork or pthread creation, now the mmap address will overflow i32 and cause some issues here. The fix should be pretty simple - just replace i32 with u32. I'll fix this later.

Besides, the test added by #67 (chain_thread.c), which was not able to run previously because it uses mmap somewhere, is verified to be able to run successfully after mmap has been integrated. Though I did some temporary tweak to make mmap is returning address from 2^31 so that it will not overflow i32. But this should at least tell us that things are pretty much there and we only have a minor issue (replace i32 to u32) to fix

[rennergade]

are we sure that this deep copies on clone?

[qianxichen233]

I actually cannot be 100% sure about this. clone method should be designed to deep-copying itself by convention from what I've read, but if the developer (in our case, developer of nodit library or its sub-dependent) is breaking the convention and doing some hacky things here, it would probably be hard for us to know. The documentation for the NoditMap only says it returns a copy of the value without mentioning shallow or deep.

[rennergade]

This is something worth just adding a test for @ChinmayShringi

[rennergade]

Looks very good. I don't have any specific requests but would like to see a few more tests.

1. more complicated situations with mmap/munmap and fork
2. Tests that use malloc instead of sbrk

[Yaxuan-w]

Great work! This is a solid implementation! Overall logic makes sense to me.

[qianxichen233]

update:

1. fixed the u32 issue
2. added two tests:
   • mmap_complicated.c: more complicated use case of mmap that uses shared memory for ipc
   • malloc_large.c: try to malloc an extremely large memory region. This will fail previously but it is working now

There is already a simple malloc test under memory_tests so I am only adding one malloc_large test

[yzhang71]

I have reviewed the implementations, and everything now looks good to me.