Porting VMMAP to RawPOSIX #56

ChinmayShringi · 2024-11-25T21:33:28Z

Please note this is taken from RawPosix: (Lind-Project/RawPOSIX#87)

Description

Issue # 45

Type of change

Porting pranav-bhatt and ruchjoshi-nyu implementation of VMMAP to RawPOSIX. Request review for porting location and if more comments are needed.

I also added few comments to make things more clear. That would be great if ruchjoshi-nyu could review their correctness.

New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Checklist:

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
Any dependent changes have been added to a pull request and/or merged in other modules (native-client, lind-glibc, lind-project)

feat: new constants folder

Yaxuan-w

Awesome! Minor changes requested.

Would it make more sense to move LIND_ROOT in fs_calls.rs and net_calls.rs to corresponding constant files as well?

src/RawPOSIX/src/safeposix/mod.rs

src/RawPOSIX/src/safeposix/syscalls/fs_calls.rs

src/RawPOSIX/src/tests/sys_tests.rs

src/RawPOSIX/src/interface/mem.rs

src/RawPOSIX/src/safeposix/vmmap.rs

src/RawPOSIX/src/interface/mem.rs

src/RawPOSIX/src/safeposix/dispatcher.rs

rennergade · 2024-12-04T20:30:57Z

@ChinmayShringi add address checking and conversion:

create function in mem.rs called check_and_convert_addr(user_addr: u64)
That function calls check_addr_mapping() in vmmap.rs and checks if address is valid, if not return Error type of result
If valid return Result(vmmap.base_addr + user_addr)
in dispatcher, if returns error, return EFAULT error
if succesful, unwrap Result and send that value to syscall like usual

rennergade · 2024-12-04T20:31:46Z

@ChinmayShringi additional tasks

read through all of this, add additional comments on understanding
add additional tests as needed

src/RawPOSIX/src/constants/net_constants.rs

rennergade · 2025-02-11T19:00:48Z

@Yaxuan-w and @JustinCappos I just changed this to approved on my end. I think its good to go. I'll need you both to change to approved to merge this so if you could take another look that would be great.

JustinCappos

In the future, let's break these down into separate PRs and make it easier to review. A lot of this isn't strictly VMMAP related.

src/wasmtime/crates/lind-multi-process/src/lib.rs

Yaxuan-w · 2025-02-12T16:33:43Z

I met error when running with latest vmmap-alice branch:

lind@d9a905abb17d:~/lind-wasm$ ./lindtool.sh run tests/unit-tests/process_tests/deterministic/hello
/home/lind/lind-wasm/src/wasmtime/target/debug/wasmtime run --allow-precompiled --wasi threads=y --wasi preview2=n tests/unit-tests/process_tests/deterministic/hello.cwasm 
warkError: failed to run main module `tests/unit-tests/process_tests/deterministic/hello.cwasm`

Caused by:
    0: failed to invoke command default
    1: error while executing at wasm backtrace:
           0: 0x38da6 - <unknown>!sysmalloc
           1: 0x36449 - <unknown>!_int_malloc
           2: 0x32d91 - <unknown>!tcache_init
           3: 0x321a4 - <unknown>!__libc_malloc
           4: 0x7ad41 - <unknown>!_IO_file_doallocate
           5: 0x7710e - <unknown>!_IO_doallocbuf
           6: 0x88936 - <unknown>!_IO_new_file_overflow
           7: 0x754fd - <unknown>!__overflow
           8: 0x729bc - <unknown>!_IO_puts
           9: 0x14d9 - <unknown>!__original_main
          10: 0x1361 - <unknown>!_start
          11: 0x114174 - <unknown>!_start.command_export
       note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information
    2: memory fault at wasm address 0xde3f00c in linear memory of size 0x30000
    3: wasm trap: out of bounds memory access

m-hemmings

PR appears well documented and code is clear. I see no changes to request.

qianxichen233 · 2025-02-12T17:04:49Z

I met error when running with latest vmmap-alice branch:

lind@d9a905abb17d:~/lind-wasm$ ./lindtool.sh run tests/unit-tests/process_tests/deterministic/hello
/home/lind/lind-wasm/src/wasmtime/target/debug/wasmtime run --allow-precompiled --wasi threads=y --wasi preview2=n tests/unit-tests/process_tests/deterministic/hello.cwasm 
warkError: failed to run main module `tests/unit-tests/process_tests/deterministic/hello.cwasm`

Caused by:
    0: failed to invoke command default
    1: error while executing at wasm backtrace:
           0: 0x38da6 - <unknown>!sysmalloc
           1: 0x36449 - <unknown>!_int_malloc
           2: 0x32d91 - <unknown>!tcache_init
           3: 0x321a4 - <unknown>!__libc_malloc
           4: 0x7ad41 - <unknown>!_IO_file_doallocate
           5: 0x7710e - <unknown>!_IO_doallocbuf
           6: 0x88936 - <unknown>!_IO_new_file_overflow
           7: 0x754fd - <unknown>!__overflow
           8: 0x729bc - <unknown>!_IO_puts
           9: 0x14d9 - <unknown>!__original_main
          10: 0x1361 - <unknown>!_start
          11: 0x114174 - <unknown>!_start.command_export
       note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information
    2: memory fault at wasm address 0xde3f00c in linear memory of size 0x30000
    3: wasm trap: out of bounds memory access

I believe current vmmap-alice branch cannot compile because there is a misresolved merge conflict with main branch previously, the getifaddrs is perserved in dispatcher.rs while it should already be removed in main branch. I guess you encountered this error probably because you didn't notice the build was failed and used the wrong build version to run the test. I just deleted the getifaddrs and I think the test should run successfully

rennergade · 2025-02-12T17:08:19Z

I met error when running with latest vmmap-alice branch:

lind@d9a905abb17d:~/lind-wasm$ ./lindtool.sh run tests/unit-tests/process_tests/deterministic/hello
/home/lind/lind-wasm/src/wasmtime/target/debug/wasmtime run --allow-precompiled --wasi threads=y --wasi preview2=n tests/unit-tests/process_tests/deterministic/hello.cwasm 
warkError: failed to run main module `tests/unit-tests/process_tests/deterministic/hello.cwasm`

Caused by:
    0: failed to invoke command default
    1: error while executing at wasm backtrace:
           0: 0x38da6 - <unknown>!sysmalloc
           1: 0x36449 - <unknown>!_int_malloc
           2: 0x32d91 - <unknown>!tcache_init
           3: 0x321a4 - <unknown>!__libc_malloc
           4: 0x7ad41 - <unknown>!_IO_file_doallocate
           5: 0x7710e - <unknown>!_IO_doallocbuf
           6: 0x88936 - <unknown>!_IO_new_file_overflow
           7: 0x754fd - <unknown>!__overflow
           8: 0x729bc - <unknown>!_IO_puts
           9: 0x14d9 - <unknown>!__original_main
          10: 0x1361 - <unknown>!_start
          11: 0x114174 - <unknown>!_start.command_export
       note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information
    2: memory fault at wasm address 0xde3f00c in linear memory of size 0x30000
    3: wasm trap: out of bounds memory access

I believe current vmmap-alice branch cannot compile because there is a misresolved merge conflict with main branch previously, the getifaddrs is perserved in dispatcher.rs while it should already be removed in main branch. I guess you encountered this error probably because you didn't notice the build was failed and used the wrong build version to run the test. I just deleted the getifaddrs and I think the test should run successfully

Yeah there seems like the build CI isn't working correctly. I mentioned this to @yashaswi2000

Yaxuan-w · 2025-02-12T17:21:11Z

I believe current vmmap-alice branch cannot compile because there is a misresolved merge conflict with main branch previously, the getifaddrs is perserved in dispatcher.rs while it should already be removed in main branch. I guess you encountered this error probably because you didn't notice the build was failed and used the wrong build version to run the test. I just deleted the getifaddrs and I think the test should run successfully

There was no error message printed on my screen previously. I still met same error with updated vmmap-alice branch

Operations I did:

git pull
./lindtool.sh make_all
./lindtool.sh cptest tests/unit-tests/process_tests/deterministic/hello
./lindtool.sh run tests/unit-tests/process_tests/deterministic/hello

Error message:

lind@d9a905abb17d:~/lind-wasm$ ./lindtool.sh run tests/unit-tests/process_tests/deterministic/hello
/home/lind/lind-wasm/src/wasmtime/target/debug/wasmtime run --allow-precompiled --wasi threads=y --wasi preview2=n tests/unit-tests/process_tests/deterministic/hello.cwasm 
warkError: failed to run main module `tests/unit-tests/process_tests/deterministic/hello.cwasm`

Caused by:
    0: failed to invoke command default
    1: error while executing at wasm backtrace:
           0: 0x38da6 - <unknown>!sysmalloc
           1: 0x36449 - <unknown>!_int_malloc
           2: 0x32d91 - <unknown>!tcache_init
           3: 0x321a4 - <unknown>!__libc_malloc
           4: 0x7ad41 - <unknown>!_IO_file_doallocate
           5: 0x7710e - <unknown>!_IO_doallocbuf
           6: 0x88936 - <unknown>!_IO_new_file_overflow
           7: 0x754fd - <unknown>!__overflow
           8: 0x729bc - <unknown>!_IO_puts
           9: 0x14d9 - <unknown>!__original_main
          10: 0x1361 - <unknown>!_start
          11: 0x114174 - <unknown>!_start.command_export
       note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information
    2: memory fault at wasm address 0x1470000c in linear memory of size 0x30000
    3: wasm trap: out of bounds memory access

The latest change seems only remove function from dispatcher.rs, which shouldn't cause any compilation error even it was still there. I also noticed that wark has been printed out, so everything related to individual syscall handling part should be correct.

Is there any other operations I need to do?

qianxichen233 · 2025-02-12T17:33:11Z

I believe current vmmap-alice branch cannot compile because there is a misresolved merge conflict with main branch previously, the getifaddrs is perserved in dispatcher.rs while it should already be removed in main branch. I guess you encountered this error probably because you didn't notice the build was failed and used the wrong build version to run the test. I just deleted the getifaddrs and I think the test should run successfully

There was no error message printed on my screen previously. I still met same error with updated vmmap-alice branch

Operations I did:
git pull
./lindtool.sh make_all
./lindtool.sh cptest tests/unit-tests/process_tests/deterministic/hello
./lindtool.sh run tests/unit-tests/process_tests/deterministic/hello
Error message:
lind@d9a905abb17d:~/lind-wasm$ ./lindtool.sh run tests/unit-tests/process_tests/deterministic/hello
/home/lind/lind-wasm/src/wasmtime/target/debug/wasmtime run --allow-precompiled --wasi threads=y --wasi preview2=n tests/unit-tests/process_tests/deterministic/hello.cwasm 
warkError: failed to run main module `tests/unit-tests/process_tests/deterministic/hello.cwasm`

Caused by:
    0: failed to invoke command default
    1: error while executing at wasm backtrace:
           0: 0x38da6 - <unknown>!sysmalloc
           1: 0x36449 - <unknown>!_int_malloc
           2: 0x32d91 - <unknown>!tcache_init
           3: 0x321a4 - <unknown>!__libc_malloc
           4: 0x7ad41 - <unknown>!_IO_file_doallocate
           5: 0x7710e - <unknown>!_IO_doallocbuf
           6: 0x88936 - <unknown>!_IO_new_file_overflow
           7: 0x754fd - <unknown>!__overflow
           8: 0x729bc - <unknown>!_IO_puts
           9: 0x14d9 - <unknown>!__original_main
          10: 0x1361 - <unknown>!_start
          11: 0x114174 - <unknown>!_start.command_export
       note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information
    2: memory fault at wasm address 0x1470000c in linear memory of size 0x30000
    3: wasm trap: out of bounds memory access
The latest change seems only remove function from dispatcher.rs, which shouldn't cause any compilation error even it was still there. I also noticed that wark has been printed out, so everything related to individual syscall handling part should be correct.

Is there any other operations I need to do?

Have you run lindtool cpwasm? make_all command actually only compiles glibc
And regarding the previous compilation error, getifaddrs is perserved in dispatcher.rs but it is removed in network_syscall, so the getifaddrs function call made by dispatcher.rs cannot find its syscall function

Yaxuan-w · 2025-02-12T17:35:57Z

Have you run lindtool cpwasm? make_all command actually only compiles glibc

Works, thanks!

Changes from vmmap alice rawposix

d9a3ba8

ChinmayShringi requested a review from Yaxuan-w November 25, 2024 21:35

ChinmayShringi and others added 5 commits November 25, 2024 16:41

fix: revert removed lines

5b13d4c

fix: formatting

70fa146

fix: rm new line

5a3ebbb

feat: new constants folder

ef9e7ad

Merge pull request #57 from Lind-Project/feat-constants-fs

a19b268

feat: new constants folder

Yaxuan-w requested changes Nov 26, 2024

View reviewed changes

src/RawPOSIX/src/safeposix/mod.rs Outdated Show resolved Hide resolved

src/RawPOSIX/src/safeposix/syscalls/fs_calls.rs Outdated Show resolved Hide resolved

src/RawPOSIX/src/tests/sys_tests.rs Outdated Show resolved Hide resolved

Yaxuan-w changed the title ~~Porting VMMAP to RawPOSIX [REVIEW for comments]~~ Porting VMMAP to RawPOSIX Nov 26, 2024

ChinmayShringi and others added 5 commits December 4, 2024 12:31

feat: update from raw-posix

38d8a32

fix: rm commented import

bd567d7

Merge branch 'main' into vmmap-alice

4114b0f

refactor: rm commented imports

5190179

refactor: rm comments

7a39e8a