Skip to content

Use CreateFileMapping\MapViewOfFile and UnmapViewOfFile\CloseHandle instead of VitualAllocEx and VirtualFreeEx on Windows - #135

Merged
simongdavies merged 10 commits into
hyperlight-dev:mainfrom
simongdavies:use-mmap-files
Jan 30, 2025
Merged

simongdavies merged 10 commits into
hyperlight-dev:mainfrom
simongdavies:use-mmap-files

Conversation

@simongdavies

Copy link
Copy Markdown
Member

This PR replaces the use of VirtualAllocEx and VirtualFreeEx with CreateFileMapping, MapViewOfFile, UnmapViewOfFile and CloseHandle on Windows.

This enables the mapping of memory into the surrogate processes without zero copying.

There are 9 commits as part of this PR each with its own description/explanation.

@simongdavies simongdavies added the kind/enhancement For PRs adding features, improving functionality, docs, tests, etc. label Jan 16, 2025

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 7 out of 10 changed files in this pull request and generated no comments.

Files not reviewed (3)
  • docs/assets/hyperlight_arch.excalidraw: Language not supported
  • src/hyperlight_host/src/hypervisor/hyperv_windows.rs: Evaluated as low risk
  • src/hyperlight_host/src/hypervisor/hypervisor_handler.rs: Evaluated as low risk
Comments suppressed due to low confidence (3)

src/hyperlight_host/src/mem/shared_mem.rs:111

  • The new behavior introduced in the Drop implementation of HostMapping should be covered by tests.
if let Err(e) = unsafe { UnmapViewOfFile(mem_mapped_address) } {

src/hyperlight_host/src/hypervisor/surrogate_process.rs:85

  • There is a spelling mistake in the error message: 'drropping' should be 'dropping'.
tracing::error!("Failed to get surrogate process manager when drropping SurrogateProcess: {:?}", e);

src/hyperlight_host/src/hypervisor/surrogate_process.rs:60

  • [nitpick] The variable 'mem_mapped_address' should follow Rust's naming convention and be renamed to 'memory_mapped_view_address'.
let mem_mapped_address = MEMORY_MAPPED_VIEW_ADDRESS {

@simongdavies
simongdavies force-pushed the use-mmap-files branch 3 times, most recently from d28e54c to 0c9a093 Compare January 16, 2025 22:12
@simongdavies
simongdavies requested a review from Copilot January 16, 2025 22:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 7 out of 10 changed files in this pull request and generated no comments.

Files not reviewed (3)
  • docs/assets/hyperlight_arch.excalidraw: Language not supported
  • src/hyperlight_host/src/hypervisor/hyperv_windows.rs: Evaluated as low risk
  • src/hyperlight_host/src/mem/mgr.rs: Evaluated as low risk
Comments suppressed due to low confidence (4)

src/hyperlight_host/src/sandbox/uninitialized.rs:1162

  • The error message should be clear and consistent. Consider rephrasing to: 'GuestBinary not found: 'some/path/that/does/not/exist': The system cannot find the path specified (os error)'.
matches!(sbox, Err(e) if e.to_string().contains("GuestBinary not found: 'some/path/that/does/not/exist': The system cannot find the path specified. (os error"))

src/hyperlight_host/src/hypervisor/surrogate_process.rs:66

  • The error message should be updated to reflect the new function name UnmapViewOfFile2.
Failed to free surrogate process resources (UnmapViewOfFile2 failed): {:?}

src/hyperlight_host/src/hypervisor/surrogate_process.rs:85

  • The error message should be updated to reflect the new function name return_surrogate_process.
Failed to get surrogate process manager when dropping SurrogateProcess: {:?}

src/hyperlight_host/src/hypervisor/surrogate_process.rs:64

  • Ensure that the new behavior introduced by UnmapViewOfFile2 is adequately covered by tests.
if let Err(e) = unsafe { UnmapViewOfFile2(process_handle, memory_mapped_view_address, flags) } {

@simongdavies
simongdavies force-pushed the use-mmap-files branch 3 times, most recently from 0e2c611 to 7f56eec Compare January 16, 2025 23:15
@simongdavies simongdavies changed the title Use CreateFileMapping\MapViewOFile and UnmapViewOfFile\CloseHandle instead of VitualAllocEx and VirtualFreeEx` on Windows Use CreateFileMapping\MapViewOfFile and UnmapViewOfFile\CloseHandle instead of VitualAllocEx and VirtualFreeEx` on Windows Jan 16, 2025
marosset
marosset previously approved these changes Jan 17, 2025

@marosset marosset left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@simongdavies simongdavies changed the title Use CreateFileMapping\MapViewOfFile and UnmapViewOfFile\CloseHandle instead of VitualAllocEx and VirtualFreeEx` on Windows Use CreateFileMapping\MapViewOfFile and UnmapViewOfFile\CloseHandle instead of VitualAllocEx and VirtualFreeEx on Windows Jan 21, 2025
ludfjig
ludfjig previously approved these changes Jan 27, 2025

@ludfjig ludfjig left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Comment thread src/hyperlight_host/src/mem/shared_mem.rs Outdated
Comment thread src/hyperlight_host/src/mem/shared_mem.rs Outdated
@simongdavies
simongdavies dismissed stale reviews from ludfjig and marosset via e5ec0a4 January 28, 2025 21:59
@simongdavies
simongdavies force-pushed the use-mmap-files branch 2 times, most recently from 850ffb2 to 08e57ea Compare January 28, 2025 23:02
ludfjig
ludfjig previously approved these changes Jan 28, 2025

@ludfjig ludfjig left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Regarding the HANDLE insteade of HandleWrapper, it just seemed unnecessary to have send/sync on handle when it wasn't needed, but if it's indeed in needed, or that made things more complicated feel free to change it back

Comment thread src/hyperlight_host/src/mem/shared_mem.rs
syntactically
syntactically previously approved these changes Jan 28, 2025
@simongdavies

Copy link
Copy Markdown
Member Author

LGTM. Regarding the HANDLE insteade of HandleWrapper, it just seemed unnecessary to have send/sync on handle when it wasn't needed, but if it's indeed in needed, or that made things more complicated feel free to change it back

It is needed in some places but not in the one where you pointed it out, since I have to make another small change I will check and see if there are other places where it can be removed.

@simongdavies
simongdavies dismissed stale reviews from syntactically and ludfjig via 60d9a17 January 29, 2025 19:47
@simongdavies
simongdavies force-pushed the use-mmap-files branch 2 times, most recently from 60d9a17 to a61ab99 Compare January 29, 2025 19:49
syntactically
syntactically previously approved these changes Jan 29, 2025
ludfjig
ludfjig previously approved these changes Jan 29, 2025
Comment thread src/hyperlight_host/src/hypervisor/surrogate_process_manager.rs Outdated
Reflect the use of CreateFileMapping/MapViewOfFile/UnmapViewOfFile/CloseHandle instead of VirtualAlloc/VirtualFree

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
…owsDriver

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
When the driver is created a handle to the FileMapping for the memory for this sandbox.

Uses that handle as an argument to the get_surrogate_process function on the SurrogateProcessManager

Removes the code that copies the Sandbox memory to and from the surrogate process memory each time the VM is run.

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
Replace use of VirtualAlloc/VirtualFree with CreateFileMapping/MapViewOFile/UnmapViewOfFile/CloseHandle to allow memry to be mapped into surrogate processes without any copying

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
… feature flag

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
…tualFreeEx

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
…ualAllocEx

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
…m_SystemServices

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/enhancement For PRs adding features, improving functionality, docs, tests, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants