Support for realms/contexts in `deno_core` (in preparation for `ShadowRealm`)

[andreubotella]

In JS, code execution is always tied to some realm (or "context" in V8 terminology), which provides a global object and a set of built-ins (Object, Array, Error, etc.) which is different from those of other realms. Multiple realms can share an event loop (= a thread), and those that do share one may be able to access objects from a different realm. In the web, a same-origin <iframe> is a different realm from the top-level page (iframe.contentWindow.Array !== window.Array). Currently there is no way to create realms or run code in them purely with JS built-ins, that's something that host environments (the web / Node.js / Deno) would have to provide, so it's fine for Deno to not support them.

But there is currently a stage-3 TC39 proposal called ShadowRealm that would allow creating realms, as a way of sandboxing untrusted code. This proposal is not close to shipping anywhere, and it's only now starting to get implemented in V8, but currently Deno assumes that there's a single V8 context available at all times, and it's better to start refactoring things to support realms with plenty of time to spare.

Since ShadowRealms are meant to be a sandbox primitive, there would be no way to make objects from the parent realm available inside the ShadowRealm and vice versa. This simplifies to some extent the requirements needed, because that way there is no need to comb through the JS code looking for wrong uses of instanceof. That said, these are the requirements that seem to be needed for ShadowRealm (and I'm sure I'm forgetting some):

• Add a Rust API to create V8 contexts associated with a JsRuntime and run scripts in them (similar to JsRuntime::execute_script), to simplify running tests. Modules can be left for later. feat(core): Add initial support for realms deno#14019
• Make sure bindings and extension scripts are initialized correctly in the context, both when using snapshots and when not. This includes making sure the ops cache in different realms can't go out of sync. feat(core): Add initial support for realms deno#14019
• Make sure any object return values of bindings and ops, as well as any exceptions they throw, are objects in the current realm (i.e. their prototype is not, say, a different realm's Object.prototype). test(core): Test that sync ops return/throw objects in the right realm deno#14750, fix(core): Have custom errors be created in the right realm deno#17050.
• Make sure async ops don't resolve a promise in the wrong realm. Originally landed on feat(core): Add support for async ops in realms deno#14734, then reverted, now relanded in feat(core): Reland support for async ops in realms deno#17204.
• js_promise_reject_cb, js_uncaught_exception_cb and js_wasm_streaming_cb should probably be realm → callback (weak)maps. Originally landed on refactor(core): Move optional callbacks from JsRuntimeState to ContextState deno#15599, then reverted, now relanded in refactor(core): Move optional callbacks from JsRuntimeState to ContextState deno#17422.
• Modules. Per the HTML spec, every realm has its own module map, although module fetches can be cached across realms. feat(core): Add support for modules in realms. deno#15760 feat(realms): Add support for modules in realms. #41
  • Add Rust APIs to load modules in a context (similar to JsRuntime::load_main_module and JsRuntime::load_side_module).
  • There should be a module map per realm, rather than one for the entire JsRuntime. (Should there be a single module loader, though? Blobs should probably not be cached across realms, but non-blob fetches probably should, so it's not clear what to do about Per-JsRuntime maps in ProcState deno#12458.)
  • Make sure module evaluation in realms integrates with the event loop.
  • Dynamic imports and import.meta.
• Review every use of serde_v8::Value to make sure objects from a realm can't leak into another.
• Implement the ShadowRealm constructor. [WIP] Implement the ShadowRealm constructor deno#16211
• Fix the inspector to properly support multiple realms. Multiplex multiple isolate/contexts over a single inspector instance #353

[andreubotella]

For the general case with arbitrary realms, not just those with the security properties of ShadowRealm, there are more requirements and questions:

• Comb through the JS runtime code to make sure instanceof checks are sound in the presence of objects from a different realm.
• HTML and WebIDL have convoluted requirements regarding in which realm JS code gets executed, and in which realm the return values of web APIs are constructed. (I confess I don't fully understand those myself.)
• serde_v8 and the bindings must support inputs from a different realm.
• Creating contexts with an empty global, even though the isolate was created from a non-empty snapshot. Also, creating contexts from arbitrary existing objects. (Both needed to implement Node.js's vm module.)

[bartlomieju]

Inspector support for multiple contexts should also be considered when working on this feature. What you already proposed is gonna be rather large undertaking, I expect even more non-trivial engineering effort for inspector integration.

[andreubotella]

Recently denoland/deno#13861 has radically changed how ops work internally, such that op synchronization is no longer needed. Also, denoland/deno#13993, although still experimental, should guarantee that async ops resolve the right promise on the right realm. Both of this PRs simplify the work needed for refactoring deno_core to support realms, and as a side effect, reduce the number of JS callbacks that need to be kept in a per-context weakmap.