[WIP] snapshot feature (issue 535) + many tiny changes

.gitignore

@@ -39,3 +39,6 @@ ipch/
 
 # macOS
 .DS_Store
+
+# Doxygen
+/doxygen/

doc/notes/snapshot.md

@@ -0,0 +1,137 @@
+## Save and restore a snapshot of a Wren VM
+
+NOTE: still in flux!
+
+### Identify an Obj independently...
+
+That's *pointer unswizzling*.
+
+#### ... from a bare C pointer
+
+References as pointers are everywhere, but we need to identify an Obj.
+
+#### ... from the order of the linked list of all Obj
+
+Iterating on the singly-linked list of all Obj from `vm->first`.
+When saving an existing VM, we iterate them from the most recently created to the oldest created (e.g. the Object class).
+When restoring a snapshot, the first item read, which was the most recently created, is now created first, and will become the oldest created.
+
+
+
+### Census
+
+The VM must be paused.
+It's related to a VM, yet it has no pointer to it.
+
+GC is forbidden meanwhile saving/restoring
+nothing is hold!
+
+
+
+Can now fork a VM :-)
+Conceptually extensible to a full hibernation/thaw
+
+
+
+The serialization file format is underspecified.
+Upon de-serialization, not enough tests are done.
+[KINDS-OF-DOC]: https://github.com/munificent/craftinginterpreters/blob/master/note/design%20breaks.md
+Values of enum ObjType are reused as is, thus no longer changeable.
+
+
+
+
+# Safety
+
+ALLOCATE is expected to succeed. (As elsewhere in the WrenVM.)
+
+
+
+# TODO Security
+
+A fuzzy snapshot is an art better suited to museums than computer science. :-)
+
+what if restoring 0 ObjFOO? Does ALLOCATE work?
+
+what if ObjFn#9 references:
+- ObjFn#1000?
+- itself?
+
+vm->modules must be a Map from ObjString to ObjModule
+
+    mkdir test/snapshot/restore/bad...
+
+
+When restoring, an absent `fn->debug->name` could be synthesized as `fn0`
+
+Validate bytecode
+- opcodes are known
+- opcode arg is valid WRT constants count
+- all jumps land on an instruction start
+- ...
+
+
+
+
+## Unit tests
+
+hijacked^Wenhanced test/test.c
+
+all but 5 tests pass
+- you must preserve line numbers (with the '1' char), otherwise many sadness, such as:
+> FAIL: test/limit/too_many_inherited_fields.wren
+>       Expected runtime error on line 139 but was on line 0.
+- The 5 failing tests are the ones of APITest_Run()
+  - they use the VM after its initialization, but the transient VM is not a replacement of the tested one.
+
+
+
+# TODO
+
+# should re-shuffle code
+
+src/snapshot/count.c
+src/snapshot/types.h
+src/snapshot/save.h
+src/snapshot/restore.h
+
+## easy, but require to unmark static or re-shuffle code
+
+String restore
+Map resize
+
+
+TODO:
+- TAG_NAN saved when WREN_NAN_TAGGING can't be restored in a VM without WREN_NAN_TAGGING.
+
+dedup strings more
+
+dedup other?
+  CONST_STRING should memoize
+sizeof(sObj) is 32; CAN shrink
+NOT fool-proof
+  src/vm/wren_vm.c:533
+    bindMethod() should IS_CLOSURE()
+RLE for:
+- empty methods
+- debugLines
+
+what if many local vars on top-level?
+
+CHAR(oneCharStr) should be CHAR(c)
+
+in src/vm/wren_vm.c, should wrenCollectGarbage() gray the 11 vm->FOOclass?
+  no
+
+
+
+# FlatBuffers in C, flatcc
+
+need a lib?
+it adds padding.
+must hook its allocation to ALLOCATE()
+
+[]: https://flatbuffers.dev/md__internals.html
+> scalars of various sizes, all aligned to their own size
+
+> Strings are simply a vector of bytes, and are always null-terminated.

doc/site/classes.markdown

@@ -174,7 +174,7 @@ You're saying "look up the method `isFancy` in the scope of the object
 the object to the left of the period is the object you want to look up the
 method on.
 
-### `this`
+### Using `this`
 
 Things get more interesting when you're inside the body of a method. When the
 method is called on some object and the body is being executed, you often need

doc/site/contributing.markdown

@@ -69,7 +69,7 @@ server from there. Python includes one:
     $ python -m http.server
 
 Running that script every time you change a line of Markdown can be slow,
-so there is also a file watching version that will automatically regenerate the
+so there is also a file-watching version that will automatically regenerate the
 docs when you edit a file:
 
     $ python util/generate_docs.py --watch

doc/site/embedding/calling-c-from-wren.markdown

@@ -10,9 +10,9 @@ flavor into our VM:
 Since Wren is object-oriented, behavior lives in methods, so for the former we
 have **foreign methods**. Likewise, data lives in objects, so for the latter, we
 define **foreign classes**. This page is about the first, foreign methods. The
-[next page][] covers foreign classes.
+[next section][] covers foreign classes.
 
-[next page]: /embedding/storing-c-data.html
+[next section]: /embedding/storing-c-data.html
 
 A foreign method looks to Wren like a regular method. It is defined on a Wren
 class, it has a name and signature, and calls to it are dynamically dispatched.

doc/site/embedding/calling-wren-from-c.markdown

@@ -73,7 +73,7 @@ arguments.
 
 This is just a regular WrenHandle, which means you can hold onto it as long as
 you like. Typically, you'd call this once outside of your application's
-performance critical loops and reuse it as long as you need. It is us up to you
+performance critical loops and reuse it as long as you need. It is up to you
 to release it when you no longer need it by calling `wrenReleaseHandle()`.
 
 ## Setting Up a Receiver
@@ -173,7 +173,7 @@ keeps running until either the method returns or a fiber [suspends][].
 
 [suspends]: ../modules/core/fiber.html#fiber.suspend()
 
-`wrenCall()` returns the same WrenInterpretResult enum as `wrenInterpret()` to
+`wrenCall()` returns the same `WrenInterpretResult` enum as `wrenInterpret()` to
 tell you if the method completed successfully or a runtime error occurred.
 (`wrenCall()` never returns `WREN_ERROR_COMPILE` since it doesn't compile
 anything.)

doc/site/embedding/configuring-the-vm.markdown

@@ -1,7 +1,7 @@
 ^title Configuring the VM
 
 When you create a Wren VM, you tweak it by passing in a pointer to a
-WrenConfiguration structure. Since Wren has no global state, you can configure
+`WrenConfiguration` structure. Since Wren has no global state, you can configure
 each VM differently if your application happens to run multiple.
 
 The struct looks like:
@@ -29,15 +29,15 @@ wrenInitConfiguration(&configuration);
 </pre>
 
 Calling this ensures that your VM doesn't get uninitialized configuration when
-new fields are added to WrenConfiguration. Here is what each field does, roughly
+new fields are added to `WrenConfiguration`. Here is what each field does, roughly
 categorized:
 
 ## Binding
 
 The VM is isolated from the outside world. These callbacks let the VM request
 access to imported code and foreign functionality.
 
-### **`loadModuleFn`**
+### The **`loadModuleFn`** callback
 
 This is the callback Wren uses to load an imported module. The VM itself does
 not know how to talk to the file system, so when an `import` statement is
@@ -56,12 +56,12 @@ host should return the source code for that module in a `WrenLoadModuleResult` s
 <pre class="snippet" data-lang="c">
 WrenLoadModuleResult myLoadModule(WrenVM* vm, const char* name) {
   WrenLoadModuleResult result = {0};
-    result.source = getSourceForModule(name);
+  result.source = getSourceForModule(name);
   return result;
 }
 </pre>
 
-The module loader is only be called once for any given module name. Wren caches
+The module loader is only called once for any given module name. Wren caches
 the result internally so subsequent imports of the same module use the
 previously loaded code.
 
@@ -80,7 +80,7 @@ static void loadModuleComplete(WrenVM* vm,
                                const char* module,
                                WrenLoadModuleResult result) 
 {
-  if(result.source) {
+  if (result.source) {
     //for example, if we used malloc to allocate
     //our source string, we use free to release it.
     free((void*)result.source);
@@ -89,23 +89,23 @@ static void loadModuleComplete(WrenVM* vm,
 
 WrenLoadModuleResult myLoadModule(WrenVM* vm, const char* name) {
   WrenLoadModuleResult result = {0};
-    result.onComplete = loadModuleComplete;
-    result.source = getSourceForModule(name);
+  result.onComplete = loadModuleComplete;
+  result.source = getSourceForModule(name);
   return result;
 }
 </pre>
 
-### **`bindForeignMethodFn`**
+### The **`bindForeignMethodFn`** callback
 
-The callback Wren uses to find a foreign method and bind it to a class. See
+This is the callback Wren uses to find a foreign method and bind it to a class. See
 [this page][foreign method] for details. If your application defines no foreign
 methods, you can leave this `NULL`.
 
 [foreign method]: /embedding/calling-c-from-wren.html
 
-### **`bindForeignClassFn`**
+### The **`bindForeignClassFn`** callback
 
-The callback Wren uses to find a foreign class and get its foreign methods. See
+This is the callback Wren uses to find a foreign class and get its foreign methods. See
 [this page][foreign class] for details. If your application defines no foreign
 classes, you can leave this `NULL`.
 
@@ -116,11 +116,11 @@ classes, you can leave this `NULL`.
 These let you wire up some minimal output so you can tell if your code is doing
 what you expect.
 
-### **`writeFn`**
+### The **`writeFn`** callback
 
 This is the callback Wren uses to output text when `System.print()` or the other
 related functions are called. This is the minimal connection the VM has with the
-outside world and lets you do rudimentary "printf debugging". Its signature is:
+outside world and lets you do rudimentary "`printf()` debugging". Its signature is:
 
 <pre class="snippet" data-lang="c">
 void write(WrenVM* vm, const char* text)
@@ -130,7 +130,7 @@ Wren does *not* have a default implementation for this. It's up to you to wire
 it up to `printf()` or some other way to show the text. If you leave it `NULL`,
 calls to `System.print()` and others silently do nothing.
 
-### **`errorFn`**
+### The **`errorFn`** callback
 
 Wren uses this callback to report compile time and runtime errors. Its signature
 is:
@@ -177,7 +177,7 @@ If you leave this `NULL`, Wren does not report any errors.
 
 These fields control how the VM allocates and manages memory.
 
-### **`reallocateFn`**
+### The **`reallocateFn`** callback
 
 This lets you provide a custom memory allocation function. Its signature is:
 
@@ -195,9 +195,9 @@ freed, this is zero. Your callback should allocate the proper amount of memory
 and return it.
 
 If you don't provide a custom allocator, the VM uses a default one that relies
-on `realloc` and `free`.
+on `realloc()` and `free()`.
 
-### **`initialHeapSize`**
+### The **`initialHeapSize`** field
 
 This defines the total number of bytes of memory the VM will allocate before
 triggering the first garbage collection. Setting this to a smaller number
@@ -206,7 +206,7 @@ to collect garbage more frequently.
 
 If you set this to zero, Wren uses a default size of 10MB.
 
-### **`minHeapSize`**
+### The **`minHeapSize`** field
 
 After a garbage collection occurs, the threshold for the *next* collection is
 determined based on the number of bytes remaining in use. This allows Wren to
@@ -219,7 +219,7 @@ back to a usable size.
 
 If zero, this defaults to 1MB.
 
-### **`heapGrowthPercent`**
+### The **`heapGrowthPercent`** field
 
 Wren tunes the rate of garbage collection based on how much memory is still in
 use after a collection. This number controls that. It determines the amount of

doc/site/embedding/index.markdown

@@ -106,8 +106,8 @@ virtual machine. To do that, you create a `WrenConfiguration` object and
 initialize it.
 
 <pre class="snippet" data-lang="c">
-    WrenConfiguration config;
-    wrenInitConfiguration(&config);
+WrenConfiguration config;
+wrenInitConfiguration(&config);
 </pre>
 
 This gives you a basic configuration that has reasonable defaults for
@@ -129,7 +129,7 @@ And then, we update the configuration to point to it.
 <pre class="snippet" data-lang="c">
   WrenConfiguration config;
   wrenInitConfiguration(&config);
-    config.writeFn = &writeFn;
+  config.writeFn = &writeFn;
 </pre>
 
 [configuration]: configuring-the-vm.html
@@ -242,8 +242,8 @@ int main()
 
   WrenConfiguration config;
   wrenInitConfiguration(&config);
-    config.writeFn = &writeFn;
-    config.errorFn = &errorFn;
+  config.writeFn = &writeFn;
+  config.errorFn = &errorFn;
   WrenVM* vm = wrenNewVM(&config);
 
   const char* module = "main";

doc/site/embedding/slots-and-handles.markdown

@@ -21,7 +21,7 @@ dynamically sized, but it's your responsibility to ensure there are enough slots
 *before* you use them. You do this by calling:
 
 <pre class="snippet" data-lang="c">
-wrenEnsureSlots(WrenVM* vm, int slotCount);
+void wrenEnsureSlots(WrenVM* vm, int numSlots);
 </pre>
 
 This grows the slot array if needed to ensure that many slots are available. If