WIP: zon

[MasonRemaley]

(wip, will clean up and make easier to review before removing draft status)

[andrewrk]

Closing drafts untouched for 30+ days.

[travisstaloch]

Hey Mason. I was looking at this work and was tempted to try and revive this. This looks really promising. Are you still interested in this? And was there anything blocking you that made you stop?

I'd be willing to try and get this running w/ master if you think its worthwhile.

[MasonRemaley]

Hey, thanks for the interest! I put this on hold to finish working on a game, but that game ships on Monday and then I'm gonna get back to this (my next game is gonna use ZON for its data files so this will be high priority for me.)

[travisstaloch]

Good news 👍 . I was just working on getting some of the tests running. Would that be helpful to get this compiling and tests passing again? Or do you imagine you'd just redo that work when you get back to it?

[MasonRemaley]

Hmm. I haven't looked at how hard the merge will be yet, but my plan is just to do an interactive rebase and step through it until it's all up to date. You could try it if you want, but I'm guessing this would be a lot easier for me do since I remember how it's all structured.

If your goal is just to get started using ZON, you might have an easier path forward--this PR has two parsers:

1. A runtime parser similar to std.json
2. A comptime parser that is run when you @import a ZON file

The first parser, the runtime one, doesn't require any changes to the compiler. You could actually just copy paste lib/std/zon/parse.zig into your project, fix any compiler errors when running it in the latest version of Zig, and you'd be able to parse ZON at runtime without a complicated merge.

[travisstaloch]

Thanks for the overview. Yeah I think it will be easier for you. I'm running into a few things where I'm not sure how to solve them.

After fixing a couple of tests, here's what I'm looking at just from at zon/parse.zig:

lib/std/zon/stringify.zig:118:38: error: runtime value contains reference to comptime var
    return typeIsRecursiveImpl(T, buf[0..0]);
                                  ~~~^~~~~~

lib/std/zon/parse.zig:1521:40: error: root struct of file 'zig.string_literal' has no member named 'multilineParser'
    var parser = std.zig.string_literal.multilineParser(writer);
                 ~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~

lib/std/zon/stringify.zig:441:24: error: root struct of file 'fmt' has no member named 'formatFloatDecimal'
            try std.fmt.formatFloatDecimal(@as(f64, @floatCast(val)), .{}, self.writer);
                ~~~~~~~^~~~~~~~~~~~~~~~~~~

Wish you the best with this. Excited to see what comes of this. I'll keep an eye out and maybe post back here if i make any progress on any these.

[MasonRemaley]

No problem! I guess I did make a few changes to the compiler to support this, they're pretty minor and I'd run through them for you if I wasn't so busy with launch right now--either way I'll reopen this and resolve them myself soon!

[travisstaloch]

The following patch gets down to 2 distinct errors when running zig test lib/std/zon.zig. Unfortunately much of the diff is formatting changes. But it does solve the runtime value contains reference to comptime var for zon/stringify.zig/typeIsRecursive() from above.

EDIT: I removed the formatting changes.

diff --git a/lib/std/zon/parse.zig b/lib/std/zon/parse.zig
index d195183d37..4cd8cfce7a 100644
--- a/lib/std/zon/parse.zig
+++ b/lib/std/zon/parse.zig
@@ -445,7 +445,7 @@ fn parseUnion(self: *@This(), comptime T: type, comptime options: ParseOptions,
         inline for (field_infos, 0..) |field, i| {
             kvs_list[i] = .{ field.name, i };
         }
-        break :b std.ComptimeStringMap(usize, kvs_list);
+        break :b std.StaticStringMap(usize).initComptime(kvs_list);
     };
 
     // Parse the union
@@ -705,7 +705,7 @@ fn parseStruct(self: *@This(), comptime T: type, comptime options: ParseOptions,
         inline for (field_infos, 0..) |field, i| {
             kvs_list[i] = .{ field.name, i };
         }
-        break :b std.ComptimeStringMap(usize, kvs_list);
+        break :b std.StaticStringMap(usize).initComptime(kvs_list);
     };
 
     // Parse the struct
@@ -1827,7 +1827,7 @@ fn parseEnumLiteral(self: @This(), comptime T: type, node: NodeIndex) error{Type
             inline for (enum_fields, 0..) |field, i| {
                 kvs_list[i] = .{ field.name, @enumFromInt(field.value) };
             }
-            const enum_tags = std.ComptimeStringMap(T, kvs_list);
+            const enum_tags = std.StaticStringMap(T).initComptime(kvs_list);
 
             // Get the tag if it exists
             const main_tokens = self.ast.nodes.items(.main_token);
@@ -1993,7 +1993,7 @@ fn parseBool(self: @This(), node: NodeIndex) error{Type}!bool {
     switch (tags[node]) {
         .identifier => {
             const bytes = self.ast.tokenSlice(token);
-            const map = std.ComptimeStringMap(bool, .{
+            const map = std.StaticStringMap(bool).initComptime(.{
                 .{ "true", true },
                 .{ "false", false },
             });
@@ -2067,7 +2067,7 @@ fn parseNumber(
                 const token = main_tokens[num_lit_node];
                 const bytes = self.ast.tokenSlice(token);
                 const Ident = enum { inf, nan };
-                const map = std.ComptimeStringMap(Ident, .{
+                const map = std.StaticStringMap(Ident).initComptime(.{
                     .{ "inf", .inf },
                     .{ "nan", .nan },
                 });
@@ -2416,10 +2416,10 @@ test "std.zon parse int" {
         defer ast.deinit(gpa);
         var status: ParseStatus = .success;
         try std.testing.expectError(error.Type, parseFromAst(u8, gpa, &ast, &status, .{}));
-        try std.testing.expectEqual(status.invalid_number_literal.reason, .{ .invalid_digit = .{
+        try std.testing.expectEqual(NumberLiteralError{ .invalid_digit = .{
             .i = 2,
             .base = @enumFromInt(10),
-        } });
+        } }, status.invalid_number_literal.reason);
         const node = status.invalid_number_literal.node;
         const main_tokens = ast.nodes.items(.main_token);
         const token = main_tokens[node];
diff --git a/lib/std/zon/stringify.zig b/lib/std/zon/stringify.zig
index dc14f55269..0d35498013 100644
--- a/lib/std/zon/stringify.zig
+++ b/lib/std/zon/stringify.zig
@@ -113,46 +113,51 @@ pub fn stringifyArbitraryDepth(val: anytype, comptime options: StringifyOptions,
 
 const RecursiveTypeBuffer = [32]type;
 
-fn typeIsRecursive(comptime T: type) bool {
-    comptime var buf: RecursiveTypeBuffer = undefined;
-    return typeIsRecursiveImpl(T, buf[0..0]);
+inline fn typeIsRecursive(comptime T: type) bool {
+    comptime {
+        var buf: RecursiveTypeBuffer = undefined;
+        const result = typeIsRecursiveImpl(T, buf[0..0]);
+        return result;
+    }
 }
 
-fn typeIsRecursiveImpl(comptime T: type, comptime visited_arg: []type) bool {
-    comptime var visited = visited_arg;
-
-    // Check if we've already seen this type
-    inline for (visited) |found| {
-        if (T == found) {
-            return true;
-        }
-    }
+inline fn typeIsRecursiveImpl(comptime T: type, comptime visited_arg: []type) bool {
+    comptime {
+        var visited = visited_arg;
 
-    // Add this type to the stack
-    if (visited.len >= @typeInfo(RecursiveTypeBuffer).Array.len) {
-        @compileError("recursion limit");
-    }
-    visited.ptr[visited.len] = T;
-    visited.len += 1;
-
-    // Recurse
-    switch (@typeInfo(T)) {
-        .Pointer => |Pointer| return typeIsRecursiveImpl(Pointer.child, visited),
-        .Array => |Array| return typeIsRecursiveImpl(Array.child, visited),
-        .Struct => |Struct| inline for (Struct.fields) |field| {
-            if (typeIsRecursiveImpl(field.type, visited)) {
-                return true;
-            }
-        },
-        .Union => |Union| inline for (Union.fields) |field| {
-            if (typeIsRecursiveImpl(field.type, visited)) {
+        // Check if we've already seen this type
+        for (visited) |found| {
+            if (T == found) {
                 return true;
             }
-        },
-        .Optional => |Optional| return typeIsRecursiveImpl(Optional.child, visited),
-        else => {},
+        }
+
+        // Add this type to the stack
+        if (visited.len >= @typeInfo(RecursiveTypeBuffer).Array.len) {
+            @compileError("recursion limit");
+        }
+        visited.ptr[visited.len] = T;
+        visited.len += 1;
+
+        // Recurse
+        switch (@typeInfo(T)) {
+            .Pointer => |Pointer| return typeIsRecursiveImpl(Pointer.child, visited),
+            .Array => |Array| return typeIsRecursiveImpl(Array.child, visited),
+            .Struct => |Struct| for (Struct.fields) |field| {
+                if (typeIsRecursiveImpl(field.type, visited)) {
+                    return true;
+                }
+            },
+            .Union => |Union| for (Union.fields) |field| {
+                if (typeIsRecursiveImpl(field.type, visited)) {
+                    return true;
+                }
+            },
+            .Optional => |Optional| return typeIsRecursiveImpl(Optional.child, visited),
+            else => {},
+        }
+        return false;
     }
-    return false;
 }
 
 test "typeIsRecursive" {
@@ -178,7 +183,7 @@ test "typeIsRecursive" {
     }));
 }

[MasonRemaley]

Alright launch crazyness has died down & this is my priority now. Have a local version of the branch rebased to use the latest zig, will push once it compiles and the tests pass leaving this one here for now because GitHub's diff view of before the rebase is a convenient reference for me for now.

[MasonRemaley]

Rebased and got all the tests passing. The PR isn't done yet--lots of TODOs in the code and I need to go through my notes to see if I left anything unfinished--but should have something soon!

Changes are in my fork, they don't show up in this PR for some reason (maybe because it's closed.) Will fix it or make a new PR and link it here when it's ready.

[MasonRemaley]

Alright, made a new PR for this! :)

#20271