-
-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ZON #20271
base: master
Are you sure you want to change the base?
ZON #20271
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Regarding &
in ZON...
You're correct that ZON definitely shouldn't include this. Our main use of ZON today (build.zig.zon
) already creates variable-length structures without this syntax. To avoid a divergence between ZON you can @import
and build.zig.zon
, I would consider this a merge blocker, but that's Andrew's call of course.
Implementation-wise, the ZIR instruction corresponding to @import
should be modified to provide a result type if one is present (you can use .none
to represent the case where there isn't one). Then, the ZON analysis logic should traverse into this result type as needed. This has the additional advantage of providing better source locations if the structure of an imported file doesn't match what's expected, since the type error will occur whilst parsing the relevant ZON expression.
Hmm I think I agree, better to knock it out now than to release it as is then immediately change the syntax. I would've done this up front but incorrectly thought it required major changes to the compiler. I'll take a look at making this change. If no result type is specified, I'll have it default to a tuple rather than a slice unless there are any objections to that default. |
Yup. I agree that it makes sense to use a tuple when there's no result type. By the way, in case you're unfamiliar with the |
There were two primary issues at play here: 1. The hex float prefix was not handled correctly when the stream was reset for the fallback parsing path, which occured when the mantissa was longer max mantissa digits. 2. The implied exponent was not adjusted for hex-floats in this branch. Additionally, some of the float parsing routines have been condensed, making use of comptime. closes ziglang#20271
There were two primary issues at play here: 1. The hex float prefix was not handled correctly when the stream was reset for the fallback parsing path, which occured when the mantissa was longer max mantissa digits. 2. The implied exponent was not adjusted for hex-floats in this branch. Additionally, some of the float parsing routines have been condensed, making use of comptime. closes ziglang#20271
In case you haven't found why this is happening, it's here: zig/ci/x86_64-linux-release.sh Lines 60 to 66 in 17f14e1
Edit: I'll address this in a PR shortly. Sit tight for a bit. |
After #20321 you should see the same failures locally when running Edit: I think if you rebase and resolve that build.zig conflict, those CI failures will disappear. |
Nice thanks! I'll do the rebase and get started on removing & from the syntax when I'm back in town next week. |
0358ab4
to
ce32047
Compare
Thanks, I'm not familiar with this code so this is helpful! I started implementing storing the result type on imports, but got a bit stuck. AstGen used to use // AstGen.builtinCall
const operand_node = params[0];
const rhs = try expr(gz, scope, .{ .rl = .none }, operand_node);
try gz.addPlNode(.import, node, Zir.Inst.Bin{
.lhs = result_type,
.rhs = rhs,
}); Zir used to use .str_tok to get the import path, that's also been updated: // Zir.zirImport
const inst_data = sema.code.instructions.items(.data)[@intFromEnum(inst)].pl_node;
const extra = sema.code.extraData(Zir.Inst.Bin, inst_data.payload_index).data;
const src = block.nodeOffset(inst_data.src_node);
const operand = try sema.resolveConstString(
block,
src,
extra.rhs,
.{ .needed_comptime_reason = "import path must be comptime-known" },
); When this code runs,
It appears that the value I get back in Sema for Any pointers would be much appreciated--I can also push the WIP commit if that would be helpful, but I imagine that I'm probably missing something that will be obvious to people more familiar with this part of the codebase. |
You probably just need to clear your cache directory -- ZIR is cached based on However, your lowering strategy for |
Ha that was it, thanks! I made the same mistake yesterday and didn't quite understand how I fixed it--that had been bugging me glad to have it resolved.
Makes sense, will do! |
This was a bit odd, because it meant that only the coercion really needed to be checked, which isn't what I was intending to test.
Will give API more thought in followup
…s resolving types fully
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This just covers std.zon.parse
, not std.zon.stringify
. I also didn't look over the tests yet, although I'm sure they're fine.
/// The type to deserialize into. May only transitively contain the following supported types: | ||
/// * bools | ||
/// * fixed sized numeric types | ||
/// * enums | ||
/// * slices | ||
/// * arrays | ||
/// * structures | ||
/// * unions | ||
/// * optionals | ||
/// * null |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After the changes we discussed to what types are accepted, it's more easy to say what's not allowed:
/// The type to deserialize into. May only transitively contain the following supported types: | |
/// * bools | |
/// * fixed sized numeric types | |
/// * enums | |
/// * slices | |
/// * arrays | |
/// * structures | |
/// * unions | |
/// * optionals | |
/// * null | |
/// The type to deserialize into. | |
/// May not be or contain any of the following types: | |
/// * Any comptime-only type | |
/// * `void`, except as a union payload | |
/// * `noreturn` | |
/// * An error set/union | |
/// * An opaque type, including `anyopaque` | |
/// * An async frame type, including `anyframe` and `anyframe->T` | |
/// All other types are valid. |
if (status) |s| s.zoir = zoir; | ||
if (zoir.hasCompileErrors()) return error.ParseZon; | ||
|
||
if (status) |s| s.* = .{}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Surely this leaks zoir
, since it won't be deinit
ed by the defer
and this line removes what would have been its new owner upon return?
EDIT: ah, I see, fromZoir
is guaranteed to re-populate it. Perhaps just comment that here for clarity.
var zoir = try ZonGen.generate(gpa, ast, .{ .parse_str_lits = false }); | ||
defer if (status == null) zoir.deinit(gpa); | ||
if (status) |s| s.zoir = zoir; | ||
if (zoir.hasCompileErrors()) return error.ParseZon; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since we call fromZoir
, isn't this redundant? Just let fromZoir
return the error.
} | ||
} | ||
|
||
fn parsePointer( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just throwing a comment in to track: remember to apply the type changes we discussed to std.zon
too. (For instance, allowing initializing data through pointers.)
if (pointer.child != u8 or | ||
pointer.size != .slice or | ||
!pointer.is_const or | ||
(pointer.sentinel_ptr != null and @as(*const u8, @ptrCast(pointer.sentinel_ptr)).* != 0) or |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(pointer.sentinel_ptr != null and @as(*const u8, @ptrCast(pointer.sentinel_ptr)).* != 0) or | |
(pointer.sentinel() != null and pointer.sentinel() != 0) or |
}); | ||
} | ||
}, | ||
else => @compileError("unreachable, should not be called for type " ++ @typeName(T)), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
else => @compileError("unreachable, should not be called for type " ++ @typeName(T)), | |
else => comptime unreachable, |
}, | ||
else => {}, | ||
} | ||
@compileError("unreachable, should not be called for type " ++ @typeName(T)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@compileError("unreachable, should not be called for type " ++ @typeName(T)); | |
comptime unreachable; |
const value_node = array_init.ast.elements[field]; | ||
break :b self.ast.firstToken(value_node); | ||
}; | ||
return self.failTokenFmt(token, 0, "cannot store runtime value in compile time variable", .{}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's sort of a weird error here, because this is about runtime fields. How about: "cannot initialize comptime field of 'Foo'"
? Or something along those lines.
switch (@typeInfo(@TypeOf(value))) { | ||
.float => {}, | ||
else => @compileError(@typeName(@TypeOf(value)) ++ " is not a runtime floating point type"), | ||
} | ||
switch (@typeInfo(T)) { | ||
.int => {}, | ||
else => @compileError(@typeName(T) ++ " is not a runtime integer type"), | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
switch (@typeInfo(@TypeOf(value))) { | |
.float => {}, | |
else => @compileError(@typeName(@TypeOf(value)) ++ " is not a runtime floating point type"), | |
} | |
switch (@typeInfo(T)) { | |
.int => {}, | |
else => @compileError(@typeName(T) ++ " is not a runtime integer type"), | |
} |
These assertions are already done thanks to the @intFromFloat
call.
return null; | ||
} | ||
|
||
return @as(T, @intFromFloat(value)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
return @as(T, @intFromFloat(value)); | |
return @intFromFloat(value); |
RLS propagates this type information automatically.
ZON
This PR implements ZON, or Zig Object Notation (think JSON, but with Zig syntax instead of Javascript.)
In particular, it implements:
A lot of files are added since I added a lot of test cases, the bulk of the work is in
src/zon.zig
andlib/std/zon
. If there's anything I can do to make this easier to review, let me know.Tests cover all new features.
Runtime
The runtime code can be found at
lib/std/zon
.Parsing
lib/std/zon/parse.zig
Detailed doc comments are provided in the source. At a high level, this module provides the following functions and some configuration options:
std.zon.parse.fromSlice
std.zon.parse.fromZoir
std.zon.parse.fromZoirNode
std.zon.parse.free
Most people will just want
std.zon.parse.fromSlice
and maybestd.zon.parse.free
, but the more fine grained functions are available for those who need them.Stringify
lib/std/zon/stringify.zig
Detailed doc comments are provided in the source. At a high level, this module provides the following functions and some configuration options:
std.zon.stringify.serialize
std.zon.stringify.serializeMaxDepth
std.zon.stringify.serializeArbitraryDepth
std.zon.stringify.serializer
Most users will just need
serialize
, or one of its variants.However,
std.zon.stringify.serializer
returns a more fine grained interface that is used in the implementation of the other functions. It may be used directly if you need to serialize something piece by piece--perhaps to apply different configuration to different parts of the value, or because the value does not actually exist laid out in memory in the same form in which you want to stringify it.Compile Time
See
src/zon.zig
.This PR implements importing ZON at compile time:
Things that may change later...
Untagged nonexhaustive enum values not yet supported
Zig does not currently have a way to represent an untagged enum value as a literal, and as such there's no way to represent one in ZON.
We could resolve this by adding a syntax for untagged enum literals to Zig, or by allowing ZON to coerce integers to untagged enum literals (provided the destination type is known.) Personally I prefer the former solution.
Allowing eliding outer braces may make sense
There's case to be made to allow eliding braces on an outer struct in ZON, like this:
Divergences from Zig
Zig has no NaN or inf literals right now. Zon does, since otherwise it would have no way to represent these values.
IMO we should add the same literals to Zig, but there are good arguments not to as well, so I'm not trying to address that here.
See Also
@import
.zon files #14531\x00
(null bytes) and empty string in@""
identifier syntax in the language specification #14534.
in tuples & anonymous struct literal syntax #5039