Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
170 changes: 116 additions & 54 deletions JsonHigh.d.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,36 @@
/**
* `JsonHigh` is called with an object which contains named event handlers that are invoked during parsing.
*
* The object may also contain [options](#options) to change the default behavior, e.g. [turn off number parsing](#parseNumbers) or turn on [buffering of strings piece-by-piece](#maxStringBufferLength).
*/
export declare const JsonHigh: JsonHigh

/**
* This is the high-level interface over {@link JsonLow} optimized for convenience. It provides similar functionality and granularity to other streaming parsers, such as [Clarinet](https://github.com/dscape/clarinet) or [creationix/jsonparse](https://github.com/creationix/jsonparse).
*/
export type JsonHigh = <Feedback, End>(
next: JsonHighHandlers<Feedback, End>,
) => {
chunk(chunk: string): Feedback,
) => JsonHighStream<End>

/**
* ### Return value
*
* {@link JsonHigh} returns a stream object with the following methods:
*/
export type JsonHighStream<End> = {
/**
* Accepts a JSON chunk to parse.
*
* Returns the stream object for chaining.
*
*/
chunk(chunk: string): JsonHighStream<End>,

/**
* Call it to signal to the stream that the current JSON document is finished.
*
* If there is no error, it will then call the corresponding `end()` event handler, and return whatever that returned.
*/
end(): End,

/**
Expand All @@ -17,57 +45,103 @@ export type JsonHigh = <Feedback, End>(
depth(): number,
// todo: type for state()
}
export type JsonHighOptions = {
/**
* ### Options
*/

/**
* ### Options
*/
export type JsonHighOptions = {
/**
* @defaultValue `Infinity`
* See also: the similar {@link JsonHighOptions.bufferOnChunk} option. These options can be combined. Differences between them are marked with a strong font.
*
* The maximum length of the key or string buffer, in code points.
*
* If set, the `key` event handler won't be called and, for strings, the `value` event handler won't be called.
* If **set to a value in range `[1;Infinity)`**, the `key` event handler won't be called and, for strings, the `value` event handler won't be called.
*
* Instead, the `bufferKey` and `bufferString` handlers will be called as soon as the given number of code points has been collected or when the key/string is finished.
* Instead, the `bufferKey` and `bufferString` handlers will be called **as soon as the given number of code points has been collected** or when the key/string is finished.
*
* In the latter case the number of code points may be smaller than `maxStringBufferLength` and a `closeKey`/`closeString` handler will be called after the last `*Buffer` event to signal the finish.
* In the latter case **the number of code points may be smaller than `maxStringBufferLength`** and a `closeKey`/`closeString` handler will be called after the last `buffer*` event to signal the finish.
*
* This is useful when dealing with long strings where it's desirable to stream them piece-by-piece, e.g. when working with LLMs (Large Language Models).
* This is useful **when dealing with long strings** where it's desirable to stream them piece-by-piece, e.g. when working with LLMs (Large Language Models).
*
* See [Add support for incomplete key and value strings #10](https://github.com/xtao-org/jsonhilo/issues/10) for more information.
*
* @default {Infinity}
*/
maxStringBufferLength?: number,

/**
* @defaultValue `8192`
* See also: the similar {@link JsonHighOptions.maxStringBufferLength} option. These options can be combined. Differences between them are marked with a strong font.
*
* If **set to `true`**, the `key` event handler won't be called and, for strings, the `value` event handler won't be called.
*
* Instead, the `bufferKey` and `bufferString` handlers will be called **as soon as the current chunk (passed in via the `.chunk()` method of the stream) has been processed** or when the key/string is finished.
*
* In the latter case a `closeKey`/`closeString` handler will be called after the last `buffer*` event to signal the finish.
*
* This is useful **when it's desirable to stream strings or keys piece-by-piece, in sync with the chunks being received, without needing to specify a fixed buffer length;** e.g. when working with LLMs (Large Language Models).
*
* See [Add support for incomplete key and value strings #10](https://github.com/xtao-org/jsonhilo/issues/10) for more information.
*
* @default {false}
*/
bufferOnChunk?: boolean,

/**
* Specifies the maximum length of a number value (in characters).
*
* @default {8192}
*/
maxNumberLength?: number,

/**
* @defaultValue `true`
*
* Controls whether numbers should be parsed (converted to JavaScript `number` type) which is the case by default.
*
* If set to `false`, the `value` event handler won't be called for numbers. Instead, the `bufferNumber` handler will be called with the number as a string.
*
* This is useful when dealing with big numbers which would lose precision when converted to the `number` type.
*
* @default {true}
*/
parseNumbers?: boolean,
}
export type JsonHighHandlers<Feedback, End> = JsonHighOptions & {

/**
* ### Basic events
*
* The basic usage of `JsonHigh` involves 4 event handlers without arguments which indicate start and end of structures:
*/

/** ### Basic events
*
* The basic usage of `JsonHigh` involves 4 event handlers without arguments which indicate start and end of structures:
* * {@link JsonHighEvents.openArray}
* * {@link JsonHighEvents.closeArray}
* * {@link JsonHighEvents.openObject}
* * {@link JsonHighEvents.closeObject}
*
* And 2 event handlers with one argument which capture primitives:
*
* * {@link JsonHighEvents.key}
* * {@link JsonHighEvents.value}
*
* Finally, there is the argumentless {@link JsonHighEvents.end} event handler.
*
* ### Extra events
*
* These handlers take no arguments:
*
* * {@link JsonHighEvents.openKey}
* * {@link JsonHighEvents.openString}
* * {@link JsonHighEvents.openNumber}
*
* ### Conditional events
*
* These handlers take no arguments:
*
* * {@link JsonHighEvents.closeKey}
* * {@link JsonHighEvents.closeString}
*
* These handlers receive the buffer that should be consumed:
*
* * {@link JsonHighEvents.bufferKey}
* * {@link JsonHighEvents.bufferString}
* * {@link JsonHighEvents.bufferNumber}
*/
export type JsonHighEvents<Feedback, End> = {
/**
* An array started (`[`).
*/
Expand All @@ -88,16 +162,14 @@ export type JsonHighHandlers<Feedback, End> = JsonHighOptions & {
*/
closeObject?: JsonHighHandler<Feedback>,

/**
* And 2 event handlers with one argument which capture primitives:
*/

/**
* An object's key ended.
*
* The argument of the handler contains the key as a JavaScript string.
*
* This event can be suppressed by setting [`maxStringBufferLength`](#maxStringBufferLength).
* `[conditional]` Called only in default (non-buffering) mode.
*
* This event can be suppressed by setting {@link JsonHighOptions.maxStringBufferLength}.
*/
key?: (key: string) => Feedback,

Expand All @@ -106,25 +178,16 @@ export type JsonHighHandlers<Feedback, End> = JsonHighOptions & {
*
* The argument of the event contains the corresponding JavaScript value: `true`, `false`, `null`, a number, or a string.
*
* This event can be suppressed for strings if [`maxStringBufferLength`](#maxStringBufferLength) is set and for numbers if [`parseNumbers`](#parseNumbers) is set to `false`.
* This event can be suppressed for strings if {@link JsonHighOptions.maxStringBufferLength} is set and for numbers if {@link JsonHighOptions.parseNumbers} is set to `false`.
*
*/
value?: (value: string | number | null | boolean) => Feedback,

/**
* Finally, there is the argumentless `end` event handler:
*/

/**
* Called by the `end` method of the stream to confirm that the parsed JSON document is complete and valid.
*/
end?: () => End,

/**
* ### Extra events
*
* These handlers take no arguments.
*/

/**
* A key started (`"`, in key position).
*/
Expand All @@ -140,34 +203,26 @@ export type JsonHighHandlers<Feedback, End> = JsonHighOptions & {
*/
openNumber?: JsonHighHandler<Feedback>,

/**
* ### Conditional events
*
* These handlers take no arguments:
*/

/**
* A key ended (`"`).
*
* Called instead of the `key` event **when `maxStringBufferLength` is set**.
* `[conditional]` Called instead of the `key` event **when `maxStringBufferLength` is set**.
*/
closeKey?: JsonHighHandler<Feedback>,

/**
/**
* A string value ended (`"`).
*
* Called instead of the `value` event **when `maxStringBufferLength` is set**.
* `[conditional]` Called instead of the `value` event **when `maxStringBufferLength` is set**.
*
* @tags conditional
*/
closeString?: JsonHighHandler<Feedback>,

/**
* These handlers receive the buffer that should be consumed:
*/


/**
* Key buffer is ready for consumption.
*
* Called instead of the `key` event **when `maxStringBufferLength` is set**.
* `[conditional]` Called instead of the `key` event **when `maxStringBufferLength` is set**.
*
* The `buffer` then contains `maxStringBufferLength` code points or possibly less if we reached the end of a key.
*/
Expand All @@ -176,7 +231,7 @@ export type JsonHighHandlers<Feedback, End> = JsonHighOptions & {
/**
* String buffer is ready for consumption.
*
* For string values, called instead of the `value` event **when `maxStringBufferLength` is set**.
* `[conditional]` For string values, called instead of the `value` event **when `maxStringBufferLength` is set**.
*
* The buffer then contains `maxStringBufferLength` code points or possibly less if we reached the end of a string value.
*/
Expand All @@ -185,10 +240,17 @@ export type JsonHighHandlers<Feedback, End> = JsonHighOptions & {
/**
* Number buffer is ready for consumption.
*
* For number values, called instead of the `value` event **when `parseNumbers` is set**.
* `[conditional]` For number values, called instead of the `value` event **when `parseNumbers` is set**.
*
* The buffer then contains the unparsed number (represented as a string).
*/
bufferNumber?: (buffer: string) => Feedback,
}

/**
* TODO: this should be called something like JsonHighParameters or JsonHighArguments or sth.
*/
export type JsonHighHandlers<Feedback, End> = JsonHighOptions & JsonHighEvents<Feedback, End>

/** TODO: doc */
export type JsonHighHandler<Feedback> = () => Feedback
96 changes: 96 additions & 0 deletions docs/all_symbols.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
<!DOCTYPE html>
<html>
<head>
<title>All Symbols - documentation</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="doc-current-file" content="">
<link rel="stylesheet" href=".&#x2F;styles.css">
<link rel="stylesheet" href=".&#x2F;page.css">
<link id="ddocResetStylesheet" rel="stylesheet" href=".&#x2F;reset.css"><link href="./comrak.css" rel="stylesheet" /><link href="./prism.css" rel="stylesheet" /><script src="./prism.js"></script><script src=".&#x2F;search_index.js" defer></script>
<script src=".&#x2F;script.js" defer></script>
<script src=".&#x2F;fuse.js" defer></script>
<script src=".&#x2F;search.js" defer></script></head>
<body>
<div class="ddoc">
<div id="categoryPanel">
<ul><li><a href=".&#x2F;.&#x2F;uncategorized.html" title="Uncategorized">Uncategorized</a></li><li>
<a class="!flex items-center gap-0.5" href=".&#x2F;.&#x2F;all_symbols.html">
<span class="leading-none">view all 6 symbols</span><svg
width="16"
height="16"
viewBox="0 0 16 16"
fill="none"
xmlns="http://www.w3.org/2000/svg">
<path fill-rule="evenodd" clip-rule="evenodd" d="M5.76748 11.8159C5.5378 11.577 5.54525 11.1972 5.78411 10.9675L8.93431 8L5.78411 5.0325C5.54525 4.80282 5.5378 4.423 5.76748 4.18413C5.99715 3.94527 6.37698 3.93782 6.61584 4.1675L10.2158 7.5675C10.3335 7.68062 10.4 7.83679 10.4 8C10.4 8.16321 10.3335 8.31938 10.2158 8.4325L6.61584 11.8325C6.37698 12.0622 5.99715 12.0547 5.76748 11.8159Z" fill="currentColor" />
</svg>
</a>
</li>
</ul>
</div>
<div><nav class="top-0 sticky bg-white z-50 py-3 h-14" id="topnav">
<div class="h-full">
<div><ul class="breadcrumbs"><li><a href=".&#x2F;" class="contextLink">index</a></li><span class="text-[#0F172A]"><svg
width="16"
height="16"
viewBox="0 0 16 16"
fill="none"
xmlns="http://www.w3.org/2000/svg">
<path fill-rule="evenodd" clip-rule="evenodd" d="M5.76748 11.8159C5.5378 11.577 5.54525 11.1972 5.78411 10.9675L8.93431 8L5.78411 5.0325C5.54525 4.80282 5.5378 4.423 5.76748 4.18413C5.99715 3.94527 6.37698 3.93782 6.61584 4.1675L10.2158 7.5675C10.3335 7.68062 10.4 7.83679 10.4 8C10.4 8.16321 10.3335 8.31938 10.2158 8.4325L6.61584 11.8325C6.37698 12.0622 5.99715 12.0547 5.76748 11.8159Z" fill="currentColor" />
</svg>
</span><li>all symbols</li></ul>
</div>

<input
type="text"
id="searchbar"
style="display: none;"
class="py-2 px-2.5 rounded text-sm border border-gray-300 bg-transparent" />
</div>
</nav>
<div id="searchResults"></div><div id="content">
<main><div class="space-y-7" id=""><section class="section" ><div class="namespaceSection"><div id="namespace_jsonhigh" class="namespaceItem" ><div class="docNodeKindIcon"><div class="text-TypeAlias bg-TypeAlias/15" title="Type Alias">T</div><div class="text-Variable bg-Variable/15" title="Variable">v</div></div>
<div class="namespaceItemContent">
<a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHigh.html" title="JsonHigh">JsonHigh</a>

<div class="namespaceItemContentDoc"><div class="markdown_summary"><p>This is the high-level interface over JsonLow optimized for convenience. It provides similar functionality and granularity to other streaming parsers, such as <a href="https://github.com/dscape/clarinet">Clarinet</a> or <a href="https://github.com/creationix/jsonparse">creationix/jsonparse</a>.</p>
</div></div></div>
</div><div id="namespace_jsonhighevents" class="namespaceItem" ><div class="docNodeKindIcon"><div class="text-TypeAlias bg-TypeAlias/15" title="Type Alias">T</div></div>
<div class="namespaceItemContent">
<a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighEvents.html" title="JsonHighEvents">JsonHighEvents</a>

<div class="namespaceItemContentDoc"><div class="markdown_summary"><h3>Basic events</h3>
</div></div><ul class="namespaceItemContentSubItems"><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighEvents.html#property_bufferkey">bufferKey</a></li><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighEvents.html#property_buffernumber">bufferNumber</a></li><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighEvents.html#property_bufferstring">bufferString</a></li><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighEvents.html#property_closearray">closeArray</a></li><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighEvents.html#property_closekey">closeKey</a></li><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighEvents.html#property_closeobject">closeObject</a></li><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighEvents.html#property_closestring">closeString</a></li><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighEvents.html#property_end">end</a></li><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighEvents.html#property_key">key</a></li><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighEvents.html#property_openarray">openArray</a></li><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighEvents.html#property_openkey">openKey</a></li><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighEvents.html#property_opennumber">openNumber</a></li><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighEvents.html#property_openobject">openObject</a></li><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighEvents.html#property_openstring">openString</a></li><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighEvents.html#property_value">value</a></li></ul></div>
</div><div id="namespace_jsonhighhandler" class="namespaceItem" ><div class="docNodeKindIcon"><div class="text-TypeAlias bg-TypeAlias/15" title="Type Alias">T</div></div>
<div class="namespaceItemContent">
<a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighHandler.html" title="JsonHighHandler">JsonHighHandler</a>

<div class="namespaceItemContentDoc"><div class="markdown_summary"><p>TODO: doc</p>
</div></div></div>
</div><div id="namespace_jsonhighhandlers" class="namespaceItem" ><div class="docNodeKindIcon"><div class="text-TypeAlias bg-TypeAlias/15" title="Type Alias">T</div></div>
<div class="namespaceItemContent">
<a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighHandlers.html" title="JsonHighHandlers">JsonHighHandlers</a>

<div class="namespaceItemContentDoc"><div class="markdown_summary"><p>TODO: this should be called something like JsonHighParameters or JsonHighArguments or sth.</p>
</div></div></div>
</div><div id="namespace_jsonhighoptions" class="namespaceItem" ><div class="docNodeKindIcon"><div class="text-TypeAlias bg-TypeAlias/15" title="Type Alias">T</div></div>
<div class="namespaceItemContent">
<a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighOptions.html" title="JsonHighOptions">JsonHighOptions</a>

<div class="namespaceItemContentDoc"><div class="markdown_summary"><h3>Options</h3>
</div></div><ul class="namespaceItemContentSubItems"><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighOptions.html#property_bufferonchunk">bufferOnChunk</a></li><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighOptions.html#property_maxnumberlength">maxNumberLength</a></li><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighOptions.html#property_maxstringbufferlength">maxStringBufferLength</a></li><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighOptions.html#property_parsenumbers">parseNumbers</a></li></ul></div>
</div><div id="namespace_jsonhighstream" class="namespaceItem" ><div class="docNodeKindIcon"><div class="text-TypeAlias bg-TypeAlias/15" title="Type Alias">T</div></div>
<div class="namespaceItemContent">
<a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighStream.html" title="JsonHighStream">JsonHighStream</a>

<div class="namespaceItemContentDoc"><div class="markdown_summary"><h3>Return value</h3>
</div></div><ul class="namespaceItemContentSubItems"><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighStream.html#method_chunk_0">chunk</a></li><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighStream.html#method_depth_0">depth</a></li><li><a href=".&#x2F;.&#x2F;.&#x2F;~&#x2F;JsonHighStream.html#method_end_0">end</a></li></ul></div>
</div></div>
</section>
</div>
</main>
</div>
</div>
</div>
</body>
</html>
Loading