Skip to content

Commit c27e3e2

Browse files
authored
Merge #26 feat!: listitem (line_li)
2 parents c418f88 + a1e606f commit c27e3e2

16 files changed

+5094
-3697
lines changed

README.md

+22-17
Original file line numberDiff line numberDiff line change
@@ -1,32 +1,33 @@
11
tree-sitter-vimdoc
22
==================
33

4-
This grammar intentionally support a subset of the vimdoc "spec"; predictable
5-
results are the primary goal, so that _output_ formats (e.g. HTML) are
6-
well-formed; the _input_ (vimdoc) is secondary. The first step should always be
7-
to try to fix the input (within reason) rather than insist on a grammar that
8-
handles vimdoc's endless quirks.
4+
This grammar intentionally support a subset of the vimdoc "spec"
5+
([ref1](https://neovim.io/doc/user/helphelp.html#help-writing),
6+
[ref2](https://github.com/nanotee/vimdoc-notes));
7+
predictable results are the primary goal, so that _output_ formats (e.g. HTML)
8+
are well-formed; the _input_ (vimdoc) is secondary. The first step should
9+
always be to try to fix the input rather than insist on a grammar that handles
10+
vimdoc's endless quirks.
911

1012
Overview
1113
--------
1214

13-
- vimdoc format "spec":
14-
- [:help help-writing](https://neovim.io/doc/user/helphelp.html#help-writing)
15-
- https://github.com/nanotee/vimdoc-notes
16-
- whitespace is intentionally captured in all atoms, because it is often used
17-
for "layout" and ascii art in legacy help files.
18-
- `block` is the main top-level node which contains `line` nodes.
19-
- ends at blank line(s) or a line starting with `<`.
15+
- `block` is the main top-level node which contains `line` and `line_li` nodes.
16+
- delimited by blank line(s) or any line starting with `<` (codeblock terminator).
2017
- `line`:
2118
- contains atoms (words, tags, taglinks, …)
22-
- contains `codeblock` because `>` can start a codeblock at the end of a line.
2319
- contains headings (`h1`, `h2`, `h3`) because `codeblock` terminated by
2420
"implicit stop" (no terminating `<`) consumes blank lines, so `block` has
2521
no way to end.
2622
- contains `column_heading` because `<` (the `codeblock` terminating char)
2723
can appear at the start of `column_heading`.
24+
- `line_li` ("list item")
25+
- consumes lines until blank line, codeblock, or next listitem.
26+
- nesting is ignored: indented listitems are parsed as siblings.
2827
- `codeblock`:
29-
- contains `line` nodes which do not contain `word` nodes, it's just the full
28+
- contained by `line` or `line_li`. Because ">" can start
29+
a codeblock at the end of any line.
30+
- contains `line` nodes without `word` nodes, it's just the full
3031
raw text line including whitespace. This is somewhat dictated by its
3132
"preformatted" nature; parsing the contents would require loading a "child"
3233
language (injection). See [#2](https://github.com/neovim/tree-sitter-vimdoc/issues/2).
@@ -38,16 +39,20 @@ Overview
3839
Known issues
3940
------------
4041

41-
- `line_li` ("list item") is experimental. It doesn't support nesting yet.
42+
- Input must end with newline/EOL (`\n`). Grammar does not support files without EOL.
43+
- Input must end with a blank line. Though this doesn't seem to matter in practice.
4244
- Spec requires that `codeblock` delimiter ">" must be preceded by a space
4345
(" >"), not a tab. But currently the grammar doesn't enforce this. Example:
4446
`:help lcs-tab`.
4547
- `url` doesn't handle _surrounding_ parens. E.g. `(https://example.com/#yay)` yields `word`
4648
- `url` doesn't handle _nested_ parens. E.g. `(https://example.com/(foo)#yay)`
49+
- `column_heading` currently only recognizes tilde "~" preceded by space (i.e.
50+
"foo ~" not "foo~"). This covers 99% of :help files, but the grammar should
51+
probably support "foo~" also.
4752

4853
TODO
4954
----
5055

51-
- `line_noeol` is a special-case to support documents that don't end in EOL.
52-
Grammar could be simpler if we require EOL at end of document.
5356
- `line_modeline` ?
57+
- `tag_heading` : line(s) containing only tags, typically implies a "heading"
58+
before a block.

corpus/arguments.txt

+20-3
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22
simple argument
33
================================================================================
44
This in an argument: {arg}
5+
6+
57
--------------------------------------------------------------------------------
68

79
(help_file
@@ -19,6 +21,7 @@ multiple arguments on the same line
1921
================================================================================
2022
{foo} {bar} {baz}
2123

24+
2225
--------------------------------------------------------------------------------
2326

2427
(help_file
@@ -31,11 +34,28 @@ multiple arguments on the same line
3134
(argument
3235
(word)))))
3336

37+
================================================================================
38+
argument in parentheses
39+
================================================================================
40+
({aaa})
41+
42+
43+
--------------------------------------------------------------------------------
44+
45+
(help_file
46+
(block
47+
(line
48+
(word)
49+
(argument
50+
(word))
51+
(word))))
52+
3453
================================================================================
3554
NOT an argument
3655
================================================================================
3756
{foo "{bar}" `{baz}` |{baz| } {}
3857

58+
3959
--------------------------------------------------------------------------------
4060

4161
(help_file
@@ -45,9 +65,6 @@ NOT an argument
4565
(word)
4666
(MISSING "}"))
4767
(word)
48-
(argument
49-
(word))
50-
(word)
5168
(codespan
5269
(word))
5370
(taglink

corpus/codeblock.txt

+10-3
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ block3:
1616
<
1717

1818
text
19+
20+
1921
--------------------------------------------------------------------------------
2022

2123
(help_file
@@ -62,6 +64,7 @@ From luaref.txt:
6264

6365
text
6466

67+
6568
--------------------------------------------------------------------------------
6669

6770
(help_file
@@ -79,6 +82,7 @@ text
7982
(word)))
8083
(block
8184
(line
85+
(word)
8286
(word)
8387
(taglink
8488
(word))
@@ -116,6 +120,7 @@ h2-headline *foo*
116120

117121
H3 HEADLINE *foo*
118122

123+
119124
--------------------------------------------------------------------------------
120125

121126
(help_file
@@ -162,8 +167,8 @@ codeblock with empty lines
162167

163168
Example: >
164169

165-
let buf = nvim_create_buf(v:false, v:true)
166-
call nvim_buf_set_lines(buf, 0, -1, v:true, ["test", "text"])
170+
- let buf = nvim_create_buf(v:false, v:true)
171+
- call nvim_buf_set_lines(buf, 0, -1, v:true, ["test", "text"])
167172

168173
let opts = {'relative': 'cursor', 'width': 10, 'height': 2, 'col': 0,
169174
\ 'row': 1, 'anchor': 'NW', 'style': 'minimal'}
@@ -172,6 +177,7 @@ Example: >
172177
call nvim_win_set_option(win, 'winhl', 'Normal:MyHighlight')
173178
x
174179

180+
175181
--------------------------------------------------------------------------------
176182

177183
(help_file
@@ -211,6 +217,7 @@ tricky codeblock
211217

212218
tricky
213219

220+
214221
--------------------------------------------------------------------------------
215222

216223
(help_file
@@ -237,7 +244,6 @@ tricky
237244
================================================================================
238245
strange codeblock
239246
================================================================================
240-
241247
Note that in the command: >
242248
:if "foo"
243249
:" NOT executed
@@ -295,6 +301,7 @@ To test for a non-empty string, use empty(): >
295301
(word)
296302
(word)
297303
(word)
304+
(word)
298305
(codeblock
299306
(line)))))
300307

corpus/codespan.txt

+4-5
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ simple codespan
44

55
a `foobar` b `:echo`
66

7+
78
--------------------------------------------------------------------------------
89

910
(help_file
@@ -23,6 +24,7 @@ codespan in text
2324
Hello `world`, I am `markup language`. But `this is
2425
an error`.
2526

27+
2628
--------------------------------------------------------------------------------
2729

2830
(help_file
@@ -52,6 +54,7 @@ NOT codespan
5254
*g'* *g'a* *g`* *g`a*
5355
g'{mark} g`{mark}
5456

57+
5558
--------------------------------------------------------------------------------
5659

5760
(help_file
@@ -76,8 +79,4 @@ g'{mark} g`{mark}
7679
(word)))
7780
(line
7881
(word)
79-
(argument
80-
(word))
81-
(word)
82-
(argument
83-
(word)))))
82+
(word))))

corpus/heading1_2.txt

+3
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ Heading two *tag2.1* *tag2.2*
1212

1313
Text2
1414

15+
1516
--------------------------------------------------------------------------------
1617

1718
(help_file
@@ -51,6 +52,7 @@ Text
5152

5253
Text
5354

55+
5456
--------------------------------------------------------------------------------
5557

5658
(help_file
@@ -104,6 +106,7 @@ Text
104106
Not-a-heading-4 *heading4*
105107
Text
106108

109+
107110
--------------------------------------------------------------------------------
108111

109112
(help_file

0 commit comments

Comments
 (0)