Data model

Under the hood, IWE represents the text as a binary tree. Where every header, paragraph, list or list item in the document is a node. And a node can have up to two references: a next-element and a child-element. For example, this paragraph is a child-element of the "Data model" header cause it's a part of the section started by this header.

Since every Markdown text block is a node in the graph, IWE internal model works with a graph of text blocks not a graph of documents. This expands the possibilities of advanced text transformations and graph traversal.

Every header is a section, every block under the header is a sub-node for the section. In other words IWE understands the hierarchy of the headers and lists and treats the document as a tree.

One special type of node IWE adds to this construct is a block-reference. It's a paragraph (block) with a single reference which represents a sub-document. IWE can extract/embed sections in a form of block-references.

Notes embedding

IWE can embed referenced notes preserving correct document structure. For example, if a note is embedded under a level 2 header all the headers levels in the embed note are going to be increased by 2. Since all the content is organized in the graph saturate, notes embedding is just a matter of graph traversal.

Indexing

Another key aspect is that every element of the text graph has a unique ID and can be referenced in an index. Like this refs index

pub struct RefIndex {
    block_references: HashMap<Key, HashSet<NodeId>>,
    inline_references: HashMap<Key, HashSet<NodeId>>,
}

Node IDs are generated when graph is loaded into memory and assigned to every bock in the graph. This allows for efficient traversal of the graph starting from any point and in any direction.

All the operations on the graph, such as extract/embed notes are build using the graph transformations. In a general sense, IWE is a library for manipulating text blocks graph which is used to build LSP and CLI utility.