README.norg

@document.meta
title: minorg
description:
authors: takuto
categories:
created: 2023-10-30
updated: 2024-02-14T11:53:05+0900
version: 1.1.1
@end

* minorg

  !{https://img.shields.io/badge/nim-2.0.0%2B-informational?style=flat&logo=nim}[Minimum supported Nim version]
  {/ LICENSE}[!{https://img.shields.io/github/license/pysan3/minorg?style=flat}[License]]

  > MyNorg: Convert all /my/ notes into {https://github.com/nvim-neorg/neorg/}[`norg`] format.

  `Pandoc json` -> `norg` converter written in {https://nim-lang.org/}[Nim] language.

  *This README is generated from {/ ./README.norg}.

** Table of Contents
   - {** Get Started}
   - {** Usage}
   -- {*** Command}
   -- {*** Example}
   - {** Known Issues}
   -- {*** Obsidian}
   - {** Compile From Source}
   - {** Contribution}
   - {** License}

** Get Started
   - Download: {https://github.com/pysan3/minorg/releases}
   - API documentation: {https://pysan3.github.io/minorg/}

** Usage
*** Command
    - `minorg help`

    @code txt
    This is a multiple-dispatch command.  -h/--help/--help-syntax is available
    for top-level/all subcommands.  Usage is like:
        minorg {SUBCMD} [subcommand-opts & args]
    where subcommand syntaxes are as follows:

      generate [optional-params]
      Options:
          --version            bool    false  print version
          -i=, --input=        string  ""     Input file. Leave it blank to use stdin.
          -o=, --output=       string  ""     Output file. Leave it blank to use stdout.
          -v, --verbose        bool    false  Outputs debug info to stderr.
          -f, --force          bool    false  Overwrite files and create parent folders if needed.
          --isObsidian         bool    false  set isObsidian
          -w=, --workRootDir=  string  ""     set workRootDir
    @end

*** Example
**** Pandoc and Usage
     {https://pandoc.org/}[Pandoc] is able to convert files from one markup format into another.
     We use this tool to parse arbitrary markup format into its AST represented in `json` format as defined in
     {https://hackage.haskell.org/package/pandoc-types-1.23/docs/Text-Pandoc-Definition.html}[Pandoc Type Definition].

     Pandoc accepts two arguments `-f` (from) and `-t` (to) to specify the format of the file. As noted above, we
     want the `json` representation so the output must be `-t json`.
     @code bash
     $ pandoc -f '<file-format>' -t json '<input-file-name>' | minorg generate -o '<output-file-name>.norg'
     @end

     The input file format depends on the file you want to convert from. Here is a list of some common files.
     - `markdown`: markdown format (`.md`), Obsidian file
     - `gfm`: GitHub-flavored Markdown
     - `html`: HTML4 / HTML5
     - `latex`: LaTeX document

     Read {https://pandoc.org/index.html}[this page] for information on more file formats.

**** Basic Usage
     - Use `-i`, `-o` flags (`--input`, `--output`)
     @code bash
     $ pandoc -f markdown -t json -o parsed.json ./your/markdown/file.md
     $ minorg generate -i parsed.json -o output.norg
     @end
     - Use `stdin`, `stdout`
     @code bash
     $ pandoc -f markdown -t json ./your/markdown/file.md | minorg generate > output.norg
     @end

**** Other Useful Flags
     - `-v`, `--verbose`
     -- Please use this flag to report issues.
     - `-f`, `--force`
     -- Force overwrite output file if exists.
     -- Also automatically creates parent directories if not exists.

**** *Obsidian* Notes
     This parser also takes some amount of care of obsidian specific format.
     Please use the `--isObsidian` flag and read {*** Obsidian} section for more information.

**** Recursive Directory of Markdown
     This is a shell script which scans the folder recursively for files with `"*.md"` extension and
     applies `pandoc -> minorg` compo to those files. This converts `foo/bar.md` into `foo/bar.md.norg`.
     However I will suggest first trying out few files manually to check if this converter generates the desired output
     for you. If you are satisfied, go for all, if you are not, feel free to
     {https://github.com/pysan3/minorg/issues/new?assignees=&labels=generate&projects=&template=converter_issues.yml&title=FEATURE%3A+or+BUG%3A+}[submit a support request].
     @code bash
     command find . -type f -name '*.md' | while read f; do
       pandoc -f markdown -t json "$f" | minorg generate -o "${f}.norg"
     done
     @end

** Known Issues

*** File Linking
    It is very hard to distinguish what link pattern should be used.

    - (x) URL: `{https://xxx.example.com}`
    -- (x) URL with name: {https://github.com/nvim-neorg/neorg/}[Neorg GitHub]
    - ( ) Norg files
    -- ( ) Relative to current file: `{:foo/bar:}` -> `foo/bar.norg`
    -- ( ) Absolute path: `{:/tmp/foo/bar:}` -> `/tmp/foo/bar.norg`. (Also works with `~/` = `$HOME/`)
    -- ( ) Relative to current workspace: `{:$/foo/bar:}` -> `~/Norg/Notes/foo/bar.norg`
    -- ( ) Relative to different workspace: `{:$work/foo/bar:}` -> `~/Norg/work/foo/bar.norg`
    - ( ) Usual files: `{/ /path/to/file}`
    - (x) Headings: {* Heading 1}
    -- (x) Any level heading: {# Heading 2}

    *AND YOU CAN COMBINE THEM*
    - ( ) `Heading 1` of `foo/bar.norg`: {:foo/bar:* Heading 1}
    - ( ) Line numbers: {:foo/bar:4}

*** `|example` Is Not Restored Correctly
    This is a problem in `norganic` / `Norg.jl`.

    - (x) I will make a PR later.
    -- {https://github.com/Klafyvel/Norg.jl/issues/41}
    - ( ) He says he needs some help {https://github.com/Klafyvel/Norg.jl/issues/41#issuecomment-1784814268}[here].
    - (x) Auto convert `@code norg` -> `|example`

*** Softbreak Not Preserved
    When the original note (especially markdown) had softbreaks (single new
    line to format the document but does not mean a new paragraph), that new line does not
    get transferred to the generated file.

    - (x) fix softbreak
    - (x) Modifier chars (i.e. *bold*, /italic/, ...) cannot be escaped when they are used as a real symbol.

*** Markdown TODO Not Working
    According to `pandoc -f markdown -t json`, bullet list with todo (`- [ \]`) is treated as a simple
    `["-", " ", "[", " ", "]"]` which cannot be converted to norg style (`- ( \)`).

    - (x) Fix markdown style todo items

*** Obsidian
**** Obsidian Style Tags Are Not Working
     It seems that when the line above a header is not empty, pandoc does not recognize it as a header
     but rather a paragraph.
     @code markdown
     #obsidiantag1 #obsidiantag2
     # A Header
     @end

     - (x) Convert them to tags in `norg` format
     -- ❗ Use the `--isObsidian` flag to enable this parse mode.
     -- Also pass `--workRootDir=/path/to/workspace` to link with other files (Required).

**** Wikilinks Are Not Working
     Obsidian uses its own format to link to other files in the directory.
     Obviously this is not an official markdown syntax, and pandoc cannot parse it.
     @code markdown
     [[file name]] -> `./somewhere/in/the/directory/file\ name.md`
     ![[file name]] -> `./somewhere/in/the/directory/file\ name.png` # Image support
     @end

     - (x) Implement a workaround
     -- See {**** Obsidian Style Tags Are Not Working} for the fix.
     --- `--isObsidian` and `--workRootDir` required.
     -- TODOs
     --- (x) `[[file name]]` -> `{:$/somewhere/in/workspace/file name:}` (norg file)
     --- (x) `[[file name.txt]]` -> `{/ $/somewhere/in/workspace/file name.txt}` (normal file)
     --- (x) `![[file name.png]]` -> `.image $/somewhere/in/workspace/file\ name.png` (image file)
     --- (x) `[[file name#header]]` -> `{:$/somewhere/in/workspace/file name:# head1}` (norg file with any level heading)
     --- (x) `[[file name#head1#head2]]` -> `{:$/somewhere/in/workspace/file name:* head1 : ** head2}` (norg file with chained headings)
     --- (x) `[[file name|display name]]` -> `{:$/somewhere/in/workspace/file name:}[display name]` (norg file with display name)
     --- (x) `[[file name.txt|display name]]` -> `{/ $/somewhere/in/workspace/file name.txt}[display name]` (normal file with display name)
     --- (x) `![[file name.png|display name]]` -> `.alt display\ name\\n.image $/somewhere/in/workspace/file\ name.png` (image file with alt text)
     --- (x) `[[unknown]]` -> `[[unknown]]` (file not found: do nothing)

** Compile From Source
   This project is written in {https://nim-lang.org/}[nim] language.
   You will need the nim compiler and the package manager `nimble`.

**** Install `choosenim` and `nim` Compiler
     If you don't have nim installed yet, I strongly suggest using {https://github.com/dom96/choosenim}[choosenim]
     to install the required toolkit.
     Please read the {https://github.com/dom96/choosenim#installation}[Choosenim Installation].
     Below is the instruction for unix systems.
     @code bash
     $ curl https://nim-lang.org/choosenim/init.sh -sSf | sh
     $ export PATH="$HOME/.nimble/bin:$PATH"
     $ choosenim stable
     # install stable version of nim and nimble
     # ... this may take a while
     $ nim -v
     # ! check nim version >= 2.0.0
     @end

**** Setup `nimble` and Compile
     `nimble` is the package manager for nim (should be installed into `$HOME/.nimble/bin` by choosenim).
     Setup for this repo is very easy.
     @code bash
     $ git clone https://github.com/pysan3/minorg.git
     $ cd minorg/
     $ nimble setup
     # ... Download dependencies and setup the environment.
     $ nimble build -d:release
     # Compile ./minorg.nim (you may omit -d:release)
     $ ./minorg help
     # Run compiled binary. Pass `help` as an argument to get started.
     @end

** Contribution
   Any contribution is welcome! And don't hesitate to {https://github.com/pysan3/minorg/issues/new/choose}[send me an issue].

**** Fix Code and Recompile
     @code bash
     $ pandoc -f markdown -t json -o parsed.json your-test-markdown-file.md
     $ nim r ./minorg.nim generate -i parsed.json -v
     @end

     Run `nim r ./minorg.nim help` for more options.

** License
   All files in this repository without annotation are licensed under the *GPL-3.0 license* as detailed in {/ LICENSE}.

** Other Projects
   - {https://github.com/boltlessengineer/norg-pandoc}[`norg-pandoc`]
   -- `norg` -> `pandoc` exporter written in lua.
   - {https://github.com/klafyvel/norganic}[`norganic`] + {https://github.com/Klafyvel/Norg.jl/}[`Norg.jl`]
   -- Another `norg` -> `pandoc` exporter written in julia.
   -- Parser is written in `Norg.jl` and `norganic` is its frontend to be used from the command line.
   - {https://github.com/nvim-neorg/tree-sitter-norg3}[`tree-sitter-norg3`]
   -- Parser of the norg format written for tree-sitter.
   -- Once this is mature, we should not require `minorg` anymore as this parser should handle the work better.
   - {https://github.com/pysan3/Norg-Tutorial}[`Norg-Tutorial`]
   -- Tutorial for the norg file format and neorg plugin.
   -- Read the {https://github.com/pysan3/Norg-Tutorial/blob/main/norg_tutorial.md#export--import}[Export / Import] section
      for more explanation to export / import your notes to and from norg files.