WIP-feat(tokenizer)

[jimersylee]

完成简单的词法分析器

[codecov-commenter]

Codecov Report

❗ No coverage uploaded for pull request base (main@73665c5). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #12   +/-   ##
=======================================
  Coverage        ?   30.00%           
=======================================
  Files           ?        1           
  Lines           ?       10           
  Branches        ?        0           
=======================================
  Hits            ?        3           
  Misses          ?        7           
  Partials        ?        0           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 73665c5...18fb36d. Read the comment docs.

[jimersylee]

bors try

[bors]

try

Build succeeded:

• build