Skip to content

Python parsers

lukomskiy edited this page Nov 11, 2020 · 2 revisions

AST Lib

Python has module for making AST's. Let's take a look on some tools to work with it

1.Parso

Pluses:

  • Community
  • Documentation
  • "Battle-tested"

2.Astroid

Pluses:

  • Community
  • Documenatation
  • Easy to use

Minuses:

  • Versions for python 2 are no longer supported

3.Astor

Pluses:

  • Easy to use

Minuses:

  • Don't support many versions
  • Stopped developing?

It seems that Parso is the best option, but i think Astroid will be better choice, cause Parso is more about grammar methods and searching for errors in code. For python 2 we can use old releases of Astroid.

Now an example how Astroid works:

Here's the code:

def f(x):
    return x + 1

x = 1
if f(x) == 2:
    print('good')
else:
    print('bad')

And here's parser output:

Module(
   name='',
   doc=None,
   file='<?>',
   path=['<?>'],
   package=False,
   pure_python=True,
   future_imports=set(),
   body=[
      FunctionDef(
         name='f',
         doc=None,
         decorators=None,
         args=Arguments(
            vararg=None,
            kwarg=None,
            args=[AssignName(name='x')],
            defaults=[],
            kwonlyargs=[],
            posonlyargs=[],
            posonlyargs_annotations=[],
            kw_defaults=[],
            annotations=[None],
            varargannotation=None,
            kwargannotation=None,
            kwonlyargs_annotations=[],
            type_comment_args=[None],
            type_comment_kwonlyargs=[],
            type_comment_posonlyargs=[]),
         returns=None,
         body=[Return(value=BinOp(
                  op='+',
                  left=Name(name='x'),
                  right=Const(value=1)))]),
      Assign(
         targets=[AssignName(name='x')],
         value=Const(value=1)),
      If(
         test=Compare(
            left=Call(
               func=Name(name='f'),
               args=[Name(name='x')],
               keywords=None),
            ops=[['==', Const(value=2)]]),
         body=[Expr(value=Call(
                  func=Name(name='print'),
                  args=[Const(value='good')],
                  keywords=None))],
         orelse=[Expr(value=Call(
                  func=Name(name='print'),
                  args=[Const(value='bad')],
                  keywords=None))])])

How to use:

pip install astroid
import astroid
tree = astroid.parse(open('src/code.py').read())

Clone this wiki locally