PyPy, Pyston, and Cython

[rocky]

Casually a I have been running things in PyPy and in Pyston. Recently though I have been using Pyston pretty regularly because it is faster.

For PyPy I don't have any hard numbers, but the load time for PyPy seems considerable. Pyston though seems to be a big win for what we do. Possibly runs things twice as fast so a 4 minute run over doctests is about 2 minutes.

The kinds of things I have been doing are mostly the developer stuff like running tests. For benchmark tests, I use combinatorica the Gries Schneider tests for which I have some timings for Python 3.8 vs pyston 3.8 (2.3.4 or 2.3.3).

I invite others though to try these and let me know what you think.

I wonder nowadays if it even makes sense to run Cython over specific Python code.

I have always found it annoying, especially in development code and the speed gain is nowhere near what we see in Pyston.

Again I am curious to know what others thing.

[rocky]

I should also say that for mathicsscript we can't use pyston yet. pyston can't run matplotlib which uses PyQT5. So there I use some version of Python.

[rocky]

It would be interesting in benchmark results running pyston code without cythonization of those specific files versus cythonization when the Pyston interpreter is used. I would not be surprised in Cython slows things down in execution time for long running programs. (For short-running programs, Cython doesn't have the JIT on startup overhead.)

[mmatera]

Actually, I was not aware that those files were still there. What we have is a line at the top of all the files that at some moment we thought deserve cythonization.

[rocky]

Not what I am seeing when I run pip install -e . or python setup.py develop. But it also not something that I really care to investigate, since right now I am not into Cython unless someone convinces me it will help in the pyston case.

Our default behavior of running Cython unless requested not to do so, should be I think do not run Cython unless requested.

[mmatera]

Agree.

[rocky]

I just ran a test with pyston with Cython and indeed we get a slowdown:

::

   load_combinatorica  482.76 ms
   test_permutations_1_1  1662.27 ms
   test_permutations_groups_1_2  12299.96 ms
   test_inversions_and_inversion_vectors_1_3  1144.22 ms
   test_special_classes_of_permutations_1_4  602.56 ms
   test_combinations_1_5  881.52 ms
   test_2_1_to_2_3  145.48 ms
   test_combinatorica_rest  180.66 ms

pyston 2.3.4 (with Cython) Note - a slowdown so not noted above

::

    load_combinatorica  513.38 ms
    test_permutations_1_1  2369.96 ms
    test_permutations_groups_1_2  17864.45 ms
    test_inversions_and_inversion_vectors_1_3  1628.33 ms
    test_special_classes_of_permutations_1_4  822.10 ms
    test_combinations_1_5  1375.82 ms
    test_2_1_to_2_3  180.88 ms
    test_combinatorica_rest  248.94 ms

These results also appear in https://github.com/Mathics3/mathics-core/blob/benchmarking%2Bfuther-improvements/bench-results.rst

GIven this we definitely shouldn't run cython over Python files now with either Pyston or PyPi. Simplifying we will just change the default.

In other words if you are running CPython and are doing something that is likely to be used a lot, e.g. inside a wheel, or inside a docker container, then you specify in an environment variable that you want Cython run over files.

[rocky]

Agree.

#398