Improve the speed

[wybert]

Wow! That's AWESOME work. I have one suggestion for the performance.
TUI is usually faster than GUI tools. I have tested euporie and vscode to open on the notebook, euporie does not faster than vscode. Also when using the preview function, euporie is slower than nbviewer

[joouha]

Hi,

Thanks!

There are various reasons that euporie is slower at rendering notebooks than VSCode and nbviewer.js:

• euporie is written in Python, while VSCode and nbviewer.js are written in JavaScript. Python is generally a slower language than Javascript, because nowadays Javascript uses just-in-time compilation to achieve high speeds.

• Both VSCode and nbviewer.js use web-browser rendering engines to display output (VSCode uses electron which uses blink, and nbviewer-app will use webkit). Web-browsers are already capable of rendering HTML and images and are extremely optimized for rendering speed.

  Terminals do not have built-in support for rendering rich output like HTML, SVG, and images. For displaying images in the terminal using ASCII-art style output, I'm relying on external libraries or programs which can convert images to text output faster than can be done in Python. Some formats can require multiple conversion steps, depending on which programs the user has installed (e.g. svg -> png -> pil -> ansi -> formatted_text). For rendering HTML, I've written a new HTML renderer for displaying HTML in the terminal, but I've not got round to optimizing this for speed yet, and Python is not an ideal language for writing an HTML rendering engine!

• I've not done much work optimizing notebook rendering yet. For example, in euporie-preview, cells are rendered sequentially; it might be much faster to cells using a thread-pool, I've just not had time to try this out yet!

In short, I think it's unlikely that euporie will be as fast as similar tools which use modern web rendering engines to display output, but there is still a lot of potential for speed-ups! It's just going to take time.

There are various (less feature-rich) alternatives which may render notebooks faster then euporie listed here in the documentation which you might be interested in trying.

[DCoderUltra]

Hi @joouha, this project is such a great idea! As someone who has terminal centric workflow this project comes in handy.
However as mentioned by @wybert there is some problems when it comes to speed.

• Do you think that a rewrite in other language would increase the speed in a significant manner or just improving the
  rendering would lead to much higher speeds?

• In terms of rich output maybe using nerdfonts as a font requirement could avoid the rendering of some images since
  there are +10 000 icons.

Congrats for all the work so far.
Thanks!

[tarsioonofrio]

@DCoderUltra and @joouha , maybe we can try a simple alternative to increase the speed: compile with Cython. I believe that is much more simpler than rewrite in another language.

I never use that extension but i will give a try.

[joouha]

@DCoderUltra

A rewrite in another language might help, but it would be a huge undertaking. I've vaguely been thinking it might be fun to port euporie to rust, but I don't think I'll have the time for to that any time soon, if ever!

@tarsioonofrio

I don't know much about Cython, but I believe using it to compile all of euporie would involve a lot of work! Targeting the bottlenecks might be a better use of time, but we'd need to profile euporie to understand exactly where these are.

There are alternatives to Cython like mypyc which might also be useful for this.

Interestingly, Euporie runs fine under pypy, but in my experience is noticeably slower then under regular CPython.

---

I've recently done some profiling and made several performance related fixes to euporie, so the latest version (v2.8.4) should feel a bit faster 🚀

[DCoderUltra]

So I've updated to the latest euporie version and indeed it does appear faster. You're right
a complete rewrite would require a lot of time, but if the speed keeps improving it
might not even be necessary. Keep up the
good work and congrats. It's an amazing
project. 😄

[joouha]

I'm going to close this issue now, as euporie is much faster than it used to be :-)