matplotlib, Qt #1291
Comments
|
Hi Lukas. I wish there was an easy solution to this. First, to debug this, I feel like we will need to have a minimum working example. I wonder if just installing matplotlib and running a simple plot will cause this behavior. Also, have you tried the LW release? That might work better. I tried working on a deeper python integration a while ago (#660), but never finished it. The first challenge is coming up with a solution that is compatible not only with Macs but also Windows and Linux. To solve this problem once and for all, I think the ultimate solution is to have klayout (GUI) communicate with a native python instance and exchange data as necessary. This would be an optional featured enabled by a setting in the GUI, where you would choose the desired python location, much like VSCode's can choose which python environment to use when running scripts. This is similar (in spirit) to Tait's https://github.com/atait/klayout-ipc project, where commands are communicated via a socket server and pya's QTcpServer. In this scenario, macros would be run by the chosen python executable, instead of Klayout's internal interpreter. Maybe we can borrow some code from the ipython/jupyter project which enable this functionality by using python kernels independent from the "code editor". @klayoutmatthias what do you think? This would require enabling a different process control pya.Application etc. Something like this project (https://github.com/tomerfiliba-org/rpyc) |
|
Hi Lukas, in gdsfactory we use the klayout python API to render plots and avoid using the python inside klayout to avoid these types of issues. This is similar to what Thomas proposes with using a native pyhton instance. https://gdsfactory.github.io/gdsfactory/notebooks/00_geometry.html We are also starting to use kweb where we can expose the klayout python API through a web interface |
|
This wouldn't work for Lukas because his use case is to launch third-party python modules from within klayout's macro environment. So a "reverse API". This sounds difficult to me at the moment -- like having klayout host a jupyter_client, with a remote python kernel, but with bidirectional communication because the scripts would need access to the host's GSI. Welcome thoughts on possible implementation. |
|
Here's an idea. Rather than burdening klayout's maintainers, what if we enable the macro editor to use an optional external python binary to execute the macro? Something along these lines:
Salt package developers and python developers can then be free to use whatever framework they choose for inter-process communication.
I think this will enhance UX significantly, much in the same way that jupyter/vscode's ability to select & remember python kernels improved my life a lot. I also think that this will simplify salt packages too by making it be simple scripts that connect to more featured python packages maintained externally by some anaconda environment or package manager. |
|
I would love having access to an external python in KLayout! For the record, there is also an effort to build the KLayout GUI for conda, such that it has access to all the packages installed in a conda environment. See here for example conda-forge/staged-recipes#20396. However, being able to access an external python might be a better, more flexible solution as long as it does not restrict which external python version can be used. |
|
@flaport fyi, we do have a klayout package in conda-eda now https://anaconda.org/LiteX-Hub/klayout/files (that does include the GUI). |
|
Coming back to the original question, I'll check why mathplotlib isn't working with KLayout. I think it has to do with the coexistence of Qt and another event loop. But if that was working once, it should be possible to revive that option. Becoming more general, I think you overestimate KLayouts "workbench" abilities somewhat. After all, that is a simple single-threaded application with all the restrictions that come with it. I created Python support for simple automation tasks. As a general solution I was actually thinking about turning KLayout into a remote-controlled engine (on a deeper level than rpyc) and running the IDE as a separate process. That avoids manifold problems connected with single-threaded applications. However this is not a small project, but it goes in the direction of having built-in RPC support with your own choice of client. As of now I - because of the simplicity of that approach - I personally favor the "build from scratch" idea that is behind the LayoutView object of the PyPI klayout module: this eventually is the atomic form of the KLayout GUI and you should be able to integrate it into whatever application you're building - even if this application is a web server (see kweb). This gives you maximum freedom of integration, but of course you need to spend some effort building the application the way you need it. I'm aware that this needs more activity. My personal focus right now is on PDK support which is a somewhat different topic. Best regards, Matthias |
I have since tested on my machine. No issues with plotting simple structures and closing windows. I suspect it has something to do with tidy3d or a particular installation.
I see your point. But I think many people already use it as their workbench. There's lots of potential in the way you implemented python/ruby-based extensions to the GUI. Like you said (simple automation). This functionality would simply extend their automation capabilities. The only question is whether this would create more technical debt than it would remove. My money is that it would be a net positive for simplicity. My proposed change would launch a new thread to run the macro, communicating only via pre-defined socket-based I/O. I can explore this idea a bit more later.
That would be awesome. And I would still use klayout as a client. :)
I always wanted to have an iPad app that would render layout files. :) Are you suggesting that folks willing to integrate klayout with other tools (like simulation tools) would need to code a new GUI? Or that the new klayout engine would need to be integrated into the other "workbench" software?
Thanks for considering my thoughts! I also like @flaport and @proppy's proposal of using anaconda for advanced python users. |
|
I like the idea of linking a separate python to facilitate for Windows and MacOs users For example, we have klayout packages for GF180nm and Skywater130nm and they only work well on Linux because they depend on gdsfactory the conda build of klayout+GUI does not work on MacOs yet, @thomasdorch did the windows one and Floris made the linux one |
|
@lukasc-ubc Coming back to the original question, I did some debugging. matplotlib comes with a number of backends. In my case (Ubuntu without PyQt installated) the default was gtk. The gtk backend does not work inside KLayout because it ends in a deadlock (nested event loops I suppose). After I installed PyQt, things worked nicely except in the macro IDE. When running macros from inside the IDE, the application crashed. The reason was that the IDE was not aware of the foreign Python events that PyQt generates out of the UI system. I can fix that problem. So basically, with PyQt installed, matplotlib seems to be functional. In generate, installing the PyQt should be easy with package management (conda, Linux, MacOS with homebrew or Anaconda). I'll check if I can include matplotlib and PyQt5 in the next Windows release (it's already heavy enough, so that should not be an issue). Matthias |
|
It would be great if you could add matplotlib as part of the Windows distribution. I like the simplicity of installation instructions that only require KLayout (with salt packages), and not a separate Conda distribution. The latter leads to tons of debugging with each computer and each individual. thank you |
@xobs recently did some work around this for conda-eda, see: hdl/conda-eda#298 |
|
@lukasc-ubc Yes, I will try that for 0.28.6. If I am lucky, the MSYS system comes with the proper packages already. I hope there aren't too many integration issues. |
|
The only funny bit is that klayout.app isn't installed into Applications, because I'm not sure if/how that's supposed to happen. But it can be natively launched, as pictured. |
|
Mustafa and I succeeded in getting plotly to work perfectly in KLayout for OSX, Windows and Linux this week. SiEPIC-Tools now includes a Photonic integrated circuit simulator "OPICS" which is able to create simulation plots. Simulations are done by extracting a netlist from KLayout, and a compact model library is included in the Technology/PDK as a Si folder "CML". Works beautifully, and we are teaching tomorrow at the OFC conference, in short course SC432. |
|
I will release 0.28.6 today and in the Windows installer I have included matplotlib and PyQt5. It is working with the examples I tried so far. Even PyQt5 works, although I doubt that it is very easy to integrate with the application. |
|
Thank you @klayoutmatthias Crash on closing matplotlib windows on M1: In the Mac OSX version, I imported matplotlib, and after creating a window and closing it via the code below, KLayout crashes with the error reports below. I get a crash with both the version I compiled (LW), and the official version from KLayout.de, klayout-0.28.5-macOS-BigSur-1-qt5Brew-RsysPhb39. @thomaslima can you see if you can get it to crash as well? LW version crash report: HW 0.28.5 vesion crash report: |
|
Hi Lukas, sorry for the late reply. I could see that crash on 0.28.5. But with 0.28.6 I cannot reproduce it any longer on Windows and Ubuntu. So I have hope it's gone on MacOS too. Good news #2: on Windows, matplotlib and PyQt5 (for the matplotlib backend) is included in the standard installer, so this sample runs out of the box without further packages and without crash :) P.S. if you run it from the debugger, the first package import takes a long time (some minutes) if the debugger is turned on. That are the trace callbacks that KLayout installs in order to catch breakpoints. You can turn off debugging (at least the first time you run the script) to mitigate this. Outside of the IDE execution is fast. Matthias |
|
On the Mac OSX, klayout-0.28.6-macOS-BigSur-1-qt5Brew-RsysPhb39, I ran the matplotlib example, and closed the window, and it crashed: |
|
That's bad ... the stack trace does not indicate anything specific. Maybe it's worth trying other combinations. The PyQt5 integration that is needed by matplotlib complicates matters. It is very important that this module is built against the same Qt5 than KLayout is - if not, these kind of crashes are likely. In the Windows case I am using Python, PyQt5, matplotlib and Qt5 from the MSYS2 package manager. Same for Linux where everything comes from the Ubuntu repo in my case. That might explain why I do not see issues there. Matthias |
|
I tried KLayout 0.28.6 (source code release-build) on Linux Mint 20.3 with another combination. Kazzz-S
|
|
Here are some more tests results. Kazzz-S [1] Using the distributed packages[1-A] plt.show(block=False)
[1-B] plt.show(block=True)
[2] Using the packages installed by pip3 (the same as above)[2-A] plt.show(block=False)
[2-B] plt.show(block=True)
|
|
@Kazzz Thanks for the notes about behavior on Mint. I could reproduce the problem, but only after fixing the backend to Qt5Agg: without this, "TkAgg" is selected. In that case, no crash happens, but also no plot shows. I debugged the problem and it is due to an unfriendly behavior of the PyQt5 version on Mint which takes ownership of the QApplication object and destroys it during the Python interpreter finalization. I do not see this behavior on the PyQt5 which comes with Ubuntu 22 in my case. That is why it does not crash. Ubuntu 22 is using matplotlib 3.5.1 while Mint 20.3 is using 3.1.2. I suspect the problem is due to this code in https://github.com/matplotlib/matplotlib/blob/v3.1.x/lib/matplotlib/backends/backend_qt5.py, line 94++: They changed this specific code somewhat in other versions, still the effect remains. If I use matplotlib from pip3 which is 3.7.1 still I see the crash on Mint while I don't see it on Ubuntu. I also don't see the same effect on the MacOS stack traces, but I cannot really read them. Maybe it's the same thing or not. Bottom line is that PyQt5 + matplotlib in KLayout isn't a stable combination apparently. It is very sensitive to details and after all, PyQt5 isn't made for embedding. I tried "WebAgg" as an alternative backend, but the tornado server also needs to embed itself into the event loop of the application - which apparently it doesn't, so the application is dead. SVG export comes to mind - instead of showing the plot interactively, one could display the SVG. Another, more elaborate solution is to supply a custom backend that integrates with KLayout's Qt binding, hence operates in the framework of the application natively. The latter is pretty similar to PyQt5, so I think it should be possible to adapt the Qt5Agg backend with a few changes. There is a template for backend development available with a description how to use it: https://matplotlib.org/stable/api/backend_template_api.html. The core Qt backend code is here: https://github.com/matplotlib/matplotlib/blob/main/lib/matplotlib/backends/backend_qt.py (it's imported via backend_qt5.py and backend_qt5agg.py). So basically that should be a feasible way to solve the problem once and for all. It's some effort though. Best regards, Matthias |
|
I was able to do a quick port of the matplotlib backend to KLayout/Qt, but not without some modifications required in KLayout (bringing it closer to PyQt). The code is here: https://github.com/klayoutmatthias/mpl_klayoutqt The changes are substantial (e.g. taking KLayout's Qt binding from Qt 5.5 to Qt 5.12). But I'll consider releasing it within the 0.28 branch. That will eventually allow the matplotlib backend to run natively inside KLayout and without PyQt and the potential side effects. |
Eight years ago, I was seeing if I could get matplotlib to work in KLayout. I tried today and it seems to work quite well on Mac OSX, and I was able to generate simple plots.
I was able to even install Flex Compute's Tidy3D simulator within the HW KLayout build and have the plots generate within KLayout.
My approach for installing packages is to do it within the KLayout IDE Python using the pip module:
Problems encountered:
I suspect the OSX version is using the MacOSX backend. Perhaps it would be better to use the Qt backend?
@thomaslima, perhaps you have already experience with this?
It would be nice to be able to create Qt dialogs in KLayout with plots.
For example, it could be used to help designers choose parameters from a PCell or display other technology information. And even display some simulation results.
@momchil-flex, @tomflexcompute, I would like to make it easy for students to use tidy3d in the edX Phot1x course. I'm curious what you think.
Thank you
Lukas
The text was updated successfully, but these errors were encountered: