java.lang.System.exit should trigger a clean exit of Python

[pelson]

I understand that java.lang.System.exit is designed to be the clean way to signal the JVM to terminate, and isn't as brutal as Runtime.getRuntime().halt in that System.exit calls the Java shutdown sequence.

Python has a similar approach in that sys.exit calls a shutdown sequence, or you can call os._exit() (private but documented...) to skip it.

It would be expected therefore for sys.exit to call the appropriate Java shutdown sequence, and for System.exit to do similar for the Python shutdown sequence.

I can confirm that I believe the sys.exit call does trigger appropriate Java shutdown (e.g. shutdown hooks):

import jpype as jp


jp.startJVM()

def some_callback():
    print('exiting inside JVM')


hook = jp.java.lang.Thread(jp.JProxy(
    'java.lang.Runnable',
    {'run': some_callback},
))

jp.java.lang.Runtime.getRuntime().addShutdownHook(hook)

import sys

sys.exit(0)

Results in the some_callback being called.

Unfortunately the same is not true for System.exit:

import atexit


def notify_exit():
    print('Exiting from python')

atexit.register(notify_exit)

import jpype as jp
jp.startJVM()
jp.java.lang.System.exit(0)

Does not lead to notify_exit being called.

Is there something we can do in JPype to rectify this? I see #254 discusses the idea of capturing System.exit calls for other purposes - perhaps this is something we can do here also?

[Thrameos]

It would be pretty difficult to pull this off. There are two ways that system exit can happen and we likely can't cover the second.

If Python were to call System.exit then we could use a customizer to redirect it back to Python pretty trivially, but this is just superficial as there is no reason for a user to call the Java version from Python.

In the flip case if Java calls System.exit it is going to lead to the start of Java shutdown sequence. This begins terminating threads until all threads but main are gone and the calls the system command to terminate the program. The problem is that Java does not know to call Python. We can add a atexit hook in Java so the first thing it does on an exit is notify Python that it needs to exit. Unfortunately this leave us in a race state. Java is already terminating and the Python is going to call jpype exit which will terminate the JVM. Only we have Java methods on the stack. Thus to make the notification we would have to create a new thread to start the notification and then delay the JVM one from proceeding until we know the Python one has gotten the message. The issue begin we can't actually complete the Java system exit because JPype shutdown needs to complete and then Python can complete the rest of its atexit calls.

I think the only way to get this to work would be to terminate the Java shutdown entirely and transfer it to Python control. We could try a security manager patch to see if we can capture the call and redirect it. Again this is a pretty brutal race condition because shutdown is only respected from the main thread.

Unfortunately this dovetails with the problems we have with the osx windowing application. To do this right I need to split the "main" thread for Python and Java. Thus startJVM should spin up its own internal thread so that Java's concept of main is not the default Python one so that I can issue exit (or gui commands) without having to take control of the Python's main thread. I can pass those off to a listener on the second thread so there is never a deadlock potential. On that front I am still waiting for the permission to perform authorized work on an open source project from my intellectual property office, now going on 5 months. Until they give me the proceed button regarding the Python contributor form that placed a hold on almost all of my JPype work, I remain in limbo.