Integrate python debugger

[kaisalmen]

I chose to integrate the debug server directly into the container.
graalpy is now started properly. All workspace files are transported over to the container on start.

The debugger client and server are generic. On the server side the container ensure what DAP capable software is available. The start command for the debugger is send from the client on init. This way it is completely exchangable.
A launch config is supplied and the files are transferred to the debugger on start.

Fixes #690

[kaisalmen]

@CGNonofr and @rubenfiszel client and server can talk to each other now, but debugging is not yet working.

[rubenfiszel]

awesome, what's blocking for the debugger?

[kaisalmen]

@rubenfiszel I have to figure that out 🙂

[kaisalmen]

@CGNonofr and @rubenfiszel I improved the logging ⬆️ The debugger does not halt at the given breakpoints:

[Graal DAP] Starting server and listening on localhost/127.0.0.1:4711
[Graal DAP] Client connected on /127.0.0.1:48538
Hello World!
Hello Moon!
[Graal DAP] Error: null

[rubenfiszel]

Looks really close!

[kaisalmen]

@rubenfiszel and @CGNonofr still not stopping at the breakpoint, but now both debugging client and server are fully generic. The container image supplies the interpreter/DAP capable software and the client provides the executable all on init.

I will continue the investigation why it does not stop at the breakpoint. That is the container logging (console log from the debugServer):

Found file: hello.py path: /home/mlc/workspace/hello.py
Found file: hello2.py path: /home/mlc/workspace/hello2.py
Found file: bad.py path: /home/mlc/workspace/bad.py
Using default file "/home/mlc/workspace/hello.py" for debugging.
DAP->Client: {"type":"event","event":"output","body":{"category":"stdout","output":"[Graal DAP] Starting server and listening on localhost/127.0.0.1:4711\n"}}
Client->DAP: {"command":"initialize","arguments":{"clientID":"vscode","clientName":"mlc-python-example","adapterID":"python","pathFormat":"path","linesStartAt1":true,"columnsStartAt1":true,"supportsVariableType":true,"supportsVariablePaging":true,"supportsRunInTerminalRequest":true,"locale":"en","supportsProgressReporting":true,"supportsInvalidatedEvent":true,"supportsMemoryReferences":true,"supportsArgsCanBeInterpretedByShell":true,"supportsMemoryEvent":true,"supportsStartDebuggingRequest":true,"supportsANSIStyling":true},"type":"request","seq":1}
DAP->Client: {"type":"event","event":"output","body":{"category":"stdout","output":"[Graal DAP] Client connected on /127.0.0.1:41684\n"}}
DAP->Client: {"event":"initialized","type":"event","seq":2}
DAP->Client: {"success":true,"type":"response","body":{"supportsConditionalBreakpoints":true,"supportsLoadedSourcesRequest":true,"supportsFunctionBreakpoints":true,"supportsExceptionInfoRequest":true,"supportsBreakpointLocationsRequest":true,"supportsHitConditionalBreakpoints":true,"supportsLogPoints":true,"supportsSetVariable":true,"supportsConfigurationDoneRequest":true,"exceptionBreakpointFilters":[{"filter":"all","label":"All Exceptions"},{"filter":"uncaught","label":"Uncaught Exceptions"}]},"request_seq":1,"command":"initialize","seq":1}
Client->DAP: {"command":"setBreakpoints","arguments":{"source":{"name":"bad.py","path":"\\home\\mlc\\workspace\\bad.py"},"lines":[],"breakpoints":[],"sourceModified":false},"type":"request","seq":2}
Client->DAP: {"command":"setBreakpoints","arguments":{"source":{"name":"hello.py","path":"\\home\\mlc\\workspace\\hello.py"},"lines":[3],"breakpoints":[{"line":3}],"sourceModified":false},"type":"request","seq":3}
Client->DAP: {"command":"setBreakpoints","arguments":{"source":{"name":"hello2.py","path":"\\home\\mlc\\workspace\\hello2.py"},"lines":[],"breakpoints":[],"sourceModified":false},"type":"request","seq":4}
Client->DAP: {"command":"launch","arguments":{"name":"Debugger: Lauch","type":"python","request":"launch","port":55555,"host":"localhost","__configurationTarget":6,"__sessionId":"41cf9933-b653-4a36-b36b-173799b7b4d8"},"type":"request","seq":5}
DAP->Client: {"success":true,"body":{"breakpoints":[]},"type":"response","request_seq":2,"command":"setBreakpoints","seq":3}
DAP->Client: {"success":true,"body":{"breakpoints":[{"line":3,"verified":false,"id":1}]},"type":"response","request_seq":3,"command":"setBreakpoints","seq":4}
DAP->Client: {"success":true,"body":{"breakpoints":[]},"type":"response","request_seq":4,"command":"setBreakpoints","seq":5}
DAP->Client: {"event":"output","body":{"output":"Debugger attached.","category":"stderr"},"type":"event","seq":6}
DAP->Client: {"success":true,"type":"response","request_seq":5,"command":"launch","seq":7}
Client->DAP: {"command":"setFunctionBreakpoints","arguments":{"breakpoints":[]},"type":"request","seq":6}
DAP->Client: {"success":true,"body":{"breakpoints":[]},"type":"response","request_seq":6,"command":"setFunctionBreakpoints","seq":8}
Client->DAP: {"command":"breakpointLocations","arguments":{"source":{"name":"hello.py","path":"\\home\\mlc\\workspace\\hello.py"},"line":3},"type":"request","seq":7}
DAP->Client: {"success":true,"body":{"breakpoints":[]},"type":"response","request_seq":7,"command":"breakpointLocations","seq":9}
Client->DAP: {"command":"setExceptionBreakpoints","arguments":{"filters":[]},"type":"request","seq":8}
Client->DAP: {"command":"breakpointLocations","arguments":{"source":{"name":"hello.py","path":"\\home\\mlc\\workspace\\hello.py"},"line":3},"type":"request","seq":9}
DAP->Client: {"success":true,"type":"response","request_seq":8,"command":"setExceptionBreakpoints","seq":10}
Client->DAP: {"command":"configurationDone","type":"request","seq":10}
DAP->Client: {"success":true,"body":{"breakpoints":[]},"type":"response","request_seq":9,"command":"breakpointLocations","seq":11}
DAP->Client: {"type":"event","event":"output","body":{"category":"stdout","output":"Hello World!\nHello Moon!\n"}}
DAP->Client: {"type":"event","event":"output","body":{"category":"stdout","output":"[Graal DAP] Error: null\n"}}
DAP->Client: {"success":true,"type":"response","request_seq":10,"command":"configurationDone","seq":12}
DAP->Client: {"event":"loadedSource","body":{"reason":"new","source":{"path":"/home/mlc/workspace/hello.py","name":"hello.py"}},"type":"event","seq":13}
DAP->Client: {"event":"loadedSource","body":{"reason":"new","source":{"path":"/home/mlc/workspace/hello2.py","name":"hello2.py"}},"type":"event","seq":14}
DAP->Client: {"event":"thread","body":{"threadId":1,"reason":"exited"},"type":"event","seq":15}
DAP->Client: {"event":"terminated","type":"event","body":{},"seq":16}
DAP->Client: {"event":"output","body":{"output":"[Graal DAP] Error: null\n","category":"stderr"},"type":"event","seq":17}
Client->DAP: {"command":"threads","type":"request","seq":11}
Client->DAP: {"command":"disconnect","arguments":{"restart":false},"type":"request","seq":12}
DAP->Client: {"success":true,"body":{"threads":[]},"type":"response","request_seq":11,"command":"threads","seq":18}
DAP->Client: {"success":true,"type":"response","request_seq":12,"command":"disconnect","seq":19}`

[kaisalmen]

@CGNonofr if I halt the debugger on start I can step through and I see the content in the app, but breakpoints are not respected, although I see the data being properly exchanged via the message login. Do you an idea what I do wrong. I took your code and condensed/generalized it. I guess I did a stupid mistake. Maybe you have a good idea. Thanks

[kaisalmen]

Even with this error still, generally this PR is ready for review

[CGNonofr]

@CGNonofr if I halt the debugger on start I can step through and I see the content in the app, but breakpoints are not respected, although I see the data being properly exchanged via the message login. Do you an idea what I do wrong. I took your code and condensed/generalized it. I guess I did a stupid mistake. Maybe you have a good idea. Thanks

Hum not really, never had such issues

[kaisalmen]

@CGNonofr When the debugger is connected it looks like this

When the debugger is disconnected it looks like this:

Does that ring a bell?

[CGNonofr]

@CGNonofr When the debugger is connected it looks like this

When the debugger is disconnected it looks like this:

Does that ring a bell?

Yes, the breakpoints are sent to the DAP server, and the server tells the client about their statuses. It looks like this one is not recognized, maybe due to some path mismatch?

[kaisalmen]

@CGNonofr arrghh, yes you that could be it:
The client sends this: \\home\\mlc\\workspace\\hello2.py and the graalpy sends this: /home/mlc/workspace/hello2.py

[kaisalmen]

@CGNonofr and @rubenfiszel problem is resolved. This ready. I would like to bring it in and then work on further improvements like current file selection.

[kaisalmen]

I should add some documentation as well (short mention on page how to launch the debugger)

[kaisalmen]

I should add some documentation as well (short mention on page how to launch the debugger)

Done ⬆️

[kaisalmen]

@CGNonofr any further feedback or is it good to go?

[CGNonofr]

👍

[kaisalmen]

@CGNonofr I will resolve the conflict and then merge.