Debug studio

[zolinthecow]

Motivation

When developing apps (especially more complex ones) with SGLang's frontend it's a bit hard to tell what is actually getting passed into the generate/ endpoint. This PR introduces a "prompt debugging studio" that allows you to turn on a debug mode and will start intercepting generate requests and saving them to a debug server, where you can go to a web interface and view all the prompts and responses for SGL functions.

For example, if you start the debug server:

python -m sglang.launch_debug_server

It will start a debug server on http://0.0.0.0:56765

Then, you can add a debug region to an sgl function:

@sgl.function
def text_qa(s, question):
    s.begin_debug_region("TEXT_QA")
    s += "Q: " + question + "\n"
    s += "A:" + sgl.gen("answer", stop="\n")

state = text_qa.run(
    question="What is the capital of France?",
    temperature=0.1,
    stream=True
)

The s.begin_debug_region will start debugging under the prompt name "TEXT_QA". You can then go to http://localhost:56765 (will have to ssh forward if you're developing on a remote server) and you should see prompts start appearing

Modifications

• Adds BeginDebugRegion and EndDebugRegion to SGLExpr for the frontend language
• Adds launch_debug_server which under the hood relies on enochian-studio (one of my packages) for the debug server
  • enochian-studio can be found here, I can say it's up to date since I maintain it, along with Etched
• Adds posting to the debug server if enabled for all backends

Checklist

• Format your code according to the Contributor Guide.
• Add unit tests as outlined in the Contributor Guide.
• Update documentation as needed, including docstrings or example tutorials.

[zolinthecow]

^ Couldn't reproduce CI error locally :(

[zhaochenyang20]

@zolinthecow Nice work! I will start to review this PR this weekend. Stay tuned!

[merrymercy]

Overall, it looks good! I left a few minor comments.
One thing we want to ensure is that it does not introduce any overhead when debug mode is turned off (e.g., use as few function calls and if statements as possible at runtime).

[merrymercy]

Suggested change

	python -m sglang.launch_debug_server
	python -m sglang.lang.launch_debug_server

[merrymercy]

This is an experimental feature for frontend language only, so please move it under python/sglang/lang.

[merrymercy]

It is better to add s.log_debug to interpreter.py so we do not need to do it for every backend file.

[merrymercy]

This is not efficient enough. When debug is turned off, you still run the code to construct the argument, which takes some time. Please minimize the overhead and do not construct any objects when debug is turned off.

[merrymercy]

closed due to inactivity and low priority