Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support VLM in reference_hf #2726

Merged
merged 1 commit into from
Jan 3, 2025
Merged

Conversation

gaocegege
Copy link
Contributor

@gaocegege gaocegege commented Jan 3, 2025

Motivation

Thanks for the project! It’s like a handy little Swiss Army knife—small but efficient.

I’ve been experimenting with QWen2 VL model and ran into some issues with bitsandbytes, so I wanted to double-check if everything works with reference_hf. Turns out, the script didn’t support VLM initially, so I spent some time adding that functionality.

I hope this PR can serve as an entry point for me to get familiar with the contribution flow of SGLang. I am glad to have the chance to play more with SGLang and explore further. (although not much time :-)

Modifications

scripts/playground/renference_hf.py

  • Implemented a new function to support VLM.
  • Introduced a new command-line argument --model-type=text,vlm to direct execution to the appropriate function based on the model type.

Checklist

  • Format your code according to the Contributor Guide.
  • Add unit tests as outlined in the Contributor Guide.
  • Update documentation as needed, including docstrings or example tutorials.

@gaocegege
Copy link
Contributor Author

By the way, the link https://github.com/sgl-project/sglang/blob/main/docs/references/contributor_guide.md in the GitHub issue template is 404.

@zhyncs zhyncs merged commit f5d0865 into sgl-project:main Jan 3, 2025
@gaocegege gaocegege deleted the vlm-reference branch January 3, 2025 14:53
XiaotongJiang pushed a commit to XiaotongJiang/sglang that referenced this pull request Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants