You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am not sure about the process of adding a model to VLLM, i saw the process is forking this repo, adding my model entire code to one of the folders here and implementing forward and load_weight, this seems like we are effectively implementing an LLM base class, does it make sense for you to allow supporting new models by deriving from some base class and having VLLM as a dependency?
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
I am not sure about the process of adding a model to VLLM, i saw the process is forking this repo, adding my model entire code to one of the folders here and implementing
forward
andload_weight
, this seems like we are effectively implementing an LLM base class, does it make sense for you to allow supporting new models by deriving from some base class and having VLLM as a dependency?Beta Was this translation helpful? Give feedback.
All reactions