-
Notifications
You must be signed in to change notification settings - Fork 26
add e5-multilingual to known configurations #533
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Max de Bayser <[email protected]>
👋 Hi! Thank you for contributing to vLLM support on Spyre.
Or this can be done with
Now you are good to go 🚀 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"num_hidden_layers": 24, | ||
"vocab_size": 250002 | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: we should add a new line at the end of file here
Signed-off-by: Christian Kadner <[email protected]>
@maxdebayser -- I accidentally pushed directly to your branch -- I intended to create a branch on top of yours :-( |
With the test running, it fails the test =================================== FAILURES ===================================
__________________________ test_find_model_by_config ___________________________
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7f0da1c79580>
caplog = <_pytest.logging.LogCaptureFixture object at 0x7f0da1c78500>
@pytest.mark.utils
@pytest.mark.cpu
def test_find_model_by_config(monkeypatch, caplog):
"""
Verify that we can find models by ModelConfigs loaded from config files.
This is important for the case where models are mounted to the local file
system instead of being loaded/cached from HuggingFace.
"""
model_configs_dir = Path(
__file__).parent.parent / "fixtures" / "model_configs"
setup_log_capture(caplog, level=logging.INFO)
with monkeypatch.context() as m:
m.setenv("VLLM_SPYRE_DYNAMO_BACKEND", "sendnn")
# m.setenv("HF_HUB_OFFLINE", "1")
for model_id in get_supported_models_list():
# TODO: get the actual FP8 model config
if "-FP8" in model_id:
continue
model_config_dir = model_configs_dir / model_id
model_config_file = model_config_dir / "config.json"
> assert model_config_file.exists(), \
(f"Missing config file for model {model_id}."
f" Use download_model_configs.py to download it.")
E AssertionError: Missing config file for model intfloat/multilingual-e5-large. Use download_model_configs.py to download it.
E assert False
E + where False = exists()
E + where exists = PosixPath('/home/runner/work/vllm-spyre/vllm-spyre/tests/fixtures/model_configs/intfloat/multilingual-e5-large/config.json').exists
tests/utils/test_model_config_validator.py:196: AssertionError
----------------------------- Captured stdout call ----------------------------- |
Signed-off-by: Christian Kadner <[email protected]>
Now I created a separate PR just to fix the known-model-config test #538 ... which revealed that we need to add more parameters to the know models config file to distinguish between
I don't intend to get my PR merged, but feel free to merge it into yours after I iron out the failing tests. |
Signed-off-by: Christian Kadner <[email protected]>
Signed-off-by: Christian Kadner <[email protected]>
Fixed the failing test in PR #538 ![]() ~ I also updated the "Supported Models" documentation page. Feel free to merge into your PR or cherry pick. |
Signed-off-by: Christian Kadner <[email protected]>
bot:test |
Thanks, @ckadner , I've merged your commits into this branch |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good!
Description
Adds e5-multilingual to known configurations. PELE tests are passing and have been added with another PR.