Skip to content

Conversation

@comeRodriguez
Copy link

@comeRodriguez comeRodriguez commented Apr 24, 2025

This PR try to add vLLM as a possible provider for batch GENERATION step in pipeline

Features

  • Added vLLM programmatic API and rest API call to generate new data
  • Created a VLLMProvider class to handle vLLM calls

Warnings
Even if PR is ready to review, work is still in progress (no vLLM batch inference for ABLATION step). Any help or advice is more than welcome :)

@comeRodriguez comeRodriguez force-pushed the add_vllm_batch_processing branch from 8391196 to 3ab8542 Compare May 3, 2025 00:51
@comeRodriguez comeRodriguez marked this pull request as ready for review May 3, 2025 00:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant