[Feat.] Refactor llm_inference/run.py to use ParallelInferenceManager with batch inference#59

yl231 merged 7 commits into

mainRouteWorks/RouterArena:mainfrom

feature/run-py-batch-inference-compatibilityRouteWorks/RouterArena:feature/run-py-batch-inference-compatibilityCopy head branch name to clipboard