Skip to content

Conversation

@sunlei1024
Copy link
Collaborator

@sunlei1024 sunlei1024 commented Nov 19, 2025

Motivation

为提升系统在多组件并行启动场景下的稳定性,本 PR 新增了 cache-queue-portengine-worker-queue-port 的端口冲突检测逻辑。当用户显式指定端口且端口已被占用时,系统将提前识别并快速失败及提示,避免在启动hang住,从而提升整体用户体验与可用性。

Modifications

  • 新增对 cache-queue-port 的端口可用性检查逻辑
  • 新增对 engine-worker-queue-port 的端口可用性检查逻辑
  • 优化错误提示信息,明确冲突来源与解决方式
  • 增强 Engine 相关模块在多实例部署时的健壮性

Command

当前服务启动时,可指定四类端口

python -m fastdeploy.entrypoints.openai.api_server \
       --model /workspace/ERNIE-4.5-0.3B-Paddle \
       --port 8680 \
       --engine-worker-queue-port 8386 \
       --cache-queue-port 8683 \
       --metrics-port 8682 \

端口冲突示例

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/workspace/FastDeploy/fastdeploy/entrypoints/openai/api_server.py", line 738, in <module>
    main()
  File "/workspace/FastDeploy/fastdeploy/entrypoints/openai/api_server.py", line 717, in main
    if not load_engine():
  File "/workspace/FastDeploy/fastdeploy/entrypoints/openai/api_server.py", line 123, in load_engine
    engine = LLMEngine.from_engine_args(engine_args)
  File "/workspace/FastDeploy/fastdeploy/engine/engine.py", line 75, in from_engine_args
    config = engine_args.create_engine_config()
  File "/workspace/FastDeploy/fastdeploy/engine/args_utils.py", line 1229, in create_engine_config
    assert is_port_available(
AssertionError: The parameter `engine_worker_queue_port`:['8386'] is already in use.

Accuracy Tests

该改动不涉及模型推理逻辑或 kernel 修改,对模型精度无影响。

Checklist

  • Add at least a tag in the PR title.
  • Code formatted; pre-commit 已通过
  • Add unit tests
  • No accuracy-related changes

@paddle-bot
Copy link

paddle-bot bot commented Nov 19, 2025

Thanks for your contribution!

@paddle-bot paddle-bot bot added the contributor External developers label Nov 19, 2025
@gongshaotian
Copy link
Collaborator

需要处理下冲突

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor External developers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants