Feat: Support production deployment mode by using gunicorn. #7845
Conversation
dosubot
bot
added
size:L
KevinHuSh
changed the title
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
|
Would you please check if the test cases can run in your local environment? |
|
I'm trying. |
dosubot
bot
added
size:XL
|
Hi team, @JinHai-CN @ZhenhangTung @KevinHuSh Apologies for the earlier PR that failed CI. I have since:
Although the test suite is green, these changes have not yet been exercised in a live production environment. Could you please run a thorough review and, if possible, a staging deployment to confirm everything behaves as expected? Thank you for your time and feedback! |
|
Appreciations! |
Sure, I will do that. |
|
@zyoung1212 @nareshrajkumar866 To avoid some unkonwn risks, it will be better to start gunicorn in sync workers mode. |
Thank you very much for your feedback. I will work on some configuration optimizations to support different Gunicorn modes. |
|
@KevinHuSh @ericwu0930
Would you mind testing again when you have a moment, especially to verify the issue you mentioned earlier? Thanks in advance! @ericwu0930 |
zyoung1212
force-pushed
the
feature/support-gunicorn
branch
from
4d6c232 to
bcc8192
Compare
|
@zyoung1212 @KevinHuSh 
|
|
@ericwu0930 thanks for raising it. |
|
@zyoung1212 @KevinHuSh @ZhenhangTung |
Sorry, I've been busy with some company projects lately and haven't had much time to think through the issue. We might need a bit more detail to get a better idea—looking forward to solving it together with you! |
@ZhenhangTung @KevinHuSh |
|
Hello, @ericwu0930 |
sync mode.
Another one has no idea. Except for being kicked out immediately after login which prevents me from operating on the ragflow website, everything goes well. |
|
Hi @ericwu0930 , I've also met a similar 401 issue on my internal deployment before, the reproduce steps from my side are:
I've fixed this issue by setting SECRET_KEY and introducing flask redis session to share the session across multiple ragflow backend instances, will take time to submit an individual PR for sharing this fix. |
|
@zyoung1212 @ericwu0930 could you run the ragflow backend service normally ? I've tried this PR and there are too many issues... For example, when it's sync mode (gevent would also have issue), it would stuck on exiting master process and must kill it via kill -9 : And also when I'm trying to add a chunk by calling |
|
@Colstuwjx @zyoung1212 I've solved this by setting SECRET_KEY. I think it's no need to introduce flask redis session. |
@KevinHuSh @zyoung1212 @Colstuwjx
|
@Colstuwjx |
|
Hi @ericwu0930 thanks for your kindly sharing. I foound the issues I mentioned before were caused by the package import of these codes , after moved them inside the post_fork function, the guicorn workers could be started normally on my local environment ( my local environment is macOS M1 PRO). However, when I try to benchmark the gunicorn workers, e.g. keep posting create chunk requests to ragflow http endpoint, the gunicorn workers would crash and the benchmark client would return 502 errors like below: And when I switch to the original development server, it could be even more stable, less timeout errors, although the process would also finally crashed. |
Hi, would you be able to share your benchmark? I'll spend some more time troubleshooting the issue later, and I'm also looking forward to optimizing it together with you. |
|
Hi @zyoung1212 , you could use this to do the benchmark, I used another internal csv data to do the perf test, but the behavior is similar to this script. |
#7995 related problem. |
|
ragflow version 0.19.0 problems:
|
|
based on main...ericwu0930:ragflow:feature/support-gunicorn to the beginning of the file,it will solve ssl recursion problem. then modify api/wsgi.py ,change |
|
When starting with Gunicorn, multiple workers are enabled (18 workers are configured locally). When using RagFlow, it redirects to the login page after one or two clicks. By adding logs, it was found that with different workers and the same query conditions, some can retrieve results while others cannot. —————————— My temporary solution is to not use a database connection pool. 
|
|
When will this feature be released? |
|
Hello everyone, I'm very sorry that this PR has been delayed for so long. It still has many issues at the moment, so it remains in the Draft status. Due to the heavy workload at my company, I haven't been able to continue optimizing the code during this period. I hope I will have time to look into it further in the days ahead. Thanks. |
Thank you, in the early versions, I achieved high availability by using Gunicorn, only needing to switch the update_process to run in a separate file. I wonder if it should provide a simpler implementation, as this would also reduce the compatibility issues that need to be considered. |
11 participants
What problem does this PR solve?
This PR adds Gunicorn support to enable production-ready deployment of the RAGFlow framework, replacing the default development server with a robust WSGI HTTP server. This change significantly improves performance, stability, and concurrency handling in real-world deployments.
Type of change