Hi Team,
We are observing an issue where long-running HTTP POST requests taking over 50 seconds are being terminated with an HTTP 499 status code and automatically retried every 50 seconds with the identical x_request_id.
Our analysis shows that the 50-second timeout and retry loop are being triggered by the CloudStack Virtual Router's default HAProxy configuration (timeout server 50s and retries 3) on the Load Balancer service.
kv-dev logs --tail 100 -f nginx-666b6675d8-m7qpt|grep "POST /v2/cr"
10.1.1.1 - - [24/Sep/2026:12:09:27 +0000] "POST /v2/cr/tenants/b8d1dc4d-5961-4ff9-9eeb-a36d2f73f1bb/call_reports HTTP/1.1" 499 0 "https://vvstei.com/app/reports/callReports" "Mozilla/5.0 (Linux; Android 15; Pixel 9) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/150.0.0.0 Mobile Safari/537.36" x_forwarded_for="-" x_request_id="534d2a13-15b6-46c0-ad9c-291f73c55053" request_time="50.002" upstream_addr="10.105.79.92:9080" upstream_status="-" upstream_response_time="50.002"real_ip="-"
10.1.1.1 - - [24/Sep/2026:12:10:17 +0000] "POST /v2/cr/tenants/b8d1dc4d-5961-4ff9-9eeb-a36d2f73f1bb/call_reports HTTP/1.1" 499 0 "https://vvstei.comreports/callReports" "Mozilla/5.0 (Linux; Android 15; Pixel 9) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/150.0.0.0 Mobile Safari/537.36" x_forwarded_for="-" x_request_id="534d2a13-15b6-46c0-ad9c-291f73c55053" request_time="50.001" upstream_addr="10.105.79.92:9080" upstream_status="-" upstream_response_time="50.001"real_ip="-"
10.1.1.1 - - [24/Sep/2026:12:11:07 +0000] "POST /v2/cr/tenants/b8d1dc4d-5961-4ff9-9eeb-a36d2f73f1bb/call_reports HTTP/1.1" 499 0 "https://vvstei.com/reports/callReports" "Mozilla/5.0 (Linux; Android 15; Pixel 9) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/150.0.0.0 Mobile Safari/537.36" x_forwarded_for="-" x_request_id="534d2a13-15b6-46c0-ad9c-291f73c55053" request_time="50.003" upstream_addr="10.105.79.92:9080" upstream_status="-" upstream_response_time="50.001"real_ip="-"
Could you please assist us with one of the following solutions?
Update HAProxy settings on the Virtual Router:
Increase timeout server and timeout client to 900s (15 minutes).
Set retries to 0.
Alternatively:
Provide guidance or update the CloudStack global parameters (haproxy.timeout.server) for the Virtual Router offering.
Please let us know if you need any additional logs or details.
Thanks,
Hi Team,
We are observing an issue where long-running HTTP POST requests taking over 50 seconds are being terminated with an HTTP 499 status code and automatically retried every 50 seconds with the identical x_request_id.
Our analysis shows that the 50-second timeout and retry loop are being triggered by the CloudStack Virtual Router's default HAProxy configuration (timeout server 50s and retries 3) on the Load Balancer service.
kv-dev logs --tail 100 -f nginx-666b6675d8-m7qpt|grep "POST /v2/cr"
10.1.1.1 - - [24/Sep/2026:12:09:27 +0000] "POST /v2/cr/tenants/b8d1dc4d-5961-4ff9-9eeb-a36d2f73f1bb/call_reports HTTP/1.1" 499 0 "https://vvstei.com/app/reports/callReports" "Mozilla/5.0 (Linux; Android 15; Pixel 9) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/150.0.0.0 Mobile Safari/537.36" x_forwarded_for="-" x_request_id="534d2a13-15b6-46c0-ad9c-291f73c55053" request_time="50.002" upstream_addr="10.105.79.92:9080" upstream_status="-" upstream_response_time="50.002"real_ip="-"
10.1.1.1 - - [24/Sep/2026:12:10:17 +0000] "POST /v2/cr/tenants/b8d1dc4d-5961-4ff9-9eeb-a36d2f73f1bb/call_reports HTTP/1.1" 499 0 "https://vvstei.comreports/callReports" "Mozilla/5.0 (Linux; Android 15; Pixel 9) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/150.0.0.0 Mobile Safari/537.36" x_forwarded_for="-" x_request_id="534d2a13-15b6-46c0-ad9c-291f73c55053" request_time="50.001" upstream_addr="10.105.79.92:9080" upstream_status="-" upstream_response_time="50.001"real_ip="-"
10.1.1.1 - - [24/Sep/2026:12:11:07 +0000] "POST /v2/cr/tenants/b8d1dc4d-5961-4ff9-9eeb-a36d2f73f1bb/call_reports HTTP/1.1" 499 0 "https://vvstei.com/reports/callReports" "Mozilla/5.0 (Linux; Android 15; Pixel 9) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/150.0.0.0 Mobile Safari/537.36" x_forwarded_for="-" x_request_id="534d2a13-15b6-46c0-ad9c-291f73c55053" request_time="50.003" upstream_addr="10.105.79.92:9080" upstream_status="-" upstream_response_time="50.001"real_ip="-"
Could you please assist us with one of the following solutions?
Update HAProxy settings on the Virtual Router:
Increase timeout server and timeout client to 900s (15 minutes).
Set retries to 0.
Alternatively:
Provide guidance or update the CloudStack global parameters (haproxy.timeout.server) for the Virtual Router offering.
Please let us know if you need any additional logs or details.
Thanks,