fix(security): harden HTTP redirects by trust boundary

[superm1]

Fixes SWSPLAT-24200 by applying scheme and redirect hardening to HttpClient's redirect-following requests, split by trust boundary so loopback backend RPC keeps working over http.

Policies (HttpSecurityPolicy):

• External hosts (HF, GitHub, release CDNs): https-only for the initial request and every redirect hop, MAXREDIRS=5.
• Lemonade-managed loopback backends: http only, redirects disabled entirely.
• Opt-in plaintext endpoints (cloud allow_insecure_http): http+https, bounded redirects.

A shared helper applies the curl options and checks each setopt return value, so a restriction that fails to apply fails closed.

Scope of protection:

• Blocks HTTPS→HTTP downgrade and non-http(s) (FTP/FILE) redirect targets on external requests.
• This is protocol/downgrade hardening, not full SSRF prevention — an https URL can still resolve to loopback/private addresses; resolved-address validation is out of scope here.
• MAXREDIRS=5 is defense-in-depth: libcurl already bounds redirects by default at our >= 8.5.0 floor, so the prior behavior did not permit infinite redirect loops.

Tests: test/cpp/test_http_client_security.cpp covers trusted-loopback GET without redirects, rejection of initial http under the external policy, the configured redirect limit, and rejection of non-http(s) redirects.

[fl0rianr]

Thanks for addressing this. I agree that external downloads should be restricted to HTTPS and that redirect chains should be bounded. However, the current change applies the HTTPS-only policy to the shared HttpClient::get() implementation, which is also used for communication with wrapped backends on http://127.0.0.1:<port>.

This is already causing real regressions rather than CI-only failures:

• Test .exe - llamacpp`fails with CURL error: Unsupported protocol
• Test .exe - audio-gen-acestep fails for the same reason

I do not think CI should be converted to HTTPS here. Production wrapped backends intentionally use loopback HTTP, so CI should continue testing that path.

I suggest separating the policies by trust boundary:

• External downloads / external requests: allow only https, follow redirects, restrict redirect protocols to https, and set MAXREDIRS.
• Lemonade-managed loopback backends: allow only http and disable redirects entirely.
• User-configured HTTP mirrors, if supported: require an explicit opt-in such as allow_insecure_http, disabled by default.

This could be implemented with a small HttpRequestOptions or HttpPolicy parameter and a shared helper that applies the corresponding curl options. The loopback callers in WrappedServer and ACE-Step should then explicitly select the trusted-loopback policy.

A few additional points:

1. Restricting protocols is useful hardening, but it does not fully prevent SSRF. HTTPS URLs can still target loopback/private addresses or resolve to them through DNS. The PR description should either narrow the claim to protocol and downgrade protection, or add resolved-address and redirect-target validation.

2. Lemonade requires libcurl >= 8.5.0, where redirects are already bounded by default. MAXREDIRS=5 is good defense in depth, but the description should not claim the existing behavior allows infinite redirect loops.

3. Please check the return values of the security-related curl_easy_setopt() calls so failure to apply a restriction fails closed.

4. Please add focused tests for:

   • trusted loopback HTTP GET without redirects,
   • rejection of initial HTTP under the external policy,
   • rejection of HTTPS-to-HTTP or non-HTTP(S) redirects,
   • the configured redirect limit.

So I support the intended hardening, but the current global HTTPS-only change should not be merged as-is.

[fl0rianr]

Thanks for the great rework. Two minor follow-ups:
1. Please update the PR title, since the implementation is no longer globally “HTTPS only.” Something like fix(security): harden HTTP redirects by trust boundary would describe it more accurately.
2. An actual HTTPS→HTTP redirect test would still be valuable. The implementation clearly blocks it through CURLOPT_REDIR_PROTOCOLS_STR="https", so I would not treat this as a blocker if CI is green.

[fl0rianr]

Thanks for rebasing and getting CI back on track. The original trust-boundary refactor still looks good, the title is now accurate, and the current workflows are green.

However, the latest loopback-provider fix introduces a new blocking issue.

The new is_loopback() logic selects TrustedLoopback based only on the parsed host. This also classifies a URL such as https://localhost:PORT/v1 as trusted loopback, but TrustedLoopback permits only the http protocol. A valid local HTTPS provider would therefore fail with Unsupported protocol.

The manual authority parsing is also not safe enough for a security-boundary decision:

const auto colon = host.find(':');
if (colon != std::string::npos) host = host.substr(0, colon);

This does not correctly handle bracketed IPv6 and can confuse URL userinfo with the hostname. For example, http://localhost:80@evil.example/... is parsed here as localhost, while the actual destination host is evil.example.

I suggest removing this automatic loopback exception entirely:

• HTTPS provider URLs should use ExternalHttpsOnly.
• HTTP provider URLs should use AllowInsecureHttp only when allow_insecure_http=true.
• TrustedLoopback should remain limited to Lemonade-generated backend URLs whose destination is already known to be 127.0.0.1.

Local cloud proxies and their tests should explicitly enable the existing insecure-HTTP opt-in. That keeps the implementation aligned with the PR description and avoids a second ad-hoc URL parser.

There is also a problem with the new HTTPS-to-HTTP downgrade test. It connects with https:// to a plain httplib::Server, so the TLS handshake fails before the /http-redirect handler can return a redirect. The test then accepts any non-CURLE_OK result, meaning it passes on an unrelated TLS or connection failure and does not verify CURLOPT_REDIR_PROTOCOLS_STR.

Please either remove that test or run a real TLS test server and verify that:

• the HTTPS route was reached,
• it returned the redirect,
• the HTTP target was not reached, and
• curl returned the expected redirect-protocol error.

The main policy implementation is otherwise in good shape, but I would fix these two points before approval.