Occasional insecure redirect from https to http

[psd]

I'm occasionally redirected from a https page to a http URL.

This happens infrequently, since launch and looks like an issue with our cloudfront configuration.

takes me to:

curl -vL 'https://www.planning.data.gov.uk/entity?dataset=local-resilience-forum-boundary'`

* Host www.planning.data.gov.uk:443 was resolved.
* IPv6: (none)
* IPv4: 18.245.162.105, 18.245.162.80, 18.245.162.114, 18.245.162.11
*   Trying 18.245.162.105:443...
* Connected to www.planning.data.gov.uk (18.245.162.105) port 443
* ALPN: curl offers h2,http/1.1
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [10 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [3838 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [264 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [36 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [36 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / x25519 / RSASSA-PSS
* ALPN: server did not agree on a protocol. Uses default.
* Server certificate:
*  subject: CN=planning.data.gov.uk
*  start date: Sep 22 00:00:00 2024 GMT
*  expire date: Oct 22 23:59:59 2025 GMT
*  subjectAltName: host "www.planning.data.gov.uk" matched cert's "*.planning.data.gov.uk"
*  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M02
*  SSL certificate verify ok.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
* using HTTP/1.x
} [5 bytes data]
> GET /entity?dataset=local-resilience-forum-boundary HTTP/1.1
> Host: www.planning.data.gov.uk
> User-Agent: curl/8.9.1
> Accept: */*
> 
* Request completely sent off
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [157 bytes data]
< HTTP/1.1 307 Temporary Redirect
< Transfer-Encoding: chunked
< Connection: keep-alive
< Date: Thu, 20 Feb 2025 11:07:07 GMT
< x-frame-options: sameorigin
< x-content-type-options: nosniff
< server: uvicorn
< location: http://www.planning.data.gov.uk/entity/?dataset=local-resilience-forum-boundary
< strict-transport-security: max-age=63072000.0; includeSubDomains; preload
< X-Cache: Miss from cloudfront
< Via: 1.1 5c5242096d35222c5309865697de769a.cloudfront.net (CloudFront)
< X-Amz-Cf-Pop: LHR5-P2
< X-Amz-Cf-Id: Z_zk_DYzI2IFQ27Y_4S1MyaGuufrhCEJgdNsQas5rUq6VA96dA_Q9g==
* Ignoring the response-body
< 
{ [5 bytes data]
* Connection #0 to host www.planning.data.gov.uk left intact
* Clear auth, redirects to port from 443 to 80
* Issue another request to this URL: 'http://www.planning.data.gov.uk/entity/?dataset=local-resilience-forum-boundary'
* Switched from HTTP to HTTPS due to HSTS => https://www.planning.data.gov.uk/entity/?dataset=local-resilience-forum-boundary
* Found bundle for host: 0x5aa62496e6c0 [serially]
* Can not multiplex, even if we wanted to
* Re-using existing connection with host www.planning.data.gov.uk
} [5 bytes data]
> GET /entity/?dataset=local-resilience-forum-boundary HTTP/1.1
> Host: www.planning.data.gov.uk
> User-Agent: curl/8.9.1
> Accept: */*
> 
* Request completely sent off
{ [5 bytes data]
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=utf-8
< Content-Length: 9570195
< Connection: keep-alive
< Date: Thu, 20 Feb 2025 11:07:17 GMT
< x-content-type-options: nosniff
< server: uvicorn
< strict-transport-security: max-age=63072000.0; includeSubDomains; preload
< x-frame-options: sameorigin
< X-Cache: Miss from cloudfront
< Via: 1.1 5c5242096d35222c5309865697de769a.cloudfront.net (CloudFront)
< X-Amz-Cf-Pop: LHR5-P2
< X-Amz-Cf-Id: es7AFtTV3J7e6SDFE3JLZFpbvP7pDU2iTYewGZdtByeQyzTW1AqHLA==
< 

[Ben-Hodgkiss]

Note for refinement: as well as fixing this, how can we set up an alert to check this doesn't reoccur?

[cpcundill]

Had to go on a bit of a detour here to get the CloudFront logs sent through to CloudWatch Logs. With that resolved, I don't unfortunately have enough information from the logs as to why there's a redirect to a URL using the http scheme instead of https. I believe the best next step will be to add some logging to the main application which prints the incoming headers. I'd also like logging on redirect responses so we can see whether the Location header is correctly set by the application.

I'd also like to know whether the X-Forwarded-For headers are coming through from CloudFront. My suspicion is that the gunicorn/Fast API application might not be respecting those headers when generating redirect responses. I note that there is some configuration which can be added to help influence this behaviour:

https://www.uvicorn.org/settings/#http

Specifically, the addition of --forwarded-allow-ips "*" is possibly required.

However, until we have logs to prove the current situation, there's no point in trying to make configuration changes.

Would you be happy @Ben-Hodgkiss for a ticket to pick up the additional logging on the main application?

[Ben-Hodgkiss]

@cpcundill - thanks for doing the research. This sounds sensible I think! Would you be able to add the ticket as a sub-issue of this one so we can then implement that and subsequently keep an eye out for URLs redirecting to http going forward. We can then keep this ticket open and check the logs after a week or so to see if we can find any evidence of it.

[cpcundill]

@Ben-Hodgkiss - Added the sub-issue which is probably best for a developer to pick up.