Unable to configure the routing-api to disable the HTTP listener in cf-deployment

[amhuber]

Issue

Unable to configure the routing-api to disable the HTTP listener in cf-deployment due to issues with the DNS healthcheck and the registration of the api.system_domain/routing route.

Affected Versions

All recent versions of cf-deployment and routing-release.

Context

As discussed at cloudfoundry/cf-deployment#906, when configuring the routing_api.enabled_api_endpoints manifest property to "mtls", the routing-api does correctly not start the listener on the HTTP port (3000 by default) but the routing release is not functional due to two issues:

• The DNS healthcheck at https://github.com/cloudfoundry/routing-release/blob/develop/jobs/routing-api/templates/dns_health_check.erb#L3 is configured to still use the (now disabled) HTTP port
• The routing-api still uses the HTTP port to register the api.system_domain/routing route and configures the route with TLS disabled which is non functional.

Steps to Reproduce

Configure routing_api.enabled_api_endpoints to "mtls" and deploy the current cf-deployment release.

Expected result

• The connections to https://routing-api.service.cf.internal from gorouter should continue to work due to the passing DNS healthcheck on the active routing-api node.
• The connection to https://api.system_domain/routing from CCNG should continue to work because the route will be successfully registered with the MTLS port (3001) and with TLS enabled.
• All traffic from the gorouter to the routing-api will be encrypted with MTLS, not exposing the UAA secret in clear text on the network.

Current result

• The DNS healthcheck is failing due to using the wrong port, leading the routing-api.service.cf-internal DNS name to not resolve, causing gorouter to fail to connect.
• The connection to https://api.system_domain/routing from CCNG fails due to the route using the wrong port and TLS setting.
• Traffic between gorouter and the routing-api requires HTTP, which exposes the secret in clear text on the network.

Possible Fix

Modify the routing-api route registration logic as necessary when enabled_api_endpoints is configured to "mtls", and use an appropriate if statement in the DNS healthcheck to use the correct port per the setting.

[ameowlia]

Thank you for writing this up @amhuber. As always, contributing a PR is the fastest way to get fixes in. Please let me know if you are interested in contributing one. Until then, I will add this to the team's backlog to fix.

Thanks,
Amelia Downs, CF Networking Engineer

[amhuber]

We've done a few pull requests previously for simpler fixes like updates to templates or spec files, but I don't have any golang developers on my team who are capable of changing the routing-api code and the unit tests.

[46bit]

I've made two commits above that I think should together address this issue. We don't actually run routing-api so I'm not going to have time to add tests and make them into full PRs. Anyone else is welcome to carry that on if they want to get this done.

[amhuber]

I was just going to start seeing if I could figure out enough golang to add your commits and see if the tests are breaking, but when I look now it looks like the commits you made are gone. I don't suppose you happen to have the code changes you made somewhere that you can share again?

[46bit]

@amhuber oh no :( I cleaned up some forks recently and it must have been in one. I'll set a reminder for next week to see if I still have a copy

[46bit]

@amhuber @ameowlia Unfortunately I don't have a copy of the commits. As I recall they were quite simple – adding the extra config option described by this issue. I'm away for the next couple of weeks.

[amhuber]

FYI this appears to be addressed now by cloudfoundry/cf-deployment#1014 and #300