Cannot fully implement mTLS

[moodiRealist]

I have a http server on a kubernetes cluster which uses a tls secret created based on the steps from this blog:
https://awkwardferny.medium.com/configuring-certificate-based-mutual-authentication-with-kubernetes-ingress-nginx-20e7e38fdfca

With the above setup, even curl command to the service won't work without providing the correct client.crt and client.key files!

My python httpx client code is instantiated this way:
async with httpx.AsyncClient(verify=False, cert=("/.../client.crt", "/.../client.key")) as http_client:
____r = await http_client.post(url=url, content=buf, headers=headers, timeout=None)

Which only gets accepted by the nginx on k8s when correct client crt and client key are provided (would receive a 400 error if certs are not provided)

However no matter what I put in the verify section (client.crt , ca.crt, server.crt) I get the following exception:
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpx/_client.py", line 1624, in post return await self.request( File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpx/_client.py", line 1361, in request response = await self.send( File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpx/_client.py", line 1396, in send response = await self._send_handling_auth( File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpx/_client.py", line 1434, in _send_handling_auth response = await self._send_handling_redirects( File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpx/_client.py", line 1466, in _send_handling_redirects response = await self._send_single_request(request, timeout) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpx/_client.py", line 1492, in _send_single_request (status_code, headers, stream, ext) = await transport.arequest( File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/contextlib.py", line 135, in __exit__ self.gen.throw(type, value, traceback) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpx/_exceptions.py", line 343, in map_exceptions raise mapped_exc(message, **kwargs) from exc # type: ignore httpx.ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)

What should be the content of verify so that client can properly validate server? I am using httpx 0.17.1 btw

[tomchristie]

Anyone able to spare some time helping get @moodiRealist rolling here?

Yes, figuring out which parameter goes where here is fiddly. Yes the error tracebacks are a bit useless.

You might want to first double check accessing with a different tool. (is everything working okay with curl? how about requests?)

[moodiRealist]

yeah the curl command works ok when --cert and --key values are provided.

Ok I can try to use requests library instead of httpx to see if that works