Not support "no_proxy" including value of ipv6 prefix style

[piamo]

Example: no_proxy=fe11::/16

How to reproduce:

no_proxy=fe11::/16 python -c 'import httpx; c = httpx.Client()'

it will raise:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/httpx/_urlparse.py", line 346, in normalize_port
    port_as_int = int(port)
ValueError: invalid literal for int() with base 10: ':'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 695, in __init__
    self._mounts: dict[URLPattern, BaseTransport | None] = {
  File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 696, in <dictcomp>
    URLPattern(key): None
  File "/usr/local/lib/python3.9/dist-packages/httpx/_utils.py", line 370, in __init__
    url = URL(pattern)
  File "/usr/local/lib/python3.9/dist-packages/httpx/_urls.py", line 115, in __init__
    self._uri_reference = urlparse(url, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/httpx/_urlparse.py", line 248, in urlparse
    parsed_port: int | None = normalize_port(port, scheme)
  File "/usr/local/lib/python3.9/dist-packages/httpx/_urlparse.py", line 348, in normalize_port
    raise InvalidURL(f"Invalid port: {port!r}")
httpx.InvalidURL: Invalid port: ':'

httpx version: 0.27.0

[matstrand]

Another test case to throw in that is affecting us:

 no_proxy=[::1] python -c 'import httpx; c = httpx.Client()'
...
httpx.InvalidURL: Invalid port: ':1]'

[Godsing]

For no_proxy=127.0.0.0/8, get_environment_proxies() will get all://127.0.0.0/8.
So, when creating a Client, in which URL class will be called, ⁠/8 will be treated as a URL path finally.

python -c 'import httpx; url = httpx.URL("all://127.0.0.0/8"); print(url.path)'
/8

httpx version: 0.27.0

[tomchristie]

Duplicate of #1536

No we don't currently support subnet masks here, yes we should at least raise an error if they're in play.

Aside... what's an example of a real-world scenario where it's useful to have a subnet masking used to toggle proxy routing on/off?

[yanyongyu]

I also encountered this issue when using proxy in company's network. We use proxy to visit outside networks and automatically set no_proxy with both ipv4/ipv6 masking to prevent proxying specific ranges of IPs, such as 127.0.0.0/8,169.254.0.0/16,100.64.0.0/10,172.16.0.0/12,192.168.0.0/16,10.0.0.0/8,::1,fe80::/10,fd00::/8.

[GreyElaina]

Here’s my take:

For no_proxy, I think we should address the issue in phases, so create a branch rather than do all in the main one.

Let’s fix the crash problem first, rather than implementing the full semantic functionality (like subnet mask support) in httpx. Since no_proxy is commonly used in VPN and Zero Trust setups, It could be to adapt the no_proxy parser to handle these cases and prevent crashes at the first step.

Of course, just preventing crashes isn’t a perfect solution, it will break the intended semantics of no_proxy and could lead to unexpected behavior.

I also suggest considering a refactor of the proxy handling in httpx, allowing dispatching at both the domain and the ip:port level. However, this might be too extreme and could cause a breaking change to the API. I’d love to hear ideas from you all.

As for some potential concerns, here are my quick responses:

1. Is ensuring stability and preventing application crashes a valid intermediate step?

I think this could change behavior between versions, which I’m not comfortable with—even with a warning mechanism in place.

2. Is limiting or configurable advanced features acceptable?

That would compromise the consistency of the API and break the intended semantics of no_proxy inside.

3. Could this introduce too much complexity?

I’m not sure, so I wander tom on this.

4. How do other HTTP libraries handle subnet masks? Anything to learn from them?

Some languages do often require extra configuration(java, nodejs), but Python’s tradition is to make things work out-of-the-box.
(I doubt other languages would adopt the idea of ignoring environment variables by default.)