Consider making the credential domain specification more robust

[aepace]

When running hop auth add, the user can optionally enter the hostname, which is useful for juggling multiple credentials across multiple hosts. However, the hostname matching in the client is not robust against host formatting specifications.

Scenario 1:

User gets prompted to optionally enter the hostname when adding a new credential. User enters kafka://kafka.scimma.org, since that's the URL that gets entered when interacting with the hop-client API. The resultant entry in auth.toml looks like:

[[auth]]
username = "alexander.pace-ecbf9436"
password = "-top-secret-"
protocol = "SASL_SSL"
mechanism = "SCRAM-SHA-512"
hostname = "kafka://kafka.scimma.org"
ssl_ca_location = "/path/to/lib/python3.8/site-packages/certifi/cacert.pem"

Then interacting with the client code results in:

In [1]: from hop.list_topics import list_topics

In [2]: list_topics('kafka://kafka.scimma.org')
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-2-6aac4c97738a> in <module>
----> 1 list_topics('kafka://kafka.scimma.org')

~/LIGO/venvs/py38-dev-env/lib/python3.8/site-packages/hop/io.py in list_topics(url, auth)
    450     if auth is True:
    451         credentials = load_auth()
--> 452         user_auth = select_matching_auth(credentials, broker_addresses[0], username)
    453     elif auth is not False:
    454         user_auth = auth

~/LIGO/venvs/py38-dev-env/lib/python3.8/site-packages/hop/auth.py in select_matching_auth(creds, hostname, username)
    288         if username is not None:
    289             err += f" with username '{username}'"
--> 290         raise RuntimeError(err)
    291
    292     if len(matches) > 1:

RuntimeError: No matching credential found for hostname 'kafka.scimma.org'

Scenario 2:

User learned their lesson after the first attempt and omitted the kafka:// scheme from the hostname entry. The resultant entry in auth.toml looks like:

[[auth]]
username = "alexander.pace-ecbf9436"
password = "-top-secret-"
protocol = "SASL_SSL"
mechanism = "SCRAM-SHA-512"
hostname = "kafka.scimma.org"
ssl_ca_location = "/path/to/lib/python3.8/site-packages/certifi/cacert.pem"

This time, the hostname look up works:

In [1]: from hop.list_topics import list_topics

In [2]: list_topics('kafka://kafka.scimma.org')
Out[2]:
{'lvalert-dev.external_swift': TopicMetadata(lvalert-dev.external_swift, 16 partitions),
 'lvalert-dev.burst_cwb_allsky': TopicMetadata(lvalert-dev.burst_cwb_allsky, 16 partitions),
 'lvalert-dev.cbc_pycbc': TopicMetadata(lvalert-dev.cbc_pycbc, 16 partitions),
 ...
...

Scenario 3:

User has the same hostname spec as in the previous scenario (no kafka:// scheme). This time, they try and be thorough in specifying the service URL and include the port number:

In [3]: list_topics('kafka://kafka.scimma.org:9092')
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-3-96e1818ea150> in <module>
----> 1 list_topics('kafka://kafka.scimma.org:9092')

~/LIGO/venvs/py38-dev-env/lib/python3.8/site-packages/hop/io.py in list_topics(url, auth)
    450     if auth is True:
    451         credentials = load_auth()
--> 452         user_auth = select_matching_auth(credentials, broker_addresses[0], username)
    453     elif auth is not False:
    454         user_auth = auth

~/LIGO/venvs/py38-dev-env/lib/python3.8/site-packages/hop/auth.py in select_matching_auth(creds, hostname, username)
    288         if username is not None:
    289             err += f" with username '{username}'"
--> 290         raise RuntimeError(err)
    291
    292     if len(matches) > 1:

RuntimeError: No matching credential found for hostname 'kafka.scimma.org:9092'

[cnweaver]

It's a good point that we're lacking input sanitization here. It says 'hostname', and I knew it meant that literally, so I always followed the instruction, but using a URL instead is indeed an easy mistake to make. We should definitely either reject things which aren't valid hostnames, or perhaps also permit URLs, but extract the hostname from them.

I'm less sure we should forbid port numbers, since that's a feature I actually use in my development/testing work (having different credentials associated with my test Kafka broker depending on the port on which I have it listening). I realize this is not going to come up very often for other people, though, so I could be convinced that we should not store port numbers in the auth data, and should correspondingly ignore them when looking up the credential to use for a given URL.