update: Give clearer message when header encoding fails

[leondz]

resolves #1035 (best effort)

HTTP under base RFC2616 only supports ASCII and latin-1 (iso-8859-1) encodings. This is echoed in Python's http lib implementation. When characters from outside these sets turn up in certain parts of an HTTP requests, an encoding exception is raised.

This PR catches that exception, wraps it in an error message, logs with a suggestion, and then tries to stop generation using a BadGeneratorException.

Design decisions:

• Because the encoding is likely in the HTTP headers or other meta-information, and this information will be static across all requests, it's likely none of the requests will succeed. In this case we may as well give up the run as soon as possible, rather than skipping.
• Because headers may contain API keys, we don't try to log or print the request that led to the failure

Verification

• python -m pytest tests/generators/test_rest::test_rest_non_latin1

[jmartin-tech]

Having read the reported issue I think the raise here may not be an appropriate type. BadGeneratorException extends PluginConfigurationError and this error is not related to the configuration at this time.

This error seems like a new exception class may be appropriate, something like BadResponseException / InvalidResponseError / MalformedResponseException. The message helps the user understand that the issue is with the response data, however I am looking for something programatic to make it clear that the target systems response is not supported vs suggesting that configuration of the generator could be changed to avoid the issue.

[leondz]

This catches an exception during request composition

…

On Thu, Jan 30, 2025, 16:55 Jeffrey Martin ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In garak/generators/rest.py <#1088 (comment)>: > @@ -212,7 +212,16 @@ def _call_model( "proxies": self.proxies, "verify": self.verify_ssl, } - resp = self.http_function(self.uri, **req_kArgs) + try: + resp = self.http_function(self.uri, **req_kArgs) + except UnicodeEncodeError as uee: + # only RFC2616 (latin-1) is guaranteed + # don't print a repr, this might leak api keys + logging.error( + "Only latin-1 encoding supported by HTTP RFC 2616, check headers and values for unusual chars", + exc_info=uee, + ) + raise BadGeneratorException from uee Having read the reported issue I think the raise here may not be an appropriate type. BadGeneratorException extends PluginConfigurationError and this error is not related to the configuration at this time. This error seems like a new exception class may be appropriate, something like BadResponseException / InvalidResponseError / MalformedResponseException. The message helps the user understand that the issue is with the response data, however I am looking for something programatic to make it clear that the target systems response is not supported vs suggesting that configuration of the generator could be changed to avoid the issue. — Reply to this email directly, view it on GitHub <#1088 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA5YTVXEK5UDNK4FGWK4UL2NJDQXAVCNFSM6AAAAABV5NRIZ2VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDKOBUGMZDAOBWGE> . You are receiving this because you authored the thread.Message ID: ***@***.***>