JSONDecodeError when dispatcher responds with duplicate_query error #172
Description
Hi there,
Since around March this year [1], Overpass API refuses to handle duplicate queries originating from the same IP within a short time span.
In that case, the content type of the response is text/html; charset=utf-8, with content
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8" lang="en"/>
<title>OSM3S Response</title>
</head>
<body>
<p>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</p>
<p><strong style="color:#FF0000">Error</strong>: runtime error: open64: 0 Success /osm3s_osm_base Dispatcher_Client::request_read_and_idx::duplicate_query </p>
</body>
</html>
API.get() does not check for text/html and tries to parse the response as JSON, which fails:
Traceback (most recent call last):
File "/home/mbugert/.../playground.py", line 326, in <module>
plot()
File "/home/mbugert/.../playground.py", line 305, in plot
rep = api.get(overpass_query, verbosity="geom")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mbugert/.local/bin/pyenv/versions/3.11/lib/python3.11/site-packages/overpass/api.py", line 114, in get
response = json.loads(r.text)
^^^^^^^^^^^^^^^^^^
File "/home/mbugert/.local/bin/pyenv/versions/3.11.10/lib/python3.11/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mbugert/.local/bin/pyenv/versions/3.11.10/lib/python3.11/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mbugert/.local/bin/pyenv/versions/3.11.10/lib/python3.11/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
This is on python 3.11 with overpass==0.7.2.