Skip to content

Commit

Permalink
Merge pull request #791 from hyperrealist/improve_json_seq_deserializ…
Browse files Browse the repository at this point in the history
…ation

Improve  JSON sequence deserialization using `newline` delimiter instead of `try except`
  • Loading branch information
jmaruland authored Jan 25, 2024
2 parents 4d20ea4 + d53e171 commit fb681ce
Showing 1 changed file with 7 additions and 5 deletions.
12 changes: 7 additions & 5 deletions databroker/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,14 +81,16 @@ def documents(self, fill=False):
handle_error(response)
tail = ""
for chunk in response.iter_bytes():
for line in chunk.decode().splitlines():
try:
for line in chunk.decode().splitlines(keepends=True):
if line[-1] == "\n":
item = json.loads(tail + line)
except json.JSONDecodeError:
tail += line
else:
yield (item["name"], _document_types[item["name"]](item["doc"]))
tail = ""
else:
tail += line
if tail:
item = json.loads(tail)
yield (item["name"], _document_types[item["name"]](item["doc"]))

def __getattr__(self, key):
"""
Expand Down

0 comments on commit fb681ce

Please sign in to comment.