Parse fails on certain documents, throws poor error

[jnu]

Get a NoneType is not iterable error on some documents with parse:openai in the pipeline.

Reproduce:

extract:azuredi
parse:openai
redact:openai

use file 002_md_baltimore_pd_CCN-5-170308460.pdf. Extract doesn't fail (or it would raise EmptyExtractionError, but there is None value in the input to the redact step. This must mean that parse failed. It should throw a better error, but it would also be nice to know why it fails in the first place.

Using prompt in kscms blind-charging-api deployment.

[chohlasa]

I can't reproduce this bug, possibly because I can't get the pipeline to run on the Karpel VM. It runs through the extract:azuredi step, but then throws a HttpResponseError: (403) Access denied due to Virtual Network/Firewall rules. error when running the parse step. Not sure how to fix networking issues like this!

When I try running the same pipeline in our HKS L1 environment, it works just fine for me.

I have a couple theories for what might be happening:

1. To my eye, it looks like the Karpel GPT-4o deployment has the default content filter on, instead of the NoFilter filter you set up. And perhaps this particular record is triggering the default content filter because it's the only example police report to talk about a homicide? See below for a (redacted) screenshot of the deployment.
2. The current parse prompt states: ... If you are unable to identify any police narratives in the output, please return an empty string. So perhaps it's just consistently not finding the narrative and returning nothing.

I have a few suggestions we can try to fix this issue:

1. Update the applied filter for GPT-4o, if the current filter is indeed the default filter.
2. Update the prompt to the more expansive prompts for parsing and redacting that I have been testing and validating.
3. Add temperature and seed arguments to the parse and redact steps for more consistent, conservative outputs.

I can push changes (2) and (3) to the kscms file in blind-charging-api deployment. I can also make the content filter change, but will wait for confirmation from @jnu before I do so.

[jnu]

oh wow, thanks so much for investigating! good catch on the content filter. I've updated that to use NoFilter and will ask them to retry right now.

[chohlasa]

Ok great! I would also love to do (2) and (3) but only when it's convenient for you.

[jnu]

fixed by #94