currently in backend/app/routes/files we have:

 # Add entry to the file index
    await index_file(es, FileOut(**new_file.dict()))

    # TODO - timing issue here, check_feed_listeners needs to happen asynchronously.
    time.sleep(1)

    # Submit file job to any qualifying feeds
    await check_feed_listeners(
        es,
        FileOut(**new_file.dict()),
        user,
        rabbitmq_client,
    )

Currently, check_feed_listeners performs queries on Elasticsearch to check if extractors should be triggered. It seems that the index_file() call takes a second to propagate and I was seeing situation where feed wasn't triggering after the index if I didn't delay for a second.

This should probably follow a different pattern:

• index the file
• issue a message to RMQ or a memory queue that says "you need to check this file for extractors"
• a separate (different thread) service handles those message checks, similar to our heartbeat/messages services