Code

Multiprocessing and timeouts#8

Code

Merged

theferrit32 merged 10 commits intomainclingen-data-model/clinvar-gk-python:mainfrom multiprocess-and-timeoutsclingen-data-model/clinvar-gk-python:multiprocess-and-timeoutsCopy head branch name to clipboardMay 7, 2024

+333-59

Commits

Commits on Apr 21, 2024

• Adding catvar_combiner script

  theferrit32

  committed
  f0eb146

  Copy full SHA for f0eb146

• Remove extra ndjson dep

  theferrit32

  committed
  6802321

  Copy full SHA for 6802321

• Genericize download_to_local_file and use local path mirror and check if exists when downloading

  theferrit32

  committed
  60efd1f

  Copy full SHA for 60efd1f

• Change output file location to a separate outputs directory

  theferrit32

  committed
  49f9c5f

  Copy full SHA for 49f9c5f

Commits on Apr 30, 2024

• Add partitioner step and multiprocessing call on each partitioned input file. Add task.py although not used

  theferrit32

  committed
  663b36d

  Copy full SHA for 663b36d

• Add start_task_with_timeout, but it's too slow to create new processes

  theferrit32

  committed
  07a72a9

  Copy full SHA for 07a72a9

• Add background process with timeout to worker process. Terminate background processes cleanly

  theferrit32

  committed
  8bacf77

  Copy full SHA for 8bacf77

• Remove task.py

  theferrit32

  committed
  42ce34c

  Copy full SHA for 42ce34c

• Fix test by extracting arg parsing to separate cli module that doesn't need to access seqrepo

  theferrit32

  committed
  cead782

  Copy full SHA for cead782

• lint

  theferrit32

  committed
  8fc6880

  Copy full SHA for 8fc6880