Code

Convert workflow to use Pegasus WMS#5

Code

Open

spigo900 wants to merge 161 commits intobaseisi-nlp/ai2:basefrom base-pegasizedisi-nlp/ai2:base-pegasizedCopy head branch name to clipboard

+950-435

Commits

Commits on Aug 18, 2020

• Add Pegasus wrapper requirement

  

  Joseph Cecil

  authored and

  spigo900

  committed
  da52c7c

  Copy full SHA for da52c7c

• Add parameter files

  

  Joseph Cecil

  authored and

  spigo900

  committed
  d7ca092

  Copy full SHA for d7ca092

• Add vistautils requirement

  

  Joseph Cecil

  authored and

  spigo900

  committed
  b2e862b

  Copy full SHA for b2e862b

• Initial Pegasus skeleton

  

  Joseph Cecil

  authored and

  spigo900

  committed
  e1adb5a

  Copy full SHA for e1adb5a

• Update root parameters file

  

  Joseph Cecil

  authored and

  spigo900

  committed
  b01a1db

  Copy full SHA for b01a1db

• Pass real job parameters to Pegasus

  

  Joseph Cecil

  authored and

  spigo900

  committed
  57fe8b2

  Copy full SHA for 57fe8b2

• Clean up some

  

  Joseph Cecil

  authored and

  spigo900

  committed
  f9cd332

  Copy full SHA for f9cd332

• Store more job info

  

  Joseph Cecil

  authored and

  spigo900

  committed
  bae796e

  Copy full SHA for bae796e

• Fix _includes in runner.params

  

  Joseph Cecil

  authored and

  spigo900

  committed
  6fcd5d1

  Copy full SHA for 6fcd5d1

• More pegasus stuff

  

  Joseph Cecil

  authored and

  spigo900

  committed
  feb5167

  Copy full SHA for feb5167

• Use simple loop to read in job parameters for Pegasus runner

  

  Joseph Cecil

  authored and

  spigo900

  committed
  ed3b77c

  Copy full SHA for ed3b77c

• Create parameters files mirroring Hydra configs

  

  Joseph Cecil

  authored and

  spigo900

  committed
  57d8c85

  Copy full SHA for 57d8c85

• Clean up SAGA code

  

  Joseph Cecil

  authored and

  spigo900

  committed
  1963323

  Copy full SHA for 1963323

• Modify train.py to take parameters files as input

  

  Joseph Cecil

  authored and

  spigo900

  committed
  0a23da0

  Copy full SHA for 0a23da0

• Don't overwrite actual parameters with combinations parameters.

  

  Joseph Cecil

  authored and

  spigo900

  committed
  fc03fb3

  Copy full SHA for fc03fb3

• Fix style issues

  

  Joseph Cecil

  authored and

  spigo900

  committed
  917c595

  Copy full SHA for 917c595

• Move Slurm configuration into configuration file

  

  Joseph Cecil

  authored and

  spigo900

  committed
  57750a8

  Copy full SHA for 57750a8

• Remove old TODO

  

  Joseph Cecil

  authored and

  spigo900

  committed
  5beadd0

  Copy full SHA for 5beadd0

• Fix usage of parameters_only_entry_point()

  

  Joseph Cecil

  authored and

  spigo900

  committed
  c5bdf99

  Copy full SHA for c5bdf99

• Fix runner params.

  

  Joseph Cecil

  authored and

  spigo900

  committed
  55ef547

  Copy full SHA for 55ef547

• Fix resource request creation.

  

  Joseph Cecil

  authored and

  spigo900

  committed
  b9e4cc4

  Copy full SHA for b9e4cc4

• Fix parameters file.

  

  Joseph Cecil

  authored and

  spigo900

  committed
  40510fd

  Copy full SHA for 40510fd

• Fix params->mapping conversion.

  

  Joseph Cecil

  authored and

  spigo900

  committed
  d4aeda1

  Copy full SHA for d4aeda1

• Fix parameter name.

  

  Joseph Cecil

  authored and

  spigo900

  committed
  5617c4a

  Copy full SHA for 5617c4a

• Update resource request creation.

  

  Joseph Cecil

  authored and

  spigo900

  committed
  8da14a2

  Copy full SHA for 8da14a2

• Fix partition in runner.params

  

  Joseph Cecil

  authored and

  spigo900

  committed
  f77bdab

  Copy full SHA for f77bdab

• No task2 goes in _default directory.

  Show description for 7709ff5
  

  Joseph Cecil

  authored and

  spigo900

  committed
  7709ff5

  Copy full SHA for 7709ff5

• Start converting ensemble.py

  

  Joseph Cecil

  authored and

  spigo900

  committed
  6022168

  Copy full SHA for 6022168

• Fix typo

  

  Joseph Cecil

  authored and

  spigo900

  committed
  63379d1

  Copy full SHA for 63379d1

• Fix another typo

  

  Joseph Cecil

  authored and

  spigo900

  committed
  74adcb6

  Copy full SHA for 74adcb6

• Fix another typo

  

  Joseph Cecil

  authored and

  spigo900

  committed
  f6a26da

  Copy full SHA for f6a26da

• Set up dependencies properly

  

  Joseph Cecil

  authored and

  spigo900

  committed
  2e7fca8

  Copy full SHA for 2e7fca8

• Remove old comment

  

  Joseph Cecil

  authored and

  spigo900

  committed
  e7ec4a0

  Copy full SHA for e7ec4a0

• Delete unnecessary parameter

  

  Joseph Cecil

  authored and

  spigo900

  committed
  b61eca8

  Copy full SHA for b61eca8

• Fix ensembling parameters setup

  

  Joseph Cecil

  authored and

  spigo900

  committed
  aa046dc

  Copy full SHA for aa046dc

• Re-add depends_on argument

  

  Joseph Cecil

  authored and

  spigo900

  committed
  2d84ed8

  Copy full SHA for 2d84ed8

• Eliminate some redundant informatoin in ensembling parameters

  

  Joseph Cecil

  authored and

  spigo900

  committed
  c15eb85

  Copy full SHA for c15eb85

• Convert ensembling script to use parameters

  

  Joseph Cecil

  authored and

  spigo900

  committed
  8bb859e

  Copy full SHA for 8bb859e

• Style fix

  spigo900

  committed
  cc1b973

  Copy full SHA for cc1b973

• Move backend parameter to another parameter file

  spigo900

  committed
  ba1ff93

  Copy full SHA for ba1ff93

• Add missing EOL at ends of files

  spigo900

  committed
  5b7937f

  Copy full SHA for 5b7937f

• Temporarily disable use of ephemeral

  spigo900

  committed
  7bb7f15

  Copy full SHA for 7bb7f15

• Fix invalid reference problem with Pegasus

  spigo900

  committed
  da07237

  Copy full SHA for da07237

• Fix another invalid reference in Pegasus script

  spigo900

  committed
  5758962

  Copy full SHA for 5758962

• Specify name in setup.py

  spigo900

  committed
  0d65f18

  Copy full SHA for 0d65f18

• Move name back into setup.py from setup.cfg.

  Show description for 4580ffe
  spigo900

  committed
  4580ffe

  Copy full SHA for 4580ffe

• Add missing comma

  spigo900

  committed
  29fc1c7

  Copy full SHA for 29fc1c7

• Fix typo

  spigo900

  committed
  6a1e9ec

  Copy full SHA for 6a1e9ec

• Fix Python references again

  spigo900

  committed
  c788a7f

  Copy full SHA for c788a7f

• Fix crash when running train.py script

  spigo900

  committed
  11d2fdf

  Copy full SHA for 11d2fdf

• Re-fix references in pegasus.py

  spigo900

  committed
  e6c7ef0

  Copy full SHA for e6c7ef0

• Try including model parameters in Pegasus train job parameters

  spigo900

  committed
  e4ab577

  Copy full SHA for e4ab577

• Fix parameter _includes reference

  spigo900

  committed
  eaca966

  Copy full SHA for eaca966

• Fix incorrect memory parameter name

  spigo900

  committed
  32d8cda

  Copy full SHA for 32d8cda

• Specify time limit

  spigo900

  committed
  08e3148

  Copy full SHA for 08e3148

• Fix typo

  spigo900

  committed
  727785d

  Copy full SHA for 727785d

• Specify ensembling output correctly in Pegasus

  spigo900

  committed
  87dae91

  Copy full SHA for 87dae91

• Fix typo

  spigo900

  committed
  dd72e90

  Copy full SHA for dd72e90

• Fix issue with "do nothing" options

  spigo900

  committed
  5ee4d2e

  Copy full SHA for 5ee4d2e

• Specify in Pegasus not to use pretrained model when training

  spigo900

  committed
  1159d87

  Copy full SHA for 1159d87

• Correctly set time limit in 'not alphanli' branch

  spigo900

  committed
  7ab0820

  Copy full SHA for 7ab0820

• Don't print parameter options (handled by vistautils)

  spigo900

  committed
  39cae0d

  Copy full SHA for 39cae0d

• Increase memory request size

  spigo900

  committed
  94a93da

  Copy full SHA for 94a93da

• Fix how task data paths are defined in parameters

  spigo900

  committed
  c11ea14

  Copy full SHA for c11ea14

• Fix typo (_include vs. _includes)

  spigo900

  committed
  fe37980

  Copy full SHA for fe37980

• train_data_slice is no longer an int

  spigo900

  committed
  c0b587d

  Copy full SHA for c0b587d

• Treat train_data_slice as an int

  spigo900

  committed
  b09a36b

  Copy full SHA for b09a36b

• Fix data params again

  spigo900

  committed
  7dbff58

  Copy full SHA for 7dbff58

• Fix typo

  spigo900

  committed
  af35c9f

  Copy full SHA for af35c9f

• Fix another typo

  spigo900

  committed
  3370ab7

  Copy full SHA for 3370ab7

• Fix yet another typo

  spigo900

  committed
  9676232

  Copy full SHA for 9676232

• Add quotes to includes for consistency

  spigo900

  committed
  2f566c1

  Copy full SHA for 2f566c1

• Add radically smaller-scale Pegasus script parameters for development

  spigo900

  committed
  9208b67

  Copy full SHA for 9208b67

• Rename Pegasus parameter file for consistency

  spigo900

  committed
  e71a4ef

  Copy full SHA for e71a4ef

• Clean up Pegasus parameter files

  spigo900

  committed
  4c78bf8

  Copy full SHA for 4c78bf8

• Specify missing configuration key

  spigo900

  committed
  21a6cb3

  Copy full SHA for 21a6cb3

• Fix typo

  spigo900

  committed
  e5004cd

  Copy full SHA for e5004cd

• Provide task data to ensemble script properly

  spigo900

  committed
  cc982d3

  Copy full SHA for cc982d3

• Fix deprecation warning

  spigo900

  committed
  6f204a4

  Copy full SHA for 6f204a4

• Fix another typo

  spigo900

  committed
  170567b

  Copy full SHA for 170567b

• Fix yet another typo

  spigo900

  committed
  0100bd1

  Copy full SHA for 0100bd1

• Explicitly convert to string when making model name

  spigo900

  committed
  dc26fe8

  Copy full SHA for dc26fe8

• More string conversions

  spigo900

  committed
  f156d68

  Copy full SHA for f156d68

• Access parameters as key, not property

  spigo900

  committed
  cf5f753

  Copy full SHA for cf5f753

• Fix: Store model name, not model map in best_model_per_seed_group

  spigo900

  committed
  921f922

  Copy full SHA for 921f922

• Fix typo

  spigo900

  committed
  3dbd882

  Copy full SHA for 3dbd882

• Fix how ensemble output file name is resolved

  spigo900

  committed
  24c8d52

  Copy full SHA for 24c8d52

• Refactor AlphaNLI special case slightly

  spigo900

  committed
  4a55d71

  Copy full SHA for 4a55d71

• Don't log config (vistautils does this already)

  spigo900

  committed
  46d37fe

  Copy full SHA for 46d37fe

• Refactor how "without {factor}" ensembling is done

  spigo900

  committed
  679fd42

  Copy full SHA for 679fd42

• Delete config directory

  spigo900

  committed
  3605367

  Copy full SHA for 3605367

• Comment Pegasus script more

  spigo900

  committed
  293c3c2

  Copy full SHA for 293c3c2

• Better typing for parameter_combinations list

  spigo900

  committed
  fd0aa9f

  Copy full SHA for fd0aa9f

• Fix eval.py to use parameters/eval.params

  Show description for db1bc9a
  spigo900

  committed
  db1bc9a

  Copy full SHA for db1bc9a

• PIQA: Switch to internal dev set to match base

  spigo900

  committed
  caeea41

  Copy full SHA for caeea41

• Fix invalid reference problem with Pegasus

  spigo900

  committed
  4b4e308

  Copy full SHA for 4b4e308

• Delete config directory

  spigo900

  committed
  7b3c794

  Copy full SHA for 7b3c794

• Use parameters better in train.py

  spigo900

  committed
  7972358

  Copy full SHA for 7972358

• Save to experiment root by default instead of project root

  spigo900

  committed
  3fae60b

  Copy full SHA for 3fae60b

• Emulate Hydra dated-saving behavior

  spigo900

  committed
  3088bd3

  Copy full SHA for 3088bd3

• Remove Hydra-related training parameters

  spigo900

  committed
  a737204

  Copy full SHA for a737204

• Convert eval.py to use parameters

  spigo900

  committed
  2e126e8

  Copy full SHA for 2e126e8

• eval.py: Fix checkpoint path parameter

  spigo900

  committed
  d16a13b

  Copy full SHA for d16a13b

• eval.py: Fix random seeding

  Show description for 294ddc2
  spigo900

  committed
  294ddc2

  Copy full SHA for 294ddc2

• Revert "eval.py: Fix random seeding"

  Show description for 5be58a2
  spigo900

  committed
  5be58a2

  Copy full SHA for 5be58a2

• eval.py: Match how train.py gets random seed

  spigo900

  committed
  25680b3

  Copy full SHA for 25680b3

• train.py: Rework dated-saving functionality

  spigo900

  committed
  244f934

  Copy full SHA for 244f934

• train.py: Fix how output directory is logged

  spigo900

  committed
  e56bcd5

  Copy full SHA for e56bcd5

• train.py: Fix call to evaluate using wrong parameters

  spigo900

  committed
  c3186e1

  Copy full SHA for c3186e1

• train.py: Fix build_on_pretrained_model parameter

  spigo900

  committed
  7464af2

  Copy full SHA for 7464af2

• Don't pass build_on_pretrained_model in pegasus.py

  spigo900

  committed
  39d2d68

  Copy full SHA for 39d2d68

• Pass save_by_date_and_parameters=False to Pegasus training jobs

  spigo900

  committed
  50ac2c4

  Copy full SHA for 50ac2c4

• Don't pass build_on_pretrained_model in training parameters file

  spigo900

  committed
  ef764e9

  Copy full SHA for ef764e9

• Try to fix SLURM scripts (except cross_task_eval)

  spigo900

  committed
  f4dfbd2

  Copy full SHA for f4dfbd2

• Grab all needed parameters at beginning of Pegasus script

  spigo900

  committed
  1c10d3f

  Copy full SHA for 1c10d3f

• Implement combination-specific overrides

  spigo900

  committed
  3ca623f

  Copy full SHA for 3ca623f

• Pollute training parameters less in Pegasus workflow

  spigo900

  committed
  abbbe4c

  Copy full SHA for abbbe4c

• dict.get() can't take default as a keyword argument

  spigo900

  committed
  172a66c

  Copy full SHA for 172a66c

• Fix: Multiply by number of possible values

  spigo900

  committed
  f8592e3

  Copy full SHA for f8592e3

• Fix override matching function

  spigo900

  committed
  f849ab3

  Copy full SHA for f849ab3

• Fix pegasus-dev overrides so that they make sense

  spigo900

  committed
  28364d6

  Copy full SHA for 28364d6

• Fix override_matches some more

  spigo900

  committed
  e75f752

  Copy full SHA for e75f752

• Fix override_matches even more

  spigo900

  committed
  c3c60d2

  Copy full SHA for c3c60d2

• Fix override complexity and matching

  spigo900

  committed
  e4801f8

  Copy full SHA for e4801f8

• Finally fix override code

  spigo900

  committed
  df1b7ce

  Copy full SHA for df1b7ce

• Clean up override code slightly

  spigo900

  committed
  4adff05

  Copy full SHA for 4adff05

• Change pegasus-dev experiment root

  spigo900

  committed
  aaa7ece

  Copy full SHA for aaa7ece

• Pegasus: Fix how model parameters are passed to training script

  spigo900

  committed
  f171767

  Copy full SHA for f171767

• Train: Pass other parameters to model

  spigo900

  committed
  412844f

  Copy full SHA for 412844f

• Add .items()

  spigo900

  committed
  1acb61e

  Copy full SHA for 1acb61e

• Fix training parameter files

  spigo900

  committed
  9fd6047

  Copy full SHA for 9fd6047

• training: Fix parameter name in files

  spigo900

  committed
  4a4f162

  Copy full SHA for 4a4f162

• Ensemble: Fix try_without

  spigo900

  committed
  69ce3b1

  Copy full SHA for 69ce3b1

• Fix dev ensembling params

  spigo900

  committed
  1d1814d

  Copy full SHA for 1d1814d

• Fix how eval.py gets model name

  spigo900

  committed
  6b77a09

  Copy full SHA for 6b77a09

• Fix how eval.py loads the model configuration

  spigo900

  committed
  cbe7e6e

  Copy full SHA for cbe7e6e

• eval.py: Include other parameters in model configuration

  Show description for b869efc
  spigo900

  committed
  b869efc

  Copy full SHA for b869efc

• Fix deprecation warning in eval.py

  Show description for c1814e0
  spigo900

  committed
  c1814e0

  Copy full SHA for c1814e0

• Override testing: Change nonsense parameter instead of real one

  Show description for 0482a81
  spigo900

  committed
  0482a81

  Copy full SHA for 0482a81

• Rename nonsense parameter

  spigo900

  committed
  4e35777

  Copy full SHA for 4e35777

• Fix typo

  spigo900

  committed
  3d9128d

  Copy full SHA for 3d9128d

• Use ephemeral in pegasus-dev configuration

  spigo900

  committed
  c04cf32

  Copy full SHA for c04cf32

• Fix partition setup in Pegasus workflow

  spigo900

  committed
  56a39b9

  Copy full SHA for 56a39b9

• Add full-workflow development parameters

  Show description for 3d7c7b4
  spigo900

  committed
  3d7c7b4

  Copy full SHA for 3d7c7b4

• Shorter time limit for AlphaNLI in pegasus-dev-full.params

  Show description for 976009e
  spigo900

  committed
  976009e

  Copy full SHA for 976009e

• Fix how gold labels are found and frontload parameter-getting

  Show description for 9183aef
  spigo900

  committed
  9183aef

  Copy full SHA for 9183aef

• Catch only FileNotFoundError in "couldn't find preds" exception block

  spigo900

  committed
  d1e9e4a

  Copy full SHA for d1e9e4a

• Fix how model_without_seed is constructed

  spigo900

  committed
  4b8f34a

  Copy full SHA for 4b8f34a

• Shorten line

  spigo900

  committed
  7e20de8

  Copy full SHA for 7e20de8

• Pegasus: Pass gold labels to ensemble script in new way

  spigo900

  committed
  da14165

  Copy full SHA for da14165

• Ensemble: Rename variable to match parameter task_to_threshold

  spigo900

  committed
  e7e8f39

  Copy full SHA for e7e8f39

• Pegasus: Remove unnecessary line

  spigo900

  committed
  24983ad

  Copy full SHA for 24983ad

• Ensemble: Include parameters in the dict of successful models

  spigo900

  committed
  5a7582c

  Copy full SHA for 5a7582c

• Pegasus: Fix how iteration over ensembling tasks is done

  spigo900

  committed
  1c69984

  Copy full SHA for 1c69984

• Pegasus: Actually add task_to_gold parameters to ensembling parameters

  spigo900

  committed
  9b192bd

  Copy full SHA for 9b192bd

• Pegasus: Do job throttling

  spigo900

  committed
  0ed048e

  Copy full SHA for 0ed048e

• Pegasus: Pass imported modules instead of strings to run

  Show description for bf5665f
  spigo900

  committed
  bf5665f

  Copy full SHA for bf5665f

• Check the list of tasks against

  Show description for 95510e2
  spigo900

  committed
  95510e2

  Copy full SHA for 95510e2

• Rearrange comment

  spigo900

  committed
  0ab6443

  Copy full SHA for 0ab6443

• Document how to run the ensembling workflow

  spigo900

  committed
  8f36dc3

  Copy full SHA for 8f36dc3

• Change AlphaNLI, HellaSwag, and SIQA to use internal dev sets

  spigo900

  committed
  0eb9806

  Copy full SHA for 0eb9806