Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide feedback when using deprecated %d specifiers in RUNPATH #4474

Merged
merged 2 commits into from
Jan 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/getting_started/configuration/poly_new/guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ We need to have each iteration run in a different folder, so that they won't
step on each others toes. Add to the config (``poly.ert``) file the following
line::

RUNPATH poly_out/realization-%d/iter-%d
RUNPATH poly_out/realization-<IENS>/iter-<ITER>

The second ``%d`` in the runpath is replaced by the iteration number of the
algorithm. This is needed since the algorithm may run several iterations during
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ NUM_REALIZATIONS 100
QUEUE_SYSTEM LOCAL
QUEUE_OPTION LOCAL MAX_RUNNING 50

RUNPATH poly_out/realization-%d/iter_%d
RUNPATH poly_out/realization-<IENS>/iter-<ITER>

GEN_KW COEFFS coeff.tmpl coeffs.json coeff_priors

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ NUM_REALIZATIONS 100
QUEUE_SYSTEM LOCAL
QUEUE_OPTION LOCAL MAX_RUNNING 50

RUNPATH poly_out/realization-%d/iter-%d
RUNPATH poly_out/realization-<IENS>/iter-<ITER>

GEN_KW COEFFS coeff.tmpl coeffs.json coeff_priors

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ NUM_REALIZATIONS 100
QUEUE_SYSTEM LOCAL
QUEUE_OPTION LOCAL MAX_RUNNING 50

RUNPATH poly_out/realization-%d/iter-%d
RUNPATH poly_out/realization-<IENS>/iter-<ITER>

GEN_KW COEFFS coeff.tmpl coeffs.json coeff_priors

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ NUM_REALIZATIONS 100
QUEUE_SYSTEM LOCAL
QUEUE_OPTION LOCAL MAX_RUNNING 50

RUNPATH poly_out/realization-%d/iter-%d
RUNPATH poly_out/realization-<IENS>/iter-<ITER>

GEN_KW COEFFS coeff.tmpl coeffs.json coeff_priors

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ NUM_REALIZATIONS 5

QUEUE_SYSTEM LOCAL

RUNPATH poly_out/realization-%d/iter-%d
RUNPATH poly_out/realization-<IENS>/iter-<ITER>

INSTALL_JOB poly_eval POLY_EVAL
SIMULATION_JOB poly_eval
22 changes: 11 additions & 11 deletions docs/reference/configuration/keywords.rst
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ Keyword name Required
:ref:`QUEUE_SYSTEM <queue_system>` NO LOCAL_DRIVER System used for running simulation jobs
:ref:`REFCASE <refcase>` NO Reference case used for observations and plotting (See HISTORY_SOURCE and SUMMARY)
:ref:`RESULT_PATH <result_path>` NO results/step_%d Define where ERT should store results
:ref:`RUNPATH <runpath>` NO simulations/realization%d Directory to run simulations
:ref:`RUNPATH <runpath>` NO realization-<IENS>/iter-<ITER> Directory to run simulations; simulations/realization-<IENS>/iter-<ITER>
:ref:`RUNPATH_FILE <runpath_file>` NO .ert_runpath_list Name of file with path for all forward models that ERT has run. To be used by user defined scripts to find the realizations
:ref:`RUN_TEMPLATE <run_template>` NO Install arbitrary files in the runpath directory
:ref:`SCHEDULE_PREDICTION_FILE <schedule_prediction_file>` NO Deprecated: Schedule prediction file
Expand Down Expand Up @@ -461,25 +461,25 @@ possible to do with ERT.
.. topic:: RUNPATH

The RUNPATH keyword should give the name of the folders where the ECLIPSE
simulations are executed. It should contain at least one %d specifier, which
will be replaced by the realization number when ERT creates the folders.
Optionally, it can contain one more %d specifier, which will be replaced by
the iteration number.
simulations are executed. It should contain <IENS> and <ITER>, which
will be replaced by the realization number and iteration number when ERT creates the folders.
By default, RUNPATH is set to "simulations/realization-<IENS>/iter-<ITER>".

By default, RUNPATH is set to "simulations/realization-%d".
Deprecated syntax still allow use of two %d specifers. Use of less than two %d is prohibited.
The behaviour is identical to the default substitution.

*Example A:*
*Example:*

::

-- Giving a RUNPATH with just one %d specifer.
RUNPATH /mnt/my_scratch_disk/realization-%d
-- Using <IENS> & <ITER> specifiers for RUNPATH.
RUNPATH /mnt/my_scratch_disk/realization-<IENS>/iter-<ITER>

*Example B:*
*Example deprecated syntax:*

::

-- Giving a RUNPATH with two %d specifers.
-- Using RUNPATH with two %d specifers.
RUNPATH /mnt/my_scratch_disk/realization-%d/iteration-%d

The RUNPATH keyword is optional.
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/queue/local_queue.ert
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ JOBNAME queue_test_%d
QUEUE_SYSTEM LOCAL
QUEUE_OPTION LOCAL MAX_RUNNING 50

RUNPATH local_testing/realization-%d/iter-%d
RUNPATH local_testing/realization-<IENS>/iter-<ITER>

NUM_REALIZATIONS 100
MIN_REALIZATIONS 1
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/queue/lsf_queue.ert
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ QUEUE_OPTION LSF LSF_SERVER be-grid01 -- Change this to a server you have access
QUEUE_OPTION LSF LSF_QUEUE mr
QUEUE_OPTION LSF PROJECT_CODE user:$USER

RUNPATH lsf_testing/realization-%d/iter-%d
RUNPATH lsf_testing/realization-<IENS>/iter-<ITER>

NUM_REALIZATIONS 1
MIN_REALIZATIONS 1
Expand Down
22 changes: 16 additions & 6 deletions src/ert/_c_wrappers/enkf/model_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

class ModelConfig:
DEFAULT_HISTORY_SOURCE = HistorySourceEnum.REFCASE_HISTORY
DEFAULT_RUNPATH = "simulations/realization-%d/iter-%d"
DEFAULT_RUNPATH = "simulations/realization-<IENS>/iter-<ITER>"
DEFAULT_GEN_KW_EXPORT_NAME = "parameters"
DEFAULT_ENSPATH = "storage"

Expand Down Expand Up @@ -40,11 +40,21 @@ def __init__(
else self.DEFAULT_HISTORY_SOURCE
)

self.runpath_format_string = (
runpath_format_string
if runpath_format_string is not None
else self.DEFAULT_RUNPATH
)
if runpath_format_string is None:
self.runpath_format_string = self.DEFAULT_RUNPATH
elif "%d" in runpath_format_string:
self.runpath_format_string = runpath_format_string
logger.warning(
"RUNPATH keyword should use syntax "
f"`{self.DEFAULT_RUNPATH}` "
"instead of deprecated syntax "
f"`{runpath_format_string}`"
)
elif not any(x in runpath_format_string for x in ["<ITER>, <IENS>"]):
self.runpath_format_string = runpath_format_string
logger.error(
"RUNPATH keyword should use syntax " f"`{self.DEFAULT_RUNPATH}`."
)

self.jobname_format_string = jobname_format_string
self.gen_kw_export_name = (
Expand Down
21 changes: 1 addition & 20 deletions src/ert/gui/tools/load_results/load_results_panel.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
import os

from PyQt5.QtWidgets import QMessageBox
from qtpy.QtWidgets import QComboBox, QFormLayout, QTextEdit, QWidget

Expand Down Expand Up @@ -50,7 +48,7 @@ def __init__(self, facade: LibresFacade):
self._active_realizations_field.setValidator(RangeStringArgument())
layout.addRow("Realizations to load:", self._active_realizations_field)

self._iterations_model = ValueModel(self.iteration_count)
self._iterations_model = ValueModel(self.facade.get_number_of_iterations())
self._iterations_field = StringBox(
self._iterations_model, "load_results_manually/iterations"
)
Expand All @@ -59,23 +57,6 @@ def __init__(self, facade: LibresFacade):

self.setLayout(layout)

@property
def iteration_count(self) -> int:
try:
self.facade.run_path % (0, 0)
except TypeError:
return 0

iteration = 0
valid_directory = True
while valid_directory:
formatted = self.facade.run_path % (0, iteration + 1)
valid_directory = os.path.exists(formatted)
if valid_directory:
iteration += 1

return iteration

def readCurrentRunPath(self):
current_case = self.facade.get_current_case_name()
run_path = self.facade.run_path
Expand Down
15 changes: 1 addition & 14 deletions src/ert/gui/tools/load_results/load_results_tool.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ def __init__(self, facade):
)
self.__import_widget = None
self.__dialog = None
self.setEnabled(self.is_valid_run_path())

def trigger(self):
if self.__import_widget is None:
Expand All @@ -34,16 +33,4 @@ def load(self, _):
self.__dialog.accept()

def is_valid_run_path(self) -> bool:
"""A run path is considered valid if we can
insert realisation and iteration numbers"""
try:
# pylint: disable=pointless-statement
self.facade.run_path % (0, 0)
return True
except TypeError:
try:
# pylint: disable=pointless-statement
self.facade.run_path % 0
return True
except TypeError:
return False
return self.facade.is_valid_runpath()
8 changes: 8 additions & 0 deletions src/ert/libres_facade.py
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,14 @@ def get_number_of_iterations(self) -> int:
def have_observations(self) -> bool:
return self._enkf_main.have_observations()

def is_valid_runpath(self) -> bool:
iter_0_paths = set(self.get_run_paths([True] * self.get_ensemble_size(), 0))
iter_1_paths = set(self.get_run_paths([True] * self.get_ensemble_size(), 1))
return (
len(iter_0_paths) == self.get_ensemble_size()
and iter_0_paths != iter_1_paths
)

@property
def run_path(self) -> str:
return self._enkf_main.getModelConfig().runpath_format_string
Expand Down
2 changes: 1 addition & 1 deletion test-data/batch_sim/batch_sim.ert
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ NUM_REALIZATIONS 25

DEFINE <STORAGE> storage/<CONFIG_FILE_BASE>

RUNPATH <STORAGE>/runpath/<CASE_NAME>/realization-%d/iter-%d
RUNPATH <STORAGE>/runpath/<CASE_NAME>/realization-<IENS>/iter-<ITER>
ENSPATH <STORAGE>/ensemble
JOBNAME SNAKE_OIL_FIELD

Expand Down
2 changes: 1 addition & 1 deletion test-data/batch_sim/batch_sim_sleep_and_fail.ert
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ QUEUE_OPTION LOCAL MAX_RUNNING 10

NUM_REALIZATIONS 10

RUNPATH runpath/realization-%d/iter-%d
RUNPATH runpath/realization-<IENS>/iter-<ITER>
JOBNAME SNAKE_OIL_FIELD

INSTALL_JOB SQUARE_PARAMS jobs/SQUARE_PARAMS
Expand Down
2 changes: 1 addition & 1 deletion test-data/batch_sim/sleepy_time.ert
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ QUEUE_OPTION LOCAL MAX_RUNNING 10

NUM_REALIZATIONS 10

RUNPATH runpath/realization-%d-<GEO_ID>/iter-%d
RUNPATH runpath/realization-<IENS>-<GEO_ID>/iter-<ITER>
JOBNAME OLE_LUKKOYE

INSTALL_JOB SLEEP jobs/SLEEP
Expand Down
2 changes: 1 addition & 1 deletion test-data/configuration_tests/ecl_config.ert
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ REFCASE input/refcase/SNAKE_OIL_FIELD
GRID input/CASE.EGRID
SCHEDULE_PREDICTION_FILE input/schedule.sch

RUNPATH /tmp/simulations/run%d
RUNPATH /tmp/simulations/realization-<IENS>/iter-<ITER>
NUM_REALIZATIONS 10
MIN_REALIZATIONS 10

Expand Down
2 changes: 1 addition & 1 deletion test-data/configuration_tests/ensemble_config.ert
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
JOBNAME Job%d
RUNPATH /tmp/simulations/run%d
RUNPATH /tmp/simulations/realization-<IENS>/iter-<ITER>
NUM_REALIZATIONS 10
MIN_REALIZATIONS 10

Expand Down
2 changes: 1 addition & 1 deletion test-data/configuration_tests/model_config.ert
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ INSTALL_JOB SNAKE_OIL_DIFF input/jobs/SNAKE_OIL_DIFF
-- ModelConfig specific input

JOBNAME model_config_test
RUNPATH /tmp/simulations/run%d
RUNPATH /tmp/simulations/realization-<IENS>/iter-<ITER>
NUM_REALIZATIONS 10
ENSPATH Ensemble

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ INSTALL_JOB SNAKE_OIL_DIFF input/jobs/SNAKE_OIL_DIFF
-- ModelConfig specific input

JOBNAME model_config_test
RUNPATH /tmp/simulations/run%d
RUNPATH /tmp/simulations/realization-<IENS>/iter-<ITER>
NUM_REALIZATIONS 10
ENSPATH Ensemble

Expand Down
2 changes: 1 addition & 1 deletion test-data/poly_example/poly-ies.ert
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ JOBNAME poly_%d
QUEUE_SYSTEM LOCAL
QUEUE_OPTION LOCAL MAX_RUNNING 50

RUNPATH poly_out/realization-%d/iter-%d
RUNPATH poly_out/realization-<IENS>/iter-<ITER>

OBS_CONFIG observations
TIME_MAP time_map
Expand Down
2 changes: 1 addition & 1 deletion test-data/poly_example/poly.ert
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ JOBNAME poly_%d
QUEUE_SYSTEM LOCAL
QUEUE_OPTION LOCAL MAX_RUNNING 50

RUNPATH poly_out/realization-%d/iter-%d
RUNPATH poly_out/realization-<IENS>/iter-<ITER>

OBS_CONFIG observations
TIME_MAP time_map
Expand Down
2 changes: 1 addition & 1 deletion test-data/poly_template/poly.ert.j2
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
QUEUE_SYSTEM LOCAL
QUEUE_OPTION LOCAL MAX_RUNNING 50

RUNPATH poly_out/realization-%d/iter-%d
RUNPATH poly_out/realization-<IENS>/iter-<ITER>

OBS_CONFIG observations

Expand Down
2 changes: 1 addition & 1 deletion test-data/simple_config/analysis_config
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
JOBNAME Job%d
RUNPATH /tmp/simulations/run%d
RUNPATH /tmp/simulations/realization-<IENS>/iter-<ITER>
NUM_REALIZATIONS 10
MIN_REALIZATIONS 10
JOB_SCRIPT script.sh
Expand Down
2 changes: 1 addition & 1 deletion test-data/simple_config/minimum_config
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
JOBNAME Job%d
RUNPATH /tmp/simulations/run%d
RUNPATH /tmp/simulations/realization-<IENS>/iter-<ITER>
NUM_REALIZATIONS 10
MIN_REALIZATIONS 10
JOB_SCRIPT script.sh
Expand Down
2 changes: 1 addition & 1 deletion test-data/simple_config/slurm_config
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
JOBNAME Job%d
RUNPATH /tmp/simulations/run%d
RUNPATH /tmp/simulations/realization-<IENS>/iter-<ITER>
NUM_REALIZATIONS 10
MIN_REALIZATIONS 10
JOB_SCRIPT script.sh
Expand Down
2 changes: 1 addition & 1 deletion test-data/snake_oil/snake_oil.ert
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ ANALYSIS_SET_VAR STD_ENKF IES_INVERSION 1
DEFINE <STORAGE> storage/<CONFIG_FILE_BASE>
RANDOM_SEED 3593114179000630026631423308983283277868

RUNPATH <STORAGE>/runpath/realization-%d/iter-%d
RUNPATH <STORAGE>/runpath/realization-<IENS>/iter-<ITER>
ENSPATH <STORAGE>/ensemble
ECLBASE SNAKE_OIL_FIELD
SUMMARY *
Expand Down
2 changes: 1 addition & 1 deletion test-data/snake_oil_field/snake_oil.ert
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ GRID grid/CASE.EGRID
DEFINE <STORAGE> storage/<CONFIG_FILE_BASE>
RANDOM_SEED 3593114179000630026631423308983283277868

RUNPATH <STORAGE>/runpath/realization-%d/iter-%d
RUNPATH <STORAGE>/runpath/realization-<IENS>/iter-<ITER>
ENSPATH <STORAGE>/ensemble
ECLBASE SNAKE_OIL_FIELD
SUMMARY *
Expand Down
2 changes: 1 addition & 1 deletion test-data/snake_oil_field/snake_oil_field.ert
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ GRID grid/CASE.EGRID
DEFINE <STORAGE> storage/<CONFIG_FILE_BASE>
RANDOM_SEED 3593114179000630026631423308983283277868

RUNPATH <STORAGE>/runpath/realization-%d/iter-%d
RUNPATH <STORAGE>/runpath/realization-<IENS>/iter-<ITER>
ENSPATH <STORAGE>/ensemble
ECLBASE SNAKE_OIL_FIELD
SUMMARY *
Expand Down
2 changes: 1 addition & 1 deletion test-data/snake_oil_field/snake_oil_surface.ert
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ DEFINE <STORAGE> storage/<CONFIG_FILE_BASE>

RANDOM_SEED 3593114179000630026631423308983283277868

RUNPATH <STORAGE>/runpath/realization-%d/iter-%d
RUNPATH <STORAGE>/runpath/realization-<IENS>/iter-<ITER>
ENSPATH <STORAGE>/ensemble
ECLBASE SNAKE_OIL_FIELD
SUMMARY *
Expand Down
2 changes: 1 addition & 1 deletion test-data/snake_oil_structure/ert/model/user_config.ert
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ DEFINE <ECLIPSE_NAME> XYZ
---------------------------------------------------
DATA_FILE ../../eclipse/model/SNAKE_OIL.DATA -- Location of the Eclipse .DATA-file
GRID ../../eclipse/include/grid/CASE.EGRID -- Name of the Eclipse .GRID-file
RUNPATH <SCRATCH>/<USER>/<CASE_DIR>/realization-%d/iter-%d -- Runpath for simulations (first %d will be replaced with realization number, second %d with iteration number
RUNPATH <SCRATCH>/<USER>/<CASE_DIR>/realization-<IENS>/iter-<ITER> -- Runpath for simulations (<IENS> will be replaced with realization number, <ITER> with iteration number)
ECLBASE eclipse/model/<ECLIPSE_NAME>-%d -- Name used for the ECLIPSE simulations (%d will be replaced with the realization number)
ENSPATH ../output/storage/<CASE_DIR> -- Storage of internal ert data
RUNPATH_FILE ../output/run_path_file/.ert-runpath-list_<CASE_DIR> -- Ert runpath file for workflows
Expand Down
Loading