Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
WHAT
Update
crossby
attribute to work the same way asgroupby
.WHY
crossby
attribute (e.g.,crossby=["param"]
) is more explicit than just havingparam
as a list of parameters in the stage config._crossable_params
.HOW
stage.submodel_ids
generator to fix a mypy errorStageConfig._crossable_params
private attributegroupby
,crossby
,subset_ids
, andparam_ids
to tuples, changed default from None to an empty tuplefield_validator
forPipeline.groupby
,Stage.groupby
, andStage.crossby
to enforce unique itemscrossby
field toStage
and removed private attribute_crossby
model_post_init
methods toRoverStage
andSPxModStage
to requiregroupby
attribute but preventcrossby
subsets
andparameters
utility modules for tuplesQUESTIONS
groupby
,crossby
,subset_ids
, andparam_ids
to be tuples instead of sets. May have some merge conflicts when merge with theUniqueList
PRgroupby
andcrossby
: Do we want a way to allow users to prevent the use of either attribute?groupby
, it will get passed to all stages. But we could have a stage (e.g., preprocessing) that we don't want to havegroupby
. Right now, the thing to do would be to NOT havegroupby
in pipeline, and then any column that we want all parallel stages to have would have to be included in each of the parallel stage's definitions. What if someone could saygroupby=False
instead? Note: Unless we do something like this, it is always possible for a stage to have subsets, so all functions (exceptcollect
) will need to havesubset_id
as an argument._crossable_params
, a user can parallelize over any parameter in a stage's config. Do we want to restrict which parameters can be parallelized over, or restrict a stage from usingcrossby
altogether? ForRoverStage
andSPxModStage
, I added a check inmodel_post_init
that does not allowcrossby
, but there could be a better way. Unless a stage does something like this, it will always be possible for a stage to have parameter sets, so all functions (exceptcollect
) will need to haveparam_id
as an argument.