Best Of ensemble: solve mixer ties with runtime as second criterion

[paxcema]

The predictor object stores an up-to-date dictionary with runtimes for each phase. This could be used in the mixer selection within BestOf to solve ties on accuracy by choosing the fastest option.

[noraa-july-stoke]

@paxcema
thought I could take a stab at this if I may bounce my initial thoughts on some first steps off of you...

1. add runtime field to the SubmodelData class:

@dataclass_json
@dataclass
class SubmodelData:
    name: str
    accuracy: float
    is_best: bool
    runtime: float  # new field for storing the runtime of each submodel
    
2. Store  the runtime in SubmodelData for each mixer:

self.submodel_data.append(SubmodelData(
    name=type(mixer).__name__,
    accuracy=avg_score,
    runtime=runtime,  # store the runtime here
    is_best=False
))

3. Add runtime to the JSON serialization of SubmodelData:

def to(self, *args, **kwargs):
    return {
        'name': self.name,
        'accuracy': self.accuracy,
        'is_best': self.is_best,
        'runtime': self.runtime,  # add this line
    }

if you wanted more from this, let me know!

[paxcema]

Looks good! Only thing that comes to mind is that perhaps there is a small functionality duplicity with the runtime_log that every predictor has and fills, but the solution above seems to me, by far, the simplest to achieve what we need.

[noraa-july-stoke]

@paxcema okay I'll send in a PR soon! Maybe I can work maybe cleaning up a bit more as a follow-up...

[paxcema]

Closing for now.