Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement merge strategy based on TYPE+TARGET #98

Merged
merged 5 commits into from
Feb 4, 2025

Conversation

Marenz
Copy link
Contributor

@Marenz Marenz commented Jan 21, 2025

No description provided.

@github-actions github-actions bot added part:tests Affects the unit, integration and performance (benchmarks) tests part:dispatcher Affects the high-level dispatcher interface labels Jan 21, 2025
@Marenz Marenz force-pushed the merge-type-target branch from 89f9260 to 46d7006 Compare January 22, 2025 14:51
@github-actions github-actions bot added the part:docs Affects the documentation label Jan 22, 2025
@Marenz Marenz force-pushed the merge-type-target branch from 46d7006 to 28dc04d Compare January 22, 2025 15:58
@Marenz Marenz marked this pull request as ready for review January 22, 2025 16:52
@Marenz Marenz requested a review from a team as a code owner January 22, 2025 16:52
@Marenz Marenz force-pushed the merge-type-target branch from 28dc04d to 74512d8 Compare January 22, 2025 16:52
@github-actions github-actions bot added the part:tooling Affects the development tooling (CI, deployment, dependency management, etc.) label Jan 22, 2025
@Marenz
Copy link
Contributor Author

Marenz commented Jan 22, 2025

Based on #99

@Marenz Marenz force-pushed the merge-type-target branch from 74512d8 to ad6bff9 Compare January 27, 2025 17:55
@Marenz Marenz mentioned this pull request Jan 27, 2025
@Marenz Marenz requested a review from llucax January 28, 2025 09:57
Copy link
Contributor

@llucax llucax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In summary:

  • I would make the MergeStrategy interface public and simplify it as suggested.
  • Make everything needed by merge strategies available as public symbols.
  • I would make merge strategies take any state they need via the constructor, not the filter function.
  • Expose the cached dispatches via the Dispatcher directly.

@Marenz Marenz force-pushed the merge-type-target branch from ad6bff9 to c6ddf66 Compare January 28, 2025 15:29
@Marenz
Copy link
Contributor Author

Marenz commented Jan 28, 2025

Updated @llucax

@Marenz
Copy link
Contributor Author

Marenz commented Jan 28, 2025

I solved avoiding passing the service now by just using the type instead

@Marenz Marenz force-pushed the merge-type-target branch from c6ddf66 to f774825 Compare January 28, 2025 15:36

@property
@abstractmethod
def filter(self) -> Callable[[Dispatch], bool]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this still returning a callable and is not the callable itself? 🤔

@@ -119,54 +200,34 @@ def new_lifecycle_events_receiver(self, type: str) -> Receiver[DispatchEvent]:
)

async def new_running_state_event_receiver(
self, type: str, *, unify_running_intervals: bool = True
self, type: str, *, merge_strategy: type[MergeStrategy] | None = None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still prefer the approach of instantiating the merge strategy from outside and passing the instance for a couple of reasons:

  1. Flexibility. A merge instance could potentially need to keep more state than just the scheduler, if we instantiate it inside the function we remove that flexibility of allowing merge strategies have the state they want.
  2. Reuse: We could potentially reuse the same merge strategy instance with many different actors (this is minor, the above is a stronger point, but still).

If the idea is to make the typical use easier, I liked the idea of adding a wrapper for common merge strategies in Dispatcher for example, like Dispatcher.mergeByWhatever(), but I think the design/architecture of merge strategies in general should be flexible.

Then I also don't think it is much worse from the user perspective to use MergeByWhatever(dispatcher) than dispatcher.mergeByWhatever(), and it might be generally useful to have dispatcher.dispatches exposed publicly as read-only.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the idea is to make the typical use easier, I liked the idea of adding a wrapper for common merge strategies in Dispatcher for example, like Dispatcher.mergeByWhatever(), but I think the design/architecture of merge strategies in general should be flexible.

I thought about this approach, but at the end it wouldn't save anything because then you have to type

  • dispatcher.new_running_state_event_receiver("TEST", dispatcher.mergeByExample) instead of
  • dispatcher.new_running_state_event_receiver("TEST", MergeByExample(dispatcher))

So that seemed a bit useless.

Only a full wrapper like

  • dispatcher.new_rst_event_receiver_merged_by_example("TEST")

would make some usage difference..

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. So combined to what we discussed in #104 I guess we agree that the end goal would be to end up with: dispatcher.manage(type="FCR", actor_factory=.., strategy=MergeByType(dispatcher)).

On the other hand... If we are merging, I guess you always need to have the full list of dispatches, otherwise how would you merge? People can still create merge strategies that require more state, but I guess the list of available dispatches is the minimum you need to do any merging on the typical case.

What about?

class MergeStrategy(ABC):
    @abstractmethod
    def filter(self, all_dispatches: Map[int, Dispatch], current_dispatch: Dispatch) -> bool: ...

...
        if merge_strategy:
            receiver = receiver.filter(functools.partial(merge_strategy.filter, self._dispatches))
            # or lambda d: merge_strategy.filter(self._dispatches, d)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated. It doesn't have the unified method manage yet, I intend to add that in a later PR

@Marenz Marenz force-pushed the merge-type-target branch 3 times, most recently from de35812 to 38c504a Compare February 3, 2025 22:04
@Marenz Marenz requested a review from llucax February 3, 2025 22:06
Copy link
Contributor

@llucax llucax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more comments, but looking preeeeety goooood!

Implement merge strategy based on TYPE+TARGET

Signed-off-by: Mathias L. Baumann <[email protected]>
@Marenz Marenz force-pushed the merge-type-target branch from 38c504a to 61043e5 Compare February 4, 2025 11:51
Signed-off-by: Mathias L. Baumann <[email protected]>
Signed-off-by: Mathias L. Baumann <[email protected]>
@Marenz Marenz force-pushed the merge-type-target branch from 61043e5 to 93683e2 Compare February 4, 2025 11:52
@Marenz
Copy link
Contributor Author

Marenz commented Feb 4, 2025

Updated, addressed all comments

Copy link
Contributor

@llucax llucax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing! 🎉

@Marenz Marenz added this pull request to the merge queue Feb 4, 2025
Merged via the queue into frequenz-floss:v0.x.x with commit 8f91403 Feb 4, 2025
14 checks passed
@Marenz Marenz deleted the merge-type-target branch February 4, 2025 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
part:dispatcher Affects the high-level dispatcher interface part:docs Affects the documentation part:tests Affects the unit, integration and performance (benchmarks) tests part:tooling Affects the development tooling (CI, deployment, dependency management, etc.)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants