Skip to content

Conversation

XN137
Copy link
Contributor

@XN137 XN137 commented Sep 12, 2025

see ML discussion:
https://lists.apache.org/thread/ot19px6t0w808pp994rrc8kk7186dfxm

the PolarisMetaStoreManager interface methods all take a PolarisCallContext parameter
because they need access to PolarisCallContext.getMetaStore aka the persistence session.
also occasionally the RealmContext and RealmConfig is needed.

this means PolarisMetaStoreManager is only usable within a request-scope.
MetaStoreManagerFactory.getOrCreateMetaStoreManager impls suggests that there needs to
be one shared PolarisMetaStoreManager instance per realm.
however if we look at the PolarisMetaStoreManager impls (AtomicOperationMetaStoreManager
and TransactionalMetaStoreManagerImpl) we can see that they dont have any state themselves, the fields only reference application-scoped instances.

Those 2 observations combined mean that we can make the PolarisMetaStoreManager request-scoped and then the PolarisCallContext parameter can be removed from all the methods without losing any functionality/flexibility.
note that if custom implementations would need any state sharing for a realm, they are still free to do so in their custom MetaStoreManagerFactory implementations, as they can
initialize the custom request-scoped PolarisMetaStoreManager instance with whatever (shared) data they need.

when PolarisMetaStoreManager is request-scoped the getOrCreateSession method also becomes a private implementation detail of the MetaStoreManagerFactory.

@github-project-automation github-project-automation bot moved this to PRs In Progress in Basic Kanban Board Sep 12, 2025
@XN137 XN137 force-pushed the request-scoped-PolarisMetaStoreManager branch 4 times, most recently from 9738d71 to c014ca0 Compare September 15, 2025 07:54
@XN137 XN137 force-pushed the request-scoped-PolarisMetaStoreManager branch 8 times, most recently from c1b2564 to 4511b5e Compare October 1, 2025 06:19
@XN137 XN137 force-pushed the request-scoped-PolarisMetaStoreManager branch 3 times, most recently from ebc2443 to e9f1079 Compare October 6, 2025 08:31
@XN137 XN137 marked this pull request as ready for review October 6, 2025 15:40
@XN137 XN137 force-pushed the request-scoped-PolarisMetaStoreManager branch from e9f1079 to 88db62e Compare October 7, 2025 13:30
@XN137 XN137 force-pushed the request-scoped-PolarisMetaStoreManager branch from 88db62e to d1dda2e Compare October 8, 2025 08:36
@dimas-b dimas-b requested review from HonahX and dennishuo October 8, 2025 21:10
Copy link
Contributor

@dimas-b dimas-b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

partial review

public PolarisMetaStoreManager createMetaStoreManager(
RealmContext realmContext, @Nullable RealmConfig realmConfig) {
if (realmConfig == null) {
realmConfig = new RealmConfigImpl(configurationStore, realmContext);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need the realmConfig parameter at all?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as we can build it ourselves its not strictly needed no, but many call sites already have it and can pass it in.
i am fine to remove the parameter if we think thats better overall.

public class InMemoryEntityCache implements EntityCache {

private final PolarisDiagnostics diagnostics;
private final PolarisMetaStoreManager polarisMetaStoreManager;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose InMemoryEntityCache must use a PolarisMetaStoreManager from the same realm. If PolarisMetaStoreManager is a method parameter it become not so obvious. WDYT about keeping this field?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

afaict we cant keep this field as PolarisMetaStoreManager now represents a (per-request) persistence session where-as InMemoryEntityCache is supposed to be a shared cache within a realm.

the question is with what persistence session do we want to populate the cache?
a shared one or the request-scoped one given by the callers as "use this for loading in case cache entry is missing" ?

@XN137 XN137 force-pushed the request-scoped-PolarisMetaStoreManager branch from d1dda2e to 37696b7 Compare October 9, 2025 09:40
@XN137
Copy link
Contributor Author

XN137 commented Oct 9, 2025

rebased after trivial a conflict and also to adjust to AccessConfigProvider (though we need it to become request-scoped as well soon)

the `PolarisMetaStoreManager` interface methods all take a `PolarisCallContext` parameter
because they need access to `PolarisCallContext.getMetaStore` aka the persistence session.
also occasionally the `RealmContext` and `RealmConfig` is needed.

this means `PolarisMetaStoreManager` is only usable within a request-scope.
`MetaStoreManagerFactory.getOrCreateMetaStoreManager` impls suggests that there needs to
be one shared `PolarisMetaStoreManager` instance per realm.
however if we look at the `PolarisMetaStoreManager` impls (`AtomicOperationMetaStoreManager`
and `TransactionalMetaStoreManagerImpl`) we can see that they dont have any state themselves,
the fields only reference application-scoped collaborators.

those 2 observations combined mean that in order to reduce the dependency on
`PolarisCallContext` the `PolarisMetaStoreManager` has to become request-scoped.
note that if custom implementations would need any state sharing for a realm, they are still
free to do so in their custom `MetaStoreManagerFactory` implementations, as they can
initialize the request-scoped `PolarisMetaStoreManager` with whatever data they need.

when `PolarisMetaStoreManager` is request-scoped the `getOrCreateSession` method also becomes
a private implementation detail of the `MetaStoreManagerFactory`.
the `EntityCache` / `InMemoryEntityCache` however can no longer operate on a single
`PolarisMetaStoreManager` and the one from the request needs to be passed as a parameter in
all the methods now.
@XN137 XN137 force-pushed the request-scoped-PolarisMetaStoreManager branch from 37696b7 to 5007ca1 Compare October 9, 2025 10:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants