NoSQL: Node IDs - API, SPI + general implementation #2728
Conversation
| return 0; | ||
| } | ||
| }; | ||
| while (true) { |
There was a problem hiding this comment.
I'd agree that permanent store/load failures are unlikely, but would it be worth adding a limit here just for general robustness?
There was a problem hiding this comment.
You mean to fail startup in case we hit a limit?
There was a problem hiding this comment.
Do you think it's worth breaking this loop on timeout (with an exception)?
There was a problem hiding this comment.
Not really. It takes a bit. Worst that can happen is that it reads 1024 (default) rows in batches of 16 and figures out that none can be leased.
Renewals are safe. Unless you stall your system for 15+ minutes during the "renewal time window" (that would fall back to a new lease attempt).
There was a problem hiding this comment.
I'm not sure.... storeManagementState() (line 153) could keep returning false (for whatever reason, we do not know what the impl. will do 🤷 ) and stall startup... WDYT?
There was a problem hiding this comment.
Ah, that's a retry-loop for having one state (think: the id-generator configuration) persisted. That configuration (for the snowflake id generator) must be immutable for the lifetime of the "repository", because changing the id-generator configuration will lead to ID collisions. Once that config is there, the loop's finished. That's orthogonal to the node-lease mechanism.
This loop is literally only for the case when concurrently starting nodes read "no state" and then attempt to persist "their" config.
A "legit DB error" bubbles up and aborts the startup anyways.
There was a problem hiding this comment.
It's a bit too fuzzy at this level (what the related components are and do depends on too many factors (build, config, etc.). I'd prefer to avoid the unlimited loop just for "general robustness".
There was a problem hiding this comment.
Added a hard coded timeout. But there's not more we can do. It has to eventually succeed or fail.
...istence/nosql/nodes/impl/src/main/java/org/apache/polaris/nodes/impl/NodeManagementImpl.java
Outdated
Show resolved
Hide resolved
This PR provides a mechanism to assign a Polaris-cluster-wide unique node-ID to each Polaris instance, which is then used when generating Polaris-cluster-wide unique Snowflake-IDs. The change is fundamental for the NoSQL work, but also demanded for the existing relational JDBC persistence. Does not include any persistence specific implementation.
Also move the expensive part to a `@PostConstruct` to not block CDI entirely from initializing.
github-project-automation
bot
moved this from PRs In Progress
to Ready to merge
in Basic Kanban Board
| Some ID generation mechanisms, | ||
| like [Snowflake-IDs](https://medium.com/@jitenderkmr/demystifying-snowflake-ids-a-unique-identifier-in-distributed-computing-72796a827c9d), | ||
| require unique integer IDs for each running node. This framework provides a mechanism to assign each running node a | ||
| unique integer ID. |
There was a problem hiding this comment.
If snowflake id generator requires such complex node id generator, maybe we should consider other options. Would it possible to use other id generators? Since we are in the persistence module already, why cannot we use something like ObjectID in mongoDB, or Java UUID?
| * `polaris-nodes-api` provides the necessary Java interfaces and immutable types. | ||
| * `polaris-nodes-impl` provides the storage agnostic implementation. | ||
| * `polaris-nodes-spi` provides the necessary interfaces to provide a storage specific implementation. | ||
| * `polaris-nodes-store-nosql` provides the storage implementation based on `polaris-persistence-nosql-api`. |
There was a problem hiding this comment.
Currently it's in the end-to-end NoSQL PR: #1189 ... to be made available for review later (to allow for smaller, easier-to-review PRs, as discussed)
| * specific language governing permissions and limitations | ||
| * under the License. | ||
| */ | ||
| package org.apache.polaris.nodes.api; |
There was a problem hiding this comment.
I don't think anywhere else in Polaris needs this. Can we rename it to org.apache.polaris.nosql.nodes.api or org.apache.polaris.nosql.snowflakeid.nodes.api?
| * `polaris-nodes-api` provides the necessary Java interfaces and immutable types. | ||
| * `polaris-nodes-impl` provides the storage agnostic implementation. | ||
| * `polaris-nodes-spi` provides the necessary interfaces to provide a storage specific implementation. |
There was a problem hiding this comment.
These modules are used by snowflake id generator only, can we merge it into the modules holding snowflake id generators? So that the snowflake id generator is more consistent and self-contained.
3 participants
This PR provides a mechanism to assign a Polaris-cluster-wide unique node-ID to each Polaris instance, which is then used when generating Polaris-cluster-wide unique Snowflake-IDs.
The change is fundamental for the NoSQL work, but also demanded for the existing relational JDBC persistence.
Does not include any persistence specific implementation.