[improve][pip] PIP-332: peek messages from topic subscription with messagePosition value #22302
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ult (apache#21652) Co-authored-by: Jiwe Guo <[email protected]>
### Motivation See pip: apache#21129 ### Modifications Add `QuotaResources`
…pache#21622) Co-authored-by: Jiwe Guo <[email protected]>
…t and webServicePortTls are set (apache#21633) Co-authored-by: Jiwe Guo <[email protected]>
… leaked when lookuping topic is not exist. (apache#21670)
…ice channel monitor interval configs. Handle AddEntry failure during topic transfer (apache#21668)
…root (apache#21084) ### Motivation Currently, the user pulsar is in the root group, it would be better to use the non-root group to keep more safety. ### Modifications - Add group pulsar (GID 10000) - Add user pulsar (UID 10000) to group pulsar
…endAsync (apache#21673) Signed-off-by: Zixuan Liu <[email protected]>
…tead of root" (apache#21691) Reverts apache#21084. Because the change breaks OpenShift support.
…pache#21684) Fixes apache#21557 ### Motivation There is a network package loss issue after enabling `haProxyProtocolEnabled`, which leads the error `Checksum failed on the broker` and `Adjusted frame length exceeds`, you can reproduce the issue by the test `testSlowNetwork`. ### Modifications Fix the bug.
…pache#21253) Signed-off-by: Lan Liang <[email protected]>
…picUpdateAsync (apache#21699) Signed-off-by: Zixuan Liu <[email protected]>
Fixes apache#21501 ### Motivation No need to `synchronized` the method `isLeader` in LeaderService See the deadlock stack : ``` "pulsar-external-listener-44525-1": at org.apache.pulsar.functions.worker.FunctionMetaDataManager.giveupLeadership(FunctionMetaDataManager.java) - waiting to lock <0x0000100013535c90> (a org.apache.pulsar.functions.worker.FunctionMetaDataManager) at org.apache.pulsar.functions.worker.LeaderService.becameInactive(LeaderService.java:167) - locked <0x000010001344c6d8> (a org.apache.pulsar.functions.worker.LeaderService) at org.apache.pulsar.client.impl.ConsumerImpl.lambda$activeConsumerChanged$27(ConsumerImpl.java:1136) at org.apache.pulsar.client.impl.ConsumerImpl$$Lambda$2606/0x00007f854ce9cb10.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1136) at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:635) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run([email protected]/Thread.java:833) "pulsar-web-44514-6": at org.apache.pulsar.functions.worker.LeaderService.isLeader(LeaderService.java) - waiting to lock <0x000010001344c6d8> (a org.apache.pulsar.functions.worker.LeaderService) at org.apache.pulsar.functions.worker.SchedulerManager.scheduleInternal(SchedulerManager.java:200) at org.apache.pulsar.functions.worker.SchedulerManager.schedule(SchedulerManager.java:229) at org.apache.pulsar.functions.worker.FunctionMetaDataManager.updateFunctionOnLeader(FunctionMetaDataManager.java:251) - locked <0x0000100013535c90> (a org.apache.pulsar.functions.worker.FunctionMetaDataManager) at org.apache.pulsar.functions.worker.rest.api.ComponentImpl.internalProcessFunctionRequest(ComponentImpl.java:1775) at org.apache.pulsar.functions.worker.rest.api.ComponentImpl.updateRequest(ComponentImpl.java:996) at org.apache.pulsar.functions.worker.rest.api.FunctionsImpl.registerFunction(FunctionsImpl.java:222) at org.apache.pulsar.broker.admin.impl.FunctionsBase.registerFunction(FunctionsBase.java:196) ```
…terOnGlobalNamespace (apache#21708)
…zk when revoke subscription permission (apache#21696) Co-authored-by: fanjianye <[email protected]> Co-authored-by: Jiwe Guo <[email protected]>
…Topic Deletion Command (apache#21664)
… inactive bundle states (apache#21721)
…ns (apache#20884) Signed-off-by: Zixuan Liu <[email protected]> Co-authored-by: Yunze Xu <[email protected]>
… Sinks and Sources (apache#22185)
Signed-off-by: Zixuan Liu <[email protected]>
…lient-tools (apache#22209) Signed-off-by: Zixuan Liu <[email protected]>
Master apache#21271 ### Motivation The proposal will introduce a new API to refresh the table view with the latest written data on the topic, ensuring that all subsequent reads are based on the refreshed data.
Co-authored-by: 道君 <[email protected]> Co-authored-by: Lari Hotari <[email protected]>
…erTask in dispatcher (apache#22279)
github-actions
bot
added
the
doc-required
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
47 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PIP: 332
Motivation
Currently we are able to peek any number of messages of a topic subscription backlog using the Pulsar Java Admin API.
However, we are only able to view these messages starting from the most recent message i.e messagePosition of 1. If we want to view the 100th message, we have to peek all the top 100 messages.
When the number of messages is large, we want to split them by paging them and displaying them on the UI. So, we would like the ability to view any batch of messages instead of all of the top messages. With the messagePosition and numberOfMessages that should be fairly straightforward to do.
Modifications
The Pulsar Admin API in Java currently provides a way to peek messages using this
This function includes a default hardcoded messagePosition value of 1. We want to give the user the option to input a different messagePosition value.
The current peekMessages(String topic, String subName, int numMessages) method will remain the same and use messagePosition of 1.
We will be overloading this method with messagePosition. It will default to 1 if not specified.
Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
Documentation
docdoc-requireddoc-not-neededdoc-completePR for pulsar-site doc updates: [improve][site] Doc changes to support peekMessages with Offset in admin-api-topics pulsar-site#840
Matching PR in forked repository
PR in forked repository: