Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BitSail][Connector] Support PB format serializer in Kafka Source #103

Open
garyli1019 opened this issue Nov 8, 2022 · 2 comments
Open

[BitSail][Connector] Support PB format serializer in Kafka Source #103

garyli1019 opened this issue Nov 8, 2022 · 2 comments
Assignees
@qidian99
Labels
difficulty-easy Easy difficulty to fix this issue enhancement New feature or request help wanted Extra attention is needed

Comments

@garyli1019
Copy link
Collaborator

garyli1019 commented Nov 8, 2022 •
edited
Loading

Is your feature request related to a problem? Please describe

Currently, kafka legacy source only support JSON format and we want to support PB format.

Describe the solution you'd like

we need to move com.bytedance.bitsail.dump.datasink.file.parser.PbBytesParser to a common place and reuse by other connector, such as kafka source

Describe alternatives you've considered

Additional context
@garyli1019 garyli1019 added help wanted Extra attention is needed enhancement New feature or request difficulty-easy Easy difficulty to fix this issue labels Nov 8, 2022
qidian99 pushed a commit to qidian99/bitsail that referenced this issue Nov 17, 2022
to bytedance#103, Support PB format serializer in Kafka Source
c19f759
@qidian99 qidian99 mentioned this issue Nov 17, 2022
Open
@qidian99
Copy link

Hi, can you assign this issue to me and help review this PR: #185

Best

@garyli1019
Copy link
Collaborator Author

sure, will take a look at the PR

qidian99 added a commit to qidian99/bitsail that referenced this issue Nov 18, 2022
@qidian99
to bytedance#103, Support PB format serializer in Kafka Source
2cb8936
qidian99 added a commit to qidian99/bitsail that referenced this issue Nov 18, 2022
@qidian99
to bytedance#103, Support PB format serializer in Kafka Source
40983a3
qidian99 added a commit to qidian99/bitsail that referenced this issue Nov 18, 2022
@qidian99
to bytedance#103, Support PB format serializer in Kafka Source
bc60582
qidian99 added a commit to qidian99/bitsail that referenced this issue Nov 19, 2022
@qidian99
to bytedance#103, pruned test cases for Kafka PB format
87e5e04
qidian99 added a commit to qidian99/bitsail that referenced this issue Nov 22, 2022
@qidian99
to bytedance#103, Support PB format serializer in Kafka Source
a46ffa4
qidian99 added a commit to qidian99/bitsail that referenced this issue Nov 22, 2022
@qidian99
to bytedance#103, Support PB format serializer in Kafka Source
00fca9e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@qidian99 qidian99

Labels
difficulty-easy Easy difficulty to fix this issue enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@qidian99 @garyli1019