feat: gitlab extractors and convertors support incremental mode #7997
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dosubot
bot
added
size:XL
dosubot
bot
added
size:XXL
abeizn
approved these changes
Sep 26, 2024
|
🤖 The current file has a conflict, and the pr cannot be automatically created. |
|
Hi @klesh, this feature is essential; I cannot update my insight version because it is missing, so I am stuck on v0.21.0 , The current stable version does not work well for big projects (missing data), and with this feature/fixes, it works. When will it be carried on the next release? |
|
@antoniomuso It is already available in the latest v1.0.x beta. |
klesh
added a commit
that referenced
this pull request
Nov 5, 2024
* feat: gitlab mr_extractor support incremental sync * feat: incr mode transformation support for deployment/issue/job/mr and others * feat: removed mr_enricher and all gitlab transformers support incr-mode * fix: some gitlab subtasks are missing connection in left join clause * fix: test cases failed due to gitlab transformers support incr-mode * fix: linting * fix: do not collect all accounts from jihulab.com * fix: typo * fix: gitlab mr comments won't be converted to domain layer till next run * feat: improve gitlab MR comments/commits collection performance * fix: gitlab issues/mrs child records are not deleted * docs: update stateful extractor doc * fix: gitlab mr detail test * refactor: unify stateful extractor and convertor helper * docs: update stateful extractor/convertor doc * refactor: remove useless type hint * fix: shoud not be deleting records not extracted by current extractor * fix: jira issue extractor should not deleting sprint_issue * refactor: remove commit related subtasks * fix: remove commit conversion e2e test * feat(gitlab): update ExtractApiChildPipelines (#8016) * fix: retransform should run in fullsync mode * fix: gitlab issue assignees are not being converted --------- Co-authored-by: Lynwee <[email protected]>
|
@antoniomuso Sorry, I just noticed that the auto-cherrypick failed. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
To reduce total number of records to be processed and produced.
Affected data entities:
gitlab
jira