Feat/relay server to spring batch #64
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hyungkishin
force-pushed
the
feat/relay-server-to-spring-batch
branch
from
861cefc to
8215dfc
Compare
hyungkishin
force-pushed
the
feat/relay-server-to-spring-batch
branch
from
8215dfc to
1ba6a6c
Compare
flab-true
reviewed
Nov 2, 2025
| private val queue = ConcurrentLinkedQueue<ClaimedRow>() | ||
|
|
||
| // 배치 로딩 Lock (한 번에 한 스레드만 로딩) | ||
| private val loadLock = ReentrantLock() |
Collaborator
There was a problem hiding this comment.
락 경쟁을 하고 있는데요. 병목이 발생할 수 있을 것 같다는 생각입니다.
Reader 에서 Lock을 걸어 중복 조회를 막는 경우라면, Partitioning을 사용하는 방안은 어떤가요?
| } | ||
|
|
||
| // Queue가 비었으면 새 배치 로드 시도 | ||
| return loadLock.withLock { |
Collaborator
There was a problem hiding this comment.
멀티스레드 Step 이라도 SynchronizedItemStreamReader를 사용하면, Reader 호출을 한번에 한 스레드만 하게 만들 수 있는 것 같습니다. 하지만 Partitioning으로 입력 자체를 나누는 방법이 더 효과적일 것 같다는 생각입니다.
| AND (next_retry_at IS NULL OR next_retry_at <= :now) | ||
| AND attempt_count < :maxAttempts | ||
| ORDER BY next_retry_at NULLS FIRST, created_at | ||
| FOR UPDATE SKIP LOCKED |
Collaborator
There was a problem hiding this comment.
트랜잭션에서 SKIP LOCKED 을 사용하는 의도는 좋은 것 같습니다. 대기가 줄고 교착 가능성이 낮아지는 효과가 있을 것 같네요.
| */ | ||
| val fixedDelayMs: Long = 1000, | ||
| /** DB 조회 배치 크기 */ | ||
| val chunkSize: Int = 500, |
Collaborator
There was a problem hiding this comment.
chunksize가 500인데요. 100,200,300 설정에서 단일 스레드와 현재 멀티 스레드 방식의 속도를 확인해 보시면 좋겠습니다.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
transfer-relay 멀티 인스턴스가 아닌, 단일 스레드에서 multiThread 방식으로 개선