Log Backup: decouple log backup resolve locks from GCWorker. (#45904)

[ti-chi-bot]

This is an automated cherry-pick of #45904

What problem does this PR solve?

Issue Number: close #40759
Problem Summary:
#35403 introduce 1 minute scan locks of the whole cluster in GCWorker. which will cost CPU when cluster has large number of regions. This PR try to decouple it.

What is changed and how it works?

1. Record checkpoint and move resolve locks in advancer.
2. Abstract GCWorkerLockResolver used for LogBackup/GCWorker/GCWorkerTest. move ResolveLocksForRange into gcutil.go to avoid cycle import.

 GCLockResolver is used for GCWorker and log backup advancer to resolve locks.
// #Note: Put it here to avoid cycle import
ype GCLockResolver interface {
 	// ResolveLocks tries to resolve expired locks.
 	// 1. For GCWorker it will scan locks for all regions before *safepoint*,
 	// and force remove locks. rollback the txn, no matter the lock is expired of not.
 	// 2. For log backup advancer, it will scan all locks for a small range.
 	// and it will check status of the txn. resolve the locks if txn is expired, Or do nothing.
 	ResolveLocks(*tikv.Backoffer, []*txnlock.Lock, tikv.RegionVerID) (bool, error)

 	// ScanLocks only used for mock test.
 	ScanLocks([]byte, uint64) []*txnlock.Lock
 	// We need to get tikvStore to build rangerunner.
 	// TODO: the most code is in client.go and the store is only used to locate end keys of a region.
 	// maybe we can move GCLockResolver into client.go.
 	GetStore() tikv.Storage
 }

3. Remove logbackup logic in GCWorker.

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
  before this PR, resolve lock every minutes

with this PR. resolve lock every 10 minutes. which will save CPU usage.

and check lock resolved.

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

Decouple log backup resolve locks from GCWorker to save TiKV CPU usage.

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (release-7.1@3888072). Click here to learn what that means.
The diff coverage is n/a.

Additional details and impacted files

@@               Coverage Diff                @@
##             release-7.1     #46894   +/-   ##
================================================
  Coverage               ?   73.4427%           
================================================
  Files                  ?       1203           
  Lines                  ?     376760           
  Branches               ?          0           
================================================
  Hits                   ?     276703           
  Misses                 ?      82393           
  Partials               ?      17664           

[MyonKeminta]

GCWorker part LGTM

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: BornChanger, Leavrth

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [Leavrth]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2023-10-07 09:58:11.107634408 +0000 UTC m=+873488.694744553: ☑️ agreed by BornChanger.
• 2023-10-09 05:27:04.699985981 +0000 UTC m=+1030022.287096126: ☑️ agreed by Leavrth.

[ti-chi-bot]

This cherry pick PR is for a release branch and has not yet been approved by release team.
Adding the do-not-merge/cherry-pick-not-approved label.

To merge this cherry pick, it must first be approved by the collaborators.

AFTER it has been approved by collaborators, please ping the release team in a comment to request a cherry pick review.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.