Skip to content

Commit

Permalink
[LEDGER STORAGE] DbLedgerStorage should do periodical flush
Browse files Browse the repository at this point in the history

Descriptions of the changes in this PR:

*Motivation*

DbLedgerStorage doesn't drive checkpoint itself. so currently DbLedgerStorage flush only
happens either when write-cache is full or entry log file rotated. The correctness is still
maintained. However the behavior is different from original yahoo behavior.

*Changes*

Revert the behavior back to original periodical flush




Reviewers: Matteo Merli <[email protected]>, Jia Zhai <None>

This closes apache#1843 from sijie/dbledgerstorage_sync
  • Loading branch information
sijie authored Nov 28, 2018
1 parent dfbfa8d commit d7fffac
Showing 1 changed file with 14 additions and 5 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@
import org.apache.bookkeeper.bookie.LedgerDirsManager.LedgerDirsListener;
import org.apache.bookkeeper.bookie.LedgerDirsManager.NoWritableLedgerDirException;
import org.apache.bookkeeper.bookie.stats.BookieStats;
import org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage;
import org.apache.bookkeeper.common.util.Watcher;
import org.apache.bookkeeper.conf.ServerConfiguration;
import org.apache.bookkeeper.discover.RegistrationManager;
Expand Down Expand Up @@ -681,14 +682,22 @@ public Bookie(ServerConfiguration conf, StatsLogger statsLogger)
LOG.info("Using ledger storage: {}", ledgerStorageClass);
ledgerStorage = LedgerStorageFactory.createLedgerStorage(ledgerStorageClass);

boolean isDbLedgerStorage = ledgerStorage instanceof DbLedgerStorage;

/*
* with this change https://github.com/apache/bookkeeper/pull/677,
* LedgerStorage drives the checkpoint logic. But with multiple entry
* logs, checkpoint logic based on a entry log is not possible, hence it
* needs to be timebased recurring thing and it is driven by SyncThread.
* SyncThread.start does that and it is started in Bookie.start method.
* LedgerStorage drives the checkpoint logic.
*
* <p>There are two exceptions:
*
* 1) with multiple entry logs, checkpoint logic based on a entry log is
* not possible, hence it needs to be timebased recurring thing and
* it is driven by SyncThread. SyncThread.start does that and it is
* started in Bookie.start method.
*
* 2) DbLedgerStorage
*/
if (entryLogPerLedgerEnabled) {
if (entryLogPerLedgerEnabled || isDbLedgerStorage) {
syncThread = new SyncThread(conf, getLedgerDirsListener(), ledgerStorage, checkpointSource) {
@Override
public void startCheckpoint(Checkpoint checkpoint) {
Expand Down

0 comments on commit d7fffac

Please sign in to comment.