EventIndexer: ensure we add initial checkpoints when the db is first opened (#31448)

* EventIndex: add some documentation

... because comments are not subject to rationing.

* EventIndex: rationalise addInitialCheckpoints logic

* EventIndex: improve logging

* use a single `logger` instance to do the prefixing
* use `JSON.stringify` on the checkpoints so that the rageshakes are useful
* distinguish between logger.warn and logger.debug
* emit some logs if the crawler loop fails

* Move check for empty database to `init`

The problem with checking if the database is empty in `onSync` is that, by the
time we get there, it won't be.

Instead let's remember if the db was empty in `init`, and then once a sync
completes, we can add the checkpoints.

* Some tests for EventIndexer

Author: richvdh

src/indexing/BaseEventIndexManager.ts

@@ -15,10 +15,23 @@ import {
 
 // The following interfaces take their names and member names from seshat and the spec
 /* eslint-disable camelcase */
+
+/** A record of a place to resume crawling events in a given room. */
 export interface ICrawlerCheckpoint {
+    /** The room to be indexed */
     roomId: string;
-    token: string | null;
+
+    /** The pagination index to resume crawling from. */
+    token: string;
+
+    /**
+     * If `fullCrawl` is false (or absent) and we find that we have already indexed the events we find, then we stop crawling.
+     *
+     * If `fullCrawl` is true, then we keep going until we reach the end of the room history.
+     */
     fullCrawl?: boolean;
+
+    /** Whether we should crawl in the forward or backward direction. */
     direction: Direction;
 }