diff --git a/developer_manual/basics/storage/migrations.rst b/developer_manual/basics/storage/migrations.rst index 70e94431239..b0dbdd0ae8a 100644 --- a/developer_manual/basics/storage/migrations.rst +++ b/developer_manual/basics/storage/migrations.rst @@ -4,257 +4,947 @@ Migrations ========== -Migrations change the database schema and operate in three steps: - -* Pre schema changes -* Schema changes -* Post schema changes - -Apps can have multiple migrations, which allows a way more flexible updating process. -For example, you can rename a column while copying all the content with 3 steps -packed in 2 migrations. - -The Nextcloud updater logic will look for your migration -files in the apps `lib/Migration` folder. - -.. note:: While in theory you can run any code in the pre- and post-steps, we - recommend not to use actual php classes. With migrations you can update - from any old version to any new version as long as the migration steps - are retained. Since they are also used for installation, you should - keep them anyway. But this also means when you change a php class which - you use in your migration, the code may be executed on different - database/file/code standings when being ran in an upgrade situation. - -.. note:: Since Nextcloud stores, which migrations have been executed already - you must not “update” migrations. The recommendation is to keep them - untouched as long as possible. You should only adjust it to make sure - it still executes, but additional changes to the database should be done - in a new migration. - -1. Migration 1: Schema change +Migrations allow apps to evolve their database schema and migrate existing +data between app versions. + +A migration can implement three phases, which run in this order during an +upgrade: + +#. ``preSchemaChange()`` +#. ``changeSchema()`` +#. ``postSchemaChange()`` + +Use ``changeSchema()`` for declarative schema changes. Use the pre- and +post-schema phases for data changes or other work that must happen before or +after the schema change. + +Apps can contain multiple migrations. Splitting a complex change across +multiple migrations makes it possible, for example, to add a replacement +column, copy existing data to it, and remove the old column safely. + +Migration location and naming ----------------------------- -With this step the new column gets created: +Migration files for an app are discovered in the app's ``lib/Migration/`` +directory. The directory is relative to the app's actual installation path, +which may be in a custom apps directory. + +Migration classes must use the app's migration namespace: .. code-block:: php - public function changeSchema(IOutput $output, \Closure $schemaClosure, array $options) { - /** @var ISchemaWrapper $schema */ - $schema = $schemaClosure(); + OCA\\Migration - $table = $schema->getTable('twofactor_backupcodes'); +Migration filenames and class names must begin with ``Version``. The migration +identifier is the remainder of the class name after removing that prefix. For +example: - $table->addColumn('user_id', \OCP\DB\Types::STRING, [ - 'notnull' => true, - 'length' => 64, - ]); +.. code-block:: text - return $schema; - } + Class: Version2404Date20220903071748 + Identifier: 2404Date20220903071748 +Files matching ``Version*.php`` are discovered recursively below the migration +directory. ``Version0`` is reserved and must not be used. -2. Migration 1: Post schema change ----------------------------------- +Migration classes must extend +``\OCP\Migration\SimpleMigrationStep``. -In this step the content gets copied from the old to the new column. +Fresh installations +------------------- -.. note:: This could also be done as part of the second migration as part of - a pre schema change +Fresh app installations run migrations in *schema-only* mode. In this mode, +Nextcloud evaluates the ``changeSchema()`` methods of all pending migrations +and applies the resulting schema, but it does not call +``preSchemaChange()`` or ``postSchemaChange()``. -.. code-block:: php +As a result: - public function postSchemaChange(IOutput $output, \Closure $schemaClosure, array $options) { - $query = $this->db->getQueryBuilder(); - $query->update('twofactor_backupcodes') - ->set('user_id', 'uid'); - $query->executeStatement(); - } +* The retained ``changeSchema()`` methods must collectively produce the + complete current schema of the app. +* An app must not depend on a pre- or post-schema method to create its final + schema. +* Data backfills in pre- or post-schema methods are upgrade operations and are + not run on a fresh installation. +* Old migration classes must remain loadable for fresh installations. -3. Migration 2: Schema change ------------------------------ +Writing durable migrations +-------------------------- -With this the old column gets removed. +Nextcloud records completed migration identifiers in the database. A migration +that has already been recorded is normally not executed again. -.. code-block:: php +Do not change a released migration to introduce additional database changes. +Create a new migration instead. A released migration should only be adjusted +when necessary to keep it executable with the currently supported code and +dependencies. - public function changeSchema(IOutput $output, \Closure $schemaClosure, array $options) { - /** @var ISchemaWrapper $schema */ - $schema = $schemaClosure(); +Migration classes may be evaluated years after they were written, including +during a fresh installation. Avoid depending on mutable application +implementation classes whose behavior or constructor may change. Prefer stable +public APIs from the ``OCP`` namespace and keep migration-specific logic within +the migration when practical. - $table = $schema->getTable('twofactor_backupcodes'); - $table->dropColumn('uid'); +Schema operations should normally be guarded with methods such as +``hasTable()``, ``hasColumn()``, and ``hasIndex()``. This makes the intended +behavior explicit and helps retained migrations tolerate compatible database +states. - return $schema; - } +Return the schema only if the migration changed it. If no schema change is +required, return ``null``. Construction of migration classes --------------------------------- -All migration classes are constructed via :ref:`dependency-injection`. So if your migration -steps need additional dependencies, these can be defined in the constructor of your migration -class. +Migration classes are normally instantiated through Nextcloud's +:ref:`dependency-injection` container. Public services required by a migration +can therefore be declared as constructor arguments. -**Example:** If your migration needs to execute SQL statements, inject a `OCP\\IDBConnection` -instance into your migration class like this: +For example, inject ``\OCP\IDBConnection`` when a migration needs to execute +database queries: .. code-block:: php + use Closure; + use OCP\DB\ISchemaWrapper; + use OCP\IDBConnection; + use OCP\Migration\IOutput; + use OCP\Migration\SimpleMigrationStep; + use Override; + class Version2404Date20220903071748 extends SimpleMigrationStep { - public function __construct( - private IDBConnection $db - ) { - } - - public function postSchemaChange(IOutput $output, \Closure $schemaClosure, array $options) { - $query = $this->db->getQueryBuilder(); - // execute some SQL ... - } + public function __construct( + private readonly IDBConnection $db, + ) { + } + + #[Override] + public function postSchemaChange( + IOutput $output, + Closure $schemaClosure, + array $options, + ): void { + $query = $this->db->getQueryBuilder(); + // Execute the required data migration. + } + } + +Use public ``OCP`` services where possible. Depending on an app service that is +later renamed, removed, or given new required constructor arguments can prevent +an old migration from being instantiated. + +Example: replacing a column +--------------------------- + +Replacing or renaming a populated column should normally be split across +multiple migrations: + +#. Add the replacement column. +#. Copy existing data in ``postSchemaChange()``. +#. Apply the final constraints and drop the old column in a later migration. + +The old column must not be removed in the same schema phase that adds the new +column because the data copy runs only after that schema phase has completed. + +Migration 1: add and populate the replacement column +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The first migration adds the replacement column and copies existing values +after the schema change: + +.. code-block:: php + + hasTable('twofactor_backupcodes')) { + return null; + } + + $table = $schema->getTable('twofactor_backupcodes'); + + if ($table->hasColumn('user_id')) { + return null; + } + + $table->addColumn('user_id', Types::STRING, [ + 'notnull' => false, + 'length' => 64, + 'default' => null, + ]); + + return $schema; + } + + #[Override] + public function postSchemaChange( + IOutput $output, + Closure $schemaClosure, + array $options, + ): void { + $schema = $schemaClosure(); + + if (!$schema->hasTable('twofactor_backupcodes')) { + return; + } + + $table = $schema->getTable('twofactor_backupcodes'); + if ( + !$table->hasColumn('uid') + || !$table->hasColumn('user_id') + ) { + return; + } + + $query = $this->db->getQueryBuilder(); + $query->update('twofactor_backupcodes') + ->set('user_id', 'uid') + ->where($query->expr()->isNull('user_id')); + $query->executeStatement(); + } + } + +Passing the source column name directly to ``set()`` creates a +column-to-column assignment. Use ``createNamedParameter()`` or +``createParameter()`` instead when assigning a literal value. + +The replacement column is nullable in this example so that it can be added +before existing rows are populated. Depending on the data and supported +databases, another valid approach is to add a non-null column with an +appropriate temporary default. + +Restricting the update to rows where ``user_id`` is null makes the backfill +safer if it is interrupted or manually repeated during development. + +Migration 2: finalize the replacement +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This example assumes that the source ``uid`` column is non-null and that every +row was copied successfully. If the source permits null values, preserve that +nullability or handle those values explicitly before applying a non-null +constraint. + +A later migration applies the final constraint and removes the old column: + +.. code-block:: php + + hasTable('twofactor_backupcodes')) { + return null; + } + + $table = $schema->getTable('twofactor_backupcodes'); + + if (!$table->hasColumn('user_id')) { + return null; + } + + $table->modifyColumn('user_id', [ + 'notnull' => true, + 'length' => 64, + ]); + + if ($table->hasColumn('uid')) { + $table->dropColumn('uid'); + } + + return $schema; + } + } + +On an upgrade, the post-schema phase of the first migration copies the existing +data before the second migration removes the old column. On a fresh +installation, only the schema phases run, and their combined result contains +the final non-null ``user_id`` column without the obsolete ``uid`` column. + +Example: modifying and transforming a column +-------------------------------------------- + +Some migrations modify a column and then transform its existing values. For +example, increasing the length of a hash column must happen before replacing +short hashes with longer hashes. + +The schema change belongs in ``changeSchema()``: + +.. code-block:: php + + use Closure; + use OCP\DB\ISchemaWrapper; + use OCP\IDBConnection; + use OCP\Migration\Attributes\ColumnType; + use OCP\Migration\Attributes\ModifyColumn; + use OCP\Migration\IOutput; + use OCP\Migration\SimpleMigrationStep; + + #[ModifyColumn( + table: 'jobs', + name: 'argument_hash', + type: ColumnType::STRING, + description: 'Increase the column size for SHA-256 hashes', + )] + class Version1000Date20260825100000 extends SimpleMigrationStep { + public function __construct( + private readonly IDBConnection $connection, + ) { + } + + public function changeSchema( + IOutput $output, + Closure $schemaClosure, + array $options, + ): ?ISchemaWrapper { + $schema = $schemaClosure(); + + if (!$schema->hasTable('jobs')) { + return null; + } + + $table = $schema->getTable('jobs'); + + if (!$table->hasColumn('argument_hash')) { + return null; + } + + $table->modifyColumn('argument_hash', [ + 'notnull' => false, + 'length' => 64, + ]); + + return $schema; + } + + // postSchemaChange() follows below. + } + +The dependent data transformation belongs in ``postSchemaChange()``. + +Processing data in batches +-------------------------- + +Data transformations can take significant time on large tables. Process large +result sets in bounded batches when practical. + +The following example uses increasing primary-key values rather than offsets. +This avoids repeatedly scanning and skipping previously processed rows: + +.. code-block:: php + + use OCP\DB\QueryBuilder\IQueryBuilder; + + public function postSchemaChange( + IOutput $output, + Closure $schemaClosure, + array $options, + ): void { + $chunkSize = 1000; + $lastId = 0; + + $update = $this->connection->getQueryBuilder(); + $update->update('jobs') + ->set( + 'argument_hash', + $update->createParameter('argument_hash'), + ) + ->where( + $update->expr()->eq( + 'id', + $update->createParameter('id'), + ), + ); + + do { + $select = $this->connection->getQueryBuilder(); + $select->select('id', 'argument') + ->from('jobs') + ->where( + $select->expr()->gt( + 'id', + $select->createNamedParameter( + $lastId, + IQueryBuilder::PARAM_INT, + ), + ), + ) + ->orderBy('id', 'ASC') + ->setMaxResults($chunkSize); + + $result = $select->executeQuery(); + $rows = $result->fetchAllAssociative(); + $result->closeCursor(); + + foreach ($rows as $row) { + $id = (int)$row['id']; + $argument = (string)$row['argument']; + + $update->setParameter( + 'argument_hash', + hash('sha256', $argument), + IQueryBuilder::PARAM_STR, + ); + $update->setParameter( + 'id', + $id, + IQueryBuilder::PARAM_INT, + ); + $update->executeStatement(); + + $lastId = $id; + } + + $output->debug( + 'Updated ' . count($rows) . ' background job hashes', + ); + } while (count($rows) === $chunkSize); + } + +Important considerations for batched migrations include: + +* Select only the columns needed by the transformation. +* Use an explicit and stable ordering. +* Prefer key-based pagination on a unique, monotonically ordered column over + increasing offsets for large tables. +* Use typed query parameters. +* Close database cursors before executing further queries when practical. +* Consider how concurrent inserts or updates could affect the selected rows. +* Keep individual queries and transactions small enough for large + installations and database clusters. +* Log progress through ``IOutput`` when a migration may take noticeable time. + +Key-based pagination prevents rows from being skipped because earlier rows +were inserted or removed. It does not isolate the migration from concurrent +updates. If the application can write to the affected table during an upgrade, +design the transformation to be idempotent and determine whether newly +inserted rows require processing. + +The correct batching strategy depends on the table and transformation. For +small tables, one set-based ``UPDATE`` can be simpler and faster than reading +and updating individual rows. + +Set-based updates +----------------- + +When every row can be transformed using the same database expression, prefer a +set-based update: + +.. code-block:: php + + $query = $this->connection->getQueryBuilder(); + $query->update('user_status') + ->set('status_message_timestamp', 'status_timestamp'); + $query->executeStatement(); + +When assigning a literal value, bind it as a parameter: + +.. code-block:: php + + $query = $this->connection->getQueryBuilder(); + $query->update('oauth2_access_tokens') + ->set( + 'token_count', + $query->createNamedParameter( + 1, + IQueryBuilder::PARAM_INT, + ), + ); + $query->executeStatement(); + +Migration metadata +------------------ + +Since Nextcloud 30, migration classes can contain repeatable PHP attributes +that describe their effects to administrators. + +The attributes are metadata only. They do not perform, validate, or +automatically infer the corresponding schema or data change. Keep the metadata +consistent with the implementation of the migration. + +For example: + +.. code-block:: php + :emphasize-lines: 7-19 + + use Closure; + use OCP\DB\ISchemaWrapper; + use OCP\Migration\Attributes\ColumnType; + use OCP\Migration\Attributes\CreateTable; + use OCP\Migration\Attributes\ModifyColumn; + use OCP\Migration\IOutput; + use OCP\Migration\SimpleMigrationStep; + + #[CreateTable( + table: 'new_table', + description: 'Stores things processed by the app', + notes: [ + 'Creation can take additional time on large installations', + ], + )] + #[ModifyColumn( + table: 'other_table', + name: 'this_field', + type: ColumnType::BIGINT, + )] + class Version30000Date20240729185117 extends SimpleMigrationStep { + public function changeSchema( + IOutput $output, + Closure $schemaClosure, + array $options, + ): ?ISchemaWrapper { + // Implement the changes described by the attributes. + } } -Migrations and Metadata ------------------------ +Available migration attributes include: + +* ``\OCP\Migration\Attributes\AddColumn`` for adding a column. +* ``\OCP\Migration\Attributes\AddIndex`` for adding an index. +* ``\OCP\Migration\Attributes\CreateTable`` for creating a table. +* ``\OCP\Migration\Attributes\DataCleansing`` for cleansing data in a table. +* ``\OCP\Migration\Attributes\DropColumn`` for dropping a column. +* ``\OCP\Migration\Attributes\DropIndex`` for dropping an index. +* ``\OCP\Migration\Attributes\DropTable`` for dropping a table. +* ``\OCP\Migration\Attributes\ModifyColumn`` for modifying a column. + +``DataCleansing`` is available since Nextcloud 32. -Since 30, details about migrations are available to administrator as metadata can be attached to your migration class by adding specific PHP Attributes: +Attributes can contain a human-readable ``description`` and a list of +``notes``. Depending on the attribute, additional properties describe the +table, column, column type, index type, or affected columns. + +It is valid to repeat an attribute when a migration performs multiple changes +of the same kind: .. code-block:: php - :emphasize-lines: 5-10 - - use OCP\Migration\Attributes\CreateTable; - use OCP\Migration\Attributes\ColumnType; - use OCP\Migration\Attributes\ModifyColumn; - - #[CreateTable( - table: 'new_table', - description: 'Table is used to store things, but also to get more things', - notes: ['this is a notice', 'and another one, if really needed'] - )] - #[ModifyColumn(table: 'other_table', name: 'this_field', type: ColumnType::BIGINT)] - class Version30000Date20240729185117 extends SimpleMigrationStep { - public function changeSchema(IOutput $output, Closure $schemaClosure, array $options) { - [...] - } - } - - -List of available Migration Attributes: - -* ``\OCP\Migration\Attributes\AddColumn`` if your migration implies the creation of a new column -* ``\OCP\Migration\Attributes\AddIndex`` if your migration adds a new index -* ``\OCP\Migration\Attributes\CreateTable`` if your migration creates a new table -* ``\OCP\Migration\Attributes\DropColumn`` if your migration drops a column -* ``\OCP\Migration\Attributes\DropIndex`` if your migration drops an index -* ``\OCP\Migration\Attributes\DropTable`` if your migration drops a table -* ``\OCP\Migration\Attributes\ModifyColumn`` if your migration modifies a column + + #[ModifyColumn( + table: 'jobs', + name: 'argument_hash', + type: ColumnType::STRING, + description: 'Increase the column size from 32 to 64', + )] + #[ModifyColumn( + table: 'jobs', + name: 'argument_hash', + type: ColumnType::STRING, + description: 'Rehash existing values using SHA-256', + )] + class Version28000Date20240828142927 extends SimpleMigrationStep { + // ... + } .. _migration_console_command: Console commands ---------------- -The following ``occ`` commands help you create and manage migrations:: +The following ``occ`` commands help create, inspect, and execute migrations:: migrations - migrations:execute execute a single migration version manually - migrations:generate generate a new migration file for an app - migrations:migrate execute migrations to a specified or the latest version - migrations:preview preview available DB migrations before an upgrade - migrations:status view the status of migrations for an app + migrations:execute Execute a single migration version manually + migrations:generate Generate a new migration file for an app + migrations:migrate Execute pending migrations up to a target + migrations:preview Preview migration metadata for an upgrade + migrations:status View a migration status summary for an app + +These commands are primarily development and administration tools. Normal app +installation and upgrade processes execute the required migrations +automatically. + +migrations:execute +~~~~~~~~~~~~~~~~~~ + +Execute one migration manually. + +The ``version`` argument is the migration identifier: the class name without +the ``Version`` prefix. For example, the identifier for +``Version2404Date20220903071748`` is ``2404Date20220903071748``: + +.. code-block:: console + + sudo -E -u www-data php occ migrations:execute myapp 2404Date20220903071748 -migrations\:execute -^^^^^^^^^^^^^^^^^^^^ +This command directly executes the selected migration's pre-schema, schema, +and post-schema phases and then records it as executed. -Execute a single migration version manually. The ``version`` argument is the -migration class name without the ``Version`` prefix — for example, if your -migration is named ``Version2404Date20220903071748``, the version is -``2404Date20220903071748``:: +Without debug mode, the command refuses identifiers that are already recorded +as executed. It also rejects the reserved values ``0`` and ``prev``. It does +not implement a migration rollback. - sudo -E -u www-data php occ migrations:execute myapp 2404Date20220903071748 +During development, debug mode can be enabled in ``config/config.php``: -.. note:: +.. code-block:: php + + 'debug' => true, + +Debug mode permits an already recorded migration to be executed again. Use +this only in a disposable development environment. Migration code is not +required to be safely repeatable, and rerunning a data migration can corrupt or +duplicate data. + +migrations:generate +~~~~~~~~~~~~~~~~~~~ + +Generate a migration class for an app: + +.. code-block:: console + + sudo -E -u www-data php occ migrations:generate myapp 1000 + +The ``version`` argument is the app-version prefix used to order migrations +from parallel development branches. The expected value is calculated as: + +.. code-block:: text + + major * 1000 + minor + +Examples include: + +.. code-block:: text + + App version 1.0.x -> 1000 + App version 2.34.x -> 2034 + App version 30.0.x -> 30000 + +Only decimal digits are accepted, with a maximum length of 16 digits. If the +provided value differs from the value calculated from the current app version, +the command emits a warning and, in interactive mode, asks whether it should +continue. + +The command appends the current timestamp and generates a class name such as: + +.. code-block:: text + + Version1000Date20260825090000 + +The file is written to: + +.. code-block:: text + + /lib/Migration/ + +The generated class contains empty ``preSchemaChange()`` and +``postSchemaChange()`` methods and a ``changeSchema()`` method that initially +returns ``null``. Remove unused methods or implement the required migration +logic. + +If the app uses a Composer-generated class map or another generated autoloader, +regenerate that autoloader after creating the class. For Composer-based +autoloaders, this can require: + +.. code-block:: console + + composer dump-autoload + +migrations:migrate +~~~~~~~~~~~~~~~~~~ + +Execute all pending migrations for an app: + +.. code-block:: console + + sudo -E -u www-data php occ migrations:migrate myapp + +Without an explicit target, the command runs all pending migrations and is +equivalent to targeting ``latest``: + +.. code-block:: console + + sudo -E -u www-data php occ migrations:migrate myapp latest + +An explicit migration identifier can be supplied to execute pending migrations +up to that ordering boundary: + +.. code-block:: console + + sudo -E -u www-data php occ migrations:migrate myapp 2404Date20220903071748 + +Use a complete migration identifier, including its app-version and ``Date`` +components. + +This command only executes migrations that are not recorded as completed. It +does not roll back migrations or remove completed migration records. Do not +use ``prev``, ``next``, or ``first`` as rollback or navigation operations. + +migrations:preview +~~~~~~~~~~~~~~~~~~ + +Preview administrator-facing migration metadata for a proposed upgrade without +executing the migrations: - Without debug mode enabled, ``migrations:execute`` will refuse to run a - version that has already been executed or that would roll back a previous - migration. Enable debug mode (``’debug’ => true`` in ``config.php``) to - override this restriction during development. +.. code-block:: console -migrations\:generate -^^^^^^^^^^^^^^^^^^^^^ + sudo -E -u www-data php occ migrations:preview 30.0.0 -Generate a new migration file for an app. The ``version`` argument is the -major version of your app as an integer. Use the major and minor digits of -your app version, mapped to 3 digits (``1.0.x => 1000``, ``2.34.x => 2034``), -to leave room for parallel branch migrations:: +The command obtains release metadata for the destination version and filters +out migrations already known to the local installation. It displays metadata +for core and relevant apps. Apps that do not provide migration metadata are +reported separately because they might still run migrations during the +upgrade. - sudo -E -u www-data php occ migrations:generate myapp 1000 +The argument can also be: -The generated file is placed in ``apps/myapp/lib/Migration/``. +* A URL from which release metadata can be downloaded. +* An absolute path, beginning with ``/``, to a local JSON metadata file. -.. note:: +For example: - After generating a migration you may need to run ``composer dump-autoload`` - before it can be executed. +.. code-block:: console -migrations\:migrate -^^^^^^^^^^^^^^^^^^^^ + sudo -E -u www-data php occ migrations:preview https://example.test/metadata.json + sudo -E -u www-data php occ migrations:preview /tmp/metadata.json -Execute all pending migrations for an app, or migrate to a specific version. -Accepts a version number (``YYYYMMDDHHMMSS``) or an alias (``first``, -``prev``, ``next``, ``latest``):: +Only use metadata from a trusted source. - sudo -E -u www-data php occ migrations:migrate myapp - sudo -E -u www-data php occ migrations:migrate myapp prev +The preview is based on published migration metadata. It does not execute the +migration classes and cannot predict unreported effects of migrations whose +attributes are absent or incomplete. -migrations\:preview -^^^^^^^^^^^^^^^^^^^^ +migrations:status +~~~~~~~~~~~~~~~~~ -Preview the DB migrations that would be applied during an upgrade to a given -version, without executing them:: +Display a migration status summary for an app: - sudo -E -u www-data php occ migrations:preview 30.0.0 +.. code-block:: console -migrations\:status -^^^^^^^^^^^^^^^^^^^ + sudo -E -u www-data php occ migrations:status myapp -Show which migrations have been executed and which are pending for an app:: +The summary includes information such as: - sudo -E -u www-data php occ migrations:status myapp +* The migrations table, namespace, and directory. +* Previous, current, next, and latest migration identifiers. +* Summary information about executed, available, unavailable, and new + migrations. +* Human-readable descriptions of pending migrations when those migrations + implement non-empty ``name()`` and ``description()`` methods. + +The command presents a summary; it does not list every executed and pending +migration identifier. Adding indices -------------- -Adding indices to existing tables can take long time, especially on large tables. Therefore it is recommended to not add the indices in the migration itself, but to indicate the index requirement to the server by adding a listener for the ``AddMissingIndicesEvent``. This way the migration can be executed in a separate step and do not block the upgrade process. For new installations the index should still be added to the migration that creates the table. +Adding an index to an existing large table can take significant time. Instead +of adding such an index in a normal app migration, an app can declare it using +``AddMissingIndicesEvent``. Administrators can then add the index separately +using the database maintenance command. + +Register a listener in the app's bootstrap class: .. code-block:: php - class AddMissingIndicesListener implements IEventListener { - public function handle(Event $event): void { - if (!$event instanceof AddMissingIndicesEvent) { - return; - } + use OCA\MyApp\Listener\AddMissingIndicesListener; + use OCP\AppFramework\Bootstrap\IRegistrationContext; + use OCP\DB\Events\AddMissingIndicesEvent; - $event->addMissingIndex('my_table', 'my_index', ['column_a', 'column_b']); - } + public function register(IRegistrationContext $context): void { + $context->registerEventListener( + AddMissingIndicesEvent::class, + AddMissingIndicesListener::class, + ); } +Implement the listener: + +.. code-block:: php + + namespace OCA\MyApp\Listener; + + use OCP\DB\Events\AddMissingIndicesEvent; + use OCP\EventDispatcher\Event; + use OCP\EventDispatcher\IEventListener; + + /** + * @template-implements IEventListener + */ + class AddMissingIndicesListener implements IEventListener { + public function handle(Event $event): void { + if (!$event instanceof AddMissingIndicesEvent) { + return; + } + + $event->addMissingIndex( + 'my_table', + 'my_index', + ['column_a', 'column_b'], + ); + } + } + +The arguments to ``addMissingIndex()`` are: + +.. code-block:: php + + addMissingIndex( + string $tableName, + string $indexName, + array $columns, + array $options = [], + bool $dropUnnamedIndex = false, + ): void + +Set ``dropUnnamedIndex`` to ``true`` when an existing unnamed index covering +exactly the same columns should be removed before the named index is added. + +To request a unique index, use ``addMissingUniqueIndex()``: + +.. code-block:: php + + $event->addMissingUniqueIndex( + 'my_table', + 'my_unique_index', + ['column_a', 'column_b'], + ); + +Declaring an index through this event does not add it during normal app +migration execution. The declaration is consumed by database setup checks and +by the following command: + +.. code-block:: console + + sudo -E -u www-data php occ db:add-missing-indices + +To print the schema operations without applying them, use: + +.. code-block:: console + + sudo -E -u www-data php occ db:add-missing-indices --dry-run + +Keep the listener registered after releasing the change. It must remain +available to installations that have not run the maintenance command yet. + Replacing indices ----------------- .. versionadded:: 29.0.0 -Similar to adding an index to an existing table, it could be necessary to replace one or more indices with a new one. To avoid a gap between dropping the old indices in a migration and adding the new one through ``AddMissingIndicesEvent``, it is possible to do both at once in ``AddMissingIndicesEvent``. +Use ``replaceIndex()`` when one or more existing indices should be replaced +with a new index. This can be used, for example, to: + +* Replace a non-unique index with a unique index. +* Merge multiple single-column indices into a multi-column index. +* Change the columns or options of an existing index while using a new name. + +The maintenance command creates the new index before removing the old indices, +avoiding a period where none of the indices exist. -If none of the previous indices are found, e.g. because they were optional and not created yet, the replacement index will be treated as *missing index*. +If the new index already exists, the replacement is skipped. Otherwise, +Nextcloud: -.. note:: Make sure to not use the same index name for the new index as for old indices. +#. Creates the new index. +#. Applies that schema change. +#. Removes any named old indices that exist. +#. Applies the removal schema change. + +If none of the old indices exist, the new index is still treated as a missing +index and is created. + +Do not reuse one of the old index names as the new index name. .. code-block:: php - class ReplaceIndicesListener implements IEventListener { - public function handle(Event $event): void { - if (!$event instanceof AddMissingIndicesEvent) { - return; - } + namespace OCA\MyApp\Listener; - $event->replaceIndex('my_table', ['my_old_index_one', 'my_old_index_two'], 'my_new_index', ['column_a', 'column_b'], false); - } + use OCP\DB\Events\AddMissingIndicesEvent; + use OCP\EventDispatcher\Event; + use OCP\EventDispatcher\IEventListener; + + /** + * @template-implements IEventListener + */ + class ReplaceIndicesListener implements IEventListener { + public function handle(Event $event): void { + if (!$event instanceof AddMissingIndicesEvent) { + return; + } + + $event->replaceIndex( + 'my_table', + ['my_old_index_one', 'my_old_index_two'], + 'my_new_index', + ['column_a', 'column_b'], + false, + ); + } } + +The arguments to ``replaceIndex()`` are: + +.. code-block:: php + + replaceIndex( + string $tableName, + array $oldIndexNames, + string $newIndexName, + array $columns, + bool $unique, + array $options = [], + ): void + +Set ``unique`` to ``true`` to create a unique replacement index. The optional +``options`` array is passed to the schema API when the new index is created.