-
Notifications
You must be signed in to change notification settings - Fork 0
Migrate from record node UUID to internal ID #119
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
SteRiccio
wants to merge
26
commits into
master
Choose a base branch
from
feat/node-iid
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 2 commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
9a21934
node internal ID: added DB migrations
SteRiccio 7f1a961
node internal ID: added DB migrations
SteRiccio cd67494
updated migrations
SteRiccio 53b9e8d
improved node i_id update
SteRiccio d308b83
fixed constraints update
SteRiccio ac81c23
db migrations: added filter by record_uuid
SteRiccio f4a3ef0
Merge branch 'master' into feat/node-iid
mergify[bot] 8340e0a
Merge branch 'master' into feat/node-iid
mergify[bot] c694c2f
Merge branch 'master' into feat/node-iid
mergify[bot] a02be0b
Merge branch 'master' into feat/node-iid
mergify[bot] 10ad793
Merge branch 'master' into feat/node-iid
mergify[bot] a8c8442
Merge branch 'master' into feat/node-iid
mergify[bot] 2c5502b
Merge branch 'master' into feat/node-iid
mergify[bot] d7b6149
added record validation migration
SteRiccio 443bdb3
Merge branch 'master' into feat/node-iid
mergify[bot] 39667c8
Merge branch 'master' into feat/node-iid
mergify[bot] 9c8a431
Merge branch 'master' into feat/node-iid
mergify[bot] 4c7a49a
Merge branch 'master' into feat/node-iid
mergify[bot] bd0a1cb
Merge branch 'master' into feat/node-iid
mergify[bot] 42bc605
Merge branch 'master' into feat/node-iid
mergify[bot] f09b247
Merge branch 'master' into feat/node-iid
mergify[bot] c45d350
Merge branch 'master' into feat/node-iid
mergify[bot] f03ac52
Merge branch 'master' into feat/node-iid
mergify[bot] 0fa0fe1
Merge branch 'master' into feat/node-iid
mergify[bot] 8bfb423
Merge branch 'master' into feat/node-iid
mergify[bot] a461ede
Merge branch 'master' into feat/node-iid
mergify[bot] File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
51 changes: 51 additions & 0 deletions
51
src/db/dbMigrator/migration/survey/migrations/20260217085512-alter-table-node-add-iid.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| 'use strict' | ||
|
|
||
| var dbm | ||
| var type | ||
| var seed | ||
| var fs = require('fs') | ||
| var path = require('path') | ||
| var Promise | ||
|
|
||
| /** | ||
| * We receive the dbmigrate dependency from dbmigrate initially. | ||
| * This enables us to not have to rely on NODE_PATH. | ||
| */ | ||
| exports.setup = function (options, seedLink) { | ||
| dbm = options.dbmigrate | ||
| type = dbm.dataType | ||
| seed = seedLink | ||
| Promise = options.Promise | ||
| } | ||
|
|
||
| exports.up = function (db) { | ||
| var filePath = path.join(__dirname, 'sqls', '20260217085512-alter-table-node-add-iid-up.sql') | ||
| return new Promise(function (resolve, reject) { | ||
| fs.readFile(filePath, { encoding: 'utf-8' }, function (err, data) { | ||
| if (err) return reject(err) | ||
| console.log('received data: ' + data) | ||
|
|
||
| resolve(data) | ||
| }) | ||
| }).then(function (data) { | ||
| return db.runSql(data) | ||
| }) | ||
| } | ||
|
|
||
| exports.down = function (db) { | ||
| var filePath = path.join(__dirname, 'sqls', '20260217085512-alter-table-node-add-iid-down.sql') | ||
| return new Promise(function (resolve, reject) { | ||
| fs.readFile(filePath, { encoding: 'utf-8' }, function (err, data) { | ||
| if (err) return reject(err) | ||
| console.log('received data: ' + data) | ||
|
|
||
| resolve(data) | ||
| }) | ||
| }).then(function (data) { | ||
| return db.runSql(data) | ||
| }) | ||
| } | ||
|
|
||
| exports._meta = { | ||
| version: 1, | ||
| } |
1 change: 1 addition & 0 deletions
1
...igrator/migration/survey/migrations/sqls/20260217085512-alter-table-node-add-iid-down.sql
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| /* Replace with your SQL commands */ | ||
123 changes: 123 additions & 0 deletions
123
...bMigrator/migration/survey/migrations/sqls/20260217085512-alter-table-node-add-iid-up.sql
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,123 @@ | ||
| ALTER TABLE node | ||
| ADD COLUMN i_id INTEGER; | ||
|
|
||
| ALTER TABLE node | ||
| ADD COLUMN p_i_id INTEGER; | ||
|
|
||
| -- Populate the i_id column with the row number for each record_uuid | ||
| WITH ranked AS ( | ||
| SELECT id, | ||
| ROW_NUMBER() OVER (PARTITION BY record_uuid ORDER BY id) AS i_id | ||
| FROM node | ||
| ) | ||
| UPDATE node | ||
| SET i_id = ranked.i_id | ||
| FROM ranked | ||
| WHERE node.id = ranked.id; | ||
|
|
||
| ALTER TABLE node | ||
| ALTER COLUMN i_id SET NOT NULL; | ||
|
|
||
| -- Populate the p_i_id column by joining the node table with itself based on the parent-child relationship | ||
| UPDATE node AS child | ||
| SET p_i_id = parent.i_id | ||
| FROM node AS parent | ||
| WHERE child.parent_uuid = parent.uuid | ||
| AND child.record_uuid = parent.record_uuid; | ||
|
|
||
| -- Replace meta.h UUID array with i_id array, preserving order | ||
| UPDATE node AS n | ||
| SET meta = jsonb_set( | ||
| meta, | ||
| '{h}', | ||
| COALESCE( | ||
| ( | ||
| SELECT jsonb_agg(p.i_id ORDER BY elems.ordinality) | ||
| FROM jsonb_array_elements_text(n.meta->'h') WITH ORDINALITY AS elems(uuid_text, ordinality) | ||
| JOIN node AS p ON p.uuid = elems.uuid_text::uuid | ||
|
SteRiccio marked this conversation as resolved.
Outdated
|
||
| ), | ||
| '[]'::jsonb | ||
| ) | ||
| ) | ||
| WHERE n.meta ? 'h'; | ||
|
|
||
| -- Update activity_log content to use i_id/p_i_id and drop meta.h | ||
| UPDATE activity_log AS al | ||
| SET content = updated.content | ||
| FROM ( | ||
| SELECT al.id, | ||
| jsonb_set( | ||
| jsonb_set( | ||
| jsonb_set( | ||
| ( | ||
| CASE | ||
| WHEN al.content ? 'meta' | ||
| THEN jsonb_set(al.content, '{meta}', (al.content->'meta') - 'h') | ||
| ELSE al.content | ||
| END | ||
| ) - 'uuid' - 'parentUuid', | ||
| '{iId}', | ||
| to_jsonb(n.i_id), | ||
| true | ||
| ), | ||
| '{pIId}', | ||
| to_jsonb(n.p_i_id), | ||
| true | ||
| ), | ||
| '{recordUuid}', | ||
| to_jsonb(n.record_uuid), | ||
| true | ||
| ) AS content | ||
| FROM activity_log AS al | ||
| JOIN node AS n ON n.uuid = (al.content->>'uuid')::uuid | ||
| WHERE al.type IN ('nodeCreate', 'nodeDelete', 'nodeValueUpdate') | ||
|
SteRiccio marked this conversation as resolved.
Outdated
|
||
| ) AS updated | ||
| WHERE al.id = updated.id; | ||
|
|
||
| DROP VIEW IF EXISTS activity_log_user_aggregate; | ||
| CREATE VIEW activity_log_user_aggregate AS | ||
| SELECT | ||
| DISTINCT ON ( | ||
| date_created::date, | ||
| user_uuid, | ||
| type, | ||
| content_uuid, | ||
| content_key, | ||
| content_record_uuid, | ||
| content_i_id | ||
| ) | ||
| id, | ||
| date_created, | ||
| user_uuid, | ||
| type, | ||
| (content->>'uuid')::uuid as content_uuid, | ||
| content->>'key' as content_key, | ||
| -- node related fields | ||
| (content->>'recordUuid')::uuid as content_record_uuid, | ||
| (content->>'iId')::integer as content_i_id, | ||
| content | ||
| FROM | ||
| activity_log | ||
| WHERE | ||
| NOT system | ||
| ORDER BY | ||
| date_created::date DESC, | ||
| user_uuid, | ||
| type, | ||
| content_uuid, | ||
| content_key, | ||
| content_record_uuid, | ||
| content_i_id, | ||
| date_created DESC; | ||
|
|
||
|
SteRiccio marked this conversation as resolved.
Outdated
|
||
| -- Add a unique constraint on the combination of record_uuid and i_id | ||
|
SteRiccio marked this conversation as resolved.
Outdated
|
||
| ALTER TABLE node | ||
| DROP CONSTRAINT IF EXISTS node_pkey; | ||
|
|
||
| -- Add a composite primary key on record_uuid and i_id | ||
| ALTER TABLE node | ||
| ADD PRIMARY KEY (record_uuid, i_id); | ||
|
SteRiccio marked this conversation as resolved.
Outdated
|
||
|
|
||
| -- Create an index on the combination of record_uuid and i_id for faster lookups | ||
| CREATE INDEX IF NOT EXISTS node_record_uuid_i_id_idx | ||
| ON node (record_uuid, i_id); | ||
|
SteRiccio marked this conversation as resolved.
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.