Deploying with SEED= wipes out existing projects #573

ychae · 2019-10-13T21:39:14Z

When deploying production with the seed set to blank or default it wipes out all existing mongo db content. However, if you try to add a new data catalog from before that has a same name, it gives a duplicate key error (see ticket #572)

The text was updated successfully, but these errors were encountered:

tschaffter · 2019-10-14T16:39:44Z

@ksuruli said in slack:

No worries. Blank sees still wipes out. Can you please test it on your end?

TODO: Try to reproduce

tschaffter · 2019-10-15T15:20:47Z

I can not reproduce this error in development environment. The difference between the following execution of the server is that once I have APP_INIT_DB_SEED_NAME=development and then APP_INIT_DB_SEED_NAME=

Thomass-MacBook-Pro:PHCCollaborationPortal tschaffter$ sudo -E npm run start:server

> [email protected] start:server /Users/tschaffter/dev/PHCCollaborationPortal
> nodemon --inspect --trace-warnings --watch server server

[nodemon] 1.19.2
[nodemon] to restart at any time, enter `rs`
[nodemon] watching dir(s): server/**/*
[nodemon] starting `node --inspect --trace-warnings server`
Debugger listening on ws://127.0.0.1:9229/a1c43faf-99cd-431f-b17a-071029d96e6b
For help, see: https://nodejs.org/en/docs/inspector
Using Primus App plugin
domain https://localhost
(node:29232) DeprecationWarning: current Server Discovery and Monitoring engine is deprecated, and will be removed in a future version. To use the new Server Discover and Monitoring engine, pass option { useUnifiedTopology: true } to the MongoClient constructor.
    at parseFn (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/mongodb/lib/operations/connect.js:312:5)
    at parseConnectionString (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/mongodb/lib/core/uri_parser.js:628:3)
    at connect (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/mongodb/lib/operations/connect.js:266:3)
    at ConnectOperation.execute (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/mongodb/lib/operations/connect.js:191:5)
    at executeOperation (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/mongodb/lib/operations/execute_operation.js:83:26)
    at MongoClient.connect (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/mongodb/lib/mongo_client.js:216:10)
    at Promise (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/mongoose/lib/connection.js:632:12)
    at Promise._execute (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/bluebird/js/release/debuggability.js:313:9)
    at Promise._resolveFromExecutor (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/bluebird/js/release/promise.js:488:18)
    at new Promise (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/bluebird/js/release/promise.js:79:10)
    at NativeConnection.Connection.openUri (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/mongoose/lib/connection.js:629:19)
    at Mongoose.connect (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/mongoose/lib/index.js:327:15)
    at Object.connect (/Users/tschaffter/dev/PHCCollaborationPortal/server/app.js:17:44)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Module._compile (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/pirates/lib/index.js:99:24)
    at Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Object.newLoader [as .js] (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/pirates/lib/index.js:104:7)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Module.require (internal/modules/cjs/loader.js:692:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at Object.<anonymous> (/Users/tschaffter/dev/PHCCollaborationPortal/server/index.js:12:28)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
Initializing db with seed: development
finished populating entity access notifications
finished populating apps
finished populating starred messages
finished populating entity attachments
finished populating articles
finished populating action permissions
finished populating organizations
finished populating dashboards
finished populating entity notifications
finished populating data catalogs
finished populating memos
finished populating threads
finished populating entity permissions
finished populating message notifications
finished populating projects
finished populating reports
finished populating states
finished populating webapps
finished populating tools
finished populating messages
finished populating users
finished populating activities
Express server listening on 9000, in development mode
GET /api/users/me 200 21.470 ms - 372
GET /api/user-notifications?archived=false 200 33.668 ms - -
GET /api/users/me 304 18.894 ms - -
GET /api/action-permissions 200 14.402 ms - 173
GET /api/entity-permissions 200 20.140 ms - 388
spark.userId 5cb7acea2d718614d81cc97e
WebSocket from 127.0.0.1:58225 connected
GET /api/projects?orderedBy=relevance&searchTerms= 200 33.781 ms - 608
^C
Thomass-MacBook-Pro:PHCCollaborationPortal tschaffter$ source envvars-dev
Thomass-MacBook-Pro:PHCCollaborationPortal tschaffter$ sudo -E npm run start:server

> [email protected] start:server /Users/tschaffter/dev/PHCCollaborationPortal
> nodemon --inspect --trace-warnings --watch server server

[nodemon] 1.19.2
[nodemon] to restart at any time, enter `rs`
[nodemon] watching dir(s): server/**/*
[nodemon] starting `node --inspect --trace-warnings server`
Debugger listening on ws://127.0.0.1:9229/0eeeb7c8-ab18-4562-8d52-dc11cb53c205
For help, see: https://nodejs.org/en/docs/inspector
Using Primus App plugin
domain https://localhost
(node:29348) DeprecationWarning: current Server Discovery and Monitoring engine is deprecated, and will be removed in a future version. To use the new Server Discover and Monitoring engine, pass option { useUnifiedTopology: true } to the MongoClient constructor.
    at parseFn (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/mongodb/lib/operations/connect.js:312:5)
    at parseConnectionString (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/mongodb/lib/core/uri_parser.js:628:3)
    at connect (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/mongodb/lib/operations/connect.js:266:3)
    at ConnectOperation.execute (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/mongodb/lib/operations/connect.js:191:5)
    at executeOperation (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/mongodb/lib/operations/execute_operation.js:83:26)
    at MongoClient.connect (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/mongodb/lib/mongo_client.js:216:10)
    at Promise (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/mongoose/lib/connection.js:632:12)
    at Promise._execute (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/bluebird/js/release/debuggability.js:313:9)
    at Promise._resolveFromExecutor (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/bluebird/js/release/promise.js:488:18)
    at new Promise (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/bluebird/js/release/promise.js:79:10)
    at NativeConnection.Connection.openUri (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/mongoose/lib/connection.js:629:19)
    at Mongoose.connect (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/mongoose/lib/index.js:327:15)
    at Object.connect (/Users/tschaffter/dev/PHCCollaborationPortal/server/app.js:17:44)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Module._compile (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/pirates/lib/index.js:99:24)
    at Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Object.newLoader [as .js] (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/pirates/lib/index.js:104:7)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Module.require (internal/modules/cjs/loader.js:692:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at Object.<anonymous> (/Users/tschaffter/dev/PHCCollaborationPortal/server/index.js:12:28)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
Express server listening on 9000, in development mode
UnauthorizedError: jwt malformed
    at /Users/tschaffter/dev/PHCCollaborationPortal/node_modules/express-jwt/lib/index.js:102:22
    at Object.module.exports [as verify] (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/jsonwebtoken/verify.js:63:12)
    at verifyToken (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/express-jwt/lib/index.js:100:13)
    at fn (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/async/lib/async.js:746:34)
    at /Users/tschaffter/dev/PHCCollaborationPortal/node_modules/async/lib/async.js:1213:16
    at /Users/tschaffter/dev/PHCCollaborationPortal/node_modules/async/lib/async.js:166:37
    at /Users/tschaffter/dev/PHCCollaborationPortal/node_modules/async/lib/async.js:706:43
    at /Users/tschaffter/dev/PHCCollaborationPortal/node_modules/async/lib/async.js:167:37
    at Immediate.<anonymous> (/Users/tschaffter/dev/PHCCollaborationPortal/node_modules/async/lib/async.js:1206:34)
    at runCallback (timers.js:705:18)
    at tryOnImmediate (timers.js:676:5)
    at processImmediate (timers.js:658:5)
GET /api/user-notifications?archived=false 401 18.635 ms - -

tschaffter · 2019-10-15T15:21:22Z

Trying the production environment... Can not reproduce

tschaffter · 2019-10-15T16:07:10Z

@ychae @jaeddy I can not reproduce this issue either in development or production environment. I have created the above PR when investigating but it only improves the readability of the seeds.

@ychae If you still have this issue, please fill in a complete bug report with version of the code used, commands executed, output of the logs, etc.

ychae · 2019-10-15T21:28:23Z

@tschaffter Thanks for the detailed update -- I think for this we'll have to schedule a call with you and @ksuruli (and also possible @Pawel-Madej) to work through this together. I'll ping you on Slack to see if I can get some time slots for this session.

UPDATE: meeting scheduled for Thursday Oct 17 @ 9:30am

tschaffter · 2019-10-18T21:37:11Z

@ychae This will be addressed by Roche/GNE when they will migrate data from their alpha4 DB to the current version of the portal. Removing from Milestone 2 as this is not an issue with the current version of the codebase.

ychae added Bug Something isn't working Priority: Critical labels Oct 13, 2019

ychae added this to the Milestone 2 milestone Oct 13, 2019

ychae assigned tschaffter Oct 13, 2019

tschaffter mentioned this issue Oct 15, 2019

Improve readability of seeds #578

Merged

tschaffter added the Review-Needed label Oct 15, 2019

tschaffter assigned ychae and ksuruli and unassigned tschaffter Oct 17, 2019

tschaffter assigned Pawel-Madej and unassigned ksuruli Oct 18, 2019

tschaffter removed this from the Milestone 2 milestone Oct 18, 2019

tschaffter unassigned Pawel-Madej and ychae Mar 23, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deploying with SEED= wipes out existing projects #573

Deploying with SEED= wipes out existing projects #573

ychae commented Oct 13, 2019

tschaffter commented Oct 14, 2019

tschaffter commented Oct 15, 2019 •

edited

Loading

tschaffter commented Oct 15, 2019 •

edited

Loading

tschaffter commented Oct 15, 2019

ychae commented Oct 15, 2019 •

edited

Loading

tschaffter commented Oct 18, 2019

Deploying with SEED= wipes out existing projects #573

Deploying with SEED= wipes out existing projects #573

Comments

ychae commented Oct 13, 2019

tschaffter commented Oct 14, 2019

tschaffter commented Oct 15, 2019 • edited Loading

tschaffter commented Oct 15, 2019 • edited Loading

tschaffter commented Oct 15, 2019

ychae commented Oct 15, 2019 • edited Loading

tschaffter commented Oct 18, 2019

tschaffter commented Oct 15, 2019 •

edited

Loading

tschaffter commented Oct 15, 2019 •

edited

Loading

ychae commented Oct 15, 2019 •

edited

Loading