Sakilla restore fix #16

rocket-3 · 2021-06-11T16:25:39Z

Alter heuristic dependencies searching mechanism in order to find dependencies among MetaSql objects more efficient
This might fix order of objects during Sakilla schema restore process

…endencies among MetaSql objects more efficient This might fix order of objects during `Sakilla` schema restore process

rocket-3 · 2021-06-11T16:27:04Z

@rultor merge, tests=*IntegrationTest*#*CustomTypes*

rultor · 2021-06-11T16:31:57Z

@rocket-3 Oops, I failed. You can see the full log here (spent 4min)

|   >         Restoring function public/_group_concat.fnc ... OK
|   >         Restoring function public/get_customer_balance.fnc ... OK
|   >         Restoring function public/inventory_held_by_customer.fnc ... OK
|   >         Restoring function public/inventory_in_stock.fnc ... OK
|   >         Restoring function public/last_day.fnc ... OK
|   >         Restoring function public/last_updated.fnc ... OK
|   >         Restoring function public/rewards_report.fnc ... OK
|   >         Restoring function public/film_in_stock.fnc ... OK
|   >         Restoring function public/film_not_in_stock.fnc ... OK
|   >         Restoring trigger public/film_fulltext_trigger.trg ... OK
|   >         Restoring trigger public/last_updated.trg ... OK
|   >         Restoring view public/actor_info.vw ... OK16:30 ERROR [App.java:41] : Error execute dbgit
|   > ru.fusionsoft.dbgit.core.ExceptionDBGitRestore: Restore objects error
|   >   at ru.fusionsoft.dbgit.adapters.DBAdapter.restoreDataBase(DBAdapter.java:160)
|   >   at ru.fusionsoft.dbgit.core.GitMetaDataManager.restoreDataBase(GitMetaDataManager.java:384)
|   >   at ru.fusionsoft.dbgit.command.CmdRestore.execute(CmdRestore.java:183)
|   >   at ru.fusionsoft.dbgit.App.executeDbGitCommand(App.java:57)
|   >   at ru.fusionsoft.dbgit.App.main(App.java:29)
|   > Caused by: ru.fusionsoft.dbgit.core.ExceptionDBGitRestore: Error restore public/actor_info.vw
|   >   at ru.fusionsoft.dbgit.postgres.DBRestoreViewPostgres.restoreMetaObject(DBRestoreViewPostgres.java:64)
|   >   at ru.fusionsoft.dbgit.adapters.DBAdapter.restoreDataBase(DBAdapter.java:140)
|   >   ... 4 common frames omitted
|   > Caused by: org.postgresql.util.PSQLException: ERROR: function group_concat(text) does not exist
|   >   Hint: No function matches the given name and argument types. You might need to add explicit type casts.
|   >   Position: 170
|   >   at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
|   >   at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
|   >   at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
|   >   at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
|   >   at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
|   >   at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
|   >   at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
|   >   at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
|   >   at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:266)
|   >   at ru.fusionsoft.dbgit.statement.StatementLogging.execute(StatementLogging.java:178)
|   >   at ru.fusionsoft.dbgit.postgres.DBRestoreViewPostgres.restoreMetaObject(DBRestoreViewPostgres.java:51)
|   >   ... 5 common frames omitted
|   java.lang.Exception: Process exited with error, code 1

rocket-3 · 2021-06-12T13:01:03Z

There is no group_concat(text) function in original schema, but a fucntion _group_concat(text, text)

View public.actor_info uses group_concat(text) function exactly in its code, but creating the same name view during restore causes error.

Further, there will be a more detailed investigation.

rocket-3 · 2021-06-12T15:01:21Z

In schema Sakilla group_concat(text) is an aggregate function which DbGit can't handle for now (fetching code for them in not the same as for common functions and procedures).

Need to implement fetching aggregate functions next.

Note: there could be window functions also

Fix DBAdapterPostgres errors on single object fetching functions Removed some unnecessary messages parts from console output

rocket-3 · 2021-06-13T15:50:54Z

OK, aggregate functions handling on Postgress is implemented

rultor · 2021-06-13T15:55:04Z

@rocket-3 Oops, I failed. You can see the full log here (spent 3min)

|   > 	at ru.fusionsoft.dbgit.command.CmdRestore.execute(CmdRestore.java:183)
|   > 	at ru.fusionsoft.dbgit.App.executeDbGitCommand(App.java:57)
|   > 	at ru.fusionsoft.dbgit.App.main(App.java:29)
|   > Caused by: ru.fusionsoft.dbgit.core.ExceptionDBGitRestore: Error restore public/actor.tbl
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.restoreTablePostgres(DBRestoreTablePostgres.java:124)
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.restoreMetaObject(DBRestoreTablePostgres.java:38)
|   > 	at ru.fusionsoft.dbgit.adapters.DBAdapter.restoreDataBase(DBAdapter.java:140)
|   > 	... 4 more
|   > Caused by: ru.fusionsoft.dbgit.core.ExceptionDBGit: Error restore public/actor.tbl#last_name
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.restoreTableFields(DBRestoreTablePostgres.java:357)
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.restoreTablePostgres(DBRestoreTablePostgres.java:112)
|   > 	... 6 more
|   > Caused by: org.postgresql.util.PSQLException: ERROR: cannot alter type of a column used by a view or rule
|   >   Detail: rule _RETURN on view nicer_but_slower_film_list depends on column "last_name"
|   > 	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
|   > 	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
|   > 	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
|   > 	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
|   > 	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
|   > 	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
|   > 	at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
|   > 	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
|   > 	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:266)
|   > 	at ru.fusionsoft.dbgit.statement.StatementLogging.execute(StatementLogging.java:178)
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.alterTypeColumn(DBRestoreTablePostgres.java:561)
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.restoreTableFields(DBRestoreTablePostgres.java:333)
|   > 	... 7 more
|   > 
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 107.985 s <<< FAILURE! - in

rocket-3 · 2021-06-13T16:43:18Z

Oops! Not the same error I have locally, but seems like there is no way changing table columns withoud dropping views...

Trying again with clean target database.

rultor · 2021-06-13T16:56:39Z

@rocket-3 Oops, I failed. You can see the full log here (spent 3min)

|   >     public/_group_concat.fnc
|   >     public/group_concat.fnc16:55 ERROR [App.java:41] : Error execute dbgit
|   > ru.fusionsoft.dbgit.core.ExceptionDBGitRestore: Restore objects error
|   > 	at ru.fusionsoft.dbgit.adapters.DBAdapter.restoreDataBase(DBAdapter.java:160)
|   > 	at ru.fusionsoft.dbgit.core.GitMetaDataManager.restoreDataBase(GitMetaDataManager.java:384)
|   > 	at ru.fusionsoft.dbgit.command.CmdRestore.execute(CmdRestore.java:183)
|   > 	at ru.fusionsoft.dbgit.App.executeDbGitCommand(App.java:57)
|   > 	at ru.fusionsoft.dbgit.App.main(App.java:29)
|   > Caused by: ru.fusionsoft.dbgit.core.ExceptionDBGit: infinite loop
|   > 	at ru.fusionsoft.dbgit.meta.SortedListMetaObject.createSortedList(SortedListMetaObject.java:126)
|   > 	at ru.fusionsoft.dbgit.meta.SortedListMetaObject.sortFromReferenced(SortedListMetaObject.java:43)
|   > 	at ru.fusionsoft.dbgit.adapters.DBAdapter.restoreDataBase(DBAdapter.java:124)
|   > 	... 4 common frames omitted
|   Process exited with error, code 1
|   Errors: 
|   > 
|   > Error execute dbgit: ru.fusionsoft.dbgit.core.ExceptionDBGitRestore: Restore objects error
|   > 	at ru.fusionsoft.dbgit.adapters.DBAdapter.restoreDataBase(DBAdapter.java:160)
|   > 	at ru.fusionsoft.dbgit.core.GitMetaDataManager.restoreDataBase(GitMetaDataManager.java:384)
|   > 	at ru.fusionsoft.dbgit.command.CmdRestore.execute(CmdRestore.java:183)
|   > 	at ru.fusionsoft.dbgit.App.executeDbGitCommand(App.java:57)
|   > 	at ru.fusionsoft.dbgit.App.main(App.java:29)
|   > Caused by: ru.fusionsoft.dbgit.core.ExceptionDBGit: infinite loop
|   > 	at ru.fusionsoft.dbgit.meta.SortedListMetaObject.createSortedList(SortedListMetaObject.java:126)
|   > 	at ru.fusionsoft.dbgit.meta.SortedListMetaObject.sortFromReferenced(SortedListMetaObject.java:43)
|   > 	at ru.fusionsoft.dbgit.adapters.DBAdapter.restoreDataBase(DBAdapter.java:124)
|   > 	... 4 more
|   > 
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 80.31 s <<< FAILURE! - in

rocket-3 · 2021-06-13T17:17:47Z

OK, got the same error I had locally

Now the current picture:

heuristic dependency search algorithm (located at SortedListMetaObject) is not stable and needs another fix
restore algorithm restores tables first, views (which depends on tables) and functions/procedures next

…o DbObjectNameInSqlPresenceTest.java with test Add dropping existing in files and database both MetaFunction, MetaProcedure, MetaView, MetaTrigger before actually dropping unneeded and restoring changed and added objects

rultor · 2021-06-16T14:33:42Z

@rocket-3 Oops, I failed. You can see the full log here (spent 3min)

|   Process exited with error, code 1
|   Errors: 
|   > 
|   > Error execute dbgit: ru.fusionsoft.dbgit.core.ExceptionDBGitRestore: Error remove public/last_updated.trg
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTriggerPostgres.removeMetaObject(DBRestoreTriggerPostgres.java:90)
|   > 	at ru.fusionsoft.dbgit.command.CmdRestore.execute(CmdRestore.java:189)
|   > 	at ru.fusionsoft.dbgit.App.executeDbGitCommand(App.java:57)
|   > 	at ru.fusionsoft.dbgit.App.main(App.java:29)
|   > Caused by: org.postgresql.util.PSQLException: ERROR: syntax error at end of input
|   >   Position: 44
|   > 	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
|   > 	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
|   > 	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
|   > 	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
|   > 	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
|   > 	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
|   > 	at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
|   > 	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
|   > 	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:266)
|   > 	at ru.fusionsoft.dbgit.statement.StatementLogging.execute(StatementLogging.java:178)
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTriggerPostgres.removeMetaObject(DBRestoreTriggerPostgres.java:87)
|   > 	... 3 more
|   >

Fix drop/restore procedure in Postgres Fix drop/restore function/aggregate function in Postgres Fix drop trigger in Postgres

rultor · 2021-06-16T17:52:13Z

@rocket-3 Oops, I failed. You can see the full log here (spent 7min)

|   > Error execute dbgit: ru.fusionsoft.dbgit.core.ExceptionDBGitRestore: Restore objects error
|   > 	at ru.fusionsoft.dbgit.adapters.DBAdapter.restoreDataBase(DBAdapter.java:160)
|   > 	at ru.fusionsoft.dbgit.core.GitMetaDataManager.restoreDataBase(GitMetaDataManager.java:384)
|   > 	at ru.fusionsoft.dbgit.command.CmdRestore.execute(CmdRestore.java:210)
|   > 	at ru.fusionsoft.dbgit.App.executeDbGitCommand(App.java:57)
|   > 	at ru.fusionsoft.dbgit.App.main(App.java:29)
|   > Caused by: ru.fusionsoft.dbgit.core.ExceptionDBGitRestore: Error restore public/staff.tbl
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.removeTableConstraintsPostgres(DBRestoreTablePostgres.java:669)
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.restoreMetaObject(DBRestoreTablePostgres.java:53)
|   > 	at ru.fusionsoft.dbgit.adapters.DBAdapter.restoreDataBase(DBAdapter.java:121)
|   > 	... 4 more
|   > Caused by: org.postgresql.util.PSQLException: ERROR: cannot drop constraint staff_pkey on table staff because other objects depend on it
|   >   Detail: constraint store_manager_staff_id_fkey on table store depends on index staff_pkey
|   >   Hint: Use DROP ... CASCADE to drop the dependent objects too.
|   > 	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
|   > 	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
|   > 	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
|   > 	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
|   > 	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
|   > 	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
|   > 	at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
|   > 	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
|   > 	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:266)
|   > 	at ru.fusionsoft.dbgit.statement.StatementLogging.execute(StatementLogging.java:178)
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.dropConstraint(DBRestoreTablePostgres.java:262)
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.removeTableConstraintsPostgres(DBRestoreTablePostgres.java:659)

rocket-3 · 2021-06-18T16:47:37Z

Seems like sorting (ordering mechanism) is OK, but...

In
dvdrental -> store depends on staff, and in
pagilla -> staff depends on store

The restore order is dvdrental -> pagilla

As you can see, when dropping constraints before restore, dbgit uses target schema dependencies instead of source...

|   >     Restoring...
|   >         Dropping constraints for all updating tables...
|   >                         public/payment.tbl ([public/rental.tbl, public/customer.tbl, public/staff.tbl])
|   >                         public/rental.tbl ([public/inventory.tbl, public/customer.tbl, public/staff.tbl])
|   >                         public/customer.tbl ([public/store.tbl, public/address.tbl])
|   >                         public/inventory.tbl ([public/store.tbl, public/film.tbl])
|   >                         public/staff.tbl ([public/store.tbl, public/address.tbl])
|   >                         public/store.tbl ([public/address.tbl])
|   >                         public/address.tbl ([public/city.tbl])
|   >                         public/film_actor.tbl ([public/actor.tbl, public/film.tbl])
|   >                         public/film_category.tbl ([public/category.tbl, public/film.tbl])
|   >                         public/city.tbl ([public/country.tbl])
|   >                         public/film.tbl ([public/language.tbl])
|   >                         public/actor.tbl ([])
|   >                         public/category.tbl ([])
|   >                         public/country.tbl ([])
|   >                         public/language.tbl ([])
|   >             Dropping constraints for table public/payment.tbl...
|   >             Dropping constraints for table public/rental.tbl...
|   >             Dropping constraints for table public/customer.tbl...
|   >             Dropping constraints for table public/inventory.tbl...
|   >             Dropping constraints for table public/staff.tbl...18:26 ERROR

… database related to .dbgit repo Add IT extension CharsDbGitIgnoreWithTypes.java to write .dbignore file with custom type extensions and without .csv (for faster execution of tests when data non needed at the moment) Enhance DbGitIntegrationTestBasic.dbToDbRestoreWorksWithCustomTypes with dropping target schema (for having the same result of each execution everywhere) Add some logging on dropping table constraints to be able to see the actual drop order and actual tables dependencies at the moment

…en ordering instead of _target_) Add `Cactoos` lib to project for EO approach learning

rocket-3 · 2021-06-21T04:04:52Z

OK, now need to specify columns explicitly on CREATE TABLE next

rultor · 2021-07-18T12:39:57Z

@rocket-3 Oops, I failed. You can see the full log here (spent 18min)

|   Errors: 
|   > 
|   > Error execute dbgit: ru.fusionsoft.dbgit.core.ExceptionDBGitRestore: Restore objects error
|   > 	at ru.fusionsoft.dbgit.adapters.DBAdapter.restoreDataBase(DBAdapter.java:146)
|   > 	at ru.fusionsoft.dbgit.core.GitMetaDataManager.restoreDataBase(GitMetaDataManager.java:384)
|   > 	at ru.fusionsoft.dbgit.command.CmdRestore.execute(CmdRestore.java:240)
|   > 	at ru.fusionsoft.dbgit.App.executeDbGitCommand(App.java:57)
|   > 	at ru.fusionsoft.dbgit.App.main(App.java:29)
|   > Caused by: ru.fusionsoft.dbgit.core.ExceptionDBGitRestore: Error restore public/payment_p2020_01.tbl
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.restoreTableIndexesPostgres(DBRestoreTablePostgres.java:200)
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.restoreMetaObject(DBRestoreTablePostgres.java:44)
|   > 	at ru.fusionsoft.dbgit.adapters.DBAdapter.restoreDataBase(DBAdapter.java:131)
|   > 	... 4 more
|   > Caused by: ru.fusionsoft.dbgit.core.ExceptionDBGitRunTime: Error load tables!
|   > 	at ru.fusionsoft.dbgit.postgres.DBAdapterPostgres.getTable(DBAdapterPostgres.java:315)
|   > 	at ru.fusionsoft.dbgit.meta.MetaTable.loadFromDB(MetaTable.java:77)
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.restoreTableIndexesPostgres(DBRestoreTablePostgres.java:150)
|   > 	... 6 more
|   > Caused by: java.lang.UnsupportedOperationException
|   > 	at java.base/java.util.AbstractCollection.add(AbstractCollection.java:267)
|   > 	at ru.fusionsoft.dbgit.postgres.DBAdapterPostgres.getTable(DBAdapterPostgres.java:303)

… when network or database is temporary unavailable

rultor · 2021-07-18T15:56:26Z

@rocket-3 Oops, I failed. You can see the full log here (spent 45min)

Error restore public/film_actor.tbl
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.restoreTableConstraintPostgres(DBRestoreTablePostgres.java:254)
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.restoreMetaObject(DBRestoreTablePostgres.java:49)
|   > 	at ru.fusionsoft.dbgit.adapters.DBAdapter.restoreDataBase(DBAdapter.java:142)
|   > 	... 4 more
|   > Caused by: org.postgresql.util.PSQLException: ERROR: could not create unique index "film_actor_pkey"
|   >   Detail: Key (actor_id, film_id)=(95, 245) is duplicated.
|   > 	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
|   > 	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
|   > 	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
|   > 	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
|   > 	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
|   > 	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
|   > 	at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
|   > 	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
|   > 	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:266)
|   > 	at ru.fusionsoft.dbgit.statement.StatementLogging.execute(StatementLogging.java:178)
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.createConstraint(DBRestoreTablePostgres.java:562)
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.restoreTableConstraintPostgres(DBRestoreTablePostgres.java:235)

rocket-3 · 2021-07-21T10:35:20Z

There are a bug in DbAdapterPostgres, that can't detect PK on a column properly, when the column is an FK at the same time.

This causes emtpy delete statements on restrore data of table with such columns. That's why we see duplicate key error in log.

…stomTypes' Change scenario to cover 'empty_db -> pagilla <-> dvdrental' sequences of restore

rultor · 2021-07-21T11:14:46Z

@rocket-3 Oops, I failed. You can see the full log here (spent 29min)

|   > Error execute dbgit: ru.fusionsoft.dbgit.core.ExceptionDBGitRestore: Restore objects error
|   > 	at ru.fusionsoft.dbgit.adapters.DBAdapter.restoreDataBase(DBAdapter.java:146)
|   > 	at ru.fusionsoft.dbgit.core.GitMetaDataManager.restoreDataBase(GitMetaDataManager.java:384)
|   > 	at ru.fusionsoft.dbgit.command.CmdRestore.execute(CmdRestore.java:240)
|   > 	at ru.fusionsoft.dbgit.App.executeDbGitCommand(App.java:57)
|   > 	at ru.fusionsoft.dbgit.App.main(App.java:29)
|   > Caused by: ru.fusionsoft.dbgit.core.ExceptionDBGitRestore: Error restore public/payment_p2020_01.tbl
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.restoreTablePostgres(DBRestoreTablePostgres.java:134)
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.restoreMetaObject(DBRestoreTablePostgres.java:39)
|   > 	at ru.fusionsoft.dbgit.adapters.DBAdapter.restoreDataBase(DBAdapter.java:131)
|   > 	... 4 more
|   > Caused by: org.postgresql.util.PSQLException: ERROR: table "payment" is not partitioned
|   > 	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
|   > 	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
|   > 	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
|   > 	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
|   > 	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
|   > 	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
|   > 	at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
|   > 	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
|   > 	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:266)
|   > 	at ru.fusionsoft.dbgit.statement.StatementLogging.execute(StatementLogging.java:178)
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.restoreTablePartition(DBRestoreTablePostgres.java:522)
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTablePostgres.restoreTablePostgres(DBRestoreTablePostgres.java:124)
|   > 	... 6 more

DBRestoreTableDataPostgres messages cleanup

rocket-3 · 2021-07-23T19:39:18Z

The reason of this error is when we find 'affected' tables in database and add them to update list when it's not needed.

This worked well until there was no case with checkout from schema with partitioned table A to schema with non-partitioned version on A.

rultor · 2021-07-23T20:08:25Z

@rocket-3 Oops, I failed. You can see the full log here (spent 25min)

[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
+ git push origin master
ERROR: Permission to databasegit/dbgit.git denied to rultor.

rultor · 2021-07-23T20:40:11Z

@rocket-3 Oops, I failed. You can see the full log here (spent 13min)

|   Errors: 
|   > 
|   > Error execute dbgit: ru.fusionsoft.dbgit.core.ExceptionDBGitRestore: Restore objects error
|   > 	at ru.fusionsoft.dbgit.adapters.DBAdapter.restoreDataBase(DBAdapter.java:146)
|   > 	at ru.fusionsoft.dbgit.core.GitMetaDataManager.restoreDataBase(GitMetaDataManager.java:384)
|   > 	at ru.fusionsoft.dbgit.command.CmdRestore.execute(CmdRestore.java:243)
|   > 	at ru.fusionsoft.dbgit.App.executeDbGitCommand(App.java:57)
|   > 	at ru.fusionsoft.dbgit.App.main(App.java:29)
|   > Caused by: ru.fusionsoft.dbgit.core.ExceptionDBGitRestore: Error restore public.actor
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTableDataPostgres.restoreTableDataPostgres(DBRestoreTableDataPostgres.java:301)
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTableDataPostgres.restoreMetaObject(DBRestoreTableDataPostgres.java:110)
|   > 	at ru.fusionsoft.dbgit.adapters.DBAdapter.restoreDataBase(DBAdapter.java:131)
|   > 	... 4 more
|   > Caused by: org.postgresql.util.PSQLException: ERROR: syntax error at or near ")"
|   >   Position: 33
|   > 	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
|   > 	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
|   > 	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
|   > 	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
|   > 	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
|   > 	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
|   > 	at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
|   > 	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
|   > 	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:266)
|   > 	at ru.fusionsoft.dbgit.statement.StatementLogging.execute(StatementLogging.java:178)
|   > 	at ru.fusionsoft.dbgit.postgres.DBRestoreTableDataPostgres.restoreTableDataPostgres(DBRestoreTableDataPostgres.java:201)
|   > 	... 6 more

rocket-3 · 2021-07-26T20:01:08Z

Seems like something's changing existing and restore tables descriptions, so they have all their fields not PK.

BackupAdapter does this some way.

Need investigate.

Alter heuristic dependencies searching mechanism in order to find dep…

e423c4e

…endencies among MetaSql objects more efficient This might fix order of objects during `Sakilla` schema restore process

rocket-3 mentioned this pull request Jun 11, 2021

Test DBGit with procedures, views and UDT's #13

Open

rocket-3 linked an issue Jun 11, 2021 that may be closed by this pull request

Fix Sakilla schema restore bug #15

Open