-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Inject postgres admin into appliance console #153
Merged
Fryguy
merged 92 commits into
ManageIQ:master
from
NickLaMuro:inject_postgres_admin_into_appliance_console
Mar 17, 2021
Merged
Changes from all commits
Commits
Show all changes
92 commits
Select commit
Hold shift + click to select a range
9fb04ed
PostgresAdmin isn't rails, move to util.
jrafanie 48c8d27
Drop Vmdb::Logging now that we're out of the Rails app.
jrafanie 513c2e0
Add common appliance pg environment methods.
jrafanie cd1c7a1
Make the bulk of the start and stop commands a testable query method.
jrafanie 71e9f3c
Fix minor bugs exposed by adding tests.
jrafanie 7532ed1
Use Rails.root, not a hardcoded appliance path for the certs location.
jrafanie fd148fd
Outside of Rails, we need to build RAILS_ROOT manually.
jrafanie 12506b8
Convert some missing constants to methods.
jrafanie 47da2bc
Rubocop cleanup.
jrafanie 6db1c21
ManageIQ code shouldn't know where the old system/TEMPLATE is
jrafanie 2ecc233
Mass reformat
1c4f7d4
Use environment instead of pgpass file
kbrock e7fc458
added PostgresAdmin.database_in_recovery?(connection) method
yrudman f28cec8
Clear up the appliance console summary screen
carbonin cca905d
Add primary/standby to appliance console database status
carbonin 9363c62
Move prep_data_directory method to PostgresAdmin
carbonin 36953c9
Get tests running
bdunne e9e2eba
Move test directories to the root
bdunne 8e4d6e6
Clean up references to GEMS_PENDING_ROOT and bundler_setup
bdunne 45516ec
Run `bundle gem` against the repo to default most files.
Fryguy ffe3d03
Switch from coveralls to codeclimate-test-reporter
Fryguy 1178621
Drop pglogical subscriptions before the restore
carbonin 3d78935
Prevent Trollop from exiting the rspec process
carbonin fe3c521
Move the trollop stubs and errors to their own spec/support file
carbonin 09412af
Original implementation of patch from #71
lmiccini 8073be1
Add specs for PostgresAdmin.mount_point
carbonin 012aaef
Alter PostgresAdmin.prep_data_directory to remove all contents
carbonin af67742
Remove the pglogical extension before attempting a database restore
carbonin 98701fb
Fix typo dropping pglogical extension
gtanzillo ef6e1a3
Move logic for dropping pglogical extension to a separate method
gtanzillo 1d7e924
Wait for "pglogical manager" db connection to go away before allowing…
gtanzillo ec38b72
Make tests pass
chessbyte bb5d906
Removed code moved to manageiq-smartstate repo
chessbyte b313143
Remove methods with no content
carbonin fed7cb8
Remove the start and stop methods from PostgresAdmin
carbonin 4612949
Remove PostgresAdmin.pg_ctl
carbonin add26e5
Remove the scl methods from PostgresAdmin
carbonin c4637b4
Move database_in_recovery? from PostgresAdmin to PostgresHaAdmin
bdunne 858a463
PostgresAdmin no longer uses 'pg' gem
bdunne 8706549
load the file in lib/
bdunne 3485cb2
reintroduce "require 'active_support/all'"
bdunne 3708de5
Use pg_basebackup rather than pg_dump for backing up the database
carbonin 427cb4b
Remove the before_restore method
carbonin 67002af
Remove annoying comment
carbonin eaec9a2
Choose restore type based on the target file type
carbonin b114ceb
Removing test because because `before_restore` method was also removed.
gtanzillo c041909
Create an empty directory for pg_basebackup
carbonin c6e2884
Fix ENOENT error from backup_pg_compress
agrare 6bedfeb
Dont mess with pglogical from PostgresAdmin
carbonin 1d16b19
Remove references to RAILS_ROOT
carbonin aeb48b1
Add back pg_dump support to PostgresAdmin
NickLaMuro a43144f
Add PostgresAdmin#combine_command_args (private)
NickLaMuro e636e24
Use #combine_command_args in #runcmd
NickLaMuro 0e51620
Handle multi-value args in PostgresAdmin.backup_pg_dump
NickLaMuro 979da90
Update PG_DUMP_MULTI_VALUE_ARGS for underscore args
NickLaMuro 21de264
Use --xlog-method=stream for pg_basebackup
carbonin 381c81a
Revert to using --xlog-method fetch for postgres backup
carbonin 74d9d4a
Use STDOUT for pg_basebackup
NickLaMuro aceff70
Use correct variable name for PostgresAdmin
NickLaMuro 2ba4ce2
Don't set empty ENV values for database dumps
kbrock b5cf1dd
Add PostgresAdmin.group
NickLaMuro e7ae10f
Add --port option config to PostgresAdmin
NickLaMuro acd69cf
Add PgEnvironmentUpdater
NickLaMuro 5d7d01b
Add PostgresRunner as a helper
NickLaMuro fd73435
Add sample pg backup and dump for testing
NickLaMuro 3144de3
Add connection helper context for db restores
NickLaMuro ba3ebba
Add postgres backup specs
NickLaMuro b424c39
Add postgresql-9.5 to travis
NickLaMuro d33ab6d
Adds PostgresAdmin file testing specs
NickLaMuro 9a9e896
Remove need for `file` shell call via .file_type
NickLaMuro 80a7efd
Handle pipes for pg_restore
NickLaMuro b371d2c
[PostgresAdmin] Add backup_type option to .restore
NickLaMuro dd32246
[PostgresAdmin] Fix backup_type being ignored
NickLaMuro 4a036f1
Use Gem::Package for tar pg restore unpacking
NickLaMuro fd4ba2a
Change pg_basebackup xlog-method option to wal-method
carbonin 2dea6ea
Update CI PG restore tests for PG 10
carbonin 816c545
Update backup and dump files for PG 10
carbonin 7d0014a
ramfs-ing speed! - restore directly to ramfs data directory
carbonin 121a5ae
Remove MiqUtil.runcmd in favor of AwesomeSpawn
Fryguy aa13289
Use Rakefile from manageiq-gems-pending
NickLaMuro fdea187
[Gemfile] Remove manageiq-gems-pending
NickLaMuro 2811360
[PostgresAdmin] Namespace to ManageIQ::ApplianceConsole
NickLaMuro 6400c62
[PostgresAdmin] Fix $log (to self.logger)
NickLaMuro 4efb575
[RestoreHelper] Use proper data dir for backups
NickLaMuro 8bfc157
[PostgresRunner] Use Dir::Tmpname.create
NickLaMuro 6590665
[.travis.yml] postgresql addon
NickLaMuro 21c6c85
[.rspec_ci] Load ci_helper
NickLaMuro ca94973
[.rspec_ci] --format documentation
NickLaMuro a83610e
[ci_helper.rb][CI] Set ENV vars earlier
NickLaMuro 84938a0
[ci_helper.rb] LinuxAdmin::Service.new override
NickLaMuro 9583441
[spec_helper.rb] Configure :with_postgres_specs
NickLaMuro 4edbc2f
[spec/] Avoid ENV overwrite in specs
NickLaMuro File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,6 @@ | ||
--require ci_helper | ||
--require spec_helper | ||
--color | ||
--order random | ||
--profile | ||
--format documentation |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,25 @@ | ||
require "bundler/gem_tasks" | ||
require "rspec/core/rake_task" | ||
|
||
desc "Run RSpec code examples (skip postgres required ones)" | ||
RSpec::Core::RakeTask.new(:spec) | ||
|
||
task :default => :spec | ||
# In CI, as part of the .rspec_ci, load a helper that sets the configuration | ||
# setting to allow the postgres specs to run. | ||
desc "Run RSpec code examples (assumes ci dependencies)" | ||
RSpec::Core::RakeTask.new("spec:ci") do |t| | ||
t.rspec_opts = "--options #{File.expand_path(".rspec_ci", __dir__)}" | ||
end | ||
|
||
desc "Run RSpec code examples (with local postgres dependencies)" | ||
RSpec::Core::RakeTask.new("spec:dev") do |t| | ||
# Load the PostgresRunner helper to facilitate a clean postgres environment | ||
# for testing locally (not necessary for CI), and enables the postgres test | ||
# via the helper. | ||
pg_runner = File.join("spec", "postgres_runner_helper.rb") | ||
t.rspec_opts = "-r #{File.expand_path(pg_runner, __dir__)}" | ||
end | ||
|
||
task :default do | ||
Rake::Task["spec#{':ci' if ENV['CI']}"].invoke | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 1 addition & 1 deletion
2
lib/manageiq/appliance_console/internal_database_configuration.rb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Even though it's not necessary for CI, this seems to over complicate the code (and making local runs "different"). Can we just run it in CI anyway presuming it's relatively inexpensive?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So there is a couple of things here, but I did explain a lot of the architecture of this in the original PR here:
ManageIQ/manageiq-gems-pending#385
So I won't fully repeat that here (and I honestly forget a lot of my decisions without using that to cheat), but the main take aways were:
vmdb_development
would be trashed every time this is run locallyThat all said, I also agree that basically adding an "
init
script written in Ruby" (seespec/postgres_runner_helper.rb
) is basically a lot of code to accomplish this, for like 2 tests...They are arguably important tests, but depending on how much we do or do not move away from this backup strategy might determine if it is still of value.
P.S. This is also still WIP, so definitely things are still subject to change a little more yet.