App-level support for sandbox overlay (on_sandbox, sandbox seeds, production-safe seeding)#1257
Draft
aaronskiba wants to merge 20 commits intointegrationfrom
Draft
App-level support for sandbox overlay (on_sandbox, sandbox seeds, production-safe seeding)#1257aaronskiba wants to merge 20 commits intointegrationfrom
on_sandbox, sandbox seeds, production-safe seeding)#1257aaronskiba wants to merge 20 commits intointegrationfrom
Conversation
Generated by 🚫 Danger |
aaronskiba
force-pushed
the
aaron/sandbox
branch
from
17a60e2 to
b0234d6
Compare
All deployed environments use `RAILS_ENV=production` and `config/environments/production.rb`.
This initializer allows the sandbox overlay to override `config/environments/${RAILS_ENV}.rb` settings.
- Prevents actual email deliveries when `Rails.application.secrets.on_sandbox.to_s == 'true'`
- Note: Uses secrets directly to avoid autoloading FeatureFlagHelper during initialization
app/controllers/users/omniauth_callbacks_controller.rb - Added before_action to block `def openid_connect` from executing when the `on_sandbox` feature flag is enabled app/views/devise/registrations/_personal_details.html.erb - Added logic to hide the 'Link your institutional credentials' link when the `on_sandbox` feature flag is enabled app/views/shared/_sign_in_form.html.erb - Added logic to hide the 'Sign in with institutional or social ID' when the `on_sandbox` feature flag is enabled
`Rails.application.secrets.funder_org_id` is not used outside of these seeds files and `Rails.application.config.default_funder_id` appears to do the same work.
This change limits the number of orgs generated via the sandbox seeds files to 3.
- The 3 orgs will be the app's default org ("Alliance") as well as the English and French test organisations.
- `Org.all.each do |org|` was previously used for generating the sandbox orgs. The code would create one sandbox org for each existing org in the db (1200+).
This change now only uses the default funder templates as well as two "test" templates for generating seed data. - The code removed from `db/seeds/sandbox/seeds_4.rb` now exists in `def sandbox_templates`.
This change enables us to remove the `super_admin_password`, `english_admin_password`, and `french_admin_password` secrets. This change will also require additional actions: - All of these accounts will now use `Rails.application.secrets.user_password.to_s` for the User password. - Password uniqueness across all user accounts will have to be performed manually within the app after the rake task has completed. - NOTE: This was already required for the multiple "regular user" accounts created. Also, doing this for the super admin and two org admin accounts isn't that much extra work, and being able to remove the extra secrets makes it worthwhile.
This secret does not appear to be referenced anywhere in the codebase.
org: `Org.find_by(abbreviation: "Portage")` is intended to be the app's default org. However, the default org has since changed its abbreviation value.
- Added variables for `default_org`, `english`, and `french`, which addresses previous repeated db calls.
- Refactored the creation of regular users.
- Previously, we had 3 separate (1..20).each to address the (org, language) groupings for users.
- Now we create the same users and (org, language) groupings via a single loop
- Removed Faker references
- Although, this seeds file will be used to populate the sandbox db, the "sandbox" environment will still use `RAILS_ENV=production`. Thus, Faker won't be available.
- Now we have firstname = "tester#{index}" (unique value for each user), surname: 'Test' (same for every user), and `api_token: Faker::Lorem.word` has been removed.
Our sandbox environment will still use RAILS_ENV=production. Thus, these seeds files will have to live in db/seeds/production.
`[20..50].include?(index)` and `[60..90].include?(index)` never matched because `[20..50]` is an array containing a single Range object, not a list of integers. Checking `include?(index)` on that array always returned false, so plans were never assigned the intended templates.
Limit to 15 plans in sandbox seeds files - Also reduced num plans assigned to test templates as a result. Update plan.template so be either default_org_template, test1_template, or test2_template.
This change ensures `bin/rails db:setup` correctly loads sandbox seed data for the sandbox overlay. Removed legacy, staging-only seed logic used "For 3.1.0 Migration only" - (`before_seeds:copy_data`, staging seed loading, and `rewrite_postgres:retrieve_data`). Enabled the sandbox seed-loading path and updated it to load `db/seeds/production/*.rb` instead of `sandbox`, since the sandbox overlay runs with `RAILS_ENV=production`.
This change results in an error being thrown when executing `bin/rails db:setup` without `ON_SANDBOX='true'` . This guard prevents db seeding of non-sandbox overlays.
aaronskiba
force-pushed
the
aaron/sandbox
branch
from
b9e1d08 to
2b5c983
Compare
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The sandbox environment runs with
RAILS_ENV=production. To create and populate the sandbox database, execute:This command must only be run against the sandbox overlay.
Running
bin/rails db:setupagainst any non-sandbox overlay will create and seed a production-mode database and may result in data loss.This PR must be coordinated with another corresponding PR (not yet published) that adds the sandbox overlay in
https://github.com/portagenetwork/roadmap-deploy.
Before merging,
rails export_production_data:build_sandbox_datashould be run one final time against an up-to-date production database dump to ensure the committed sandbox seed files reflect current production data.Summary
This PR introduces application-level support for a sandbox overlay and prepares the codebase to safely run sandbox environments using
RAILS_ENV=production.Disabling Unwanted Features via
FeatureFlagHelper/ on_sandboxNote: For these changes to fully take effect,
onSandbox: 'true'must be set in the sandbox overlay's config.Sandbox seed generation
Limit num of Orgs in sandbox seed data & refactor
Org.all.each do |org|was previously used for generating the sandbox orgs. The code would create one sandbox org for each existing org in the db (1200+).Update generation/handling of sandbox templates
db/seeds/sandbox/seeds_4.rbnow exists indef sandbox_templates.Filter Guidances/Groups by default + test orgs
Rename db/seeds/sandbox -> db/seeds/production
RAILS_ENV=production. Thus, these seeds files will have to live in db/seeds/production.Enable seed-loading in db/seeds/production.rb
bin/rails db:setupcorrectly loads sandbox seed data for the sandbox overlay.before_seeds:copy_data, staging seed loading, andrewrite_postgres:retrieve_data).db/seeds/production/*.rbinstead ofsandbox, since the sandbox overlay runs withRAILS_ENV=production.Remove deprecated secrets
As a result of this PR's changes, a number of secrets have been removed from
charts/dmp-roadmap/templates/app/configmap.yamlandconfig/secrets.ymlAdditional Notes
Reference: Sandbox Seed Contents
The sandbox seed files generate a constrained, production-safe dataset intended for demo, testing, and statistics only.
Seeded Data Overview
Orgs
Question Formats
Guidance Groups & Guidances
Templates
Plans
Languages
Users
Reference: Notes For PR Reviewers
This PR includes new sandbox database seed generation. The commands below can be used as a reference to test sandbox database creation and verify that its contents are generated correctly.
Local setup tips for reviewers
Environment variables
Before running the setup command, add the following to your .env file:
ON_SANDBOX=trueenables sandbox-specific behavior required by the new seeds.USER_PASSWORDensures seed-generated users are created with a valid password that satisfies validation rules.Validating sandbox DB and seed generation
To test the full sandbox database initialization and seed process:
This executes the sandbox database creation and ensures the new seed logic runs as intended.