From 2becd1271c51a509078f6b41ca13e12ac76ac61b Mon Sep 17 00:00:00 2001 From: Craig Zaccaro Date: Mon, 5 Sep 2022 23:48:55 -0400 Subject: [PATCH] Documentation updates. --- Gemfile | 2 +- README.md | 16 +++++++--------- application_overview.md | 16 +++++++++------- blazer_reporting.md | 14 +++++++------- 4 files changed, 24 insertions(+), 24 deletions(-) diff --git a/Gemfile b/Gemfile index fb7a55ba..ff9b993f 100644 --- a/Gemfile +++ b/Gemfile @@ -70,7 +70,7 @@ end # Report builder gem 'reports_kit' # Replaced by blazer reporting - 1/24/21 -# Dependency on Blazer version 2.4.7 due to app-specific overrides living in +# Dependency on Blazer version 2.5.0 due to app-specific overrides living in # app/overrides that depend on the Blazer code at this version of the gem. Updating # this gem will require review of our overridden classes with changes to code in the # desired Blazer gem version. diff --git a/README.md b/README.md index 65078f5e..423b1437 100644 --- a/README.md +++ b/README.md @@ -10,16 +10,16 @@ The Puget Sound Restoration Fund works to raise and outplant hatchery-reared [Pi This application will enable groups to add data either through CSV upload or through the web interface. Groups can view reports and visual representations of key data. Future plans include giving groups the ability to generate custom reports on the fly. -## Welcome Contributors!! + ## Getting Started @@ -88,10 +88,10 @@ This application uses a modified implementation of the [Blazer](https://github.c ### Docker -We are currently experimenting with Docker for development. While we would love for more people to try it out be forewarned - Docker functionality may not be maintained moving forward. You will need Docker and docker-compose. +We are currently experimenting with Docker for development. While we would love for more people to try it out, be forewarned - Docker functionality may not be maintained moving forward. You will need Docker and docker-compose. * [Docker Desktop](https://www.docker.com/products/docker-desktop) is recommended for Windows and Mac computers. -* The `make` utility can also make your development life easier. It it usually already installed on Linux and Mac computers. For Windows, an easy way to install it is via [Chocolatey](https://chocolatey.org/install), a software package management system similar to Homebrew for Windows. Once Chocolatey is installed, install make with `choco install make` in a command prompt running as Administrator. +* The `make` utility can also make your development life easier. It is usually already installed on Linux and Mac computers. For Windows, an easy way to install it is via [Chocolatey](https://chocolatey.org/install), a software package management system similar to Homebrew for Windows. Once Chocolatey is installed, install make with `choco install make` in a command prompt running as Administrator. * If you run into issues using Docker Desktop on windows, we recommend you view [this page](https://github.com/mdworken/MKD-Docker-Windows-Rails) for troubleshooting info. **Starting Fresh** @@ -148,7 +148,7 @@ We want it to be easy to understand and contribute to this app, which means we l [Application Overview](https://github.com/rubyforgood/abalone/blob/main/application_overview.md) -### Current Status + ### The Problem Our stakeholders, the Bodega Marine Laboratory and the Puget Sound Restoration Fund work with large amounts of data collected as part of their abalone captive breeding programs. They need a system that can act as a central data repository for all of this data and provide robust reporting capabilities to help them examine trends and combine data collected across their research efforts. @@ -181,8 +181,6 @@ We are building a multi-tenant application which has the following capabilities: [See a full data dictionary here.](https://github.com/rubyforgood/abalone/wiki/Abalone-Data-Dictionary) -## Deployment -The application is currently deployed on Heroku at https://abalone-staging.herokuapp.com/. ## And Don't Forget... diff --git a/application_overview.md b/application_overview.md index 21c9549d..5e4c685d 100644 --- a/application_overview.md +++ b/application_overview.md @@ -1,8 +1,8 @@ # Application Overview -__Last updated to the document: August 2021__ +__Last updated to the document: September 2022__ -This document provides a general overview of the key parts of the Abalone Analytics application. It is primarily intended to help facilitate on-boarding of developers that are new to the application. As the application evolves, so will this document. PRs with updates and improvements are welcome!! +This document provides a general overview of the key parts of the Abalone Analytics application. It is primarily intended to help facilitate on-boarding of developers that are new to the application. As the application evolves, so will this document. PRs with updates and improvements are welcome! ## Purpose @@ -18,7 +18,7 @@ The application also accepts direct imports for several models in the system. Th ### Data Storage -Imported data is currently stored in the database and presented using a combination of standard rails views (Animals index, Cohorts index, etc.) and the reporting interface described below. We expect the storage and presentation of data to evolve to suit the needs of the application's initial stakeholders. +Imported data is stored in the database and presented using a combination of standard rails views (Animals index, Cohorts index, etc.) and the reporting interface described below. We expect the storage and presentation of data to evolve to suit the needs of the application's initial stakeholders. ### Reporting @@ -37,7 +37,7 @@ Currently, an organization's users can have one of two roles - `admin` and `user #### Row level security -The application makes use of row level security to protect each organizations data in the reporting interface. The technical process for applying and updating the row level security can be found [here](https://github.com/rubyforgood/abalone/blob/main/blazer_reporting.md). The application has a rake task in place that handles the SQL commands needed. This task is updated when new tables that should be included in row level security or made available to an org_user are added. +The application makes use of row level security to protect each organizations data in the reporting interface. The technical process for applying and updating the row level security can be found [here](https://github.com/rubyforgood/abalone/blob/main/blazer_reporting.md). The application has a rake task that handles the SQL commands needed. This task is updated when new tables that should be included in row level security or made available to an org_user are added. #### Other models @@ -52,21 +52,23 @@ Currently, the following models access the `organization` model directly. This i operations facilities animals + processed_files ``` ### Data Import There are two primary paths for importing data into the system. -The first is through the File Upload interface. This allows users to import csv files containing measurement data. Organizations can enter custom measurement types, for example, `mm`, `count`, `gonad_score`. In the future, additional categories beyond `measurement` could be added to this interface. Organizations can upload existing files or download a template file that they can populate. +The first is through the File Upload interface. This allows users to import csv files containing measurement event data or mortality event date. Organizations can enter custom measurement types, for example, `mm`, `count`, `gonad_score` and custom exit types, for example, `incidental`, `olutplanted`, `sacrificed`. In the future, additional categories beyond `measurement` and `mortalityEvent` could be added to this interface. Organizations can upload existing files or download a template file that they can populate. Application Flow: - Files uploaded at FileUploads#new - Upload is sent to [FileUploder](https://github.com/rubyforgood/abalone/blob/main/app/lib/file_uploader.rb) - If the category is valid, a background job is created to process the files +- The job for the relevant `UPLOADABLE_MODEL` is run. This job will include `ImportJob` - [ImportJob](https://github.com/rubyforgood/abalone/blob/main/app/jobs/concerns/import_job.rb) runs -- The job is performed, validating headers against the model and calling `import_records` -- The import information is passed to [CSVImporter.new](https://github.com/rubyforgood/abalone/blob/main/app/lib/csv_importer.rb) to create the measurement record from the import. +- The job is performed, validating headers and calling `import_records` +- The import information is passed to [CSVImporter.new](https://github.com/rubyforgood/abalone/blob/main/app/lib/csv_importer.rb) to create the record from the import. The second is through the CSV Upload path available on certain index pages. These allow users to import model specific csv files such as a file of basic animal data. These are most likely to be used when onboarding an organization. diff --git a/blazer_reporting.md b/blazer_reporting.md index ef2ddbf4..8a8d56b6 100644 --- a/blazer_reporting.md +++ b/blazer_reporting.md @@ -1,6 +1,6 @@ ## Abalone Analytics Blazer Reporting -This application utilizes a modified implementation of the [Blazer](https://github.com/ankane/blazer) gem to provide direct SQL access to specific tables with data scoped to an organizational level. Modifications to the Blazer gem are performed in app/overrides directories per [Rails documented practices](https://edgeguides.rubyonrails.org/engines.html#overriding-models-and-controllers). The Blazer gem version has been locked at 2.4.7 to ensure these overrides remain dependable. +This application utilizes a modified implementation of the [Blazer](https://github.com/ankane/blazer) gem to provide direct SQL access to specific tables with data scoped to an organizational level. Modifications to the Blazer gem are performed in app/overrides directories per [Rails documented practices](https://edgeguides.rubyonrails.org/engines.html#overriding-models-and-controllers). The Blazer gem version has been locked at 2.5.0 to ensure these overrides remain dependable. Use of the Blazer gem requires setup in both the production and development environments. @@ -8,7 +8,7 @@ Use of the Blazer gem requires setup in both the production and development envi #### Initial Setup -This setup assumes Heroku hosting. Guidance for setup on other hosting platforms is welcome via PR. The maintainers will also update this document as the application evolves. +This setup assumes Heroku hosting. Guidance for setup on other hosting platforms is welcome via pull request. The maintainers will also update this document as the application evolves. Note: Requires, at least, a Standard-0 Postgres instance in Heroku to allow for the creation of additional roles. @@ -27,10 +27,10 @@ Note: For consistency, each org credential should be named with that organizatio 6. Attach that credential to the application. 7. Leave the permissions blank. They will be set by the Blazer rake task. 8. Repeat steps 5-7 for each organization in the application. -9. In Heroku, add the environment variables for the blazer credential and all org credentials. The urls will be available under each credential in the Postgres add-on. - `BLAZER_DATABASE_URL: add credential uri here` - `ORG1_DATABASE_URL: add credential uri here` - `ORG2_DATABASE_URL: add credential uri here` +9. In Heroku, add the environment variables for the blazer credential and all org credentials. The urls will be available under each credential in the Postgres add-on. + `BLAZER_DATABASE_URL: add credential uri here` + `ORG1_DATABASE_URL: add credential uri here` + `ORG2_DATABASE_URL: add credential uri here` 10. Run a backup of the production database 11. Run the Blazer rake task: `heroku run SAVE=1 rake blazer:add_database_security`. 12. Log in as an organization user and confirm reporting is now functioning and scoped. @@ -84,7 +84,7 @@ This process assumes initial setup, migration and seeding of a local postgres da ### Refreshing the staging database - Run manual backup of the database -- Confirm that we have a backup of each SQL query. (these will need to be re-entered) +- Confirm that we have a backup of each SQL query (these will need to be re-entered). - Reset the database through the Heroku interface - `heroku run rails db:migrate --app abalone-staging` - `heroku run rails db:seed --app abalone-staging`