Merge pull request #290 from Cingulara/develop

OpenRMF OSS v1.8

README.md

@@ -1,5 +1,7 @@
-# OpenRMF<sup>&reg;</sup> Documentation (v 1.7.0)
-OpenRMF<sup>&reg;</sup> is an open source tool for managing, viewing, and reporting of your DoD STIG checklists, SCAP Scans and Nessus Patch Scans in one web-based interface using your browser. It also generates a compliance listing of all your checklists across a whole system based on NIST 800-53 for your Risk Management Framework (RMF) documentation and process. This tool helps you manage multiple systems going through the RMF process and allows you to structure your data in a clean interface all in one location for your group or program. 
+# OpenRMF<sup>&reg;</sup> Documentation (v 1.8.0)
+OpenRMF<sup>&reg;</sup> is an open source application for managing, viewing, and reporting of your DoD STIG checklists, SCAP Scans and Nessus Patch Scans in one web-based interface using your browser. It also generates a compliance listing of all your checklists across a whole system based on NIST 800-53 for your Risk Management Framework (RMF) documentation and process. This tool helps you manage multiple systems going through the RMF process and allows you to structure your data in a clean interface all in one location for your group or program. 
+
+You can export your checklists as CKL files and your test plan and POAM as MS Excel properly formatted files as well.
 
 ## Get OpenRMF<sup>&reg;</sup> Core Running Locally
 If you want to get it running on your local laptop, desktop, or server follow these instructions below. You need a fairly good internet connection and Docker Desktop / Docker Community Edition to get this going. And then go to the [latest release](https://github.com/Cingulara/openrmf-docs/releases) and download the Keycloak zip file and OpenRMF<sup>&reg;</sup> zip file. 

architecture/README.md

@@ -1,5 +1,5 @@
 # OpenRMF Architecture
-This has the current architecture information for the OpenRMF application as of version 0.11 and beyond to include the current 1.5.x version.
+This has the current architecture information for the OpenRMF<sup>&reg;</sup> OSS application as of version 1.8.
 
 ![Image](./openRMF-Tool-Architecture.png?raw=true)
 
@@ -13,20 +13,16 @@ The architecture was setup to do a few things for this tool and for the team act
 
 * https://github.com/Cingulara/openrmf-web is the web UI pointing to all these APIs below to render checklists listings, data, vulnerabilities, reports, and allowing saving 
 of chart data and XLSX downloads.
-* https://github.com/Cingulara/openrmf-api-read is for listing, getting, and downloading a checklist and its metadata of title, description, type, and future user info. It also has an export to Excel function that is color coded for status thanks to a request by a good IA/CS friend of mine that needed that.
-* https://github.com/Cingulara/openrmf-api-save is for saving checklist data by posting it ALL in a form, including the raw checklist data (not a file). This publishes an "openrmf.save.xxxx" type of event to NATS.
+* https://github.com/Cingulara/openrmf-api-read is for listing, getting, and downloading a checklist and its metadata of title, description, type, and future user info. It also has an export to Excel function that is color coded for status thanks to a request by a good IA/CS friend of mine that needed that. As of v1.8 it also is where you can upload CKL and XML data, save checklist and system package information, as well as generate compliance.
 * https://github.com/Cingulara/openrmf-api-template is for uploading, listing, and getting checklist file templates to start from.
 * https://github.com/Cingulara/openrmf-msg-score is a NATS messaging subscriber listening to "openrmf.save.*" events from Save and Upload to score the checklist and putting that score into the Mongo DB for the scoring API
 * https://github.com/Cingulara/openrmf-api-scoring is for reading a score of a checklist as well as scoring a checklist based on a file posted (at runtime).
-* https://github.com/Cingulara/openrmf-api-upload is for uploading a .CKL checklist file with metadata and saving the result. This publishes an "openrmf.save.xxxx" type of event.
 * https://github.com/Cingulara/openrmf-api-controls is a read-only lookup of NIST controls to match to CCI for the compliance API and other pieces that need to pull the NIST control descriptions for 800-53.
-* https://github.com/Cingulara/openrmf-api-compliance is for generating the compliance listing, matching NIST controls via CCI to 1 or more checklists in a System. This generates a table of controls and the checklists corresponding to the control from the system's group of checklists. The checklist is linked to the Checklist service and color coded by status.
 * https://github.com/Cingulara/openrmf-api-audit is a read-only lookup of Audit information for OpenRMF that only Administrators can access.
 * https://github.com/Cingulara/openrmf-api-reports is a read-only lookup of OpenRMF data for certain reports that use caching and eventual consistency of data (Nessus Patch Report and Host Vulnerability).
 * https://github.com/Cingulara/openrmf-msg-controls is a NATS client for responding to request/reply on a list of all RMF controls or get the information on a specific control (i.e. AC-1).
 * https://github.com/Cingulara/openrmf-msg-compliance is a NATS client for responding to request/reply on a list of all compliance listings mapping STIG vulnerability IDs to controls. Use this for a full listing based on a low/moderate/high level as well as if you are using personally identifiable information (PII) or similar data.
 * https://github.com/Cingulara/openrmf-msg-template is a NATS client for responding to request/reply on a request for a System template based on the title passed in.
-* https://github.com/Cingulara/openrmf-msg-checklist is a NATS client for responding to request/reply on a request for a checklist based on the Mongo DB record Id passed in.
 * https://github.com/Cingulara/openrmf-msg-system is a NATS client for responding to published messages for updating a System based on title, number of checklists, or running a compliance check.
 * https://github.com/Cingulara/openrmf-msg-audit is a NATS client for responding to published messages for recording auditable events through OpenRMF.
 * https://github.com/Cingulara/openrmf-msg-reports is a NATS client for responding to published messages for eventual consistency of OpenRMF data used for reporting.
@@ -39,8 +35,8 @@ OpenRMF uses NATS messaging to work eventual consistency as well as API-to-API c
 
 | Subject | Msg Type | Calling API |     Receiving Client  | Description |
 |---------|----------|-------------|-----------------------|-------------|
-| openrmf.checklist.read | Request/Reply | Score (Msg Client), Compliance  | openrmf-msg-checklist | Ask for a full checklist/artifact record based on the ID passed in |
-| openrmf.system.checklists.read | Request/Reply | Compliance          | openrmf-msg-checklist | Ask for all checklist records for a given system title passed in |
+| openrmf.checklist.read | Request/Reply | Score (Msg Client), Compliance  | openrmf-msg-system | Ask for a full checklist/artifact record based on the ID passed in |
+| openrmf.system.checklists.read | Request/Reply | Compliance          | openrmf-msg-system | Ask for all checklist records for a given system title passed in |
 | openrmf.checklist.save.new | Subscribe | Upload | openrmf-msg-score | Grab the new uploaded checklist ID sent and generate the score of open, not applicable, not a finding, and not reviewed items across categories |
 | openrmf.checklist.save.new | Subscribe | Upload | openrmf-msg-reports | Grab the new uploaded checklist ID sent and generate the vulnerability data in the reports database, separated out by vulnerability ID |
 | openrmf.checklist.save.update | Subscribe | Upload | openrmf-msg-score | Grab the updated checklist ID sent and generate the score of open, not applicable, not a finding, and not reviewed items across categories |
@@ -52,8 +48,8 @@ OpenRMF uses NATS messaging to work eventual consistency as well as API-to-API c
 | openrmf.controls | Request/Reply | Compliance |  openrmf-msg-controls| Send back the list of all controls. |
 | openrmf.controls.search | Request/Reply | Controls | openrmf-msg-controls | Send back a single record for the passed in control (i.e. AC-2). |
 | openrmf.template.read | Request/Reply | Upload | openrmf-msg-template | Send back a single template checklist record for the passed in title. Used when you upload an XCCDF SCAP scan result to create a checklist. |
-| openrmf.checklist.read | Request/Reply | Score | openrmf-msg-checklist | Send back a single checklist record for the passed in Mongo DB InternalId title. Used when you score a checklist in eventual consistency to pull the checklist and create the structure so we can do a count on status. |
-| openrmf.system.checklists.read | Request/Reply | Read | openrmf-msg-checklist | Send back the list of checklists so we can export them into XLSX from the System page. |
+| openrmf.checklist.read | Request/Reply | Score | openrmf-msg-system | Send back a single checklist record for the passed in Mongo DB InternalId title. Used when you score a checklist in eventual consistency to pull the checklist and create the structure so we can do a count on status. |
+| openrmf.system.checklists.read | Request/Reply | Read | openrmf-msg-system | Send back the list of checklists so we can export them into XLSX from the System page. |
 | openrmf.system.update.{Id} | Subscribe | Save | openrmf-msg-system | When a system title is updated, make sure all references throughout the checklists are updated. We save the system group Id and the title with the checklists for easier usage throughout OpenRMF. The source-of-truth is the systemgroups collection in MongoDB. |
 | openrmf.system.count.> | Subscribe | Upload (add) and Save (delete) | openrmf-msg-system | Increments with a ".add" at the end of the subject or decrements if there is a ".delete" at the end of the subject. The payload is the system group Id. |
 | openrmf.system.compliance | Subscribe | Compliance | openrmf-msg-system | Stores the date of the last compliance check run into the system group record for display later. |

database/Dockerfile.MongoDB

@@ -1,2 +1,2 @@
-FROM mongo:4.4.4
+FROM mongo:5.0.6
 USER mongodb

database/makeDatabase.cmd

@@ -1 +1 @@
-docker build -f ./Dockerfile.MongoDB -t mongo:4.4.4-nonroot .
+docker build -f ./Dockerfile.MongoDB -t mongo:5.0.6-nonroot .

database/makeDatabase.sh

@@ -1 +1 @@
-docker build -f ./Dockerfile.MongoDB -t mongo:4.4.4-nonroot .
+docker build -f ./Dockerfile.MongoDB -t mongo:5.0.6-nonroot .

docs/cleanup.md

@@ -0,0 +1,37 @@
+---
+layout: default
+title: Upgrades and Cleanup
+nav_order: 2000
+---
+
+# Upgrading from older OpenRMF OSS versions
+As you upgrade from version 1.0, to 1.2, to 1.6 and then 1.8 and beyond, you will have older container images for the application you no longer need. Most of the time, these are saved in the /var file structure on Linux. There is an open source tool called Portainer that is great for cleanup of older containers. If you have never used this or have had OpenRMF since its inception, you will want to perform this to clean up the older non-used images. On Linux, the /var folder eventually may fill up with these old images.  So it is good to prune older ones that are not needed.
+
+Before you do any of these steps please take a full backup or snapshot of your server, virtual machine or workstation that is running OpenRMF as a precaution. 
+
+# Portainer Registry Screen
+
+To run it perform the following command. You may need “sudo” in front of it depending on your login and permissions. This will load the Portainer interface on port 9005. You can change that to whatever port you wish that is accessible to you and is not already running. 
+
+```
+docker run -p 9005:9000 --rm --name=portainer -v /var/run/docker.sock:/var/run/docker.sock -v ~/container_data/portainer:/data portainer/portainer-ce:2.9.0-alpine
+```
+
+Then open a browser window to that hostname / IP address using http://xxxxxxxxxxx:9005/ or whatever port you used. On first login you will need to create an administrator account and password to connect to your local Docker registry. Once done click the Local instance and you will see a menu like below. There are a few things to review for cleaning up. It will be best if you are running OpenRMF fully when doing this so you do not remove any running container. If you accidentally remove one, you can always log into SoteriaSoft.Jfrog.io and pull the image again. 
+
+Once the admin login is set up, click on the Get Started button and then on the “local” listing to connect to the local registry. 
+
+## Portainer Volumes Listing
+On the next screen click on the Volumes menu on the left and see the volumes on your computer. As long as you are fully running OpenRMF , all the ones marked “Unused” are the ones you can remove. The rest are in use or are needed. You can change the list per page from “10” in the table listing to 50 or 100 depending on how long your listing is. 
+To remove older unused volumes, click the checkbox next to the volumes and then click the Remove button above the listing. An example is shown below. Be careful NOT TO remove any specifically named “openrmf” volumes as that is more than likely your real data. 
+
+## Portainer Images Listing
+Next, click on the Images menu on the left. Depending on when you began using OpenRMF there may be a list of older images here that are no longer used that are marked “Unused” as well. You can change the list per page from “10” in the table listing to 50 or 100 depending on how long your listing is. 
+
+To remove these older images, click the checkbox next to the images no longer needed and then click the Remove button above the listing. There can be images such as older Prometheus, Grafana, and Vault images as well as several that look like “openrmf-xxx-xxxxxxx”. There are some images listed here such as the openrmf-api-external, mongo-express, ubuntu:latest and portainer images that you should keep. 
+
+## Portainer Cleanup
+When all is done you can log out of Portainer to finish the cleanup process. Additionally, when you are done you can remove the “~/container_data” folder in your home directory to wipe all information on Portainer and clean up after this process as well.  
+
+When you want to run Portainer later to clean this up again, repeat these processes.
+If you accidentally remove an image that is needed, the “start” procedures will look for them and pull them down again.

docs/index.md

@@ -7,9 +7,9 @@ nav_order: 1
 # OpenRMF Introduction
 
 Welcome to the OpenRMF Docs site. This site contains help screens, scenarios, screenshots and 
-other useful information to use the OpenRMF tool fo you and your team. 
+other useful information to use the OpenRMF tool for you and your team. This is a container-based application you can download locally and run on a laptop, server, VM, cloud instance, even Kubernetes if you wish.
 
-OpenRMF is the only web-based open source tool to manage your DoD STIG checklists, generate NIST compliance, keep track of your security items that are Open or Not Reviewed, and massively shrink your timeline to collect data and submit for an ATO!
+OpenRMF is the only web-based open source tool to manage your DoD STIG checklists, generate NIST compliance, keep track of your security items that are Open or Not Reviewed, and massively shrink your timeline to collect data and submit for an ATO! You can use the application to manage your checklists, track compliance, and then export CKL files, a test plan XLSX and a POAM XLSX for your use. 
 
 Compare this to the manual way you have to manage STIG Checklists and SCAP scans, outlined in Tutela's [blog post here](https://medium.com/@dgould_43957/how-to-use-disa-stig-viewer-tool-907358d17cea). 
 
@@ -72,6 +72,6 @@ With coming updates such as automating the Risk Assessment Report, automating th
 More information can be found in our <a href="https://www.openrmf.io/doc/OpenRMF-Product-Information.pdf" target="_blank">product PDF</a> on our website.
 
 ## Architecture
-Below is the top level architecture as of version 0.11 and beyond, drawn via <a href="https://app.diagrams.net/" target="_blank">Draw.io</a>'s great tool (now Diagrams.net). 
+Below is the top level architecture as of version 1.8 and beyond, drawn via <a href="https://app.diagrams.net/" target="_blank">Draw.io</a>'s great tool (now Diagrams.net). 
 
 ![OpenRMF v0.11 Architecture and beyond](/assets/openRMF-Tool-Architecture.png)