Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: cfpb/hmda-platform
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v2.3.12
Choose a base ref
...
head repository: cfpb/hmda-platform
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref
Loading
Showing 560 changed files with 202,816 additions and 115,627 deletions.
22 changes: 22 additions & 0 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
## Releasing Image to ECR

> **_NOTE:_** This document contains info on how to publish the `hmda-platform` image to ECR.
> This job, located in `ecr-push.yaml`, can **only** be run locally on your machine.
### Prerequisites

- `act` - version >=0.2.71
- https://nektosact.com/installation/index.html
- Logged in and authenticated to AWS

### Copy Job
At the root of the repo, run:

```shell
act -j 'release_to_ecr' --env=IMAGE_TAG=v0.1.2 --env=AWS_URL=aws.com --env-file <(aws configure export-credentials --format env)
```
Make sure to set:
- `IMAGE_TAG`
- The tag of what image you want to copy from Dockerhub to ECR
- `AWS_URL`
- The URL to our AWS instance
42 changes: 42 additions & 0 deletions .github/workflows/cve-scan-pr.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: HMDA PR CVE Scan

on:
pull_request:
types: [opened, synchronize, reopened]

permissions:
pull-requests: write

jobs:
hmda-platform-cve-scan:
runs-on: ubuntu-latest

steps:
- name: Check out repo
uses: actions/checkout@v4

- name: Setup sbt launcher
uses: sbt/setup-sbt@v1

- name: Build Docker image of HMDA platform
run: |
env JAVA_OPTS="-Xss256m -Xmx4096m" sbt "project hmda-platform" dockerPublishLocalSkipTests
- name: Tag and name Docker image
run: docker tag hmda/hmda-platform:latest pr-cve-scan:latest

- name: Scan image
uses: anchore/scan-action@v6
with:
image: pr-cve-scan:latest
output-file: grype-report.txt
fail-build: false
severity-cutoff: high
output-format: table

- name: Upload Grype report to artifact
uses: actions/upload-artifact@v4
with:
name: cve-report
path: |
grype-report.txt
37 changes: 37 additions & 0 deletions .github/workflows/dockerhub-push.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: HMDA Docker Hub Image Push

on:
push:
tags:
- '*' # Push events to every tag not containing /

env:
REGISTRY: hmda/hmda-platform

jobs:
push_to_dockerhub:
name: Push Docker image to Docker Hub
runs-on: ubuntu-latest
permissions: write-all
steps:
- name: Check out the repo
uses: actions/checkout@v4

- name: Setup sbt launcher
uses: sbt/setup-sbt@v1

- name: Build image of HMDA Platform only
run: |
sbt "project hmda-platform" dockerPublishLocalSkipTests
- name: Tag Docker image
run: docker tag $(docker images --filter=reference="hmda/hmda-platform:latest" --format "{{.ID}}") ${{ env.REGISTRY }}:${{ github.ref_name }}

- name: Log in to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Push image to Docker Hub
run: docker push ${{ env.REGISTRY }}:${{ github.ref_name }}
30 changes: 30 additions & 0 deletions .github/workflows/ecr-push.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: HMDA ECR Image Push

on: workflow_dispatch

jobs:
release_to_ecr:
name: Copy Docker image from Dockerhub to ECR
runs-on: ubuntu-latest
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ env.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ env.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1

- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v2

- name: Copy Image to ECR
run: |
docker pull hmda/hmda-platform:${{ env.IMAGE_TAG }}
docker tag hmda/hmda-platform:${{ env.IMAGE_TAG }} ${{ env.AWS_URL }}/hmda/hmda-platform:${{ env.IMAGE_TAG }}
docker push ${{ env.AWS_URL }}/hmda/hmda-platform:${{ env.IMAGE_TAG }}
- name: Cleanup images
run: |
docker rmi hmda/hmda-platform:${{ env.IMAGE_TAG }}
docker rmi ${{ env.AWS_URL }}/hmda/hmda-platform:${{ env.IMAGE_TAG }}
42 changes: 42 additions & 0 deletions .github/workflows/helper-functions.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: "Helper Functions"
on:
workflow_call:
inputs:
service:
required: true
type: string
output-file:
required: true
type: string
jobs:
test-service:
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Setup sbt launcher
uses: sbt/setup-sbt@v1

- name: Setup JDK
uses: actions/setup-java@v4
with:
java-version: '11'
distribution: 'adopt'

- name: Run hmda-platform Tests
run: |
sbt "project ${{ inputs.service }}" "testOnly -- -l actions-ignore" 2>&1 | tee -a ${{ inputs.output-file }}
continue-on-error: true

- name: Check Test Results
run: |
if [ $(grep 'All tests passed.' ${{ inputs.output-file }} | wc -l) -ne 1 ]; then
echo "One or more projects had failures. Please review the logs."
cat ${{ inputs.output-file }}
exit 1
else
echo "All tests passed."
exit 0
fi
86 changes: 86 additions & 0 deletions .github/workflows/unit-test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
name: HMDA Unit Tests

on:
pull_request:
branches:
- master

jobs:
hmda_platform_tests:
uses: ./.github/workflows/helper-functions.yaml
with:
service: "hmda-platform"
output-file: "output-log"

check_digit_tests:
uses: ./.github/workflows/helper-functions.yaml
with:
service: "check-digit"
output-file: "output-log"

common_tests:
uses: ./.github/workflows/helper-functions.yaml
with:
service: "common"
output-file: "output-log"

data_browser_tests:
uses: ./.github/workflows/helper-functions.yaml
with:
service: "data-browser"
output-file: "output-log"

hmda_analytics_tests:
uses: ./.github/workflows/helper-functions.yaml
with:
service: "hmda-analytics"
output-file: "output-log"


hmda_data_publisher_tests:
uses: ./.github/workflows/helper-functions.yaml
with:
service: "hmda-data-publisher"
output-file: "output-log"

hmda_quarterly_data_service_tests:
uses: ./.github/workflows/helper-functions.yaml
with:
service: "hmda-quarterly-data-service"
output-file: "output-log"

hmda_reporting_tests:
uses: ./.github/workflows/helper-functions.yaml
with:
service: "hmda-reporting"
output-file: "output-log"

institutions_api_tests:
uses: ./.github/workflows/helper-functions.yaml
with:
service: "institutions-api"
output-file: "output-log"

modified_lar_tests:
uses: ./.github/workflows/helper-functions.yaml
with:
service: "modified-lar"
output-file: "output-log"

rate_limit_tests:
uses: ./.github/workflows/helper-functions.yaml
with:
service: "rate-limit"
output-file: "output-log"

ratespread_calculator_tests:
uses: ./.github/workflows/helper-functions.yaml
with:
service: "ratespread-calculator"
output-file: "output-log"

submission_errors_tests:
uses: ./.github/workflows/helper-functions.yaml
with:
service: "submission-errors"
output-file: "output-log"
12 changes: 12 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -85,6 +85,18 @@ src_managed/
project/boot/
project/plugins/project/

## sbt project specifics
check-digit/project/
data-browser/project/
email-service/project/
file-proxy/project/
hmda-reporting/project/
institutions-api/project/
irs-publisher/project/
modified-lar/project/
rate-limit/project/
ratespread-calculator/project/

## Lightbend
.lightbend/

Loading