Skip to content

Command line tool (could later be used as lambda function) to download CSV files from SBA and generate JSON

Notifications You must be signed in to change notification settings

NguyenDa18/PPP-Data-Tool

Repository files navigation

PPP Data Tool

  • Tool to read PPP CSVs and generate JSON

Table of PPP loans exceeding 150k - big datasets for a state had to be split to multiple views

  • Table of PPP loans < 150k : coming soon

SBA Download Issue

  • The downloadUrl in constants.js might not work because SBA refreshes download session or something
  • To fix this issue and get the right download link, go to the SBA Download Link, open a Developer Console and inspect the network when you click "Download" -that URL fetching the download is the one to replace for downloadUrl

Image of Simple CLI Tool

Image of Simple CLI generating data for US States

Using Tool

  • npm i : Install dependencies first
  • npm start : Start CLI tool
  • Make sure following dirs are in the data dir:
    • csv
    • json
    • raw
  • Pick option for operation to do

Loading to Airtable

  • Following CSV files exceed 5MB : CA, FL, IL, NY, TX
  • Split the files using split-data/split.py and use those to upload

DONE

  • CLI tool basics
  • download zip by url
  • extract zip and save to data repo
  • parse CSV and generate JSON for PPP files of bailous > 150k
  • parse CSVs and generate JSON files for PPP files of bailouts > 150k

TODO

  • parse CSVs and generate JSON files for PPP files of bailouts < 150k

About

Command line tool (could later be used as lambda function) to download CSV files from SBA and generate JSON

Topics

Resources

Stars

Watchers

Forks