Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
0d5d682
Fix column naming inconsistency in traffic safety module (GEOID vs fips)
Apr 19, 2025
5ac79e7
Add comprehensive traffic safety module documentation
Apr 19, 2025
42ff710
Fix inconsistent column naming in fetch_digital_access_data.r
Apr 19, 2025
9ce5e10
\Add debug logging to identify column naming issues\n\nEnhanced the d…
Apr 20, 2025
7617bd6
\Fix parameter mismatch in traffic safety integration module\n\nFixed…
Apr 20, 2025
c3030ea
Fix parameter passing in safe_fetch_extended function
Apr 20, 2025
709d603
Add fallback dataset for traffic safety when API fails
Apr 20, 2025
afd5038
Remove simulated data fallback for traffic safety, per requirements
Apr 20, 2025
15ffda9
Add robust fallback methods for FARS data retrieval
Apr 20, 2025
3773c4e
Add traffic safety integration and comprehensive data caching
Apr 20, 2025
3b7a3b1
Fix pipeline issues: Update variable count to 178, rewrite fetch_hist…
Apr 20, 2025
8b13e56
Update data dictionaries to enumerate all 178 variables by domain
Apr 20, 2025
02c61c0
Re-add IHME life expectancy variables to cross-walk files and documen…
Apr 20, 2025
f13a2e8
Add traffic safety variables to crosswalks and update DATA_DICTIONARY.md
Apr 20, 2025
7774318
Fix map generation: Handle database schema differences and column cas…
Apr 20, 2025
7a7d021
Consolidate all crosswalk functions into a single, definitive impleme…
Apr 20, 2025
6eeda20
Create modular pipeline architecture for better organization and main…
Apr 20, 2025
9264205
Fix pipeline module initialization errors
Apr 20, 2025
98f88a4
Add data fetching module and fix pipeline structure
Apr 20, 2025
f40db31
Update README and variable crosswalk with 255 variables total
Apr 20, 2025
e3d9662
Update documentation to reflect modular pipeline and 255 variables
Apr 20, 2025
a6cd9d6
Add modular pipeline architecture and restore missing variables
Apr 21, 2025
02526df
Fix modular pipeline for traffic safety and IHME integration
Apr 21, 2025
a174f13
Add YAML configuration and fix IHME life expectancy integration
Apr 21, 2025
100b0ae
Fix IHME life expectancy data processing to handle all 29 variables a…
Apr 21, 2025
8a7e8b4
Implement strict no-simulated-data policy across all data sources
Apr 21, 2025
af42823
Fix IHME life expectancy data processing by correcting column name re…
Apr 21, 2025
61c5432
Optimize IHME data processing and add automatic package installation
Apr 24, 2025
44d3f2c
Add support for new users with sample data generation
davidlary Apr 24, 2025
8b5af7f
Comprehensive documentation and repository updates
davidlary Apr 24, 2025
4b9196e
Fix database duplicate key constraint violation
davidlary Apr 24, 2025
2e0bbd1
Implement incremental processing to prevent redundant data reprocessing
davidlary Apr 24, 2025
8ae50ba
Implement advanced database optimizations for improved performance
davidlary Apr 24, 2025
a000ead
Fix unexpected end of input in database module
davidlary Apr 24, 2025
832d8bf
Fix module sourcing issue in database module
davidlary Apr 24, 2025
0291396
Simplify module sourcing in database module
davidlary Apr 24, 2025
b84239c
Add explicit EOF marker to database module
davidlary Apr 25, 2025
6a54cdf
Completely rewrite database module ending
davidlary Apr 25, 2025
572fd75
Implement robust database module loading with fallback mechanism
davidlary Apr 25, 2025
98ecc30
Add restart capability to unified pipeline
davidlary Apr 25, 2025
bc525fc
Fix skip_steps variable initialization in pipeline
davidlary Apr 25, 2025
60e6fb1
Implement more robust database function loading
davidlary Apr 25, 2025
72cc132
Implement atomic upsert operations with INSERT OR REPLACE for robust …
davidlary May 2, 2025
b273a7d
Complete database module implementation and fix map generation
davidlary May 2, 2025
61fc2bf
Add database rebuild script
davidlary May 2, 2025
f5caa9c
Fix unified pipeline to ensure database population
davidlary May 2, 2025
0ef391d
Add comprehensive summary of database and pipeline fixes
davidlary May 2, 2025
e806fac
Complete traffic safety implementation and documentation
davidlary May 5, 2025
0837f6e
Complete comprehensive implementation with all 255+ variables includi…
davidlary May 7, 2025
b300478
Extend database year range to 1970-2022 with full county coverage
davidlary May 7, 2025
53a98e3
Add traffic safety implementation summary document
davidlary May 7, 2025
c0288ba
Update map generation to include all US states and territories
davidlary May 7, 2025
7b73aaf
Implement advanced temporal interpolation for all variables
davidlary May 7, 2025
29153e0
Complete traffic safety implementation with comprehensive testing
davidlary May 7, 2025
109d32d
Fix traffic safety implementation issues
davidlary May 8, 2025
86c8918
Add summary of traffic safety implementation fixes
davidlary May 8, 2025
fe1d23a
Update rebuild script to use enhanced traffic safety verification
davidlary May 8, 2025
a60f80b
Enhance map generation and add comprehensive logging
davidlary May 9, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Pull Request Template

## Description
Describe the purpose of this pull request, the changes made, and any issues it addresses.

## Type of change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Checklist
- [ ] I have tested these changes locally
- [ ] I have updated the documentation accordingly
- [ ] My code follows the style guidelines of this project
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing tests pass with my changes
- [ ] Any dependent changes have been merged and published

## Performance Impact
- [ ] No performance impact
- [ ] Minor performance improvement
- [ ] Significant performance improvement
- [ ] Potential performance regression (explain below)

## Memory Usage Impact
- [ ] No memory usage impact
- [ ] Reduced memory requirements
- [ ] Increased memory requirements (explain below)

## Additional Information
Add any other relevant information about the changes, including screenshots if appropriate.
63 changes: 21 additions & 42 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,48 +1,27 @@
# R specific ignores
.Rproj.user
.Rhistory
.RData
.Ruserdata

# Large data files
data/cdc_places/*.csv
data/census_historical/*.txt
data/crime/*.csv
data/economic/*.csv
data/education/*.csv
data/epa/tri/*.csv
data/epa/*.csv
data/epa/*.zip
data/healthcare/*.csv
data/housing/*.csv
data/ihme/CSV/*.CSV
data/ihme/Docs/*.PDF
data/ihme/Docs/*.XLSX
data/nhgis/*.CSV
data/seer/*.gz
data/shapefiles/*.rds
data/shapefiles/*.txt
data/social_cohesion/*.csv
data/transportation/*.csv
data/usda_food_atlas/*.csv
data/usda_food_atlas/*.xls
data/built_environment/*.csv

# Preserve smaller index and metadata files
!data/shapefiles/shapefile_index.csv

# Database files
*.duckdb
*.duckdb.wal
*.csv.gz
*.csv.zip
*.rds
*.parquet
*.feather

# Cache and generated files
data/cache/*
output/maps/*
logs/*.log

# OS specific
# Temporary files
*.log
*.png
Rplots.pdf
.RData
.Rhistory
__pycache__/
*.py[cod]
*$py.class
.DS_Store
Thumbs.db
backup_*/*

# Keep empty directories
\!**/.gitkeep

# Allow .gitkeep files in otherwise empty directories
!**/.gitkeep
# Don't ignore specific README files
\!output/maps/README.md
\!*/README.md
349 changes: 349 additions & 0 deletions BackupFiles/DATA_DICTIONARY.md.bak

Large diffs are not rendered by default.

Loading