Traffic Safety Integration and Comprehensive Caching System - #2

Open

davidlary wants to merge 58 commits into

mainfrom

complete-traffic-safety-implementation

Commits on Apr 19, 2025

Fix column naming inconsistency in traffic safety module (GEOID vs fips)

authored and
David Lary
committed
0d5d682
Add comprehensive traffic safety module documentation

authored and
David Lary
committed
5ac79e7
Fix inconsistent column naming in fetch_digital_access_data.r

authored and
David Lary
committed
42ff710

Commits on Apr 20, 2025

\Add debug logging to identify column naming issues\n\nEnhanced the data processing modules with detailed error logging to identify\nthe source of GEOID vs geoid/fips column naming inconsistencies.…

David Lary
authored and
David Lary
committed
9ce5e10
\Fix parameter mismatch in traffic safety integration module\n\nFixed an error when calling fetch_traffic_safety_data that was causing\nthe pipeline to fail with the error:\n\nError during executio…

David Lary
authored and
David Lary
committed
7617bd6
Fix parameter passing in safe_fetch_extended function

David Lary
authored and
David Lary
committed
c3030ea
Add fallback dataset for traffic safety when API fails

David Lary
authored and
David Lary
committed
709d603
Remove simulated data fallback for traffic safety, per requirements

David Lary
authored and
David Lary
committed
afd5038
Add robust fallback methods for FARS data retrieval

David Lary
authored and
David Lary
committed
15ffda9
Add traffic safety integration and comprehensive data caching

authored and
David Lary
committed
3773c4e
Fix pipeline issues: Update variable count to 178, rewrite fetch_historical_data.r, add CONUS maps

David Lary
authored and
David Lary
committed
3b7a3b1
Update data dictionaries to enumerate all 178 variables by domain

David Lary
authored and
David Lary
committed
8b13e56
Re-add IHME life expectancy variables to cross-walk files and documentation

David Lary
authored and
David Lary
committed
02c61c0
Add traffic safety variables to crosswalks and update DATA_DICTIONARY.md

David Lary
authored and
David Lary
committed
f13a2e8
Fix map generation: Handle database schema differences and column case sensitivity

authored and
David Lary
committed
7774318
Consolidate all crosswalk functions into a single, definitive implementation

authored and
David Lary
committed
7a7d021
Create modular pipeline architecture for better organization and maintenance

authored and
David Lary
committed
6eeda20
Fix pipeline module initialization errors

David Lary
authored and
David Lary
committed
9264205
Add data fetching module and fix pipeline structure

David Lary
authored and
David Lary
committed
98f88a4
Update README and variable crosswalk with 255 variables total

David Lary
authored and
David Lary
committed
f40db31
Update documentation to reflect modular pipeline and 255 variables

David Lary
authored and
David Lary
committed
e3d9662

Commits on Apr 21, 2025

Add modular pipeline architecture and restore missing variables

authored and
David Lary
committed
a6cd9d6
Fix modular pipeline for traffic safety and IHME integration

authored and
David Lary
committed
02526df
Add YAML configuration and fix IHME life expectancy integration

David Lary
authored and
David Lary
committed
a174f13
Fix IHME life expectancy data processing to handle all 29 variables and multiple file formats

David Lary
authored and
David Lary
committed
100b0ae
Implement strict no-simulated-data policy across all data sources

David Lary
authored and
David Lary
committed
8a7e8b4
Fix IHME life expectancy data processing by correcting column name references from 'lower'/'upper' to 'le_lower_ci'/'le_upper_ci' in process_race_data function

authored and
David Lary
committed
af42823

Commits on Apr 24, 2025

Optimize IHME data processing and add automatic package installation

authored and
David Lary
committed
61c5432
Add support for new users with sample data generation

davidlary
and
claude
committed
44d3f2c
Comprehensive documentation and repository updates

davidlary
and
claude
committed
8b5af7f
Fix database duplicate key constraint violation

davidlary
and
claude
committed
4b9196e
Implement incremental processing to prevent redundant data reprocessing

davidlary
and
claude
committed
2e0bbd1
Implement advanced database optimizations for improved performance

davidlary
and
claude
committed
8ae50ba
Fix unexpected end of input in database module

davidlary
and
claude
committed
a000ead
Fix module sourcing issue in database module

davidlary
and
claude
committed
832d8bf
Simplify module sourcing in database module

davidlary
and
claude
committed
0291396

Commits on Apr 25, 2025

Add explicit EOF marker to database module

davidlary
and
claude
committed
b84239c
Completely rewrite database module ending

davidlary
and
claude
committed
6a54cdf
Implement robust database module loading with fallback mechanism

davidlary
and
claude
committed
572fd75
Add restart capability to unified pipeline

davidlary
and
claude
committed
98ecc30
Fix skip_steps variable initialization in pipeline

davidlary
and
claude
committed
bc525fc
Implement more robust database function loading

davidlary
and
claude
committed
60e6fb1

Commits on May 2, 2025

Implement atomic upsert operations with INSERT OR REPLACE for robust database handling
davidlary
committed
72cc132
Complete database module implementation and fix map generation
davidlary
committed
b273a7d
Add database rebuild script
davidlary
committed
61fc2bf
Fix unified pipeline to ensure database population
davidlary
committed
f5caa9c
Add comprehensive summary of database and pipeline fixes
davidlary
committed
0ef391d

Commits on May 5, 2025

Complete traffic safety implementation and documentation

davidlary
and
claude
committed
e806fac

Commits on May 7, 2025

Complete comprehensive implementation with all 255+ variables including traffic safety data

davidlary
and
claude
committed
0837f6e
Extend database year range to 1970-2022 with full county coverage

davidlary
and
claude
committed
b300478
Add traffic safety implementation summary document

davidlary
and
claude
committed
53a98e3
Update map generation to include all US states and territories

davidlary
and
claude
committed
c0288ba
Implement advanced temporal interpolation for all variables

davidlary
and
claude
committed
7b73aaf
Complete traffic safety implementation with comprehensive testing

davidlary
and
claude
committed
29153e0

Commits on May 8, 2025

Fix traffic safety implementation issues
davidlary
committed
109d32d
Add summary of traffic safety implementation fixes
davidlary
committed
86c8918
Update rebuild script to use enhanced traffic safety verification
davidlary
committed
fe1d23a

Commits on May 9, 2025

Enhance map generation and add comprehensive logging

davidlary
and
claude
committed
a60f80b