publicdata/j40-cejst-2
1
0
Fork 0
mirror of https://github.com/DOI-DO/j40-cejst-2.git synced 2025-02-23 10:04:18 -08:00
Code Issues Projects Releases Packages Wiki Activity Actions
j40-cejst-2/data/data-pipeline/data_pipeline/etl/sources
History
Travis Newby 6f39033dde
Add ability to cache ETL data sources (#2169) 
* Add a rough prototype allowing a developer to pre-download data sources for all ETLs

* Update code to be more production-ish

* Move fetch to Extract part of ETL
* Create a downloader to house all downloading operations
* Remove unnecessary "name" in data source

* Format source files with black

* Fix issues from pylint and get the tests working with the new folder structure

* Clean up files with black

* Fix unzip test

* Add caching notes to README

* Fix tests (linting and case sensitivity bug)

* Address PR comments and add API keys for census where missing

* Merging comparator changes from main into this branch for the sake of the PR

* Add note on using cache (-u) during pipeline
2023-03-03 12:26:24 -06:00
..
calenviroscreen Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
cdc_life_expectancy Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
cdc_places Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
cdc_svi_index Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
census Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
census_acs Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
census_acs_2010 Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
census_acs_median_income Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
census_decennial Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
child_opportunity_index Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
doe_energy_burden Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
dot_travel_composite Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
eamlis Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
ejscreen Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
ejscreen_areas_of_concern Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
energy_definition_alternative_draft Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
epa_rsei Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
fsf_flood_risk Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
fsf_wildfire_risk Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
geocorr Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
historic_redlining Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
housing_and_transportation Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
hud_housing Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
hud_recap Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
mapping_for_ej Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
mapping_inequality Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
maryland_ejscreen Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
michigan_ejscreen Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
national_risk_index Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
nlcd_nature_deprived Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
persistent_poverty Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
tree_equity_score Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
tribal Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
tribal_overlap Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
us_army_fuds Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
__init__.py Data directory should adopt standard Poetry-suggested python package structure (#457) 2021-08-05 15:35:54 -04:00
geo_utils.py User Story 2152 – Clean up logging (#2155) 2023-02-08 13:08:55 -06:00