publicdata/j40-cejst-2
1
0
Fork 0
mirror of https://github.com/DOI-DO/j40-cejst-2.git synced 2025-02-23 10:04:18 -08:00
Code Issues Projects Releases Packages Wiki Activity Actions
j40-cejst-2/data/data-pipeline/data_pipeline/etl/sources/historic_redlining
History
Travis Newby 6f39033dde
Add ability to cache ETL data sources (#2169) 
* Add a rough prototype allowing a developer to pre-download data sources for all ETLs

* Update code to be more production-ish

* Move fetch to Extract part of ETL
* Create a downloader to house all downloading operations
* Remove unnecessary "name" in data source

* Format source files with black

* Fix issues from pylint and get the tests working with the new folder structure

* Clean up files with black

* Fix unzip test

* Add caching notes to README

* Fix tests (linting and case sensitivity bug)

* Address PR comments and add API keys for census where missing

* Merging comparator changes from main into this branch for the sake of the PR

* Add note on using cache (-u) during pipeline
2023-03-03 12:26:24 -06:00
..
__init__.py Backend release branch to main (#1822) 2022-12-01 21:50:54 -05:00
etl.py Add ability to cache ETL data sources (#2169) 2023-03-03 12:26:24 -06:00
README.md Backend release branch to main (#1822) 2022-12-01 21:50:54 -05:00

README.md