j40-cejst-2/data/data-pipeline/data_pipeline
Billy Daly f0900f7b69
Adds National Risk Index data to ETL pipeline (#549)
* Adds dev dependencies to requirements.txt and re-runs black on codebase

* Adds test and code for national risk index etl, still in progress

* Removes test_data from .gitignore

* Adds test data to nation_risk_index tests

* Creates tests and ETL class for NRI data

* Adds tests for load() and transform() methods of NationalRiskIndexETL

* Updates README.md with info about the NRI dataset

* Adds to dos

* Moves tests and test data into a tests/ dir in national_risk_index

* Moves tmp_dir for tests into data/tmp/tests/

* Promotes fixtures to conftest and relocates national_risk_index tests:
The relocation of national_risk_index tests is necessary because tests 
can only use fixtures specified in conftests within the same package

* Fixes issue with df.equals() in test_transform()

* Files reformatted by black

* Commit changes to other files after re-running black

* Fixes unused import that caused lint checks to fail

* Moves tests/ directory to app root for data_pipeline
2021-09-07 20:51:34 -04:00
..
data AWS Sync Public Read (#508) 2021-08-12 14:17:25 -04:00
etl Adds National Risk Index data to ETL pipeline (#549) 2021-09-07 20:51:34 -04:00
ipython Score F, testing methodology (#510) 2021-08-24 16:40:54 -04:00
tests Adds National Risk Index data to ETL pipeline (#549) 2021-09-07 20:51:34 -04:00
tile adding layer to mvts (#503) 2021-08-12 10:56:54 -04:00
__init__.py Data directory should adopt standard Poetry-suggested python package structure (#457) 2021-08-05 15:35:54 -04:00
application.py adding additional missed import (#477) 2021-08-06 11:48:11 -04:00
config.py Data directory should adopt standard Poetry-suggested python package structure (#457) 2021-08-05 15:35:54 -04:00
utils.py Score F, testing methodology (#510) 2021-08-24 16:40:54 -04:00