Adds National Risk Index data to ETL pipeline (#549)

* Adds dev dependencies to requirements.txt and re-runs black on codebase

* Adds test and code for national risk index etl, still in progress

* Removes test_data from .gitignore

* Adds test data to nation_risk_index tests

* Creates tests and ETL class for NRI data

* Adds tests for load() and transform() methods of NationalRiskIndexETL

* Updates README.md with info about the NRI dataset

* Adds to dos

* Moves tests and test data into a tests/ dir in national_risk_index

* Moves tmp_dir for tests into data/tmp/tests/

* Promotes fixtures to conftest and relocates national_risk_index tests:
The relocation of national_risk_index tests is necessary because tests 
can only use fixtures specified in conftests within the same package

* Fixes issue with df.equals() in test_transform()

* Files reformatted by black

* Commit changes to other files after re-running black

* Fixes unused import that caused lint checks to fail

* Moves tests/ directory to app root for data_pipeline
This commit is contained in:
Billy Daly 2021-09-07 20:51:34 -04:00 committed by GitHub
commit f0900f7b69
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
14 changed files with 307 additions and 7 deletions

View file

@ -39,6 +39,11 @@ DATASET_LIST = [
"module_dir": "cdc_places",
"class_name": "CDCPlacesETL",
},
{
"name": "national_risk_index",
"module_dir": "national_risk_index",
"class_name": "NationalRiskIndexETL",
},
]
CENSUS_INFO = {
"name": "census",