mirror of
https://github.com/DOI-DO/j40-cejst-2.git
synced 2025-07-28 01:01:16 -07:00
Adds National Risk Index data to ETL pipeline (#549)
* Adds dev dependencies to requirements.txt and re-runs black on codebase * Adds test and code for national risk index etl, still in progress * Removes test_data from .gitignore * Adds test data to nation_risk_index tests * Creates tests and ETL class for NRI data * Adds tests for load() and transform() methods of NationalRiskIndexETL * Updates README.md with info about the NRI dataset * Adds to dos * Moves tests and test data into a tests/ dir in national_risk_index * Moves tmp_dir for tests into data/tmp/tests/ * Promotes fixtures to conftest and relocates national_risk_index tests: The relocation of national_risk_index tests is necessary because tests can only use fixtures specified in conftests within the same package * Fixes issue with df.equals() in test_transform() * Files reformatted by black * Commit changes to other files after re-running black * Fixes unused import that caused lint checks to fail * Moves tests/ directory to app root for data_pipeline
This commit is contained in:
parent
94298635c2
commit
f0900f7b69
14 changed files with 307 additions and 7 deletions
|
@ -39,6 +39,11 @@ DATASET_LIST = [
|
|||
"module_dir": "cdc_places",
|
||||
"class_name": "CDCPlacesETL",
|
||||
},
|
||||
{
|
||||
"name": "national_risk_index",
|
||||
"module_dir": "national_risk_index",
|
||||
"class_name": "NationalRiskIndexETL",
|
||||
},
|
||||
]
|
||||
CENSUS_INFO = {
|
||||
"name": "census",
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue