j40-cejst-2/data/data-pipeline/data_pipeline/etl/sources/national_risk_index
Jorge Escobar 1c448a77f9
NRI dataset and initial score YAML configuration (#1534)
* update be staging gha

* NRI dataset and initial score YAML configuration

* checkpoint

* adding data checks for release branch

* passing tests

* adding INPUT_EXTRACTED_FILE_NAME to base class

* lint

* columns to keep and tests

* update be staging gha

* checkpoint

* update be staging gha

* NRI dataset and initial score YAML configuration

* checkpoint

* adding data checks for release branch

* passing tests

* adding INPUT_EXTRACTED_FILE_NAME to base class

* lint

* columns to keep and tests

* checkpoint

* PR Review

* renoving source url

* tests

* stop execution of ETL if there's a YAML schema issue

* update be staging gha

* adding source url as class var again

* clean up

* force cache bust

* gha cache bust

* dynamically set score vars from YAML

* docsctrings

* removing last updated year - optional reverse percentile

* passing tests

* sort order

* column ordening

* PR review

* class level vars

* Updating DatasetsConfig

* fix pylint errors

* moving metadata hint back to code

Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov>
2022-08-09 16:37:10 -04:00
..
__init__.py Adds National Risk Index data to ETL pipeline (#549) 2021-09-07 20:51:34 -04:00
etl.py NRI dataset and initial score YAML configuration (#1534) 2022-08-09 16:37:10 -04:00
README.md Adds National Risk Index data to ETL pipeline (#549) 2021-09-07 20:51:34 -04:00

FEMA National Risk Index

Description

The National Risk Index is a new, online mapping application from FEMA that identifies communities most at risk to 18 natural hazards. This application visualizes natural hazard risk metrics and includes data about expected annual losses from natural hazards, social vulnerability and community resilience.

The National Risk Index's interactive web maps are at the county and Census tract level and made available via geographic information system (GIS) services for custom analyses. For this project, we've utilized the NRI data collected at the Census tract level

Data Transformation Summary

The following transformations were applied to the NRI data during the ETL process:

  • The TRACTFIPS column was renamed to GEOID10_TRACT to match the name of columns that hold the Census Tract FIPS code in other data sets
  • The NRI score values for each Census tract were applied to each of the Census block groups inside of that Census tract so that the unit of analysis would match that of other datasets like the American Communities Survey