mirror of
https://github.com/DOI-DO/j40-cejst-2.git
synced 2025-02-23 01:54:18 -08:00
CDC SVI Index: Additions to data-pipeline and comparison tool (#1096)
* wip * working * working * rename * documentation * add link * add readme * update fieldnames * typo * add comparison tool * revise wording * variable change for FIPS * workding * wording in readme * cleanup wording * update comparison tool * final tune up * grammar and punctuation in the documentation * period * cleanup comments * added revisions * parallelism * PR feedback from Lucas * remove extraneous fields from comparison tool * style * updates * remove themes * formatting * remove referenes to percentile rank * remove referenes to percentile rank * typo in fieldnames * updates based on feedback from Lucas * fieldnames formatting * fix broken markdown link Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov>
This commit is contained in:
parent
95a14adb35
commit
87e08f5fe1
7 changed files with 337 additions and 36 deletions
|
@ -109,6 +109,11 @@ DATASET_LIST = [
|
|||
"module_dir": "michigan_ejscreen",
|
||||
"class_name": "MichiganEnviroScreenETL",
|
||||
},
|
||||
{
|
||||
"name": "cdc_svi_index",
|
||||
"module_dir": "cdc_svi_index",
|
||||
"class_name": "CDCSVIIndex",
|
||||
},
|
||||
{
|
||||
"name": "maryland_ejscreen",
|
||||
"module_dir": "maryland_ejscreen",
|
||||
|
|
|
@ -0,0 +1,69 @@
|
|||
# CDC SVI Index
|
||||
|
||||
For additional information on the methodology, please refer to the following paper for the [SVI Index](https://www.atsdr.cdc.gov/placeandhealth/svi/documentation/pdf/SVI2018Documentation-H.pdf)
|
||||
|
||||
CDC SVI Index utilizes American Community Survey (ACS), 2014-2018 (5-year) data for the following estimates (also known as themes in the [documentation](https://www.atsdr.cdc.gov/placeandhealth/svi/documentation/pdf/SVI2018Documentation-H.pdf)):
|
||||
|
||||
**Socioeconomic Status**
|
||||
1. Below Poverty
|
||||
2. Unemployed
|
||||
3. Income
|
||||
4. No High School Diploma
|
||||
|
||||
**Household Composition & Disability**
|
||||
1. Aged 65 or Older
|
||||
2. Aged 17 or Younger
|
||||
3. Civilian with a Disability
|
||||
4. Single-Parent Households
|
||||
|
||||
**Minority Status & Language**
|
||||
1. Minority
|
||||
2. Speaks English “Less than Well”
|
||||
|
||||
**Housing Type & Transportation**
|
||||
1. Multi-Unit Structures
|
||||
2. Mobile Homes
|
||||
3. Crowding
|
||||
4. No Vehicle
|
||||
5. Group Quarters
|
||||
|
||||
|
||||
### Rankings
|
||||
Census tracts were ranked within each state and the District of Columbia. In addition, a ranking of all tracts for the entire United States was used relative to one another. Tract rankings are based on percentiles. _Percentile ranking_ values range from 0 to 1, with higher values indicating greater vulnerability.
|
||||
|
||||
For each tract, a percentile rank is generated among all tracts for 1) the fifteen individual variables, 2) the four themes, and 3) its overall position.
|
||||
|
||||
**Theme rankings**
|
||||
|
||||
For each of the four themes, the percentiles for the variables comprising each theme were computed. The authors ordered the summed percentiles for each theme to determine theme-specific percentile rankings.
|
||||
|
||||
The four summary theme ranking variables, detailed in the Data Dictionary below, are:
|
||||
|
||||
* Socioeconomic - RPL_THEME1
|
||||
* Household Composition & Disability - RPL_THEME2
|
||||
* Minority Status & Language - RPL_THEME3
|
||||
* Housing Type & Transportation - RPL_THEME4
|
||||
|
||||
**Overall tract rankings:**
|
||||
|
||||
The sums of each theme were summed, ordered the tracts, and then calculated overall percentile rankings. Please note; taking the sum of the sums for each theme is the same as summing individual variable rankings. The overall tract summary ranking variable is RPL_THEMES.
|
||||
|
||||
For SVI 2018, the authors also included two adjunct variables, 1) 2014-2018 ACS estimates for persons without health insurance, and 2) an estimate of the daytime population derived from LandScan 2018 estimates. These adjunct variables are excluded from SVI rankings. At this time, these are not included in the output in the current methodology.
|
||||
|
||||
|
||||
**Important Notes**
|
||||
|
||||
1. Tracts with zero estimates for the total population (N = 645 for the U.S.) were removed during the ranking process. These tracts were added back to the SVI databases after ranking.
|
||||
|
||||
2. The TOTPOP field value is 0, but the percentile ranking fields (RPL_THEME1, RPL_THEME2, RPL_THEME3, RPL_THEME4, and RPL_THEMES) were set to -999.
|
||||
|
||||
Whenever available, they use Census-calculated MOEs. If Census MOEs are unavailable, for instance when aggregating variables within a table, the authors use approximation formulas provided by the Census in Appendix A (pages A-14 through A-17) of _A Compass for Understanding and Using American Community Survey Data here_:
|
||||
|
||||
https://www.census.gov/content/dam/Census/library/publications/2008/acs/ACSGeneralHandbook.pdf
|
||||
|
||||
If more precise MOEs are required, see Census methods and data regarding Variance Replicate Tables
|
||||
here: https://www.census.gov/programs-surveys/acs/data/variance-tables.html.
|
||||
|
||||
For selected ACS 5-year Detailed Tables, “Users can calculate margins of error for aggregated data by using the variance replicates. Unlike available approximation formulas, this method results in an exact margin of error by using the covariance term.”
|
||||
|
||||
MOEs are _not_ included nor considered during this data processing nor for the scoring comparison tool.
|
|
@ -0,0 +1,112 @@
|
|||
import pandas as pd
|
||||
import numpy as np
|
||||
|
||||
from data_pipeline.etl.base import ExtractTransformLoad
|
||||
from data_pipeline.utils import get_module_logger
|
||||
from data_pipeline.score import field_names
|
||||
|
||||
logger = get_module_logger(__name__)
|
||||
|
||||
|
||||
class CDCSVIIndex(ExtractTransformLoad):
|
||||
"""CDC SVI Index class ingests 2018 dataset located
|
||||
here: https://www.atsdr.cdc.gov/placeandhealth/svi/index.html
|
||||
Please see the README in this module for further details.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self.OUTPUT_PATH = self.DATA_PATH / "dataset" / "cdc_svi_index"
|
||||
|
||||
self.CDC_SVI_INDEX_URL = "https://svi.cdc.gov/Documents/Data/2018_SVI_Data/CSV/SVI2018_US.csv"
|
||||
|
||||
self.CDC_RPL_THEMES_THRESHOLD = 0.90
|
||||
|
||||
self.CDC_SVI_INDEX_TRACTS_FIPS_CODE = "FIPS"
|
||||
|
||||
self.COLUMNS_TO_KEEP = [
|
||||
self.GEOID_TRACT_FIELD_NAME,
|
||||
field_names.CDC_SVI_INDEX_SE_THEME_FIELD
|
||||
+ field_names.PERCENTILE_FIELD_SUFFIX, # SE prefix references "socioeconomic"
|
||||
field_names.CDC_SVI_INDEX_HOUSEHOLD_THEME_COMPOSITION_FIELD
|
||||
+ field_names.PERCENTILE_FIELD_SUFFIX,
|
||||
field_names.CDC_SVI_INDEX_LANGUAGE_THEME_FIELD
|
||||
+ field_names.PERCENTILE_FIELD_SUFFIX,
|
||||
field_names.CDC_SVI_INDEX_HOUSING_TRANSPORTATION_FIELD
|
||||
+ field_names.PERCENTILE_FIELD_SUFFIX,
|
||||
field_names.CDC_SVI_INDEX_RPL_THEMES_OVERALL_FIELD
|
||||
+ field_names.PERCENTILE_FIELD_SUFFIX,
|
||||
# Derived columns
|
||||
field_names.CDC_SVI_INDEX_THEMES_PRIORITY_COMMUNITY,
|
||||
]
|
||||
|
||||
self.df: pd.DataFrame
|
||||
|
||||
def extract(self) -> None:
|
||||
logger.info("Extracting 43 MB CDC SVI INDEX")
|
||||
self.df = pd.read_csv(
|
||||
filepath_or_buffer=self.CDC_SVI_INDEX_URL,
|
||||
dtype={self.CDC_SVI_INDEX_TRACTS_FIPS_CODE: "string"},
|
||||
low_memory=False,
|
||||
)
|
||||
|
||||
def transform(self) -> None:
|
||||
logger.info("Starting CDC SVI INDEX transform")
|
||||
# Note: In this dataset all US census tracts are ranked against one another.
|
||||
# Puerto Rico is not included in this dataset
|
||||
self.df.rename(
|
||||
columns={
|
||||
self.CDC_SVI_INDEX_TRACTS_FIPS_CODE: self.GEOID_TRACT_FIELD_NAME,
|
||||
"RPL_THEME1": field_names.CDC_SVI_INDEX_SE_THEME_FIELD
|
||||
+ field_names.PERCENTILE_FIELD_SUFFIX,
|
||||
"RPL_THEME2": field_names.CDC_SVI_INDEX_HOUSEHOLD_THEME_COMPOSITION_FIELD
|
||||
+ field_names.PERCENTILE_FIELD_SUFFIX,
|
||||
"RPL_THEME3": field_names.CDC_SVI_INDEX_LANGUAGE_THEME_FIELD
|
||||
+ field_names.PERCENTILE_FIELD_SUFFIX,
|
||||
"RPL_THEME4": field_names.CDC_SVI_INDEX_HOUSING_TRANSPORTATION_FIELD
|
||||
+ field_names.PERCENTILE_FIELD_SUFFIX,
|
||||
"RPL_THEMES": field_names.CDC_SVI_INDEX_RPL_THEMES_OVERALL_FIELD
|
||||
+ field_names.PERCENTILE_FIELD_SUFFIX,
|
||||
},
|
||||
inplace=True,
|
||||
errors="raise",
|
||||
)
|
||||
|
||||
# From page 2 in the documentation:
|
||||
# Tracts with zero estimates for total population (N = 645 for the U.S.) were removed during the ranking
|
||||
# process. These tracts were added back to the SVI databases after ranking. The TOTPOP field value is 0,
|
||||
# but the percentile ranking fields (RPL_THEME1, RPL_THEME2, RPL_THEME3, RPL_THEME4, and
|
||||
# RPL_THEMES) were set to -999.
|
||||
# ▪ For tracts with > 0 TOTPOP, a value of -999 in any field either means the value was unavailable from the
|
||||
# original census data or we could not calculate a derived value because of unavailable census data.
|
||||
# ▪ Any cells with a -999 were not used for further calculations. For example, total flags do not include fields
|
||||
# with a -999 value.
|
||||
self.df = self.df.replace(-999, np.nan)
|
||||
|
||||
# At or above 90 for percentile rank
|
||||
self.df[field_names.CDC_SVI_INDEX_THEMES_PRIORITY_COMMUNITY] = (
|
||||
self.df[
|
||||
field_names.CDC_SVI_INDEX_RPL_THEMES_OVERALL_FIELD
|
||||
+ field_names.PERCENTILE_FIELD_SUFFIX
|
||||
]
|
||||
>= self.CDC_RPL_THEMES_THRESHOLD
|
||||
)
|
||||
expected_census_tract_field_length = 11
|
||||
self.df[self.GEOID_TRACT_FIELD_NAME] = (
|
||||
self.df[self.GEOID_TRACT_FIELD_NAME]
|
||||
.astype(str)
|
||||
.apply(lambda x: x.zfill(expected_census_tract_field_length))
|
||||
)
|
||||
|
||||
if len(self.df[self.GEOID_TRACT_FIELD_NAME].str.len().unique()) != 1:
|
||||
raise ValueError(
|
||||
f"GEOID Tract must be length of {expected_census_tract_field_length}"
|
||||
)
|
||||
|
||||
def load(self) -> None:
|
||||
logger.info("Saving CDC SVI Index Data")
|
||||
|
||||
self.OUTPUT_PATH.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
self.df[self.COLUMNS_TO_KEEP].to_csv(
|
||||
path_or_buf=self.OUTPUT_PATH / "usa.csv", index=False
|
||||
)
|
|
@ -50,7 +50,9 @@
|
|||
}
|
||||
],
|
||||
"source": [
|
||||
"df_raw = pd.read_csv('master_raw_data.csv') #file from the 'Data collection.ipynb'\n",
|
||||
"df_raw = pd.read_csv(\n",
|
||||
" \"master_raw_data.csv\"\n",
|
||||
") # file from the 'Data collection.ipynb'\n",
|
||||
"print(df_raw.shape)\n",
|
||||
"print(df_raw.columns)"
|
||||
]
|
||||
|
@ -296,8 +298,8 @@
|
|||
}
|
||||
],
|
||||
"source": [
|
||||
"#More advanced imputation method\n",
|
||||
"df_imputed_x = pd.read_excel('20200420_input_final.xlsx')\n",
|
||||
"# More advanced imputation method\n",
|
||||
"df_imputed_x = pd.read_excel(\"20200420_input_final.xlsx\")\n",
|
||||
"df_imputed_x.head()"
|
||||
]
|
||||
},
|
||||
|
@ -389,22 +391,48 @@
|
|||
}
|
||||
],
|
||||
"source": [
|
||||
"data = df_imputed_x.copy() #data\n",
|
||||
"model1 = Model(data) #Initializing class variable\n",
|
||||
"data = df_imputed_x.copy() # data\n",
|
||||
"model1 = Model(data) # Initializing class variable\n",
|
||||
"\n",
|
||||
"# variables list to include in our regression model.\n",
|
||||
"columns_regress = ['HE_FOOD', 'HE_WALK', 'HE_VACANCY', 'HE_SUPRFND','HE_HLTHINS','BINGE_CrudePrev',\n",
|
||||
" 'CHECKUP_CrudePrev','BPHIGH_CrudePrev','SLEEP_CrudePrev', 'STROKE_CrudePrev', 'Drug Test', 'Pedalcyclist']\n",
|
||||
"columns_regress = [\n",
|
||||
" \"HE_FOOD\",\n",
|
||||
" \"HE_WALK\",\n",
|
||||
" \"HE_VACANCY\",\n",
|
||||
" \"HE_SUPRFND\",\n",
|
||||
" \"HE_HLTHINS\",\n",
|
||||
" \"BINGE_CrudePrev\",\n",
|
||||
" \"CHECKUP_CrudePrev\",\n",
|
||||
" \"BPHIGH_CrudePrev\",\n",
|
||||
" \"SLEEP_CrudePrev\",\n",
|
||||
" \"STROKE_CrudePrev\",\n",
|
||||
" \"Drug Test\",\n",
|
||||
" \"Pedalcyclist\",\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# multipliers for each variable to rescale the variables as higher the value better for the health score.\n",
|
||||
"multiply_cols = {'HE_FOOD': -1,'HE_WALK': 1,'HE_VACANCY': -1,'HE_SUPRFND':-1 , 'HE_HLTHINS': 1 ,'BINGE_CrudePrev': -1 , \n",
|
||||
" 'CHECKUP_CrudePrev': 1,'BPHIGH_CrudePrev': -1,'SLEEP_CrudePrev': -1,'STROKE_CrudePrev' : -1, 'Drug Test' : -1, 'Pedalcyclist' : -1,}\n",
|
||||
"multiply_cols = {\n",
|
||||
" \"HE_FOOD\": -1,\n",
|
||||
" \"HE_WALK\": 1,\n",
|
||||
" \"HE_VACANCY\": -1,\n",
|
||||
" \"HE_SUPRFND\": -1,\n",
|
||||
" \"HE_HLTHINS\": 1,\n",
|
||||
" \"BINGE_CrudePrev\": -1,\n",
|
||||
" \"CHECKUP_CrudePrev\": 1,\n",
|
||||
" \"BPHIGH_CrudePrev\": -1,\n",
|
||||
" \"SLEEP_CrudePrev\": -1,\n",
|
||||
" \"STROKE_CrudePrev\": -1,\n",
|
||||
" \"Drug Test\": -1,\n",
|
||||
" \"Pedalcyclist\": -1,\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"#target variable\n",
|
||||
"target = 'life expectancy'\n",
|
||||
"# target variable\n",
|
||||
"target = \"life expectancy\"\n",
|
||||
"\n",
|
||||
"#storing the data and model weights to calculate health score.\n",
|
||||
"multiplied_zscore_data_le, params_le = model1.model_output(columns_regress,target,multiply_cols)"
|
||||
"# storing the data and model weights to calculate health score.\n",
|
||||
"multiplied_zscore_data_le, params_le = model1.model_output(\n",
|
||||
" columns_regress, target, multiply_cols\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
@ -456,16 +484,42 @@
|
|||
}
|
||||
],
|
||||
"source": [
|
||||
"columns_regress = ['HE_FOOD', 'HE_WALK', 'HE_VACANCY', 'HE_SUPRFND','HE_HLTHINS','BINGE_CrudePrev',\n",
|
||||
" 'CHECKUP_CrudePrev','BPHIGH_CrudePrev','SLEEP_CrudePrev', 'STROKE_CrudePrev', 'Drug Test', 'Pedalcyclist']\n",
|
||||
"columns_regress = [\n",
|
||||
" \"HE_FOOD\",\n",
|
||||
" \"HE_WALK\",\n",
|
||||
" \"HE_VACANCY\",\n",
|
||||
" \"HE_SUPRFND\",\n",
|
||||
" \"HE_HLTHINS\",\n",
|
||||
" \"BINGE_CrudePrev\",\n",
|
||||
" \"CHECKUP_CrudePrev\",\n",
|
||||
" \"BPHIGH_CrudePrev\",\n",
|
||||
" \"SLEEP_CrudePrev\",\n",
|
||||
" \"STROKE_CrudePrev\",\n",
|
||||
" \"Drug Test\",\n",
|
||||
" \"Pedalcyclist\",\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"multiply_cols = {'HE_FOOD': -1,'HE_WALK': 1,'HE_VACANCY': -1,'HE_SUPRFND':-1 , 'HE_HLTHINS': 1 ,'BINGE_CrudePrev': -1 , \n",
|
||||
" 'CHECKUP_CrudePrev': 1,'BPHIGH_CrudePrev': -1,'SLEEP_CrudePrev': -1,'STROKE_CrudePrev' : -1, 'Drug Test' : -1, 'Pedalcyclist' : -1,}\n",
|
||||
"multiply_cols = {\n",
|
||||
" \"HE_FOOD\": -1,\n",
|
||||
" \"HE_WALK\": 1,\n",
|
||||
" \"HE_VACANCY\": -1,\n",
|
||||
" \"HE_SUPRFND\": -1,\n",
|
||||
" \"HE_HLTHINS\": 1,\n",
|
||||
" \"BINGE_CrudePrev\": -1,\n",
|
||||
" \"CHECKUP_CrudePrev\": 1,\n",
|
||||
" \"BPHIGH_CrudePrev\": -1,\n",
|
||||
" \"SLEEP_CrudePrev\": -1,\n",
|
||||
" \"STROKE_CrudePrev\": -1,\n",
|
||||
" \"Drug Test\": -1,\n",
|
||||
" \"Pedalcyclist\": -1,\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"target = 'PHLTH_CrudePrev'\n",
|
||||
"target = \"PHLTH_CrudePrev\"\n",
|
||||
"target_multiplier = -1\n",
|
||||
"\n",
|
||||
"multiplied_zscore_data_1, params_1 = model1.model_output(columns_regress,target,multiply_cols,target_multiplier = target_multiplier)"
|
||||
"multiplied_zscore_data_1, params_1 = model1.model_output(\n",
|
||||
" columns_regress, target, multiply_cols, target_multiplier=target_multiplier\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
@ -517,20 +571,46 @@
|
|||
}
|
||||
],
|
||||
"source": [
|
||||
"#regression of MHLTH_CrudePrev \n",
|
||||
"# regression of MHLTH_CrudePrev\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"columns_regress = ['HE_FOOD', 'HE_WALK', 'HE_VACANCY', 'HE_SUPRFND','HE_HLTHINS','BINGE_CrudePrev',\n",
|
||||
" 'CHECKUP_CrudePrev','BPHIGH_CrudePrev','SLEEP_CrudePrev', 'STROKE_CrudePrev', 'Drug Test', 'Pedalcyclist']\n",
|
||||
"columns_regress = [\n",
|
||||
" \"HE_FOOD\",\n",
|
||||
" \"HE_WALK\",\n",
|
||||
" \"HE_VACANCY\",\n",
|
||||
" \"HE_SUPRFND\",\n",
|
||||
" \"HE_HLTHINS\",\n",
|
||||
" \"BINGE_CrudePrev\",\n",
|
||||
" \"CHECKUP_CrudePrev\",\n",
|
||||
" \"BPHIGH_CrudePrev\",\n",
|
||||
" \"SLEEP_CrudePrev\",\n",
|
||||
" \"STROKE_CrudePrev\",\n",
|
||||
" \"Drug Test\",\n",
|
||||
" \"Pedalcyclist\",\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"multiply_cols = {'HE_FOOD': -1,'HE_WALK': 1,'HE_VACANCY': -1,'HE_SUPRFND':-1 , 'HE_HLTHINS': 1 ,'BINGE_CrudePrev': -1 , \n",
|
||||
" 'CHECKUP_CrudePrev': 1,'BPHIGH_CrudePrev': -1,'SLEEP_CrudePrev': -1,'STROKE_CrudePrev' : -1, 'Drug Test' : -1, 'Pedalcyclist' : -1,}\n",
|
||||
"multiply_cols = {\n",
|
||||
" \"HE_FOOD\": -1,\n",
|
||||
" \"HE_WALK\": 1,\n",
|
||||
" \"HE_VACANCY\": -1,\n",
|
||||
" \"HE_SUPRFND\": -1,\n",
|
||||
" \"HE_HLTHINS\": 1,\n",
|
||||
" \"BINGE_CrudePrev\": -1,\n",
|
||||
" \"CHECKUP_CrudePrev\": 1,\n",
|
||||
" \"BPHIGH_CrudePrev\": -1,\n",
|
||||
" \"SLEEP_CrudePrev\": -1,\n",
|
||||
" \"STROKE_CrudePrev\": -1,\n",
|
||||
" \"Drug Test\": -1,\n",
|
||||
" \"Pedalcyclist\": -1,\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"target = 'MHLTH_CrudePrev'\n",
|
||||
"target = \"MHLTH_CrudePrev\"\n",
|
||||
"target_multiplier = -1\n",
|
||||
"\n",
|
||||
"multiplied_zscore_data_2, params_2 = model1.model_output(columns_regress,target,multiply_cols,target_multiplier = target_multiplier)"
|
||||
"multiplied_zscore_data_2, params_2 = model1.model_output(\n",
|
||||
" columns_regress, target, multiply_cols, target_multiplier=target_multiplier\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
@ -579,10 +659,12 @@
|
|||
}
|
||||
],
|
||||
"source": [
|
||||
"health_scores1 = HealthScores(params_1,params_2,params_le,multiplied_zscore_data_1,data['geoid'])\n",
|
||||
"final_data,weights_table = health_scores1.final_scaled_data()\n",
|
||||
"health_scores1 = HealthScores(\n",
|
||||
" params_1, params_2, params_le, multiplied_zscore_data_1, data[\"geoid\"]\n",
|
||||
")\n",
|
||||
"final_data, weights_table = health_scores1.final_scaled_data()\n",
|
||||
"\n",
|
||||
"health_scores1.weights #Can access the weights directly like this."
|
||||
"health_scores1.weights # Can access the weights directly like this."
|
||||
]
|
||||
},
|
||||
{
|
||||
|
|
|
@ -276,6 +276,21 @@
|
|||
"mapping_inequality_df"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "fceb3136",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"cdc_svi_index_path = DATA_DIR / \"dataset\" / \"cdc_svi_index\" / \"usa.csv\"\n",
|
||||
"cdc_svi_index_df = pd.read_csv(\n",
|
||||
" cdc_svi_index_path,\n",
|
||||
" dtype={ExtractTransformLoad.GEOID_TRACT_FIELD_NAME: \"string\"},\n",
|
||||
")\n",
|
||||
"cdc_svi_index_df.head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
|
@ -302,7 +317,7 @@
|
|||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Load alternative energy-related definition \n",
|
||||
"# Load alternative energy-related definition\n",
|
||||
"energy_definition_alternative_draft_path = (\n",
|
||||
" DATA_DIR / \"dataset\" / \"energy_definition_alternative_draft\" / \"usa.csv\"\n",
|
||||
")\n",
|
||||
|
@ -370,7 +385,8 @@
|
|||
" epa_rsei_aggregate_df,\n",
|
||||
" maryland_ejscreen_df,\n",
|
||||
" energy_definition_alternative_draft_df,\n",
|
||||
" michigan_ejscreen_df\n",
|
||||
" michigan_ejscreen_df,\n",
|
||||
" cdc_svi_index_df,\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"merged_df = functools.reduce(\n",
|
||||
|
@ -493,8 +509,8 @@
|
|||
" ),\n",
|
||||
" Index(\n",
|
||||
" method_name=\"EPA RSEI Aggregate Microdata\",\n",
|
||||
" priority_communities_field=field_names.EPA_RSEI_SCORE_THRESHOLD_FIELD\n",
|
||||
" ), \n",
|
||||
" priority_communities_field=field_names.EPA_RSEI_SCORE_THRESHOLD_FIELD,\n",
|
||||
" ),\n",
|
||||
" Index(\n",
|
||||
" method_name=\"Persistent Poverty\",\n",
|
||||
" priority_communities_field=PERSISTENT_POVERTY_TRACT_LEVEL_FIELD,\n",
|
||||
|
@ -502,15 +518,19 @@
|
|||
" Index(\n",
|
||||
" method_name=\"Maryland EJSCREEN\",\n",
|
||||
" priority_communities_field=field_names.MARYLAND_EJSCREEN_BURDENED_THRESHOLD_FIELD,\n",
|
||||
" ), \n",
|
||||
" ),\n",
|
||||
" Index(\n",
|
||||
" method_name=field_names.ENERGY_RELATED_COMMUNITIES_DEFINITION_ALTERNATIVE,\n",
|
||||
" priority_communities_field=field_names.ENERGY_RELATED_COMMUNITIES_DEFINITION_ALTERNATIVE,\n",
|
||||
" ),\n",
|
||||
" Index(\n",
|
||||
" method_name=\"CDC SVI Index\",\n",
|
||||
" priority_communities_field=field_names.CDC_SVI_INDEX_THEMES_PRIORITY_COMMUNITY,\n",
|
||||
" ),\n",
|
||||
" Index(\n",
|
||||
" method_name=\"Michigan EJSCREEN\",\n",
|
||||
" priority_communities_field=field_names.MICHIGAN_EJSCREEN_PRIORITY_COMMUNITY_FIELD,\n",
|
||||
" ), \n",
|
||||
" ),\n",
|
||||
" ]\n",
|
||||
" # Insert indices for each of the HOLC factors.\n",
|
||||
" # Note: since these involve no renaming, we write them using list comprehension.\n",
|
||||
|
|
|
@ -238,13 +238,26 @@ MICHIGAN_EJSCREEN_PRIORITY_COMMUNITY_FIELD: str = (
|
|||
"Michigan EJSCREEN Priority Community"
|
||||
)
|
||||
|
||||
# CDC SVI INDEX percentile fields
|
||||
CDC_SVI_INDEX_SE_THEME_FIELD: str = "SVI - Socioeconomic Index"
|
||||
CDC_SVI_INDEX_HOUSEHOLD_THEME_COMPOSITION_FIELD: str = (
|
||||
"SVI - Household Composition Index"
|
||||
)
|
||||
CDC_SVI_INDEX_LANGUAGE_THEME_FIELD: str = "SVI- Minority Status/Language Index"
|
||||
CDC_SVI_INDEX_HOUSING_TRANSPORTATION_FIELD: str = (
|
||||
"SVI- Housing Type/Transportation Index"
|
||||
)
|
||||
CDC_SVI_INDEX_RPL_THEMES_OVERALL_FIELD: str = (
|
||||
"Overall rank for Social Vulnerability Indices"
|
||||
)
|
||||
CDC_SVI_INDEX_THEMES_PRIORITY_COMMUNITY: str = "At or above 90 for overall percentile ranking according to Social Vulnerability Indices"
|
||||
|
||||
# Maryland EJSCREEN Data.
|
||||
MARYLAND_EJSCREEN_SCORE_FIELD: str = "Maryland Environmental Justice Score"
|
||||
|
||||
MARYLAND_EJSCREEN_BURDENED_THRESHOLD_FIELD: str = (
|
||||
"Maryland EJSCREEN Priority Community"
|
||||
)
|
||||
|
||||
# Child Opportunity Index data
|
||||
# Summer days with maximum temperature above 90F.
|
||||
EXTREME_HEAT_FIELD = "Summer days above 90F"
|
||||
|
|
Loading…
Add table
Reference in a new issue