Refactor DOE Energy Burden and COI to use YAML (#1796)

* added tribalId for Supplemental dataset (#1804)

* Setting zoom levels for tribal map (#1810)

* NRI dataset and initial score YAML configuration (#1534)

* update be staging gha

* NRI dataset and initial score YAML configuration

* checkpoint

* adding data checks for release branch

* passing tests

* adding INPUT_EXTRACTED_FILE_NAME to base class

* lint

* columns to keep and tests

* update be staging gha

* checkpoint

* update be staging gha

* NRI dataset and initial score YAML configuration

* checkpoint

* adding data checks for release branch

* passing tests

* adding INPUT_EXTRACTED_FILE_NAME to base class

* lint

* columns to keep and tests

* checkpoint

* PR Review

* renoving source url

* tests

* stop execution of ETL if there's a YAML schema issue

* update be staging gha

* adding source url as class var again

* clean up

* force cache bust

* gha cache bust

* dynamically set score vars from YAML

* docsctrings

* removing last updated year - optional reverse percentile

* passing tests

* sort order

* column ordening

* PR review

* class level vars

* Updating DatasetsConfig

* fix pylint errors

* moving metadata hint back to code

Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov>

* Correct copy typo (#1809)

* Add basic test suite for COI (#1518)

* Update COI to use new yaml (#1518)

* Add tests for DOE energy budren (1518

* Add dataset config for energy budren (1518)

* Refactor ETL to use datasets.yml (#1518)

* Add fake GEOIDs to COI tests (#1518)

* Refactor _setup_etl_instance_and_run_extract to base (#1518)

For the three classes we've done so far, a generic
_setup_etl_instance_and_run_extract will work fine, for the moment we
can reuse the same setup method until we decide future classes need more
flexibility --- but they can also always subclass so...

* Add output-path tests (#1518)

* Update YAML to match constant (#1518)

* Don't blindly set float format (#1518)

* Add defaults for extract (#1518)

* Run YAML load on all subclasses (#1518)

* Update description fields (#1518)

* Update YAML per final format (#1518)

* Update fixture tract IDs (#1518)

* Update base class refactor (#1518)

Now that NRI is final I needed to make a small number of updates to my
refactored code.

* Remove old comment (#1518)

* Fix type signature and return (#1518)

* Update per code review (#1518)

Co-authored-by: Jorge Escobar <83969469+esfoobar-usds@users.noreply.github.com>
Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov>
Co-authored-by: Vim <86254807+vim-usds@users.noreply.github.com>

.github/workflows/data-checks.yml

@@ -2,7 +2,9 @@
 name: Data Checks
 on:
   pull_request:
-    branches: [main] # runs on any PR against main
+    branches:
+      - main
+      - "**/release/**"
     paths:
       - "data/**"
 jobs:
@@ -16,7 +18,7 @@ jobs:
         # checks all of the versions allowed in pyproject.toml
         python-version: [3.8, 3.9]
     steps:
-      # installs python
+      # installs Python
       # one execution of the tests per version listed above
       - uses: actions/checkout@v2
       - name: Set up Python ${{ matrix.python-version }}

.github/workflows/deploy_be_staging.yml

@@ -62,7 +62,7 @@ jobs:
       - name: Update PR with deployed Score URLs
         uses: mshick/add-pr-comment@v1
         with:
-          # Deploy to S3 for the staging URL
+          # Deploy to S3 for the Staging URL
           message: |
             ** Score Deployed! **   
             Find it here: 

client/src/components/AreaDetail/tests/__snapshots__/areaDetail.test.tsx.snap

@@ -519,7 +519,7 @@ exports[`rendering of the AreaDetail checks if indicators for NATION is present
             <div>
               Expected building loss rate
               <div>
-                Economic loss rate to agricultural value resulting from natural hazards each year
+                Economic loss rate to building value resulting from natural hazards each year
               </div>
             </div>
             <div>

client/src/data/copy/explore.tsx

@@ -590,7 +590,7 @@ export const SIDE_PANEL_INDICATOR_DESCRIPTION = defineMessages({
   },
   EXP_BLD_LOSS: {
     id: 'explore.map.page.side.panel.indicator.description.exp.bld.loss',
-    defaultMessage: 'Economic loss rate to agricultural value resulting from natural hazards each year',
+    defaultMessage: 'Economic loss rate to building value resulting from natural hazards each year',
     description: `Navigate to the explore the map page. When the map is in view, click on the map. The side 
     panel will show an indicator description of Economic loss rate to buildings resulting from natural hazards`,
   },

client/src/intl/en.json

@@ -496,7 +496,7 @@
     "description": "Navigate to the explore the map page. When the map is in view, click on the map. The side panel will show an indicator description of Economic loss rate to agriculture resulting from natural hazards\n    "
   },
   "explore.map.page.side.panel.indicator.description.exp.bld.loss": {
-    "defaultMessage": "Economic loss rate to agricultural value resulting from natural hazards each year",
+    "defaultMessage": "Economic loss rate to building value resulting from natural hazards each year",
     "description": "Navigate to the explore the map page. When the map is in view, click on the map. The side \n    panel will show an indicator description of Economic loss rate to buildings resulting from natural hazards"
   },
   "explore.map.page.side.panel.indicator.description.exp.pop.loss": {

client/src/intl/es.json

@@ -125,7 +125,7 @@
     "explore.map.page.side.panel.indicator.description.dieselPartMatter": "Descarga de gas de motor diésel en el aire",
     "explore.map.page.side.panel.indicator.description.energyBurden": "Costo promedio anual de la energía dividido por el ingreso familiar",
     "explore.map.page.side.panel.indicator.description.exp.ag.loss": "Tasa de pérdida económica en relación con el valor agrícola resultante de peligros naturales cada año",
-    "explore.map.page.side.panel.indicator.description.exp.bld.loss": "Tasa de pérdida económica en relación con el valor agrícola resultante de peligros naturales cada año",
+    "explore.map.page.side.panel.indicator.description.exp.bld.loss": "Tasa de pérdida económica en relación con el valor construcción resultante de peligros naturales cada año",
     "explore.map.page.side.panel.indicator.description.exp.pop.loss": "Tasa de muertes y lesiones resultantes de peligros naturales cada año",
     "explore.map.page.side.panel.indicator.description.heartDisease": "Personas con 18 años cumplidos a quienes se les ha diagnosticado una cardiopatía",
     "explore.map.page.side.panel.indicator.description.high.ed": "Porcentaje de la población con 15 años cumplidos del grupo de bloques del censo que no está inscrita en la universidad, escuela superior o escuela de posgrado",

data/data-pipeline/data_pipeline/etl/base.py

@@ -1,12 +1,15 @@
 import enum
 import pathlib
+import sys
 import typing
 from typing import Optional
 
 import pandas as pd
 
 from data_pipeline.config import settings
+from data_pipeline.etl.score.schemas.datasets import DatasetsConfig
 from data_pipeline.utils import (
+    load_yaml_dict_from_file,
     unzip_file_from_url,
     remove_all_from_dir,
     get_module_logger,
@@ -30,6 +33,9 @@ class ExtractTransformLoad:
     Attributes:
         DATA_PATH (pathlib.Path): Local path where all data will be stored
         TMP_PATH (pathlib.Path): Local path where temporary data will be stored
+
+        TODO: Fill missing attrs here
+
         GEOID_FIELD_NAME (str): The common column name for a Census Block Group identifier
         GEOID_TRACT_FIELD_NAME (str): The common column name for a Census Tract identifier
     """
@@ -40,6 +46,8 @@ class ExtractTransformLoad:
     DATA_PATH: pathlib.Path = APP_ROOT / "data"
     TMP_PATH: pathlib.Path = DATA_PATH / "tmp"
     CONTENT_CONFIG: pathlib.Path = APP_ROOT / "content" / "config"
+    DATASET_CONFIG_PATH: pathlib.Path = APP_ROOT / "etl" / "score" / "config"
+    DATASET_CONFIG: Optional[dict] = None
 
     # Parameters
     GEOID_FIELD_NAME: str = "GEOID10"
@@ -55,6 +63,9 @@ class ExtractTransformLoad:
     # SOURCE_URL is used to extract source data in extract().
     SOURCE_URL: str = None
 
+    # INPUT_EXTRACTED_FILE_NAME is the name of the file after extract().
+    INPUT_EXTRACTED_FILE_NAME: str = None
+
     # GEO_LEVEL is used to identify whether output data is at the unit of the tract or
     # census block group.
     # TODO: add tests that enforce seeing the expected geographic identifier field
@@ -64,6 +75,13 @@ class ExtractTransformLoad:
     # COLUMNS_TO_KEEP is used to identify which columns to keep in the output df.
     COLUMNS_TO_KEEP: typing.List[str] = None
 
+    # INPUT_GEOID_TRACT_FIELD_NAME is the field name that identifies the Census Tract ID
+    # on the input file
+    INPUT_GEOID_TRACT_FIELD_NAME: str = None
+
+    # NULL_REPRESENTATION is how nulls are represented on the input field
+    NULL_REPRESENTATION: str = None
+
     # Thirteen digits in a census block group ID.
     EXPECTED_CENSUS_BLOCK_GROUPS_CHARACTER_LENGTH: int = 13
     # TODO: investigate. Census says there are only 217,740 CBGs in the US. This might
@@ -77,8 +95,56 @@ class ExtractTransformLoad:
     #  periods. https://github.com/usds/justice40-tool/issues/964
     EXPECTED_MAX_CENSUS_TRACTS: int = 74160
 
+    # We use output_df as the final dataframe to use to write to the CSV
+    # It is used on the "load" base class method
     output_df: pd.DataFrame = None
 
+    def __init_subclass__(cls) -> None:
+        cls.DATASET_CONFIG = cls.yaml_config_load()
+
+    @classmethod
+    def yaml_config_load(cls) -> Optional[dict]:
+        """Generate config dictionary and set instance variables from YAML dataset."""
+        if cls.NAME is not None:
+            # check if the class instance has score YAML definitions
+            datasets_config = load_yaml_dict_from_file(
+                cls.DATASET_CONFIG_PATH / "datasets.yml",
+                DatasetsConfig,
+            )
+
+            # get the config for this dataset
+            try:
+                dataset_config = next(
+                    item
+                    for item in datasets_config.get("datasets")
+                    if item["module_name"] == cls.NAME
+                )
+            except StopIteration:
+                # Note: it'd be nice to log the name of the dataframe, but that's not accessible in this scope.
+                logger.error(
+                    f"Exception encountered while extracting dataset config for dataset {cls.NAME}"
+                )
+                sys.exit()
+
+            # set some of the basic fields
+            cls.INPUT_GEOID_TRACT_FIELD_NAME = dataset_config[
+                "input_geoid_tract_field_name"
+            ]
+
+            # get the columns to write on the CSV
+            # and set the constants
+            cls.COLUMNS_TO_KEEP = [
+                cls.GEOID_TRACT_FIELD_NAME,  # always index with geoid tract id
+            ]
+            for field in dataset_config["load_fields"]:
+                cls.COLUMNS_TO_KEEP.append(field["long_name"])
+                setattr(cls, field["df_field_name"], field["long_name"])
+
+                # set the constants for the class
+                setattr(cls, field["df_field_name"], field["long_name"])
+            return dataset_config
+        return None
+
     # This is a classmethod so it can be used by `get_data_frame` without
     # needing to create an instance of the class. This is a use case in `etl_score`.
     @classmethod
@@ -87,16 +153,10 @@ class ExtractTransformLoad:
         if cls.NAME is None:
             raise NotImplementedError(
                 f"Child ETL class needs to specify `cls.NAME` (currently "
-                f"{cls.NAME}) and `cls.LAST_UPDATED_YEAR` (currently "
+                f"{cls.NAME})."
-                f"{cls.LAST_UPDATED_YEAR})."
             )
 
-        output_file_path = (
+        output_file_path = cls.DATA_PATH / "dataset" / f"{cls.NAME}" / "usa.csv"
-            cls.DATA_PATH
-            / "dataset"
-            / f"{cls.NAME}_{cls.LAST_UPDATED_YEAR}"
-            / "usa.csv"
-        )
         return output_file_path
 
     def get_tmp_path(self) -> pathlib.Path:
@@ -120,14 +180,18 @@ class ExtractTransformLoad:
         to get the file from a source url, unzips it and stores it on an
         extract_path."""
 
-        # this can be accessed via super().extract()
+        if source_url is None:
-        if source_url and extract_path:
+            source_url = self.SOURCE_URL
-            unzip_file_from_url(
+
-                file_url=source_url,
+        if extract_path is None:
-                download_path=self.get_tmp_path(),
+            extract_path = self.get_tmp_path()
-                unzipped_file_path=extract_path,
+
-                verify=verify,
+        unzip_file_from_url(
-            )
+            file_url=source_url,
+            download_path=self.get_tmp_path(),
+            unzipped_file_path=extract_path,
+            verify=verify,
+        )
 
     def transform(self) -> None:
         """Transform the data extracted into a format that can be consumed by the
@@ -229,8 +293,7 @@ class ExtractTransformLoad:
 
         Data is written in the specified local data folder or remote AWS S3 bucket.
 
-        Uses the directory from `self.OUTPUT_DIR` and the file name from
+        Uses the directory and the file name from `self._get_output_file_path`.
-        `self._get_output_file_path`.
         """
         logger.info(f"Saving `{self.NAME}` CSV")
 

data/data-pipeline/data_pipeline/etl/score/config/datasets.yml

@@ -0,0 +1,131 @@
+---
+datasets:
+  - long_name: "FEMA National Risk Index"
+    short_name: "nri"
+    module_name: national_risk_index
+    input_geoid_tract_field_name: "TRACTFIPS"
+    load_fields:
+      - short_name: "ex_loss"
+        df_field_name: "RISK_INDEX_EXPECTED_ANNUAL_LOSS_SCORE_FIELD_NAME"
+        long_name: "FEMA Risk Index Expected Annual Loss Score"
+        field_type: float
+        number_of_decimals_in_output: 6
+
+      - short_name: "ex_pop_loss"
+        df_field_name: "EXPECTED_POPULATION_LOSS_RATE_FIELD_NAME"
+        long_name: "Expected population loss rate (Natural Hazards Risk Index)"
+        description_short:
+          "Rate of fatalities and injuries resulting from natural hazards each year"
+        description_long:
+          "Rate relative to the population of fatalities and injuries due to fourteen
+          types of natural hazards each year that have some link to climate change:
+          avalanche, coastal flooding, cold wave, drought, hail, heat wave, hurricane,
+          ice storm, landslide, riverine flooding, strong wind, tornado, wildfire, and
+          winter weather. Population loss is defined as the Spatial Hazard Events and
+          Losses and National Centers for Environmental Information’s (NCEI) reported
+          number of fatalities and injuries caused by the hazard occurrence. To combine
+          fatalities and injuries for the computation of population loss value, an
+          injury is counted as one-tenth (1/10) of a fatality. The NCEI Storm Events
+          Database classifies injuries and fatalities as direct or indirect. Both direct
+          and indirect injuries and fatalities are counted as population loss. This
+          total number of injuries and fatalities is then divided by the population in
+          the census tract to get a per-capita rate of population risk."
+        field_type: float
+        number_of_decimals_in_output: 6
+        include_in_tiles: true
+        include_in_downloadable_files: true
+        create_percentile: true
+
+      - short_name: "ex_ag_loss"
+        df_field_name: "EXPECTED_AGRICULTURE_LOSS_RATE_FIELD_NAME"
+        long_name: "Expected agricultural loss rate (Natural Hazards Risk Index)"
+        description_short:
+          "Economic loss rate to agricultural value resulting from natural hazards each
+          year"
+        description_long:
+          "Percent of agricultural value at risk from losses due to fourteen types of
+          natural hazards that have some link to climate change: avalanche, coastal
+          flooding, cold wave, drought, hail, heat wave, hurricane, ice storm,
+          landslide, riverine flooding, strong wind, tornado, wildfire, and winter
+          weather. Rate calculated by dividing the agricultural value at risk in a
+          census tract by the total agricultural value in that census tract."
+        field_type: float
+        number_of_decimals_in_output: 6
+        include_in_tiles: true
+        include_in_downloadable_files: true
+        create_percentile: true
+
+      - short_name: "ex_bldg_loss"
+        df_field_name: "EXPECTED_BUILDING_LOSS_RATE_FIELD_NAME"
+        long_name: "Expected building loss rate (Natural Hazards Risk Index)"
+        description_short:
+          "Economic loss rate to building value resulting from natural hazards each year"
+        description_long:
+          "Percent of building value at risk from losses due to fourteen types of
+          natural hazards that have some link to climate change: avalanche, coastal
+          flooding, cold wave, drought, hail, heat wave, hurricane, ice storm,
+          landslide, riverine flooding, strong wind, tornado, wildfire, and winter
+          weather. Rate calculated by dividing the building value at risk in a census
+          tract by the total building value in that census tract."
+        field_type: float
+        number_of_decimals_in_output: 6
+        include_in_tiles: true
+        include_in_downloadable_files: true
+        create_percentile: true
+
+      - short_name: "has_ag_val"
+        df_field_name: "CONTAINS_AGRIVALUE"
+        long_name: "Contains agricultural value"
+        field_type: bool
+  - long_name: "Child Opportunity Index 2.0 database"
+    short_name: "coi"
+    module_name: "child_opportunity_index"
+    input_geoid_tract_field_name: "geoid"
+    load_fields:
+      - short_name: "he_heat"
+        df_field_name: "EXTREME_HEAT_FIELD"
+        long_name: "Summer days above 90F" 
+        field_type: float
+        include_in_downloadable_files: true
+        include_in_tiles: true
+      - short_name: "he_food"
+        long_name: "Percent low access to healthy food"
+        df_field_name: "HEALTHY_FOOD_FIELD"
+        field_type: float
+        include_in_downloadable_files: true
+        include_in_tiles: true
+      - short_name: "he_green"
+        long_name: "Percent impenetrable surface areas" 
+        df_field_name: "IMPENETRABLE_SURFACES_FIELD"
+        field_type: float
+        include_in_downloadable_files: true
+        include_in_tiles: true
+      - short_name: "ed_reading"
+        df_field_name: "READING_FIELD"
+        long_name: "Third grade reading proficiency"
+        field_type: float
+        include_in_downloadable_files: true
+        include_in_tiles: true
+  - long_name: "Low-Income Energy Affordabililty Data"
+    short_name: "LEAD"
+    module_name: "doe_energy_burden"
+    input_geoid_tract_field_name: "FIP"
+    load_fields:
+      - short_name: "EBP_PFS"
+        df_field_name: "REVISED_ENERGY_BURDEN_FIELD_NAME"
+        long_name: "Energy burden" 
+        field_type: float
+        include_in_downloadable_files: true
+        include_in_tiles: true
+  - long_name: "Example ETL"
+    short_name: "Example"
+    module_name: "example_dataset"
+    input_geoid_tract_field_name: "GEOID10_TRACT"
+    load_fields:
+      - short_name: "EXAMPLE_FIELD"
+        df_field_name: "Input Field 1"
+        long_name: "Example Field 1" 
+        field_type: float
+        include_in_tiles: true
+        include_in_downloadable_files: true
+

data/data-pipeline/data_pipeline/etl/score/etl_score.py

@@ -442,6 +442,7 @@ class ScoreETL(ExtractTransformLoad):
             # for instance, 3rd grade reading level : Low 3rd grade reading level.
             # This low field will not exist yet, it is only calculated for the
             # percentile.
+            # TODO: This will come from the YAML dataset config
             ReversePercentile(
                 field_name=field_names.READING_FIELD,
                 low_field_name=field_names.LOW_READING_FIELD,

data/data-pipeline/data_pipeline/etl/score/schemas/datasets.py

@@ -0,0 +1,83 @@
+from dataclasses import dataclass, field
+from enum import Enum
+from typing import List, Optional
+
+
+class FieldType(Enum):
+    STRING = "string"
+    INT64 = "int64"
+    BOOL = "bool"
+    FLOAT = "float"
+    PERCENTAGE = "percentage"
+
+
+@dataclass
+class DatasetsConfig:
+    @dataclass
+    class Dataset:
+        """A class that defines a dataset and its load variables.
+
+        Attributes:
+            long_name (str): A human readable title for the dataset.
+            short_name (str): used to compose the short variable names for tiles/arcgis. All short variable names will be prepended
+            with the short name of the data set it comes from, i.e. `nri__ex_loss`.
+            module_name (str): A string that matches both the Python module name for the dataset and the `NAME` property on the ETL class.
+            load_fields (LoadField): A list of type LoadField that will drive the score ETL and side effects (tiles, downloadables).
+        """
+
+        @dataclass
+        class LoadField:
+            """A class to define the fields to be saved on the dataset's output.
+
+            These fields will be then imported by the score generation ETL.
+
+            Attributes:
+                short_name (str): Used in conjunction with the dataset's `short_name` for files where short names are needed.
+                df_field_name (str): Name for the field in the etl class.
+                long_name (str): Column name for the dataset's output csv.
+                field_type (FieldType): An enum that dictates what type of field this is.
+                description_short (Optional str): Description used if the field appears in the side panel.
+                description_long (Optional str): Description used if the field appears in the Methodology page.
+                number_of_decimals_in_output (Optional int): Used to represent number of decimals in side effects, like Excel. Defaults to 2 decimals.
+                include_in_tiles (Optional bool): Include this field on the tile export. Defaults to False.
+                include_in_downloadable_files (Optional bool): Include this field on the CSV and Excel exports. Defaults to False.
+                create_percentile (Optional bool): Whether or not the backend processing should create a percentile field (ranked in ascending order)
+                    from the values in this field. Defaults to False.
+                create_reverse_percentile (Optional bool): Whether or not the backend processing should create a "reverse percentile" field (ranked in
+                    descending order) from the values in this field. Defaults to False.
+                include_in_comparison_tool_as_index (Optional bool): Whether or not to include this field in the comparison tool
+                    as an index used as comparison (e.g., this field might be a state or national index that identifies priority communities).
+                    The field itself must be a boolean for the comparison tool to work appropriately. Defaults to False.
+                include_in_comparison_tool_as_statistical_descriptor (Optional bool): Whether or not to include this field in the comparison tool as a
+                    statistical descriptor of census tracts (e.g., this field might income levels, life expectancy, etc). This will be
+                    used to generate reports that produce information such as, tracts identified by Index A but not Index B have higher
+                    income levels but lower life expectancy. Defaults to False.
+            """
+
+            short_name: str
+            df_field_name: str
+            long_name: str
+            field_type: FieldType = field(
+                metadata={"by_value": True}
+            )  # This will be used on the `etl_score_post` for the
+            # data manipulation. The `by_value` metadata prop will load the field type's Enum value instead of the index, i.e. "string"
+            # and not STRING
+            description_short: Optional[str] = None
+            description_long: Optional[str] = None
+            number_of_decimals_in_output: Optional[int] = 2
+            include_in_tiles: Optional[bool] = False
+            include_in_downloadable_files: Optional[bool] = False
+            create_percentile: Optional[bool] = False
+            create_reverse_percentile: Optional[bool] = False
+            include_in_comparison_tool_as_index: Optional[bool] = False
+            include_in_comparison_tool_as_statistical_descriptor: Optional[
+                bool
+            ] = False
+
+        long_name: str
+        short_name: str
+        module_name: str
+        input_geoid_tract_field_name: str
+        load_fields: List[LoadField]
+
+    datasets: List[Dataset]

data/data-pipeline/data_pipeline/etl/sources/child_opportunity_index/etl.py

@@ -1,9 +1,8 @@
 from pathlib import Path
 import pandas as pd
 
-from data_pipeline.etl.base import ExtractTransformLoad
+from data_pipeline.etl.base import ExtractTransformLoad, ValidGeoLevel
-from data_pipeline.score import field_names
+from data_pipeline.utils import get_module_logger
-from data_pipeline.utils import get_module_logger, unzip_file_from_url
 
 logger = get_module_logger(__name__)
 
@@ -21,15 +20,27 @@ class ChildOpportunityIndex(ExtractTransformLoad):
     Full technical documents: https://www.diversitydatakids.org/sites/default/files/2020-02/ddk_coi2.0_technical_documentation_20200212.pdf.
 
     Github repo: https://github.com/diversitydatakids/COI/
-
     """
 
+    # Metadata for the baseclass
+    NAME = "child_opportunity_index"
+    GEO_LEVEL = ValidGeoLevel.CENSUS_TRACT
+
+    # Define these for easy code completion
+    EXTREME_HEAT_FIELD: str
+    HEALTHY_FOOD_FIELD: str
+    IMPENETRABLE_SURFACES_FIELD: str
+    READING_FIELD: str
+
     def __init__(self):
-        self.COI_FILE_URL = (
+        self.SOURCE_URL = (
             "https://data.diversitydatakids.org/datastore/zip/f16fff12-b1e5-4f60-85d3-"
             "3a0ededa30a0?format=csv"
         )
 
+        # TODO: Decide about nixing this
+        self.TRACT_INPUT_COLUMN_NAME = self.INPUT_GEOID_TRACT_FIELD_NAME
+
         self.OUTPUT_PATH: Path = (
             self.DATA_PATH / "dataset" / "child_opportunity_index"
         )
@@ -40,31 +51,19 @@ class ChildOpportunityIndex(ExtractTransformLoad):
         self.IMPENETRABLE_SURFACES_INPUT_FIELD = "HE_GREEN"
         self.READING_INPUT_FIELD = "ED_READING"
 
-        # Constants for output
-        self.COLUMNS_TO_KEEP = [
-            self.GEOID_TRACT_FIELD_NAME,
-            field_names.EXTREME_HEAT_FIELD,
-            field_names.HEALTHY_FOOD_FIELD,
-            field_names.IMPENETRABLE_SURFACES_FIELD,
-            field_names.READING_FIELD,
-        ]
-
-        self.raw_df: pd.DataFrame
         self.output_df: pd.DataFrame
 
     def extract(self) -> None:
         logger.info("Starting 51MB data download.")
-
+        super().extract(
-        unzip_file_from_url(
+            source_url=self.SOURCE_URL,
-            file_url=self.COI_FILE_URL,
+            extract_path=self.get_tmp_path(),
-            download_path=self.get_tmp_path(),
-            unzipped_file_path=self.get_tmp_path() / "child_opportunity_index",
         )
 
-        self.raw_df = pd.read_csv(
+    def transform(self) -> None:
-            filepath_or_buffer=self.get_tmp_path()
+        logger.info("Starting transforms.")
-            / "child_opportunity_index"
+        raw_df = pd.read_csv(
-            / "raw.csv",
+            filepath_or_buffer=self.get_tmp_path() / "raw.csv",
             # The following need to remain as strings for all of their digits, not get
             # converted to numbers.
             dtype={
@@ -73,16 +72,13 @@ class ChildOpportunityIndex(ExtractTransformLoad):
             low_memory=False,
         )
 
-    def transform(self) -> None:
+        output_df = raw_df.rename(
-        logger.info("Starting transforms.")
-
-        output_df = self.raw_df.rename(
             columns={
                 self.TRACT_INPUT_COLUMN_NAME: self.GEOID_TRACT_FIELD_NAME,
-                self.EXTREME_HEAT_INPUT_FIELD: field_names.EXTREME_HEAT_FIELD,
+                self.EXTREME_HEAT_INPUT_FIELD: self.EXTREME_HEAT_FIELD,
-                self.HEALTHY_FOOD_INPUT_FIELD: field_names.HEALTHY_FOOD_FIELD,
+                self.HEALTHY_FOOD_INPUT_FIELD: self.HEALTHY_FOOD_FIELD,
-                self.IMPENETRABLE_SURFACES_INPUT_FIELD: field_names.IMPENETRABLE_SURFACES_FIELD,
+                self.IMPENETRABLE_SURFACES_INPUT_FIELD: self.IMPENETRABLE_SURFACES_FIELD,
-                self.READING_INPUT_FIELD: field_names.READING_FIELD,
+                self.READING_INPUT_FIELD: self.READING_FIELD,
             }
         )
 
@@ -95,8 +91,8 @@ class ChildOpportunityIndex(ExtractTransformLoad):
 
         # Convert percents from 0-100 to 0-1 to standardize with our other fields.
         percent_fields_to_convert = [
-            field_names.HEALTHY_FOOD_FIELD,
+            self.HEALTHY_FOOD_FIELD,
-            field_names.IMPENETRABLE_SURFACES_FIELD,
+            self.IMPENETRABLE_SURFACES_FIELD,
         ]
 
         for percent_field_to_convert in percent_fields_to_convert:
@@ -105,11 +101,3 @@ class ChildOpportunityIndex(ExtractTransformLoad):
             )
 
         self.output_df = output_df
-
-    def load(self) -> None:
-        logger.info("Saving CSV")
-
-        self.OUTPUT_PATH.mkdir(parents=True, exist_ok=True)
-        self.output_df[self.COLUMNS_TO_KEEP].to_csv(
-            path_or_buf=self.OUTPUT_PATH / "usa.csv", index=False
-        )

data/data-pipeline/data_pipeline/etl/sources/doe_energy_burden/etl.py

@@ -2,63 +2,48 @@ from pathlib import Path
 import pandas as pd
 
 from data_pipeline.config import settings
-from data_pipeline.etl.base import ExtractTransformLoad
+from data_pipeline.etl.base import ExtractTransformLoad, ValidGeoLevel
-from data_pipeline.utils import get_module_logger, unzip_file_from_url
+from data_pipeline.utils import get_module_logger
 
 logger = get_module_logger(__name__)
 
 
 class DOEEnergyBurden(ExtractTransformLoad):
-    def __init__(self):
+    NAME = "doe_energy_burden"
-        self.DOE_FILE_URL = (
+    SOURCE_URL: str = (
-            settings.AWS_JUSTICE40_DATASOURCES_URL
+        settings.AWS_JUSTICE40_DATASOURCES_URL
-            + "/DOE_LEAD_AMI_TRACT_2018_ALL.csv.zip"
+        + "/DOE_LEAD_AMI_TRACT_2018_ALL.csv.zip"
-        )
+    )
+    GEO_LEVEL = ValidGeoLevel.CENSUS_TRACT
 
+    REVISED_ENERGY_BURDEN_FIELD_NAME: str
+
+    def __init__(self):
         self.OUTPUT_PATH: Path = (
             self.DATA_PATH / "dataset" / "doe_energy_burden"
         )
-
-        self.TRACT_INPUT_COLUMN_NAME = "FIP"
         self.INPUT_ENERGY_BURDEN_FIELD_NAME = "BURDEN"
-        self.REVISED_ENERGY_BURDEN_FIELD_NAME = "Energy burden"
-
-        # Constants for output
-        self.COLUMNS_TO_KEEP = [
-            self.GEOID_TRACT_FIELD_NAME,
-            self.REVISED_ENERGY_BURDEN_FIELD_NAME,
-        ]
 
         self.raw_df: pd.DataFrame
         self.output_df: pd.DataFrame
 
-    def extract(self) -> None:
+    def transform(self) -> None:
-        logger.info("Starting data download.")
+        logger.info("Starting DOE Energy Burden transforms.")
-
+        raw_df: pd.DataFrame = pd.read_csv(
-        unzip_file_from_url(
-            file_url=self.DOE_FILE_URL,
-            download_path=self.get_tmp_path(),
-            unzipped_file_path=self.get_tmp_path() / "doe_energy_burden",
-        )
-
-        self.raw_df = pd.read_csv(
             filepath_or_buffer=self.get_tmp_path()
-            / "doe_energy_burden"
             / "DOE_LEAD_AMI_TRACT_2018_ALL.csv",
             # The following need to remain as strings for all of their digits, not get converted to numbers.
             dtype={
-                self.TRACT_INPUT_COLUMN_NAME: "string",
+                self.INPUT_GEOID_TRACT_FIELD_NAME: "string",
             },
             low_memory=False,
         )
 
-    def transform(self) -> None:
+        logger.info("Renaming columns and ensuring output format is correct")
-        logger.info("Starting transforms.")
+        output_df = raw_df.rename(
-
-        output_df = self.raw_df.rename(
             columns={
                 self.INPUT_ENERGY_BURDEN_FIELD_NAME: self.REVISED_ENERGY_BURDEN_FIELD_NAME,
-                self.TRACT_INPUT_COLUMN_NAME: self.GEOID_TRACT_FIELD_NAME,
+                self.INPUT_GEOID_TRACT_FIELD_NAME: self.GEOID_TRACT_FIELD_NAME,
             }
         )
 
@@ -75,7 +60,4 @@ class DOEEnergyBurden(ExtractTransformLoad):
     def load(self) -> None:
         logger.info("Saving DOE Energy Burden CSV")
 
-        self.OUTPUT_PATH.mkdir(parents=True, exist_ok=True)
+        super().load()
-        self.output_df[self.COLUMNS_TO_KEEP].to_csv(
-            path_or_buf=self.OUTPUT_PATH / "usa.csv", index=False
-        )

data/data-pipeline/data_pipeline/etl/sources/national_risk_index/etl.py

@@ -15,10 +15,16 @@ class NationalRiskIndexETL(ExtractTransformLoad):
     """ETL class for the FEMA National Risk Index dataset"""
 
     NAME = "national_risk_index"
-    LAST_UPDATED_YEAR = 2020
     SOURCE_URL = "https://hazards.fema.gov/nri/Content/StaticDocuments/DataDownload//NRI_Table_CensusTracts/NRI_Table_CensusTracts.zip"
     GEO_LEVEL = ValidGeoLevel.CENSUS_TRACT
 
+    # Output score variables (values set on datasets.yml) for linting purposes
+    RISK_INDEX_EXPECTED_ANNUAL_LOSS_SCORE_FIELD_NAME: str
+    EXPECTED_BUILDING_LOSS_RATE_FIELD_NAME: str
+    EXPECTED_AGRICULTURE_LOSS_RATE_FIELD_NAME: str
+    EXPECTED_POPULATION_LOSS_RATE_FIELD_NAME: str
+    CONTAINS_AGRIVALUE: str
+
     ## TEMPORARILY HERE
     ## To get this value up in time for launch, we've hard coded it. We would like
     ## to, in the future, have this pull the 10th percentile (or nth percentile)
@@ -27,54 +33,31 @@ class NationalRiskIndexETL(ExtractTransformLoad):
     AGRIVALUE_LOWER_BOUND = 408000
 
     def __init__(self):
+        # define the full path for the input CSV file
         self.INPUT_CSV = self.get_tmp_path() / "NRI_Table_CensusTracts.csv"
 
+        # this is the main dataframe
+        self.df: pd.DataFrame
+
+        # Start dataset-specific vars here
         self.RISK_INDEX_EXPECTED_ANNUAL_LOSS_SCORE_INPUT_FIELD_NAME = (
             "EAL_SCORE"
         )
-
-        self.RISK_INDEX_EXPECTED_ANNUAL_LOSS_SCORE_FIELD_NAME = (
-            "FEMA Risk Index Expected Annual Loss Score"
-        )
-
         self.EXPECTED_ANNUAL_LOSS_BUILDING_VALUE_INPUT_FIELD_NAME = "EAL_VALB"
-
         self.EXPECTED_ANNUAL_LOSS_AGRICULTURAL_VALUE_INPUT_FIELD_NAME = (
             "EAL_VALA"
         )
         self.EXPECTED_ANNUAL_LOSS_POPULATION_VALUE_INPUT_FIELD_NAME = "EAL_VALP"
-
         self.AGRICULTURAL_VALUE_INPUT_FIELD_NAME = "AGRIVALUE"
         self.POPULATION_INPUT_FIELD_NAME = "POPULATION"
         self.BUILDING_VALUE_INPUT_FIELD_NAME = "BUILDVALUE"
 
-        self.EXPECTED_BUILDING_LOSS_RATE_FIELD_NAME = (
-            "Expected building loss rate (Natural Hazards Risk Index)"
-        )
-        self.EXPECTED_AGRICULTURE_LOSS_RATE_FIELD_NAME = (
-            "Expected agricultural loss rate (Natural Hazards Risk Index)"
-        )
-        self.EXPECTED_POPULATION_LOSS_RATE_FIELD_NAME = (
-            "Expected population loss rate (Natural Hazards Risk Index)"
-        )
-        self.CONTAINS_AGRIVALUE = "Contains agricultural value"
-
-        self.COLUMNS_TO_KEEP = [
-            self.GEOID_TRACT_FIELD_NAME,
-            self.RISK_INDEX_EXPECTED_ANNUAL_LOSS_SCORE_FIELD_NAME,
-            self.EXPECTED_POPULATION_LOSS_RATE_FIELD_NAME,
-            self.EXPECTED_AGRICULTURE_LOSS_RATE_FIELD_NAME,
-            self.EXPECTED_BUILDING_LOSS_RATE_FIELD_NAME,
-            self.CONTAINS_AGRIVALUE,
-        ]
-
-        self.df: pd.DataFrame
-
     def extract(self) -> None:
         """Unzips NRI dataset from the FEMA data source and writes the files
         to the temporary data folder for use in the transform() method
         """
         logger.info("Downloading 405MB National Risk Index Data")
+
         super().extract(
             source_url=self.SOURCE_URL,
             extract_path=self.get_tmp_path(),
@@ -90,19 +73,18 @@ class NationalRiskIndexETL(ExtractTransformLoad):
         """
         logger.info("Transforming National Risk Index Data")
 
-        NRI_TRACT_COL = "TRACTFIPS"  # Census Tract Column in NRI data
-
         # read in the unzipped csv from NRI data source then rename the
         # Census Tract column for merging
         df_nri: pd.DataFrame = pd.read_csv(
             self.INPUT_CSV,
-            dtype={NRI_TRACT_COL: "string"},
+            dtype={self.INPUT_GEOID_TRACT_FIELD_NAME: "string"},
             na_values=["None"],
             low_memory=False,
         )
+
         df_nri.rename(
             columns={
-                NRI_TRACT_COL: self.GEOID_TRACT_FIELD_NAME,
+                self.INPUT_GEOID_TRACT_FIELD_NAME: self.GEOID_TRACT_FIELD_NAME,
                 self.RISK_INDEX_EXPECTED_ANNUAL_LOSS_SCORE_INPUT_FIELD_NAME: self.RISK_INDEX_EXPECTED_ANNUAL_LOSS_SCORE_FIELD_NAME,
             },
             inplace=True,
@@ -170,6 +152,7 @@ class NationalRiskIndexETL(ExtractTransformLoad):
         ].clip(
             lower=self.AGRIVALUE_LOWER_BOUND
         )
+
         # This produces a boolean that is True in the case of non-zero agricultural value
         df_nri[self.CONTAINS_AGRIVALUE] = (
             df_nri[self.AGRICULTURAL_VALUE_INPUT_FIELD_NAME] > 0
@@ -185,6 +168,7 @@ class NationalRiskIndexETL(ExtractTransformLoad):
         # Note: `round` is smart enough to only apply to float columns.
         df_nri = df_nri.round(10)
 
+        # Assign the final df to the class' output_df for the load method
         self.output_df = df_nri
 
     def load(self) -> None:

data/data-pipeline/data_pipeline/etl/sources/tribal/etl.py

@@ -81,13 +81,13 @@ class TribalETL(ExtractTransformLoad):
         bia_aian_supplemental_df = gpd.read_file(tribal_geojson_path)
 
         bia_aian_supplemental_df.drop(
-            ["OBJECTID", "GISAcres", "Source", "Shape_Length", "Shape_Area"],
+            ["GISAcres", "Source", "Shape_Length", "Shape_Area"],
             axis=1,
             inplace=True,
         )
 
         bia_aian_supplemental_df.rename(
-            columns={"Land_Area_": "landAreaName"},
+            columns={"OBJECTID": "tribalId", "Land_Area_": "landAreaName"},
             inplace=True,
         )
 

data/data-pipeline/data_pipeline/tests/sources/child_opportunity_index/data/extract.csv

@@ -0,0 +1,11 @@
+geoid,year,in100,msaid15,msaname15,countyfips,statefips,stateusps,pop,ED_APENR,ED_ATTAIN,ED_COLLEGE,ED_ECENROL,ED_HSGRAD,ED_MATH,ED_READING,ED_SCHPOV,ED_TEACHXP,ED_PRXECE,ED_PRXHQECE,HE_FOOD,HE_GREEN,HE_HEAT,HE_HLTHINS,HE_OZONE,HE_PM25,HE_VACANCY,HE_WALK,HE_SUPRFND,HE_RSEI,SE_POVRATE,SE_PUBLIC,SE_HOME,SE_OCC,SE_MHE,SE_EMPRAT,SE_JOBPROX,SE_SINGLE
+34023007402,2010,1,35620,"New York-Newark-Jersey City, NY-NJ-PA Metro Area",34023,34,NJ,750,.24224862,30.624092,,100,69.515396,263.57706,236.18246,27.868332,9.545229,4.3510399,1.0986123,3.9372432,29.1,10.666667,94.001648,40.704281,9.0991697,5.2830191,12.461183,.79894525,10.256244,3.0262272,.69044882,90.909096,43.792515,83043,83.874458,19.936565,15.571776
+36081033700,2015,1,35620,"New York-Newark-Jersey City, NY-NJ-PA Metro Area",36081,36,NY,567,.48479512,18.983355,,17.9,61.892601,220.9837,206.39664,72.493996,9.9560204,6.6488576,3.0244465,0,83.5,12.333333,76.111687,32.268497,9.9404745,15.087041,12.130791,-13.815511,6.071692,15.781346,11.366246,52.009182,23.888559,69967,84.174629,28.09762,44.444447
+36047001500,2010,1,35620,"New York-Newark-Jersey City, NY-NJ-PA Metro Area",36047,36,NY,675,.14864263,52.80785,,30,71.530525,185.22771,165.57692,67.353996,14.186028,6.9858408,4.08319,0,83.5,14.333333,84.398552,34.934345,10.56029,32.567932,13.031661,0,8.0978775,28.949436,17.139614,13.110103,66.617065,74066,72.894844,14.008394,58.309036
+48141000109,2010,1,21340,"El Paso, TX Metro Area",48141,48,TX,1131,.37170947,6.937799,38.419178,51.400002,87.275864,232.89197,200.59705,78.853096,14.936053,3.8554859,1.7132878,6.9115214,42.599998,106,59.259258,43.708778,8.8110294,4.1181045,5.9151278,-13.815511,8.7751179,35.326088,25.688818,63.533222,21.570066,33227,63.284718,7.3966646,41.100918
+29007950100,2010,0,33020,"Mexico, MO Micro Area",29007,29,MO,676,.037339322,6.8397722,14.420464,75.599998,90.849075,218.05362,201.03607,54.0625,6.4516129,.69314718,-13.815511,12.788422,5,38,83.116882,40.412949,10.307842,16.764277,8.6579781,-13.815511,3.9731553,16.500185,22.075638,72.471413,21.940929,37446,32.747749,2.5800712,26.517572
+06059032048,2015,1,31080,"Los Angeles-Long Beach-Anaheim, CA Metro Area",06059,06,CA,977,.49177772,55.322414,52.337311,0,94.574394,302.98648,301.14258,16.570803,2.6474135,4.409451,2.4849067,.0067947675,39.599998,4.3333335,93.603134,45.231636,11.814985,4.1246161,12.046153,-13.815511,7.9175715,5.0019093,.64073229,90.114418,51.413792,111061,91.015175,10.40153,13.380281
+06071006203,2015,1,40140,"Riverside-San Bernardino-Ontario, CA Metro Area",06071,06,CA,1721,.39206344,10.373445,37.920834,3.8,65.640205,70.388351,87.780144,95.931778,24.099722,4.5038018,.7687456,14.522655,44.599998,71.666664,88.938148,54.547478,10.972307,6.8592057,8.1460381,-13.815511,8.009738,36.548023,45.025841,26.29199,15.143603,31926,56.200939,10.074074,63.414635
+17031020802,2015,1,16980,"Chicago-Naperville-Elgin, IL-IN-WI Metro Area",17031,17,IL,1735,.62798566,32.851238,46.603329,12.7,80.669044,135.75914,176.19102,80.124969,15.568522,5.7649512,3.8511307,0,69,19.666666,73.180443,34.496658,12.666351,8.2152977,14.281524,-13.815511,10.020544,24.757034,30.288029,38.703873,30.388424,47207,74.281876,14.190094,37.284233
+55059001200,2015,1,16980,"Chicago-Naperville-Elgin, IL-IN-WI Metro Area",55059,55,WI,1170,.25817108,9.1463413,35.304192,26.700001,87.450645,149.80862,151.35675,72.033096,21.983971,4.2610936,-13.815511,3.1195433,69.400002,5.6666665,90.574654,37.383728,11.085336,16.062801,9.8132915,-13.815511,9.6796389,33.292171,34.388489,53.669064,20.963005,42031,67.647057,4.5596504,51.340996
+18097357600,2015,1,26900,"Indianapolis-Carmel-Anderson, IN Metro Area",18097,18,IN,1949,0,7.0523305,37.289841,76.800003,56.344406,137.10654,137.40863,82.021439,87.151176,3.9076374,2.5138195,6.7697067,37.400002,15,80.642738,36.055035,12.187755,14.801825,12.55805,-13.815511,8.2662792,28.008997,41.786743,46.191849,12.268188,28658,59.289223,2.8684118,64.355064

data/data-pipeline/data_pipeline/tests/sources/child_opportunity_index/data/output.csv

@@ -0,0 +1,16 @@
+GEOID10_TRACT,Summer days above 90F,Percent low access to healthy food,Percent impenetrable surface areas,Third grade reading proficiency
+15001021010,,0.0321765850,,132.7394000000
+15001021101,,0.0584802530,,152.4008000000
+15001021402,,0.0468876600,,138.3826600000
+15001021800,,0.0461176160,,186.3202800000
+15003010201,,0.0698271750,,203.2172200000
+15007040603,,0.0303736330,,192.8455400000
+15007040604,,0.0283296820,,183.0814800000
+15007040700,,0.0207271430,,179.8546400000
+15009030100,,0.0440458350,,158.1934500000
+15009030201,,0.0000000000,,183.1507700000
+15009030402,,0.0167432700,,165.8253500000
+15009030800,,0.0145107520,,189.8833900000
+06027000800,,0.0150000000,,200.0000000000
+06061021322,,0.0150000000,,200.0000000000
+06069000802,,0.0150000000,,200.0000000000

data/data-pipeline/data_pipeline/tests/sources/child_opportunity_index/data/raw.csv

@@ -0,0 +1,28 @@
+geoid,year,in100,msaid15,A,msaname15,countyfips,statefips,stateusps,pop,ED_APENR,ED_ATTAIN,ED_COLLEGE,ED_ECENROL,ED_HSGRAD,ED_MATH,ED_READING,ED_SCHPOV,ED_TEACHXP,ED_PRXECE,ED_PRXHQECE,HE_FOOD,HE_GREEN,HE_HEAT,HE_HLTHINS,HE_OZONE,HE_PM25,HE_VACANCY,HE_WALK,HE_SUPRFND,HE_RSEI,SE_POVRATE,SE_PUBLIC,SE_HOME,SE_OCC,SE_MHE,SE_EMPRAT,SE_JOBPROX,SE_SINGLE
+15001021010,2010,0,25900,1,"Hilo, HI Micro Area",15001,15,HI,1631,0.0736903030,19.5522060000,38.5793110000,82.0999980000,73.1855090000,204.0279700000,179.6740100000,83.3590090000,15.2101190000,1.2236612000,-3.2645943000,2.1897779000,,,85.4777530000,,,12.2256100000,1.9031702000,-13.8155110000,2.8101339000,31.1270680000,25.8038250000,70.6552660000,31.4786000000,39396,63.3027500000,8.8585281000,55.5727540000
+15001021010,2015,0,25900,2,"Hilo, HI Micro Area",15001,15,HI,1528,0.1899261800,20.9306340000,40.2508050000,58.0000000000,75.0283360000,169.2188100000,132.7394000000,76.8540880000,15.6538250000,1.0273454000,-5.3525348000,3.2176585000,,,91.0719530000,,,10.0154490000,1.9031702000,-13.8155110000,2.6162012000,24.8258710000,31.7182030000,78.6101380000,43.7814290000,30360,58.2020030000,5.0404000000,32.4159010000
+15001021101,2010,0,25900,3,"Hilo, HI Micro Area",15001,15,HI,535,0.0533180940,32.0215420000,39.8683090000,46.4000020000,70.3282170000,224.0678700000,174.5002100000,77.7506030000,13.0946430000,1.7718583000,-0.0300022310,4.3048000000,,,87.8565600000,,,11.0146500000,3.7242184000,-13.8155110000,2.2041564000,27.0347920000,27.0693510000,74.3475040000,34.1918300000,37855,61.0904580000,13.4843590000,35.0230410000
+15001021101,2015,0,25900,4,"Hilo, HI Micro Area",15001,15,HI,433,0.2015089800,39.5533680000,37.1644330000,43.7999990000,66.2127690000,152.8964500000,152.4008000000,86.2977980000,6.9529324000,1.3696530000,-0.0197537470,5.8480253000,,,87.3994670000,,,8.1250000000,3.7242184000,-13.8155110000,2.0477815000,28.9167980000,26.3636360000,76.8484800000,39.1722790000,30968,59.7030750000,24.3119280000,51.2953340000
+15001021402,2010,0,25900,5,"Hilo, HI Micro Area",15001,15,HI,929,0.1647190500,25.1356240000,8.7702494000,42.0000000000,77.7994690000,192.0977000000,182.9247600000,60.5234150000,13.9293730000,2.0509064000,-1.3752689000,4.3765292000,,,90.2309570000,,,4.5751634000,6.9091167000,-13.8155110000,-0.7224926400,9.1823902000,16.9319820000,53.1837920000,32.6673320000,52381,82.3242870000,6.3340807000,40.6818200000
+15001021402,2015,0,25900,6,"Hilo, HI Micro Area",15001,15,HI,825,0.2525457700,25.1578960000,29.2249470000,59.5000000000,73.7396470000,155.7005000000,138.3826600000,67.2758410000,15.3872220000,2.0137198000,-1.3752689000,4.6887660000,,,95.2972790000,,,4.8293624000,6.9091167000,-13.8155110000,-0.3878991900,11.4337100000,19.0618020000,65.3015590000,30.9687670000,61926,89.9835820000,3.8922157000,46.8085100000
+15001021800,2010,0,25900,7,"Hilo, HI Micro Area",15001,15,HI,1748,0.2083706300,19.7838120000,17.6447330000,34.7999990000,90.5556340000,178.7575700000,172.6336700000,60.7752300000,12.0777750000,0.6661295300,-0.0573484860,2.1838551000,,,92.4866940000,,,8.6256704000,5.2165289000,-13.8155110000,-1.9711585000,17.6570170000,13.7801600000,73.8337780000,21.2060700000,53017,74.5713120000,11.2527480000,30.1675970000
+15001021800,2015,0,25900,8,"Hilo, HI Micro Area",15001,15,HI,1399,0.1826577300,24.7472530000,26.1164550000,16.2999990000,76.7413410000,163.1518600000,186.3202800000,69.2822570000,2.4274187000,0.6304196700,-0.1245087700,4.6117616000,,,93.4117200000,,,13.6399060000,5.2165289000,-13.8155110000,-1.9774016000,14.3425040000,12.0707070000,68.7373730000,17.8891470000,71209,72.9902110000,14.1710300000,31.3364070000
+15003010201,2010,1,46520,9,"Urban Honolulu, HI Metro Area",15003,15,HI,1609,0.2816832100,24.8328270000,37.7788580000,50.5000000000,79.8380890000,221.4442300000,214.4257200000,68.6519700000,17.9115310000,1.7652657000,-13.8155110000,4.9072938000,,,91.1199340000,,,9.0680103000,10.0983580000,-13.8155110000,4.0513158000,18.6613730000,13.9044080000,52.2656750000,38.9115640000,69896,74.8474880000,24.4086570000,30.8544310000
+15003010201,2015,1,46520,10,"Urban Honolulu, HI Metro Area",15003,15,HI,1586,0.3262355900,31.2335190000,38.1880150000,53.0999980000,81.9343190000,263.1299700000,203.2172200000,58.4180180000,18.3911150000,1.8023838000,-0.2093867800,6.9827175000,,,90.3112640000,,,8.8028173000,10.0983580000,-13.8155110000,3.9325697000,9.5938101000,15.3647510000,51.4525490000,32.5877190000,72930,76.5251310000,32.7527810000,33.8129500000
+15007040603,2010,0,28180,11,"Kapaa, HI Micro Area",15007,15,HI,455,0.2938456200,32.7254300000,25.0950970000,42.5000000000,83.3289790000,239.3781600000,211.4160000000,44.8172530000,20.6196160000,1.9461737000,-13.8155110000,6.6860218000,,,91.4368590000,,,4.1306438000,6.2141290000,-13.8155110000,-2.5745666000,3.4628704000,10.9958510000,65.8713680000,31.5982400000,72065,79.6573870000,1.8644068000,29.7101440000
+15007040603,2015,0,28180,12,"Kapaa, HI Micro Area",15007,15,HI,710,0.3217573500,34.5935750000,26.9131070000,29.9000000000,83.9192810000,219.1101400000,192.8455400000,45.7621310000,18.8732510000,1.3572797000,-13.8155110000,3.0373633000,,,90.8629460000,,,5.7142859000,6.2141290000,-13.8155110000,-3.3995571000,9.5723677000,8.5572138000,55.7213900000,27.3037550000,70639,80.0589370000,1.0885341000,43.5294110000
+15007040604,2010,0,28180,13,"Kapaa, HI Micro Area",15007,15,HI,732,0.2915806200,24.5812400000,29.6286280000,24.0000000000,83.3070600000,236.0561400000,209.3688000000,42.8187870000,18.0469950000,1.9836926000,-13.8155110000,6.2032728000,,,92.2233960000,,,5.7585139000,6.4129410000,-13.8155110000,-2.5975277000,10.0240100000,11.3287910000,63.2027280000,33.0645180000,66262,80.3639140000,2.3361454000,29.1237110000
+15007040604,2015,0,28180,14,"Kapaa, HI Micro Area",15007,15,HI,639,0.3213492000,28.7210830000,21.3032890000,47.2000010000,84.0398100000,211.0873300000,183.0814800000,44.8632510000,16.4886360000,1.3866930000,-13.8155110000,2.8329682000,,,95.4563450000,,,8.3898830000,6.4129410000,-13.8155110000,-3.4551327000,18.2676660000,15.7275020000,56.6021840000,32.9291650000,69750,79.8629000000,2.5787966000,49.1289220000
+15007040700,2010,0,28180,15,"Kapaa, HI Micro Area",15007,15,HI,2011,0.1132875600,25.4790360000,25.8629530000,54.2999990000,83.0592730000,237.8249200000,206.2347100000,42.3221820000,13.7346280000,2.0951800000,-6.8926415000,4.2036662000,,,93.5513230000,,,9.0025101000,6.2517805000,-13.8155110000,-1.9463800000,15.2579730000,11.8912790000,66.1004100000,30.3776680000,68381,78.7573240000,1.8246027000,42.0395430000
+15007040700,2015,0,28180,16,"Kapaa, HI Micro Area",15007,15,HI,2101,0.2257254300,37.3111080000,21.7736360000,31.1000000000,83.5062480000,225.5370800000,179.8546400000,45.9002040000,13.3931940000,1.6256510000,-1.1560694000,2.0727143000,,,90.6197200000,,,9.1895151000,6.2517805000,-13.8155110000,-3.0142822000,7.5953355000,6.4549179000,67.2814180000,28.9595490000,82467,83.4867320000,4.3556356000,24.4994110000
+15009030100,2010,0,27980,17,"Kahului-Wailuku-Lahaina, HI Metro Area",15009,15,HI,535,0.0947706330,21.3217950000,20.9204140000,82.5000000000,87.5318760000,222.5835400000,190.5396400000,68.0386050000,14.1374330000,0.6221639500,-13.8155110000,0.4278838900,,,82.2360310000,,,11.7456890000,5.5044246000,-13.8155110000,1.1383853000,9.7967691000,7.9934745000,62.1533470000,23.3065430000,62972,67.7938840000,7.9391890000,40.9302330000
+15009030100,2015,0,27980,18,"Kahului-Wailuku-Lahaina, HI Metro Area",15009,15,HI,361,0.1140028300,25.3853150000,16.3010810000,80.0000000000,89.5757980000,163.8899400000,158.1934500000,74.5342790000,12.4208780000,0.6730875400,-0.0356756300,4.4045835000,,,92.7496570000,,,12.5000000000,5.5044246000,-13.8155110000,0.6983343400,10.9501740000,7.5901327000,65.8444060000,28.7472040000,66161,87.4015730000,10.1990050000,31.4960630000
+15009030201,2010,0,27980,19,"Kahului-Wailuku-Lahaina, HI Metro Area",15009,15,HI,333,0.3042595100,34.7983170000,27.3489970000,100.0000000000,77.2663500000,264.8783000000,235.5722700000,55.4464300000,9.2995071000,2.1566634000,-0.2553466900,1.5144995000,,,75.7439580000,,,9.2709274000,3.4708600000,-13.8155110000,1.4976557000,10.4116220000,10.0529100000,67.5132290000,46.3436930000,57163,78.9054720000,7.2706938000,37.2881360000
+15009030201,2015,0,27980,20,"Kahului-Wailuku-Lahaina, HI Metro Area",15009,15,HI,304,0.3610763000,23.8664990000,27.8072380000,100.0000000000,81.1670530000,213.0963400000,183.1507700000,58.9944080000,6.7822700000,2.2015967000,0.0413851890,0.0000000000,,,90.6270220000,,,3.1697340000,3.4708600000,-13.8155110000,1.0144377000,10.5798570000,8.5106382000,68.0851060000,36.8266410000,61048,70.6093220000,4.2774568000,49.1525420000
+15009030402,2010,0,27980,21,"Kahului-Wailuku-Lahaina, HI Metro Area",15009,15,HI,1893,0.3215200000,24.1058310000,22.8293340000,86.3000030000,74.7603760000,273.5332000000,238.2146800000,54.6146470000,10.3586990000,2.8028412000,0.0000000000,1.6413509000,,,85.0915530000,,,7.9121537000,7.3098612000,-13.8155110000,1.9970932000,2.7091255000,6.3772955000,62.7712860000,35.3862000000,81832,89.7885360000,7.0808282000,38.4026260000
+15009030402,2015,0,27980,22,"Kahului-Wailuku-Lahaina, HI Metro Area",15009,15,HI,1685,0.3768204500,25.8798220000,34.9604720000,69.9000020000,79.8403630000,208.5855100000,165.8253500000,56.9929960000,12.2689750000,2.8912144000,0.6931471800,1.6743270000,,,94.9152530000,,,3.1778426000,7.3098612000,-13.8155110000,1.4903787000,7.6546302000,5.0015731000,69.9591060000,27.6414090000,80619,85.7456740000,4.9374290000,27.2051010000
+15009030800,2010,0,27980,23,"Kahului-Wailuku-Lahaina, HI Metro Area",15009,15,HI,1725,0.2588196400,25.8761410000,25.7107850000,59.5999980000,79.6580120000,222.2627900000,185.5475600000,50.3295820000,15.1613000000,3.4776812000,1.7540390000,1.0601417000,,,91.7269360000,,,6.5111756000,4.7154179000,-13.8155110000,4.9853492000,1.8906569000,8.0818968000,83.3512880000,34.2178340000,96516,83.9158250000,4.3561821000,30.5000000000
+15009030800,2015,0,27980,24,"Kahului-Wailuku-Lahaina, HI Metro Area",15009,15,HI,2051,0.3132624300,28.6977020000,25.2963310000,58.4000020000,78.9907610000,184.0429400000,189.8833900000,50.4299890000,8.5740423000,3.5176146000,2.1515093000,1.4510752000,,,95.4700240000,,,2.7706735000,4.7154179000,-13.8155110000,4.4182396000,5.1701570000,5.6275120000,82.7601620000,33.0580600000,100117,85.9390640000,3.3368926000,23.8437820000
+6027000800,2015,0,25900,98,"Made up, Micro Area",704,6,CA,12,0.2000000000,25.0000000000,25.0000000000,25.0000000000,190.0000000000,200.0000000000,200.0000000000,50.0000000000,7.0000000000,1.5000000000,1.5000000000,1.5000000000,,,90.1000000000,,,3.1000000000,3.1000000000,-13.4400000000,4.4400000000,5.1700000000,5.6200000000,72.8000000000,33.5000000000,33333,58.0100000000,9.3300000000,32.3300000000
+6061021322,2015,0,25900,87,"Made up, Micro Area",102,6,CA,12,0.2000000000,25.0000000000,25.0000000000,25.0000000000,190.0000000000,200.0000000000,200.0000000000,50.0000000000,7.0000000000,1.5000000000,1.5000000000,1.5000000000,,,90.1000000000,,,3.1000000000,3.1000000000,-13.4400000000,4.4400000000,5.1700000000,5.6200000000,72.8000000000,33.5000000000,33333,58.0100000000,9.3300000000,32.3300000000
+6069000802,2015,0,25900,84,"Made up, Micro Area",704,6,CA,12,0.2000000000,25.0000000000,25.0000000000,25.0000000000,190.0000000000,200.0000000000,200.0000000000,50.0000000000,7.0000000000,1.5000000000,1.5000000000,1.5000000000,,,90.1000000000,,,3.1000000000,3.1000000000,-13.4400000000,4.4400000000,5.1700000000,5.6200000000,72.8000000000,33.5000000000,33333,58.0100000000,9.3300000000,32.3300000000

data/data-pipeline/data_pipeline/tests/sources/child_opportunity_index/data/transform.csv

@@ -0,0 +1,16 @@
+GEOID10_TRACT,year,in100,msaid15,A,msaname15,countyfips,statefips,stateusps,pop,ED_APENR,ED_ATTAIN,ED_COLLEGE,ED_ECENROL,ED_HSGRAD,ED_MATH,Third grade reading proficiency,ED_SCHPOV,ED_TEACHXP,ED_PRXECE,ED_PRXHQECE,Percent low access to healthy food,Percent impenetrable surface areas,Summer days above 90F,HE_HLTHINS,HE_OZONE,HE_PM25,HE_VACANCY,HE_WALK,HE_SUPRFND,HE_RSEI,SE_POVRATE,SE_PUBLIC,SE_HOME,SE_OCC,SE_MHE,SE_EMPRAT,SE_JOBPROX,SE_SINGLE
+15001021010,2015,0,25900,2,"Hilo, HI Micro Area",15001,15,HI,1528,0.1899261800,20.9306340000,40.2508050000,58.0000000000,75.0283360000,169.2188100000,132.7394000000,76.8540880000,15.6538250000,1.0273454000,-5.3525348000,0.0321765850,,,91.0719530000,,,10.0154490000,1.9031702000,-13.8155110000,2.6162012000,24.8258710000,31.7182030000,78.6101380000,43.7814290000,30360,58.2020030000,5.0404000000,32.4159010000
+15001021101,2015,0,25900,4,"Hilo, HI Micro Area",15001,15,HI,433,0.2015089800,39.5533680000,37.1644330000,43.7999990000,66.2127690000,152.8964500000,152.4008000000,86.2977980000,6.9529324000,1.3696530000,-0.0197537470,0.0584802530,,,87.3994670000,,,8.1250000000,3.7242184000,-13.8155110000,2.0477815000,28.9167980000,26.3636360000,76.8484800000,39.1722790000,30968,59.7030750000,24.3119280000,51.2953340000
+15001021402,2015,0,25900,6,"Hilo, HI Micro Area",15001,15,HI,825,0.2525457700,25.1578960000,29.2249470000,59.5000000000,73.7396470000,155.7005000000,138.3826600000,67.2758410000,15.3872220000,2.0137198000,-1.3752689000,0.0468876600,,,95.2972790000,,,4.8293624000,6.9091167000,-13.8155110000,-0.3878991900,11.4337100000,19.0618020000,65.3015590000,30.9687670000,61926,89.9835820000,3.8922157000,46.8085100000
+15001021800,2015,0,25900,8,"Hilo, HI Micro Area",15001,15,HI,1399,0.1826577300,24.7472530000,26.1164550000,16.2999990000,76.7413410000,163.1518600000,186.3202800000,69.2822570000,2.4274187000,0.6304196700,-0.1245087700,0.0461176160,,,93.4117200000,,,13.6399060000,5.2165289000,-13.8155110000,-1.9774016000,14.3425040000,12.0707070000,68.7373730000,17.8891470000,71209,72.9902110000,14.1710300000,31.3364070000
+15003010201,2015,1,46520,10,"Urban Honolulu, HI Metro Area",15003,15,HI,1586,0.3262355900,31.2335190000,38.1880150000,53.0999980000,81.9343190000,263.1299700000,203.2172200000,58.4180180000,18.3911150000,1.8023838000,-0.2093867800,0.0698271750,,,90.3112640000,,,8.8028173000,10.0983580000,-13.8155110000,3.9325697000,9.5938101000,15.3647510000,51.4525490000,32.5877190000,72930,76.5251310000,32.7527810000,33.8129500000
+15007040603,2015,0,28180,12,"Kapaa, HI Micro Area",15007,15,HI,710,0.3217573500,34.5935750000,26.9131070000,29.9000000000,83.9192810000,219.1101400000,192.8455400000,45.7621310000,18.8732510000,1.3572797000,-13.8155110000,0.0303736330,,,90.8629460000,,,5.7142859000,6.2141290000,-13.8155110000,-3.3995571000,9.5723677000,8.5572138000,55.7213900000,27.3037550000,70639,80.0589370000,1.0885341000,43.5294110000
+15007040604,2015,0,28180,14,"Kapaa, HI Micro Area",15007,15,HI,639,0.3213492000,28.7210830000,21.3032890000,47.2000010000,84.0398100000,211.0873300000,183.0814800000,44.8632510000,16.4886360000,1.3866930000,-13.8155110000,0.0283296820,,,95.4563450000,,,8.3898830000,6.4129410000,-13.8155110000,-3.4551327000,18.2676660000,15.7275020000,56.6021840000,32.9291650000,69750,79.8629000000,2.5787966000,49.1289220000
+15007040700,2015,0,28180,16,"Kapaa, HI Micro Area",15007,15,HI,2101,0.2257254300,37.3111080000,21.7736360000,31.1000000000,83.5062480000,225.5370800000,179.8546400000,45.9002040000,13.3931940000,1.6256510000,-1.1560694000,0.0207271430,,,90.6197200000,,,9.1895151000,6.2517805000,-13.8155110000,-3.0142822000,7.5953355000,6.4549179000,67.2814180000,28.9595490000,82467,83.4867320000,4.3556356000,24.4994110000
+15009030100,2015,0,27980,18,"Kahului-Wailuku-Lahaina, HI Metro Area",15009,15,HI,361,0.1140028300,25.3853150000,16.3010810000,80.0000000000,89.5757980000,163.8899400000,158.1934500000,74.5342790000,12.4208780000,0.6730875400,-0.0356756300,0.0440458350,,,92.7496570000,,,12.5000000000,5.5044246000,-13.8155110000,0.6983343400,10.9501740000,7.5901327000,65.8444060000,28.7472040000,66161,87.4015730000,10.1990050000,31.4960630000
+15009030201,2015,0,27980,20,"Kahului-Wailuku-Lahaina, HI Metro Area",15009,15,HI,304,0.3610763000,23.8664990000,27.8072380000,100.0000000000,81.1670530000,213.0963400000,183.1507700000,58.9944080000,6.7822700000,2.2015967000,0.0413851890,0.0000000000,,,90.6270220000,,,3.1697340000,3.4708600000,-13.8155110000,1.0144377000,10.5798570000,8.5106382000,68.0851060000,36.8266410000,61048,70.6093220000,4.2774568000,49.1525420000
+15009030402,2015,0,27980,22,"Kahului-Wailuku-Lahaina, HI Metro Area",15009,15,HI,1685,0.3768204500,25.8798220000,34.9604720000,69.9000020000,79.8403630000,208.5855100000,165.8253500000,56.9929960000,12.2689750000,2.8912144000,0.6931471800,0.0167432700,,,94.9152530000,,,3.1778426000,7.3098612000,-13.8155110000,1.4903787000,7.6546302000,5.0015731000,69.9591060000,27.6414090000,80619,85.7456740000,4.9374290000,27.2051010000
+15009030800,2015,0,27980,24,"Kahului-Wailuku-Lahaina, HI Metro Area",15009,15,HI,2051,0.3132624300,28.6977020000,25.2963310000,58.4000020000,78.9907610000,184.0429400000,189.8833900000,50.4299890000,8.5740423000,3.5176146000,2.1515093000,0.0145107520,,,95.4700240000,,,2.7706735000,4.7154179000,-13.8155110000,4.4182396000,5.1701570000,5.6275120000,82.7601620000,33.0580600000,100117,85.9390640000,3.3368926000,23.8437820000
+06027000800,2015,0,25900,98,"Made up, Micro Area",704,6,CA,12,0.2000000000,25.0000000000,25.0000000000,25.0000000000,190.0000000000,200.0000000000,200.0000000000,50.0000000000,7.0000000000,1.5000000000,1.5000000000,0.0150000000,,,90.1000000000,,,3.1000000000,3.1000000000,-13.4400000000,4.4400000000,5.1700000000,5.6200000000,72.8000000000,33.5000000000,33333,58.0100000000,9.3300000000,32.3300000000
+06061021322,2015,0,25900,87,"Made up, Micro Area",102,6,CA,12,0.2000000000,25.0000000000,25.0000000000,25.0000000000,190.0000000000,200.0000000000,200.0000000000,50.0000000000,7.0000000000,1.5000000000,1.5000000000,0.0150000000,,,90.1000000000,,,3.1000000000,3.1000000000,-13.4400000000,4.4400000000,5.1700000000,5.6200000000,72.8000000000,33.5000000000,33333,58.0100000000,9.3300000000,32.3300000000
+06069000802,2015,0,25900,84,"Made up, Micro Area",704,6,CA,12,0.2000000000,25.0000000000,25.0000000000,25.0000000000,190.0000000000,200.0000000000,200.0000000000,50.0000000000,7.0000000000,1.5000000000,1.5000000000,0.0150000000,,,90.1000000000,,,3.1000000000,3.1000000000,-13.4400000000,4.4400000000,5.1700000000,5.6200000000,72.8000000000,33.5000000000,33333,58.0100000000,9.3300000000,32.3300000000

data/data-pipeline/data_pipeline/tests/sources/child_opportunity_index/test_etl.py

@@ -0,0 +1,69 @@
+# pylint: disable=protected-access
+import pathlib
+
+from data_pipeline.etl.sources.child_opportunity_index.etl import (
+    ChildOpportunityIndex,
+)
+from data_pipeline.tests.sources.example.test_etl import TestETL
+from data_pipeline.utils import get_module_logger
+
+logger = get_module_logger(__name__)
+
+
+class TestChildOpportunityIndexETL(TestETL):
+    """Tests the COI ETL.
+
+    This uses pytest-snapshot.
+    To update individual snapshots: $ poetry run pytest
+            data_pipeline/tests/sources/child_opportunity_index/test_etl.py::TestClassNameETL::<testname>
+            --snapshot-update
+    """
+
+    _ETL_CLASS = ChildOpportunityIndex
+
+    _SAMPLE_DATA_PATH = pathlib.Path(__file__).parents[0] / "data"
+    _SAMPLE_DATA_FILE_NAME = "raw.csv"
+    _SAMPLE_DATA_ZIP_FILE_NAME = "coi.zip"
+    _EXTRACT_TMP_FOLDER_NAME = "ChildOpportunityIndex"
+    _EXTRACT_CSV_FILE_NAME = "raw.csv"
+
+    def setup_method(self, _method, filename=__file__):
+        """Invoke `setup_method` from Parent, but using the current file name.
+
+        This code can be copied identically between all child classes.
+        """
+        super().setup_method(_method=_method, filename=filename)
+
+    def test_init(self, mock_etl, mock_paths):
+        """Tests that the ChildOpportunityIndexETL class was initialized
+        correctly.
+        """
+
+        etl = ChildOpportunityIndex()
+        data_path, _ = mock_paths
+        assert etl.DATA_PATH == data_path
+        assert etl.COLUMNS_TO_KEEP == [
+            "GEOID10_TRACT",
+            "Summer days above 90F",
+            "Percent low access to healthy food",
+            "Percent impenetrable surface areas",
+            "Third grade reading proficiency",
+        ]
+        assert etl.GEOID_FIELD_NAME == "GEOID10"
+        assert etl.GEOID_TRACT_FIELD_NAME == "GEOID10_TRACT"
+        assert etl.TRACT_INPUT_COLUMN_NAME == "geoid"
+        assert etl.EXTREME_HEAT_INPUT_FIELD == "HE_HEAT"
+        assert etl.HEALTHY_FOOD_INPUT_FIELD == "HE_FOOD"
+        assert etl.IMPENETRABLE_SURFACES_INPUT_FIELD == "HE_GREEN"
+        assert etl.READING_INPUT_FIELD == "ED_READING"
+
+    def test_get_output_file_path(self, mock_etl, mock_paths):
+        """Tests the right file name is returned."""
+        etl = self._ETL_CLASS()
+        data_path, tmp_path = mock_paths
+
+        output_file_path = etl._get_output_file_path()
+        expected_output_file_path = (
+            data_path / "dataset" / "child_opportunity_index" / "usa.csv"
+        )
+        assert output_file_path == expected_output_file_path

data/data-pipeline/data_pipeline/tests/sources/doe_energy_burden/data/extract.csv

@@ -0,0 +1,16 @@
+ABV,FIP,BURDEN,QUANTILE
+HI,15001021010,0.0380000000,30
+HI,15001021101,0.0410000000,25
+HI,15001021402,0.0240000000,66
+HI,15001021800,0.0290000000,51
+HI,15003010201,0.0270000000,58
+HI,15007040603,0.0440000000,21
+HI,15007040604,0.0330000000,40
+HI,15007040700,0.0260000000,59
+HI,15009030100,0.0350000000,37
+HI,15009030201,0.0220000000,71
+HI,15009030402,0.0200000000,75
+HI,15009030800,0.0190000000,80
+CA,6069000802,0.2000000000,70
+CA,6061021322,0.5000000000,50
+CA,6027000800,0.1990000000,30

data/data-pipeline/data_pipeline/tests/sources/doe_energy_burden/data/output.csv

@@ -0,0 +1,16 @@
+GEOID10_TRACT,Energy burden
+15001021010,0.0380000000
+15001021101,0.0410000000
+15001021402,0.0240000000
+15001021800,0.0290000000
+15003010201,0.0270000000
+15007040603,0.0440000000
+15007040604,0.0330000000
+15007040700,0.0260000000
+15009030100,0.0350000000
+15009030201,0.0220000000
+15009030402,0.0200000000
+15009030800,0.0190000000
+06069000802,0.2000000000
+06061021322,0.5000000000
+06027000800,0.1990000000

data/data-pipeline/data_pipeline/tests/sources/doe_energy_burden/data/transform.csv

@@ -0,0 +1,16 @@
+ABV,GEOID10_TRACT,Energy burden,QUANTILE
+HI,15001021010,0.0380000000,30
+HI,15001021101,0.0410000000,25
+HI,15001021402,0.0240000000,66
+HI,15001021800,0.0290000000,51
+HI,15003010201,0.0270000000,58
+HI,15007040603,0.0440000000,21
+HI,15007040604,0.0330000000,40
+HI,15007040700,0.0260000000,59
+HI,15009030100,0.0350000000,37
+HI,15009030201,0.0220000000,71
+HI,15009030402,0.0200000000,75
+HI,15009030800,0.0190000000,80
+CA,06069000802,0.2000000000,70
+CA,06061021322,0.5000000000,50
+CA,06027000800,0.1990000000,30

data/data-pipeline/data_pipeline/tests/sources/doe_energy_burden/test_etl.py

@@ -0,0 +1,61 @@
+# pylint: disable=protected-access
+import pathlib
+
+from data_pipeline.etl.sources.doe_energy_burden.etl import (
+    DOEEnergyBurden,
+)
+from data_pipeline.tests.sources.example.test_etl import TestETL
+from data_pipeline.utils import get_module_logger
+
+logger = get_module_logger(__name__)
+
+
+class TestDOEEnergyBurdenETL(TestETL):
+    """Tests the COI ETL.
+
+    This uses pytest-snapshot.
+    To update individual snapshots: $ poetry run pytest
+            data_pipeline/tests/sources/doe_energy_burden/test_etl.py::TestClassNameETL::<testname>
+            --snapshot-update
+    """
+
+    _ETL_CLASS = DOEEnergyBurden
+
+    _SAMPLE_DATA_PATH = pathlib.Path(__file__).parents[0] / "data"
+    _SAMPLE_DATA_FILE_NAME = "DOE_LEAD_AMI_TRACT_2018_ALL.csv"
+    _SAMPLE_DATA_ZIP_FILE_NAME = "DOE_LEAD_AMI_TRACT_2018_ALL.csv.zip"
+    _EXTRACT_TMP_FOLDER_NAME = "DOEEnergyBurden"
+    _EXTRACT_CSV_FILE_NAME = "extract.csv"
+
+    def setup_method(self, _method, filename=__file__):
+        """Invoke `setup_method` from Parent, but using the current file name.
+
+        This code can be copied identically between all child classes.
+        """
+        super().setup_method(_method=_method, filename=filename)
+
+    def test_init(self, mock_etl, mock_paths):
+        """Tests that the ChildOpportunityIndexETL class was initialized
+        correctly.
+        """
+
+        etl = DOEEnergyBurden()
+        data_path, _ = mock_paths
+        assert etl.DATA_PATH == data_path
+        assert etl.COLUMNS_TO_KEEP == ["GEOID10_TRACT", "Energy burden"]
+        assert etl.GEOID_FIELD_NAME == "GEOID10"
+        assert etl.GEOID_TRACT_FIELD_NAME == "GEOID10_TRACT"
+        assert etl.INPUT_GEOID_TRACT_FIELD_NAME == "FIP"
+        assert etl.INPUT_ENERGY_BURDEN_FIELD_NAME == "BURDEN"
+        assert etl.REVISED_ENERGY_BURDEN_FIELD_NAME == "Energy burden"
+
+    def test_get_output_file_path(self, mock_etl, mock_paths):
+        """Tests the right file name is returned."""
+        etl = self._ETL_CLASS()
+        data_path, tmp_path = mock_paths
+
+        output_file_path = etl._get_output_file_path()
+        expected_output_file_path = (
+            data_path / "dataset" / "doe_energy_burden" / "usa.csv"
+        )
+        assert output_file_path == expected_output_file_path

data/data-pipeline/data_pipeline/tests/sources/example/data/extract.csv

@@ -1,7 +1,7 @@
 GEOID10_TRACT,Input Field 1
-06007040300,2.0000000000
+06027000800,2.0000000000
-06001020100,6.1000000000
+06069000802,6.1000000000
-06007040500,-7.8000000000
+06061021322,-7.8000000000
 15001021010,12.0000000000
 15001021101,12.0552478300
 15007040603,13.5141757800

data/data-pipeline/data_pipeline/tests/sources/example/data/output.csv

@@ -1,7 +1,7 @@
 GEOID10_TRACT,Example Field 1
-06007040300,4.0000000000
+06027000800,4.0000000000
-06001020100,12.2000000000
+06069000802,12.2000000000
-06007040500,-15.6000000000
+06061021322,-15.6000000000
 15001021010,24.0000000000
 15001021101,24.1104956600
 15007040603,27.0283515600

data/data-pipeline/data_pipeline/tests/sources/example/data/transform.csv

@@ -1,7 +1,7 @@
 GEOID10_TRACT,Input Field 1,Example Field 1
-06007040300,2.0000000000,4.0000000000
+06027000800,2.0000000000,4.0000000000
-06001020100,6.1000000000,12.2000000000
+06069000802,6.1000000000,12.2000000000
-06007040500,-7.8000000000,-15.6000000000
+06061021322,-7.8000000000,-15.6000000000
 15001021010,12.0000000000,24.0000000000
 15001021101,12.0552478300,24.1104956600
 15007040603,13.5141757800,27.0283515600

data/data-pipeline/data_pipeline/tests/sources/example/test_etl.py

@@ -3,8 +3,10 @@ import copy
 import os
 import pathlib
 from typing import Type
+from unittest import mock
 import pytest
 
+import requests
 import numpy as np
 import pandas as pd
 
@@ -47,9 +49,9 @@ class TestETL:
     # we use the same tract IDs across fixtures.
     # The test fixtures may also contain other tract IDs that are not on this list.
     _FIXTURES_SHARED_TRACT_IDS = [
-        "06007040300",
+        "06027000800",
-        "06001020100",
+        "06069000802",
-        "06007040500",
+        "06061021322",
         "15001021010",
         "15001021101",
         "15007040603",
@@ -98,18 +100,32 @@ class TestETL:
         In order to re-implement this method, usually it will involve a
         decent amount of work to monkeypatch `requests` or another method that's
         used to retrieve data in order to force that method to retrieve the fixture
-        data.
+        data. A basic version of that patching is included here for classes that can use it.
         """
-        # When running this in child classes, make sure the child class re-implements
+        with mock.patch("data_pipeline.utils.requests") as requests_mock:
-        # this method.
+            zip_file_fixture_src = (
-        if self._ETL_CLASS is not ExampleETL:
+                self._DATA_DIRECTORY_FOR_TEST / self._SAMPLE_DATA_ZIP_FILE_NAME
-            raise NotImplementedError(
-                "Prepare and run extract method not defined for this class."
             )
+            tmp_path = mock_paths[1]
 
-        # The rest of this method applies for `ExampleETL` only.
+            # Create mock response.
-        etl = self._get_instance_of_etl_class()
+            with open(zip_file_fixture_src, mode="rb") as file:
-        etl.extract()
+                file_contents = file.read()
+            response_mock = requests.Response()
+            response_mock.status_code = 200
+            # pylint: disable=protected-access
+            response_mock._content = file_contents
+            # Return text fixture:
+            requests_mock.get = mock.MagicMock(return_value=response_mock)
+
+            # Instantiate the ETL class.
+            etl = self._ETL_CLASS()
+
+            # Monkey-patch the temporary directory to the one used in the test
+            etl.TMP_PATH = tmp_path
+
+            # Run the extract method.
+            etl.extract()
 
         return etl
 
@@ -119,6 +135,8 @@ class TestETL:
         """
         # Setup
         etl = self._get_instance_of_etl_class()
+        etl.__init__()
+
         data_path, tmp_path = mock_paths
 
         assert etl.DATA_PATH == data_path
@@ -126,8 +144,6 @@ class TestETL:
 
         # Also make sure all parameters that need to be non-null are non-null
         assert etl.NAME is not None
-        assert etl.LAST_UPDATED_YEAR is not None
-        assert etl.SOURCE_URL is not None
         assert etl.GEO_LEVEL is not None
         assert etl.COLUMNS_TO_KEEP is not None
         assert len(etl.COLUMNS_TO_KEEP) > 0
@@ -148,14 +164,10 @@ class TestETL:
         etl = self._get_instance_of_etl_class()
         data_path, tmp_path = mock_paths
 
+        etl.__init__()
         actual_file_path = etl._get_output_file_path()
 
-        expected_file_path = (
+        expected_file_path = data_path / "dataset" / etl.NAME / "usa.csv"
-            data_path
-            / "dataset"
-            / f"{etl.NAME}_{etl.LAST_UPDATED_YEAR}"
-            / "usa.csv"
-        )
 
         logger.info(f"Expected: {expected_file_path}")
 
@@ -255,6 +267,7 @@ class TestETL:
         etl = self._setup_etl_instance_and_run_extract(
             mock_etl=mock_etl, mock_paths=mock_paths
         )
+        etl.__init__()
         etl.transform()
 
         assert etl.output_df is not None
@@ -272,6 +285,7 @@ class TestETL:
         """
         # setup - input variables
         etl = self._get_instance_of_etl_class()
+        etl.__init__()
 
         # setup - mock transform step
         df_transform = pd.read_csv(
@@ -370,9 +384,14 @@ class TestETL:
             etl_with_duplicate_geo_field.output_df = actual_output_df.copy(
                 deep=True
             )
+            etl_with_duplicate_geo_field.output_df.reset_index(inplace=True)
             etl_with_duplicate_geo_field.output_df.loc[
                 0:1, ExtractTransformLoad.GEOID_TRACT_FIELD_NAME
-            ] = "06007040300"
+            ] = etl_with_duplicate_geo_field.output_df[
+                ExtractTransformLoad.GEOID_TRACT_FIELD_NAME
+            ].iloc[
+                0
+            ]
             with pytest.raises(ValueError) as error:
                 etl_with_duplicate_geo_field.validate()
             assert str(error.value).startswith("Duplicate values:")
@@ -443,7 +462,7 @@ class TestETL:
 
         # Remove another column to keep and make sure error occurs.
         etl_with_missing_column = copy.deepcopy(etl)
-        columns_to_keep = actual_output_df.columns[:-1]
+        columns_to_keep = etl.COLUMNS_TO_KEEP[:-1]
         etl_with_missing_column.output_df = actual_output_df[columns_to_keep]
         with pytest.raises(ValueError) as error:
             etl_with_missing_column.validate()

data/data-pipeline/data_pipeline/tests/sources/national_risk_index/data/extract.csv

@@ -1,7 +1,7 @@
 OID_,NRI_ID,STATE,STATEABBRV,STATEFIPS,COUNTY,COUNTYTYPE,COUNTYFIPS,STCOFIPS,TRACT,TRACTFIPS,POPULATION,BUILDVALUE,AGRIVALUE,AREA,RISK_SCORE,RISK_RATNG,RISK_NPCTL,RISK_SPCTL,EAL_SCORE,EAL_RATNG,EAL_NPCTL,EAL_SPCTL,EAL_VALT,EAL_VALB,EAL_VALP,EAL_VALPE,EAL_VALA,SOVI_SCORE,SOVI_RATNG,SOVI_NPCTL,SOVI_SPCTL,SOVI_VALUE,RESL_SCORE,RESL_RATNG,RESL_NPCTL,RESL_SPCTL,RESL_VALUE,AVLN_EVNTS,AVLN_AFREQ,AVLN_EXPB,AVLN_EXPP,AVLN_EXPPE,AVLN_EXPT,AVLN_HLRB,AVLN_HLRP,AVLN_HLRR,AVLN_EALB,AVLN_EALP,AVLN_EALPE,AVLN_EALT,AVLN_EALS,AVLN_EALR,AVLN_RISKS,AVLN_RISKR,CFLD_EVNTS,CFLD_AFREQ,CFLD_EXPB,CFLD_EXPP,CFLD_EXPPE,CFLD_EXPT,CFLD_HLRB,CFLD_HLRP,CFLD_HLRR,CFLD_EALB,CFLD_EALP,CFLD_EALPE,CFLD_EALT,CFLD_EALS,CFLD_EALR,CFLD_RISKS,CFLD_RISKR,CWAV_EVNTS,CWAV_AFREQ,CWAV_EXPB,CWAV_EXPP,CWAV_EXPPE,CWAV_EXPA,CWAV_EXPT,CWAV_HLRB,CWAV_HLRP,CWAV_HLRA,CWAV_HLRR,CWAV_EALB,CWAV_EALP,CWAV_EALPE,CWAV_EALA,CWAV_EALT,CWAV_EALS,CWAV_EALR,CWAV_RISKS,CWAV_RISKR,DRGT_EVNTS,DRGT_AFREQ,DRGT_EXPA,DRGT_EXPT,DRGT_HLRA,DRGT_HLRR,DRGT_EALA,DRGT_EALT,DRGT_EALS,DRGT_EALR,DRGT_RISKS,DRGT_RISKR,ERQK_EVNTS,ERQK_AFREQ,ERQK_EXPB,ERQK_EXPP,ERQK_EXPPE,ERQK_EXPT,ERQK_HLRB,ERQK_HLRP,ERQK_HLRR,ERQK_EALB,ERQK_EALP,ERQK_EALPE,ERQK_EALT,ERQK_EALS,ERQK_EALR,ERQK_RISKS,ERQK_RISKR,HAIL_EVNTS,HAIL_AFREQ,HAIL_EXPB,HAIL_EXPP,HAIL_EXPPE,HAIL_EXPA,HAIL_EXPT,HAIL_HLRB,HAIL_HLRP,HAIL_HLRA,HAIL_HLRR,HAIL_EALB,HAIL_EALP,HAIL_EALPE,HAIL_EALA,HAIL_EALT,HAIL_EALS,HAIL_EALR,HAIL_RISKS,HAIL_RISKR,HWAV_EVNTS,HWAV_AFREQ,HWAV_EXPB,HWAV_EXPP,HWAV_EXPPE,HWAV_EXPA,HWAV_EXPT,HWAV_HLRB,HWAV_HLRP,HWAV_HLRA,HWAV_HLRR,HWAV_EALB,HWAV_EALP,HWAV_EALPE,HWAV_EALA,HWAV_EALT,HWAV_EALS,HWAV_EALR,HWAV_RISKS,HWAV_RISKR,HRCN_EVNTS,HRCN_AFREQ,HRCN_EXPB,HRCN_EXPP,HRCN_EXPPE,HRCN_EXPA,HRCN_EXPT,HRCN_HLRB,HRCN_HLRP,HRCN_HLRA,HRCN_HLRR,HRCN_EALB,HRCN_EALP,HRCN_EALPE,HRCN_EALA,HRCN_EALT,HRCN_EALS,HRCN_EALR,HRCN_RISKS,HRCN_RISKR,ISTM_EVNTS,ISTM_AFREQ,ISTM_EXPB,ISTM_EXPP,ISTM_EXPPE,ISTM_EXPT,ISTM_HLRB,ISTM_HLRP,ISTM_HLRR,ISTM_EALB,ISTM_EALP,ISTM_EALPE,ISTM_EALT,ISTM_EALS,ISTM_EALR,ISTM_RISKS,ISTM_RISKR,LNDS_EVNTS,LNDS_AFREQ,LNDS_EXPB,LNDS_EXPP,LNDS_EXPPE,LNDS_EXPT,LNDS_HLRB,LNDS_HLRP,LNDS_HLRR,LNDS_EALB,LNDS_EALP,LNDS_EALPE,LNDS_EALT,LNDS_EALS,LNDS_EALR,LNDS_RISKS,LNDS_RISKR,LTNG_EVNTS,LTNG_AFREQ,LTNG_EXPB,LTNG_EXPP,LTNG_EXPPE,LTNG_EXPT,LTNG_HLRB,LTNG_HLRP,LTNG_HLRR,LTNG_EALB,LTNG_EALP,LTNG_EALPE,LTNG_EALT,LTNG_EALS,LTNG_EALR,LTNG_RISKS,LTNG_RISKR,RFLD_EVNTS,RFLD_AFREQ,RFLD_EXPB,RFLD_EXPP,RFLD_EXPPE,RFLD_EXPA,RFLD_EXPT,RFLD_HLRB,RFLD_HLRP,RFLD_HLRA,RFLD_HLRR,RFLD_EALB,RFLD_EALP,RFLD_EALPE,RFLD_EALA,RFLD_EALT,RFLD_EALS,RFLD_EALR,RFLD_RISKS,RFLD_RISKR,SWND_EVNTS,SWND_AFREQ,SWND_EXPB,SWND_EXPP,SWND_EXPPE,SWND_EXPA,SWND_EXPT,SWND_HLRB,SWND_HLRP,SWND_HLRA,SWND_HLRR,SWND_EALB,SWND_EALP,SWND_EALPE,SWND_EALA,SWND_EALT,SWND_EALS,SWND_EALR,SWND_RISKS,SWND_RISKR,TRND_EVNTS,TRND_AFREQ,TRND_EXPB,TRND_EXPP,TRND_EXPPE,TRND_EXPA,TRND_EXPT,TRND_HLRB,TRND_HLRP,TRND_HLRA,TRND_HLRR,TRND_EALB,TRND_EALP,TRND_EALPE,TRND_EALA,TRND_EALT,TRND_EALS,TRND_EALR,TRND_RISKS,TRND_RISKR,TSUN_EVNTS,TSUN_AFREQ,TSUN_EXPB,TSUN_EXPP,TSUN_EXPPE,TSUN_EXPT,TSUN_HLRB,TSUN_HLRP,TSUN_HLRR,TSUN_EALB,TSUN_EALP,TSUN_EALPE,TSUN_EALT,TSUN_EALS,TSUN_EALR,TSUN_RISKS,TSUN_RISKR,VLCN_EVNTS,VLCN_AFREQ,VLCN_EXPB,VLCN_EXPP,VLCN_EXPPE,VLCN_EXPT,VLCN_HLRB,VLCN_HLRP,VLCN_HLRR,VLCN_EALB,VLCN_EALP,VLCN_EALPE,VLCN_EALT,VLCN_EALS,VLCN_EALR,VLCN_RISKS,VLCN_RISKR,WFIR_EVNTS,WFIR_AFREQ,WFIR_EXPB,WFIR_EXPP,WFIR_EXPPE,WFIR_EXPA,WFIR_EXPT,WFIR_HLRB,WFIR_HLRP,WFIR_HLRA,WFIR_HLRR,WFIR_EALB,WFIR_EALP,WFIR_EALPE,WFIR_EALA,WFIR_EALT,WFIR_EALS,WFIR_EALR,WFIR_RISKS,WFIR_RISKR,WNTW_EVNTS,WNTW_AFREQ,WNTW_EXPB,WNTW_EXPP,WNTW_EXPPE,WNTW_EXPA,WNTW_EXPT,WNTW_HLRB,WNTW_HLRP,WNTW_HLRA,WNTW_HLRR,WNTW_EALB,WNTW_EALP,WNTW_EALPE,WNTW_EALA,WNTW_EALT,WNTW_EALS,WNTW_EALR,WNTW_RISKS,WNTW_RISKR,NRI_VER
-1,T06001020100,Hawaii,HI,15,Kauai,County,7,15007,40300,6001020100,8385,992658000.0000000000,147860.5647200878,3.6108521589,18.0705830803,Relatively Low,63.0775787404,63.4969325153,18.6199401875,Relatively Low,59.6420077263,70.5521472393,324935.2155714268,98076.5248682368,0.0296790442,225560.7358958097,1297.9548073803,31.6808724993,Relatively Moderate,48.7278745931,51.8518518519,-0.1330000000,52.5091980000,Relatively Low,23.5125676106,100.0000000000,2.6254599000,,,,,,,,,Not Applicable,,,,,,Not Applicable,,Not Applicable,,0.0699710120,202742385.5800542533,1862.6855876887,14156410466.4337959290,14359152852.0138511658,0.0000357579,0.0000000020,Very Low,507.2650077305,0.0000002606,1.9802850905,509.2452928210,2.6321796000,Very Low,2.5538810410,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000021774,0.0000022062,0.0080465986,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0030589604,No Rating,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,0.0005345855,912658000.0000000000,8385.0000000000,63726000000.0000000000,64638658000.0000000000,0.0167507621,0.0001397988,Very Low,22512.2000000000,0.0001541200,1171.3120000000,23683.5120000000,11.8920653303,Relatively Low,13.0147002820,Relatively Low,0.0000000000,0.0000000000,912658000.0000000000,8385.0000000000,63726000000.0000000000,147860.5647200878,64638805860.5647201538,0.0000180224,0.0000000760,0.0002275779,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000016,0.0000001005,0.0000761839,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,2.0000000000,0.0343605913,912658000.0000000000,8385.0000000000,63726000000.0000000000,147860.5647200878,64638805860.5647201538,0.0000255348,0.0000003276,0.0002460797,Very Low,788.9305592758,0.0000968737,736.2401254130,1.3226671624,1526.4933518512,4.6757862953,Very Low,6.1662913066,Very Low,0.0000000000,0.0148900000,912658000.0000000000,8385.0000000000,63726000000.0000000000,64638658000.0000000000,0.0000058883,0.0000028944,Relatively Low,80.0189118426,0.0003613770,2746.4650635800,2826.4839754226,19.2773661946,Relatively Low,15.4429446232,Relatively Low,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,142.0000000000,5.9166666660,59632790.0585851222,418.9266599156,3183842615.3584799767,51591.3125103788,3243526996.7295761108,0.0001804370,0.0000114831,0.0042466231,Very Low,63663.1136805333,0.0284625391,216315.2971586954,1296.2757495066,281274.6865887354,29.5879096062,Relatively High,26.9708819409,Relatively High,1.0000000000,0.0312500000,912658000.0000000000,8385.0000000000,63726000000.0000000000,147860.5647200878,64638805860.5647201538,0.0000032387,0.0000018297,0.0000727233,Very Low,92.3692287258,0.0004794348,3643.7043933928,0.3360282071,3736.4096503256,14.9734902768,Relatively Low,16.6070545485,Relatively Low,0.0000000000,0.0000653310,912658000.0000000000,8385.0000000000,63726000000.0000000000,147860.5647200878,64638805860.5647201538,0.0089662390,0.0000059784,0.0021079463,Very Low,534.6107152638,0.0000032750,24.8896914625,0.0203625042,559.5207692305,5.8706925202,Very Low,6.7469108145,Very Low,7.0000000000,0.0319693090,198555247.5326173902,978.4678896234,7436355961.1380958557,7634911208.6707134247,0.0015593140,0.0000038734,Very Low,9898.0167648649,0.0001211641,920.8471781755,10818.8639430404,23.6580872265,Relatively High,20.2115884136,Relatively Moderate,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0005411070,0.0000037371,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,,,,,,,,,,Insufficient Data,,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000851,0.0000001057,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,November 2021
+1,T06069000802,Hawaii,HI,15,Kauai,County,7,15007,40300,6069000802,8385,992658000.0000000000,147860.5647200878,3.6108521589,18.0705830803,Relatively Low,63.0775787404,63.4969325153,18.6199401875,Relatively Low,59.6420077263,70.5521472393,324935.2155714268,98076.5248682368,0.0296790442,225560.7358958097,1297.9548073803,31.6808724993,Relatively Moderate,48.7278745931,51.8518518519,-0.1330000000,52.5091980000,Relatively Low,23.5125676106,100.0000000000,2.6254599000,,,,,,,,,Not Applicable,,,,,,Not Applicable,,Not Applicable,,0.0699710120,202742385.5800542533,1862.6855876887,14156410466.4337959290,14359152852.0138511658,0.0000357579,0.0000000020,Very Low,507.2650077305,0.0000002606,1.9802850905,509.2452928210,2.6321796000,Very Low,2.5538810410,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000021774,0.0000022062,0.0080465986,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0030589604,No Rating,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,0.0005345855,912658000.0000000000,8385.0000000000,63726000000.0000000000,64638658000.0000000000,0.0167507621,0.0001397988,Very Low,22512.2000000000,0.0001541200,1171.3120000000,23683.5120000000,11.8920653303,Relatively Low,13.0147002820,Relatively Low,0.0000000000,0.0000000000,912658000.0000000000,8385.0000000000,63726000000.0000000000,147860.5647200878,64638805860.5647201538,0.0000180224,0.0000000760,0.0002275779,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000016,0.0000001005,0.0000761839,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,2.0000000000,0.0343605913,912658000.0000000000,8385.0000000000,63726000000.0000000000,147860.5647200878,64638805860.5647201538,0.0000255348,0.0000003276,0.0002460797,Very Low,788.9305592758,0.0000968737,736.2401254130,1.3226671624,1526.4933518512,4.6757862953,Very Low,6.1662913066,Very Low,0.0000000000,0.0148900000,912658000.0000000000,8385.0000000000,63726000000.0000000000,64638658000.0000000000,0.0000058883,0.0000028944,Relatively Low,80.0189118426,0.0003613770,2746.4650635800,2826.4839754226,19.2773661946,Relatively Low,15.4429446232,Relatively Low,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,142.0000000000,5.9166666660,59632790.0585851222,418.9266599156,3183842615.3584799767,51591.3125103788,3243526996.7295761108,0.0001804370,0.0000114831,0.0042466231,Very Low,63663.1136805333,0.0284625391,216315.2971586954,1296.2757495066,281274.6865887354,29.5879096062,Relatively High,26.9708819409,Relatively High,1.0000000000,0.0312500000,912658000.0000000000,8385.0000000000,63726000000.0000000000,147860.5647200878,64638805860.5647201538,0.0000032387,0.0000018297,0.0000727233,Very Low,92.3692287258,0.0004794348,3643.7043933928,0.3360282071,3736.4096503256,14.9734902768,Relatively Low,16.6070545485,Relatively Low,0.0000000000,0.0000653310,912658000.0000000000,8385.0000000000,63726000000.0000000000,147860.5647200878,64638805860.5647201538,0.0089662390,0.0000059784,0.0021079463,Very Low,534.6107152638,0.0000032750,24.8896914625,0.0203625042,559.5207692305,5.8706925202,Very Low,6.7469108145,Very Low,7.0000000000,0.0319693090,198555247.5326173902,978.4678896234,7436355961.1380958557,7634911208.6707134247,0.0015593140,0.0000038734,Very Low,9898.0167648649,0.0001211641,920.8471781755,10818.8639430404,23.6580872265,Relatively High,20.2115884136,Relatively Moderate,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0005411070,0.0000037371,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,,,,,,,,,,Insufficient Data,,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000851,0.0000001057,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,November 2021
-2,T06007040300,Hawaii,HI,15,Hawaii,County,1,15001,20100,6007040300,5213,409283000.0000000000,30161527.9142542519,97.0642891247,26.0474557835,Relatively High,89.4815710967,87.4233128834,24.6571275391,Relatively Moderate,83.8106105391,87.4233128834,754552.3595077734,510222.1167381129,0.0320334258,243454.0359926557,876.2067770047,33.3455935266,Relatively Moderate,67.0519519602,65.5270655271,0.9080000000,50.7751980000,Relatively Low,9.3859370029,40.0000000000,2.5387599000,,,,,,,,,Not Applicable,,,,,,Not Applicable,,Not Applicable,,0.0579710120,1082842.5920536572,13.7920666932,104819706.8679994941,105902549.4600531608,0.0000313713,0.0000000025,Very Low,1.9692852387,0.0000000020,0.0151413322,1.9844265710,0.4142077200,Very Low,0.4374499910,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000021774,0.0000022062,0.0080465986,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000541,No Rating,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,0.0099874373,409283000.0000000000,5213.0000000000,39618800000.0000000000,40028083000.0000000000,0.0008505842,0.0000116917,Very Low,509627.8000000000,0.0314233600,238817.5360000000,748445.3360000000,37.5977579168,Very High,44.7882310288,Very High,1.0000000000,0.0312500000,409283000.0000000000,5213.0000000000,39618800000.0000000000,30161527.9142542407,40058244527.9142456055,0.0000180224,0.0000000760,0.0002275779,Very Low,230.5075462219,0.0000123856,94.1304164907,214.5030827638,539.1410454765,5.2311349597,Very Low,5.8932581207,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000016,0.0000001005,0.0000761839,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0100000000,409283000.0000000000,5213.0000000000,39618800000.0000000000,30161527.9142542407,40058244527.9142456055,0.0000255348,0.0000003276,0.0002460797,Very Low,104.5094165573,0.0000170798,129.8064434247,74.2213962963,308.5372562783,2.7440512545,Very Low,3.9390063490,Very Low,0.0000000000,0.0148900000,409283000.0000000000,5213.0000000000,39618800000.0000000000,40028083000.0000000000,0.0000058883,0.0000013610,Very Low,35.8846142757,0.0001056430,802.8864714520,838.7710857277,12.8581949229,Relatively Low,11.2121138672,Relatively Low,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0006245044,0.0000038327,0.0003492485,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,1.0000000000,0.0312500000,409283000.0000000000,5213.0000000000,39618800000.0000000000,30161527.9142542407,40058244527.9142456055,0.0000083601,0.0000003102,0.0006212585,Very Low,106.9261414761,0.0000505411,384.1124266061,585.5657605523,1076.6043286345,9.8898625798,Very Low,11.9394659724,Relatively Low,0.0000000000,0.0006781468,409283000.0000000000,5213.0000000000,39618800000.0000000000,30161527.9142542519,40058244527.9142456055,0.0003985575,0.0000002657,0.0000937001,Very Low,110.6212172132,0.0000009395,7.1398535480,1.9165373923,119.6776081535,3.5109250974,Very Low,4.3917261130,Very Low,4.0000000000,0.0182681760,315888.8587620233,2.2117928286,16809625.4977076985,17125514.3564697206,0.0006654598,0.0000038734,Very Low,3.8401775301,0.0000001565,1.1894532216,5.0296307517,1.8327269938,Very Low,1.7042906890,Very Low,4.0000000000,0.0204021391,407903840.5845158696,5201.9799937840,39535047952.7582778931,39942951793.3427886963,0.0000000070,0.0000040043,Very Low,0.0583395999,0.0004233184,3217.2197865804,3217.2781261802,17.0524727301,Relatively Low,17.9932135371,Relatively Low,,,,,,,,,,,Insufficient Data,,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000851,0.0000001057,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,November 2021
+2,T06061021322,Hawaii,HI,15,Hawaii,County,1,15001,20100,6061021322,5213,409283000.0000000000,30161527.9142542519,97.0642891247,26.0474557835,Relatively High,89.4815710967,87.4233128834,24.6571275391,Relatively Moderate,83.8106105391,87.4233128834,754552.3595077734,510222.1167381129,0.0320334258,243454.0359926557,876.2067770047,33.3455935266,Relatively Moderate,67.0519519602,65.5270655271,0.9080000000,50.7751980000,Relatively Low,9.3859370029,40.0000000000,2.5387599000,,,,,,,,,Not Applicable,,,,,,Not Applicable,,Not Applicable,,0.0579710120,1082842.5920536572,13.7920666932,104819706.8679994941,105902549.4600531608,0.0000313713,0.0000000025,Very Low,1.9692852387,0.0000000020,0.0151413322,1.9844265710,0.4142077200,Very Low,0.4374499910,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000021774,0.0000022062,0.0080465986,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000541,No Rating,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,0.0099874373,409283000.0000000000,5213.0000000000,39618800000.0000000000,40028083000.0000000000,0.0008505842,0.0000116917,Very Low,509627.8000000000,0.0314233600,238817.5360000000,748445.3360000000,37.5977579168,Very High,44.7882310288,Very High,1.0000000000,0.0312500000,409283000.0000000000,5213.0000000000,39618800000.0000000000,30161527.9142542407,40058244527.9142456055,0.0000180224,0.0000000760,0.0002275779,Very Low,230.5075462219,0.0000123856,94.1304164907,214.5030827638,539.1410454765,5.2311349597,Very Low,5.8932581207,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000016,0.0000001005,0.0000761839,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0100000000,409283000.0000000000,5213.0000000000,39618800000.0000000000,30161527.9142542407,40058244527.9142456055,0.0000255348,0.0000003276,0.0002460797,Very Low,104.5094165573,0.0000170798,129.8064434247,74.2213962963,308.5372562783,2.7440512545,Very Low,3.9390063490,Very Low,0.0000000000,0.0148900000,409283000.0000000000,5213.0000000000,39618800000.0000000000,40028083000.0000000000,0.0000058883,0.0000013610,Very Low,35.8846142757,0.0001056430,802.8864714520,838.7710857277,12.8581949229,Relatively Low,11.2121138672,Relatively Low,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0006245044,0.0000038327,0.0003492485,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,1.0000000000,0.0312500000,409283000.0000000000,5213.0000000000,39618800000.0000000000,30161527.9142542407,40058244527.9142456055,0.0000083601,0.0000003102,0.0006212585,Very Low,106.9261414761,0.0000505411,384.1124266061,585.5657605523,1076.6043286345,9.8898625798,Very Low,11.9394659724,Relatively Low,0.0000000000,0.0006781468,409283000.0000000000,5213.0000000000,39618800000.0000000000,30161527.9142542519,40058244527.9142456055,0.0003985575,0.0000002657,0.0000937001,Very Low,110.6212172132,0.0000009395,7.1398535480,1.9165373923,119.6776081535,3.5109250974,Very Low,4.3917261130,Very Low,4.0000000000,0.0182681760,315888.8587620233,2.2117928286,16809625.4977076985,17125514.3564697206,0.0006654598,0.0000038734,Very Low,3.8401775301,0.0000001565,1.1894532216,5.0296307517,1.8327269938,Very Low,1.7042906890,Very Low,4.0000000000,0.0204021391,407903840.5845158696,5201.9799937840,39535047952.7582778931,39942951793.3427886963,0.0000000070,0.0000040043,Very Low,0.0583395999,0.0004233184,3217.2197865804,3217.2781261802,17.0524727301,Relatively Low,17.9932135371,Relatively Low,,,,,,,,,,,Insufficient Data,,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000851,0.0000001057,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,November 2021
-3,T06007040500,Hawaii,HI,15,Kauai,County,7,15007,40500,6007040500,5943,1030806000.0000000000,459516.6731830848,6.1500338151,19.0467198618,Relatively Moderate,67.4534981234,69.3251533742,18.7719774304,Relatively Low,60.4118835838,72.0858895706,332959.9571449574,167792.7734322688,0.0217301935,165149.4709508616,17.7127618271,33.1217117362,Relatively Moderate,64.7826443794,63.5327635328,0.7680000000,52.5091980000,Relatively Low,23.5125676106,100.0000000000,2.6254599000,,,,,,,,,Not Applicable,,,,,,Not Applicable,,Not Applicable,,0.0699710120,66594737.2848528028,383.9447225607,2917979891.4611377716,2984574628.7459902763,0.0000063169,0.0000000003,Very Low,29.4350693631,0.0000000083,0.0628428106,29.4979121737,1.0184434918,Very Low,1.0330889632,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000021774,0.0000022062,0.0080465986,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,56.0000000000,3.1111111110,0.0000000000,0.0000000000,0.0030589604,No Rating,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,0.0005860614,1030806000.0000000000,5943.0000000000,45166800000.0000000000,46197606000.0000000000,0.0167507621,0.0001397988,Very Low,120075.0000000000,0.0011438300,8693.1080000000,128768.1080000000,20.9111551033,Relatively Moderate,23.9260247408,Relatively Moderate,0.0000000000,0.0000000000,1030806000.0000000000,5943.0000000000,45166800000.0000000000,459516.6731830846,46198065516.6731948853,0.0000180224,0.0000000760,0.0002275779,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000016,0.0000001005,0.0000761839,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,2.0000000000,0.0289855072,1030806000.0000000000,5943.0000000000,45166800000.0000000000,459516.6731830846,46198065516.6731948853,0.0000255348,0.0000003276,0.0002460797,Very Low,762.9385502884,0.0000564393,428.9386307213,3.2776151707,1195.1547961804,4.3095415029,Very Low,5.9417734791,Very Low,0.0000000000,0.0148900000,1030806000.0000000000,5943.0000000000,45166800000.0000000000,46197606000.0000000000,0.0000058883,0.0000028944,Relatively Low,90.3777476786,0.0002561316,1946.6001040973,2036.9778517759,17.2833380202,Relatively Low,14.4752368977,Relatively Low,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,142.0000000000,5.9166666660,40606220.8832914308,293.0385094863,2227092672.0956783295,530.1707312656,2267699423.1497006416,0.0001804370,0.0000114831,0.0042466231,Very Low,43350.6205845832,0.0199094993,151312.1945288390,13.3209920158,194676.1361054380,26.1722849103,Relatively High,24.9423944801,Relatively High,1.0000000000,0.0312500000,1030806000.0000000000,5943.0000000000,45166800000.0000000000,459516.6731830846,46198065516.6731948853,0.0000032387,0.0000018297,0.0000727233,Very Low,104.3268729204,0.0003398069,2582.5325235382,1.0442984855,2687.9036949441,13.4166096589,Relatively Low,15.5570766452,Relatively Low,0.0000000000,0.0001223370,1030806000.0000000000,5943.0000000000,45166800000.0000000000,459516.6731830848,46198065516.6731948853,0.0052856261,0.0000035243,0.0012426410,Very Low,666.5475081608,0.0000025623,19.4736228040,0.0698561550,686.0909871197,6.2836381633,Very Low,7.5500148235,Very Low,9.0000000000,0.0411033970,42337272.9888006300,137.6534442030,1046166175.9429297447,1088503448.9317302704,0.0015593140,0.0000038734,Very Low,2713.5270992744,0.0000219159,166.5606980512,2880.0877973256,15.2190537663,Relatively Moderate,13.5932751503,Relatively Moderate,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0005411070,0.0000037371,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,,,,,,,,,,Insufficient Data,,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000851,0.0000001057,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,November 2021
+3,T06027000800,Hawaii,HI,15,Kauai,County,7,15007,40500,6027000800,5943,1030806000.0000000000,459516.6731830848,6.1500338151,19.0467198618,Relatively Moderate,67.4534981234,69.3251533742,18.7719774304,Relatively Low,60.4118835838,72.0858895706,332959.9571449574,167792.7734322688,0.0217301935,165149.4709508616,17.7127618271,33.1217117362,Relatively Moderate,64.7826443794,63.5327635328,0.7680000000,52.5091980000,Relatively Low,23.5125676106,100.0000000000,2.6254599000,,,,,,,,,Not Applicable,,,,,,Not Applicable,,Not Applicable,,0.0699710120,66594737.2848528028,383.9447225607,2917979891.4611377716,2984574628.7459902763,0.0000063169,0.0000000003,Very Low,29.4350693631,0.0000000083,0.0628428106,29.4979121737,1.0184434918,Very Low,1.0330889632,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000021774,0.0000022062,0.0080465986,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,56.0000000000,3.1111111110,0.0000000000,0.0000000000,0.0030589604,No Rating,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,0.0005860614,1030806000.0000000000,5943.0000000000,45166800000.0000000000,46197606000.0000000000,0.0167507621,0.0001397988,Very Low,120075.0000000000,0.0011438300,8693.1080000000,128768.1080000000,20.9111551033,Relatively Moderate,23.9260247408,Relatively Moderate,0.0000000000,0.0000000000,1030806000.0000000000,5943.0000000000,45166800000.0000000000,459516.6731830846,46198065516.6731948853,0.0000180224,0.0000000760,0.0002275779,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000016,0.0000001005,0.0000761839,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,2.0000000000,0.0289855072,1030806000.0000000000,5943.0000000000,45166800000.0000000000,459516.6731830846,46198065516.6731948853,0.0000255348,0.0000003276,0.0002460797,Very Low,762.9385502884,0.0000564393,428.9386307213,3.2776151707,1195.1547961804,4.3095415029,Very Low,5.9417734791,Very Low,0.0000000000,0.0148900000,1030806000.0000000000,5943.0000000000,45166800000.0000000000,46197606000.0000000000,0.0000058883,0.0000028944,Relatively Low,90.3777476786,0.0002561316,1946.6001040973,2036.9778517759,17.2833380202,Relatively Low,14.4752368977,Relatively Low,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,142.0000000000,5.9166666660,40606220.8832914308,293.0385094863,2227092672.0956783295,530.1707312656,2267699423.1497006416,0.0001804370,0.0000114831,0.0042466231,Very Low,43350.6205845832,0.0199094993,151312.1945288390,13.3209920158,194676.1361054380,26.1722849103,Relatively High,24.9423944801,Relatively High,1.0000000000,0.0312500000,1030806000.0000000000,5943.0000000000,45166800000.0000000000,459516.6731830846,46198065516.6731948853,0.0000032387,0.0000018297,0.0000727233,Very Low,104.3268729204,0.0003398069,2582.5325235382,1.0442984855,2687.9036949441,13.4166096589,Relatively Low,15.5570766452,Relatively Low,0.0000000000,0.0001223370,1030806000.0000000000,5943.0000000000,45166800000.0000000000,459516.6731830848,46198065516.6731948853,0.0052856261,0.0000035243,0.0012426410,Very Low,666.5475081608,0.0000025623,19.4736228040,0.0698561550,686.0909871197,6.2836381633,Very Low,7.5500148235,Very Low,9.0000000000,0.0411033970,42337272.9888006300,137.6534442030,1046166175.9429297447,1088503448.9317302704,0.0015593140,0.0000038734,Very Low,2713.5270992744,0.0000219159,166.5606980512,2880.0877973256,15.2190537663,Relatively Moderate,13.5932751503,Relatively Moderate,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0005411070,0.0000037371,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,,,,,,,,,,Insufficient Data,,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000851,0.0000001057,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,November 2021
 4,T15001021010,Hawaii,HI,15,Hawaii,County,1,15001,21010,15001021010,7884,737712000.0000000000,8711454.3090733420,58.4401512286,43.1066279987,Very High,99.4459643383,98.1595092025,42.6674572964,Very High,99.2741170486,99.0797546012,3909779.1321200719,2582125.8111252696,0.1746532017,1327364.3330713348,288.9879234675,31.8903618889,Relatively Moderate,51.0956693021,54.4159544160,-0.0020000000,50.7751980000,Relatively Low,9.3859370029,40.0000000000,2.5387599000,,,,,,,,,Not Applicable,,,,,,Not Applicable,,Not Applicable,,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000478451,0.0000000048,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000021774,0.0000022062,0.0080465986,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000541,No Rating,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,0.0099998852,737712000.0000000000,7884.0000000000,59918400000.0000000000,60656112000.0000000000,0.0008505842,0.0000116917,Very Low,2580741.3999999999,0.1736765400,1319941.7039999999,3900683.1039999998,65.1861714882,Very High,74.2640163391,Very High,1.0000000000,0.0312500000,737712000.0000000000,7884.0000000000,59918400000.0000000000,8711454.3090733420,60664823454.3090744019,0.0000180224,0.0000000760,0.0002275779,Very Low,415.4782459486,0.0000187316,142.3602922696,61.9542156517,619.7927538699,5.4799587665,Very Low,5.9041560145,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000016,0.0000001005,0.0000761839,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0269344664,737712000.0000000000,7884.0000000000,59918400000.0000000000,8711454.3090733420,60664823454.3090744019,0.0000255348,0.0000003276,0.0002460797,Very Low,473.5051910310,0.0000651127,494.8567057547,57.2461948490,1025.6080916347,4.0952789981,Very Low,5.6221049906,Very Low,0.0000000000,0.0148900000,737712000.0000000000,7884.0000000000,59918400000.0000000000,60656112000.0000000000,0.0000058883,0.0000013610,Very Low,64.6802104328,0.0001597715,1214.2637523360,1278.9439627688,14.7995789625,Relatively Low,12.3417814165,Relatively Low,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0006245044,0.0000038327,0.0003492485,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,1.0000000000,0.0312500000,737712000.0000000000,7884.0000000000,59918400000.0000000000,8711454.3090733420,60664823454.3090744019,0.0000083601,0.0000003102,0.0006212585,Very Low,192.7289862509,0.0000764370,580.9212298706,169.1270211135,942.7772372349,9.4618177655,Very Low,10.9242145239,Very Low,1.0000000000,0.0004673635,737712000.0000000000,7884.0000000000,59918400000.0000000000,8711454.3090733420,60664823454.3090744019,0.0006900376,0.0000004601,0.0001622266,Very Low,237.9109428670,0.0000016953,12.8843062101,0.6604918534,251.4557409305,4.4968090785,Very Low,5.3796416501,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0006654598,0.0000038734,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,4.0000000000,0.0207448000,737708710.8628113270,7883.9591351862,59918089427.4153671265,60655798138.2781677246,0.0000000070,0.0000040043,Very Low,0.1075487398,0.0006549135,4977.3427848938,4977.4503336337,19.7224171343,Relatively Low,19.9022650650,Relatively Low,,,,,,,,,,,Insufficient Data,,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000851,0.0000001057,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,November 2021
 5,T15001021101,Hawaii,HI,15,Hawaii,County,1,15001,21101,15001021101,3531,365469000.0000000000,1115552.9463470120,41.0551206444,39.6369371498,Very High,99.0514029613,96.6257668712,35.4631324234,Relatively High,97.7453635601,94.4785276074,2244880.4514211570,1569603.2441089998,0.0888473124,675239.5743199890,37.6329921689,35.2805718581,Relatively High,83.0000273575,82.3361823362,2.1180000000,50.7751980000,Relatively Low,9.3859370029,40.0000000000,2.5387599000,,,,,,,,,Not Applicable,,,,,,Not Applicable,,Not Applicable,,0.0679710120,53358423.6905883327,515.5255139327,3917993905.8884677887,3971352329.5790557861,0.0000009778,0.0000000001,Very Low,3.5462107144,0.0000000023,0.0178004814,3.5640111958,0.5034846073,Very Low,0.5625920420,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000021774,0.0000022062,0.0080465986,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000541,No Rating,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,0.0099998512,365469000.0000000000,3531.0000000000,26835600000.0000000000,27201069000.0000000000,0.0008505842,0.0000116917,Very Low,1549795.8000000000,0.0875910700,665692.1319999999,2215487.9320000000,53.9839983966,Very High,68.0399795668,Very High,1.0000000000,0.0312500000,365469000.0000000000,3531.0000000000,26835600000.0000000000,1115552.9463470120,27202184552.9463424683,0.0000180224,0.0000000760,0.0002275779,Very Low,205.8315698678,0.0000083893,63.7587762572,7.9336015953,277.5239477203,4.1923926160,Very Low,4.9971070139,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000016,0.0000001005,0.0000761839,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0289855072,365469000.0000000000,3531.0000000000,26835600000.0000000000,1115552.9463470120,27202184552.9463424683,0.0000255348,0.0000003276,0.0002460797,Very Low,270.4974447523,0.0000335331,254.8514731746,7.9569545004,533.3058724274,3.2931774779,Very Low,5.0015747332,Very Low,0.0000000000,0.0148900000,365469000.0000000000,3531.0000000000,26835600000.0000000000,27201069000.0000000000,0.0000058883,0.0000013610,Very Low,32.0431439731,0.0000715567,543.8312163240,575.8743602971,11.3433526973,Very Low,10.4651653429,Relatively Low,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,142.0000000000,5.9166666660,4828130.5279219840,35.1384012388,267051849.4150594473,0.0000000000,271879979.9429814219,0.0006245044,0.0000038327,0.0003492485,Very Low,17839.8663537918,0.0007968309,6055.9146131274,0.0000000000,23895.7809669192,13.0070200492,Relatively Moderate,13.6546608024,Relatively Moderate,1.0000000000,0.0312500000,365469000.0000000000,3531.0000000000,26835600000.0000000000,1115552.9463470120,27202184552.9463424683,0.0000083601,0.0000003102,0.0006212585,Very Low,95.4796314509,0.0000342338,260.1766695466,21.6577094941,377.3140104915,6.9727783560,Very Low,8.9063071715,Very Low,0.0000000000,0.0003634330,365469000.0000000000,3531.0000000000,26835600000.0000000000,1115552.9463470120,27202184552.9463424683,0.0008889061,0.0000005927,0.0002089802,Very Low,118.0676167774,0.0000007606,5.7804922284,0.0847265791,123.9328355849,3.5520526364,Very Low,4.7010550308,Very Low,13.0000000000,0.0593715740,31437177.7921413518,196.0173546829,1489731895.5901708603,1521169073.3823122978,0.0006654598,0.0000038734,Very Low,1242.0638448472,0.0000450783,342.5948426489,1584.6586874961,12.4708959075,Relatively Moderate,12.2698912376,Relatively Moderate,3.0000000000,0.0188028000,365467633.7354047298,3530.9854379618,26835489328.5099411011,27200956962.2453422546,0.0000000070,0.0000040043,Very Low,0.0482928249,0.0002658574,2020.5164362008,2020.5647290257,14.6032241308,Relatively Low,16.3029908639,Relatively Low,,,,,,,,,,,Insufficient Data,,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000851,0.0000001057,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,November 2021
 6,T15007040603,Hawaii,HI,15,Kauai,County,7,15007,40603,15007040603,2544,509507000.0000000000,3763051.3782403329,15.9289735326,23.8613675670,Relatively Moderate,84.6148558545,84.9693251534,22.2413255033,Relatively Moderate,75.9028856597,83.7423312883,553788.5026946985,159866.0053362669,0.0465200191,353552.1448416797,40370.3525167520,35.0215086434,Relatively Moderate,81.3161710393,79.7720797721,1.9560000000,52.5091980000,Relatively Low,23.5125676106,100.0000000000,2.6254599000,,,,,,,,,Not Applicable,,,,,,Not Applicable,,Not Applicable,,0.0699710120,59268365.9828897715,295.9306212878,2249072721.7871074677,2308341087.7699966431,0.0000020063,0.0000000001,Very Low,8.3203647759,0.0000000014,0.0109218690,8.3312866448,0.6682062552,Very Low,0.7166933897,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000021774,0.0000022062,0.0080465986,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,119.0000000000,6.6111111110,1994468.3763317089,1994468.3763317089,0.0030589604,Relatively Moderate,40334.3876510453,40334.3876510453,9.3173396900,Relatively Moderate,10.0118819196,Relatively Moderate,,0.0006288023,509507000.0000000000,2544.0000000000,19334400000.0000000000,19843907000.0000000000,0.0167507621,0.0001397988,Very Low,29888.8000000000,0.0002046000,1554.9600000000,31443.7600000000,13.0703357152,Relatively Low,15.8125293377,Relatively Low,0.0000000000,0.0000000000,509507000.0000000000,2544.0000000000,19334400000.0000000000,3763051.3782403329,19847670051.3782386780,0.0000180224,0.0000000760,0.0002275779,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000016,0.0000001005,0.0000761839,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,2.0000000000,0.0289855072,509500026.7867159247,2543.9789504995,19334240023.7962799072,3763051.3782403329,19847503101.9612274170,0.0000255348,0.0000003276,0.0002460797,Very Low,377.1002611632,0.0000241596,183.6127961654,26.8408852286,587.5539425572,3.4012529352,Very Low,4.9584510525,Very Low,0.0000000000,0.0148900000,509507000.0000000000,2544.0000000000,19334400000.0000000000,19843907000.0000000000,0.0000058883,0.0000028944,Relatively Low,44.6719315627,0.0001096414,833.2745523849,877.9464839477,13.0553404852,Relatively Low,11.5613443431,Relatively Low,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,142.0000000000,5.9166666660,119566421.2469792366,677.5008183296,5149006219.3049850464,0.0000000000,5268572640.5519647598,0.0001804370,0.0000114831,0.0042466231,Very Low,127647.4010480262,0.0460304759,349831.6169989206,0.0000000000,477479.0180469467,35.2957296359,Relatively High,35.5664685650,Very High,1.0000000000,0.0312500000,509507000.0000000000,2544.0000000000,19334400000.0000000000,3763051.3782403329,19847670051.3782386780,0.0000032387,0.0000018297,0.0000727233,Very Low,51.5667080334,0.0001454600,1105.4960019992,8.5519178837,1165.6146279163,10.1552327033,Very Low,12.4507973241,Relatively Low,0.0000000000,0.0002990171,509507000.0000000000,2544.0000000000,19334400000.0000000000,3763051.3782403329,19847670051.3782386780,0.0021625099,0.0000014419,0.0005084021,Very Low,329.4612383326,0.0000010968,8.3360081463,0.5720625944,338.3693090733,4.9645617720,Very Low,6.3071150891,Very Low,3.0000000000,0.0137011320,71084897.0818793178,86.3741073938,656443216.1930950880,727528113.2749742270,0.0015593140,0.0000038734,Relatively Low,1518.6837843730,0.0000045839,34.8375621943,1553.5213465673,12.3886737842,Relatively Moderate,11.6999323670,Relatively Moderate,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0005411070,0.0000037371,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,,,,,,,,,,Insufficient Data,,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000851,0.0000001057,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,November 2021

data/data-pipeline/data_pipeline/tests/sources/national_risk_index/data/output.csv

@@ -1,7 +1,7 @@
 GEOID10_TRACT,FEMA Risk Index Expected Annual Loss Score,Expected population loss rate (Natural Hazards Risk Index),Expected agricultural loss rate (Natural Hazards Risk Index),Expected building loss rate (Natural Hazards Risk Index),Contains agricultural value
-06001020100,18.6199401875,0.0000035067,0.0031812618,0.0000661520,True
+06069000802,18.6199401875,0.0000035067,0.0031812618,0.0000661520,True
-06007040300,24.6571275391,0.0000000358,0.0000290505,0.0000014426,True
+06061021322,24.6571275391,0.0000000358,0.0000290505,0.0000014426,True
-06007040500,18.7719774304,0.0000034603,0.0000385465,0.0000436593,True
+06027000800,18.7719774304,0.0000034603,0.0000385465,0.0000436593,True
 15001021010,42.6674572964,0.0000000408,0.0000331733,0.0000018765,True
 15001021101,35.4631324234,0.0000002677,0.0000337348,0.0000507987,True
 15007040603,22.2413255033,0.0000182039,0.0107280896,0.0002521232,True

data/data-pipeline/data_pipeline/tests/sources/national_risk_index/data/transform.csv

@@ -1,7 +1,7 @@
 OID_,NRI_ID,STATE,STATEABBRV,STATEFIPS,COUNTY,COUNTYTYPE,COUNTYFIPS,STCOFIPS,TRACT,GEOID10_TRACT,POPULATION,BUILDVALUE,AGRIVALUE,AREA,RISK_SCORE,RISK_RATNG,RISK_NPCTL,RISK_SPCTL,FEMA Risk Index Expected Annual Loss Score,EAL_RATNG,EAL_NPCTL,EAL_SPCTL,EAL_VALT,EAL_VALB,EAL_VALP,EAL_VALPE,EAL_VALA,SOVI_SCORE,SOVI_RATNG,SOVI_NPCTL,SOVI_SPCTL,SOVI_VALUE,RESL_SCORE,RESL_RATNG,RESL_NPCTL,RESL_SPCTL,RESL_VALUE,AVLN_EVNTS,AVLN_AFREQ,AVLN_EXPB,AVLN_EXPP,AVLN_EXPPE,AVLN_EXPT,AVLN_HLRB,AVLN_HLRP,AVLN_HLRR,AVLN_EALB,AVLN_EALP,AVLN_EALPE,AVLN_EALT,AVLN_EALS,AVLN_EALR,AVLN_RISKS,AVLN_RISKR,CFLD_EVNTS,CFLD_AFREQ,CFLD_EXPB,CFLD_EXPP,CFLD_EXPPE,CFLD_EXPT,CFLD_HLRB,CFLD_HLRP,CFLD_HLRR,CFLD_EALB,CFLD_EALP,CFLD_EALPE,CFLD_EALT,CFLD_EALS,CFLD_EALR,CFLD_RISKS,CFLD_RISKR,CWAV_EVNTS,CWAV_AFREQ,CWAV_EXPB,CWAV_EXPP,CWAV_EXPPE,CWAV_EXPA,CWAV_EXPT,CWAV_HLRB,CWAV_HLRP,CWAV_HLRA,CWAV_HLRR,CWAV_EALB,CWAV_EALP,CWAV_EALPE,CWAV_EALA,CWAV_EALT,CWAV_EALS,CWAV_EALR,CWAV_RISKS,CWAV_RISKR,DRGT_EVNTS,DRGT_AFREQ,DRGT_EXPA,DRGT_EXPT,DRGT_HLRA,DRGT_HLRR,DRGT_EALA,DRGT_EALT,DRGT_EALS,DRGT_EALR,DRGT_RISKS,DRGT_RISKR,ERQK_EVNTS,ERQK_AFREQ,ERQK_EXPB,ERQK_EXPP,ERQK_EXPPE,ERQK_EXPT,ERQK_HLRB,ERQK_HLRP,ERQK_HLRR,ERQK_EALB,ERQK_EALP,ERQK_EALPE,ERQK_EALT,ERQK_EALS,ERQK_EALR,ERQK_RISKS,ERQK_RISKR,HAIL_EVNTS,HAIL_AFREQ,HAIL_EXPB,HAIL_EXPP,HAIL_EXPPE,HAIL_EXPA,HAIL_EXPT,HAIL_HLRB,HAIL_HLRP,HAIL_HLRA,HAIL_HLRR,HAIL_EALB,HAIL_EALP,HAIL_EALPE,HAIL_EALA,HAIL_EALT,HAIL_EALS,HAIL_EALR,HAIL_RISKS,HAIL_RISKR,HWAV_EVNTS,HWAV_AFREQ,HWAV_EXPB,HWAV_EXPP,HWAV_EXPPE,HWAV_EXPA,HWAV_EXPT,HWAV_HLRB,HWAV_HLRP,HWAV_HLRA,HWAV_HLRR,HWAV_EALB,HWAV_EALP,HWAV_EALPE,HWAV_EALA,HWAV_EALT,HWAV_EALS,HWAV_EALR,HWAV_RISKS,HWAV_RISKR,HRCN_EVNTS,HRCN_AFREQ,HRCN_EXPB,HRCN_EXPP,HRCN_EXPPE,HRCN_EXPA,HRCN_EXPT,HRCN_HLRB,HRCN_HLRP,HRCN_HLRA,HRCN_HLRR,HRCN_EALB,HRCN_EALP,HRCN_EALPE,HRCN_EALA,HRCN_EALT,HRCN_EALS,HRCN_EALR,HRCN_RISKS,HRCN_RISKR,ISTM_EVNTS,ISTM_AFREQ,ISTM_EXPB,ISTM_EXPP,ISTM_EXPPE,ISTM_EXPT,ISTM_HLRB,ISTM_HLRP,ISTM_HLRR,ISTM_EALB,ISTM_EALP,ISTM_EALPE,ISTM_EALT,ISTM_EALS,ISTM_EALR,ISTM_RISKS,ISTM_RISKR,LNDS_EVNTS,LNDS_AFREQ,LNDS_EXPB,LNDS_EXPP,LNDS_EXPPE,LNDS_EXPT,LNDS_HLRB,LNDS_HLRP,LNDS_HLRR,LNDS_EALB,LNDS_EALP,LNDS_EALPE,LNDS_EALT,LNDS_EALS,LNDS_EALR,LNDS_RISKS,LNDS_RISKR,LTNG_EVNTS,LTNG_AFREQ,LTNG_EXPB,LTNG_EXPP,LTNG_EXPPE,LTNG_EXPT,LTNG_HLRB,LTNG_HLRP,LTNG_HLRR,LTNG_EALB,LTNG_EALP,LTNG_EALPE,LTNG_EALT,LTNG_EALS,LTNG_EALR,LTNG_RISKS,LTNG_RISKR,RFLD_EVNTS,RFLD_AFREQ,RFLD_EXPB,RFLD_EXPP,RFLD_EXPPE,RFLD_EXPA,RFLD_EXPT,RFLD_HLRB,RFLD_HLRP,RFLD_HLRA,RFLD_HLRR,RFLD_EALB,RFLD_EALP,RFLD_EALPE,RFLD_EALA,RFLD_EALT,RFLD_EALS,RFLD_EALR,RFLD_RISKS,RFLD_RISKR,SWND_EVNTS,SWND_AFREQ,SWND_EXPB,SWND_EXPP,SWND_EXPPE,SWND_EXPA,SWND_EXPT,SWND_HLRB,SWND_HLRP,SWND_HLRA,SWND_HLRR,SWND_EALB,SWND_EALP,SWND_EALPE,SWND_EALA,SWND_EALT,SWND_EALS,SWND_EALR,SWND_RISKS,SWND_RISKR,TRND_EVNTS,TRND_AFREQ,TRND_EXPB,TRND_EXPP,TRND_EXPPE,TRND_EXPA,TRND_EXPT,TRND_HLRB,TRND_HLRP,TRND_HLRA,TRND_HLRR,TRND_EALB,TRND_EALP,TRND_EALPE,TRND_EALA,TRND_EALT,TRND_EALS,TRND_EALR,TRND_RISKS,TRND_RISKR,TSUN_EVNTS,TSUN_AFREQ,TSUN_EXPB,TSUN_EXPP,TSUN_EXPPE,TSUN_EXPT,TSUN_HLRB,TSUN_HLRP,TSUN_HLRR,TSUN_EALB,TSUN_EALP,TSUN_EALPE,TSUN_EALT,TSUN_EALS,TSUN_EALR,TSUN_RISKS,TSUN_RISKR,VLCN_EVNTS,VLCN_AFREQ,VLCN_EXPB,VLCN_EXPP,VLCN_EXPPE,VLCN_EXPT,VLCN_HLRB,VLCN_HLRP,VLCN_HLRR,VLCN_EALB,VLCN_EALP,VLCN_EALPE,VLCN_EALT,VLCN_EALS,VLCN_EALR,VLCN_RISKS,VLCN_RISKR,WFIR_EVNTS,WFIR_AFREQ,WFIR_EXPB,WFIR_EXPP,WFIR_EXPPE,WFIR_EXPA,WFIR_EXPT,WFIR_HLRB,WFIR_HLRP,WFIR_HLRA,WFIR_HLRR,WFIR_EALB,WFIR_EALP,WFIR_EALPE,WFIR_EALA,WFIR_EALT,WFIR_EALS,WFIR_EALR,WFIR_RISKS,WFIR_RISKR,WNTW_EVNTS,WNTW_AFREQ,WNTW_EXPB,WNTW_EXPP,WNTW_EXPPE,WNTW_EXPA,WNTW_EXPT,WNTW_HLRB,WNTW_HLRP,WNTW_HLRA,WNTW_HLRR,WNTW_EALB,WNTW_EALP,WNTW_EALPE,WNTW_EALA,WNTW_EALT,WNTW_EALS,WNTW_EALR,WNTW_RISKS,WNTW_RISKR,NRI_VER,Expected population loss rate (Natural Hazards Risk Index),Expected agricultural loss rate (Natural Hazards Risk Index),Contains agricultural value,Expected building loss rate (Natural Hazards Risk Index)
-1,T06001020100,Hawaii,HI,15,Kauai,County,7,15007,40300,06001020100,8385,992658000.0000000000,147860.5647200878,3.6108521589,18.0705830803,Relatively Low,63.0775787404,63.4969325153,18.6199401875,Relatively Low,59.6420077263,70.5521472393,324935.2155714268,98076.5248682368,0.0296790442,225560.7358958097,1297.9548073803,31.6808724993,Relatively Moderate,48.7278745931,51.8518518519,-0.1330000000,52.5091980000,Relatively Low,23.5125676106,100.0000000000,2.6254599000,,,,,,,,,Not Applicable,,,,,,Not Applicable,,Not Applicable,,0.0699710120,202742385.5800542533,1862.6855876887,14156410466.4337959290,14359152852.0138511658,0.0000357579,0.0000000020,Very Low,507.2650077305,0.0000002606,1.9802850905,509.2452928210,2.6321796000,Very Low,2.5538810410,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000021774,0.0000022062,0.0080465986,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0030589604,No Rating,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,0.0005345855,912658000.0000000000,8385.0000000000,63726000000.0000000000,64638658000.0000076294,0.0167507621,0.0001397988,Very Low,22512.2000000000,0.0001541200,1171.3120000000,23683.5120000000,11.8920653303,Relatively Low,13.0147002820,Relatively Low,0.0000000000,0.0000000000,912658000.0000000000,8385.0000000000,63726000000.0000000000,147860.5647200878,64638805860.5647201538,0.0000180224,0.0000000760,0.0002275779,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000016,0.0000001005,0.0000761839,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,2.0000000000,0.0343605913,912658000.0000000000,8385.0000000000,63726000000.0000000000,147860.5647200878,64638805860.5647201538,0.0000255348,0.0000003276,0.0002460797,Very Low,788.9305592758,0.0000968737,736.2401254130,1.3226671624,1526.4933518512,4.6757862953,Very Low,6.1662913066,Very Low,0.0000000000,0.0148900000,912658000.0000000000,8385.0000000000,63726000000.0000000000,64638658000.0000076294,0.0000058883,0.0000028944,Relatively Low,80.0189118426,0.0003613770,2746.4650635800,2826.4839754226,19.2773661946,Relatively Low,15.4429446232,Relatively Low,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,142.0000000000,5.9166666660,59632790.0585851297,418.9266599156,3183842615.3584799767,51591.3125103788,3243526996.7295761108,0.0001804370,0.0000114831,0.0042466231,Very Low,63663.1136805333,0.0284625391,216315.2971586954,1296.2757495066,281274.6865887354,29.5879096062,Relatively High,26.9708819409,Relatively High,1.0000000000,0.0312500000,912658000.0000000000,8385.0000000000,63726000000.0000000000,147860.5647200878,64638805860.5647201538,0.0000032387,0.0000018297,0.0000727233,Very Low,92.3692287258,0.0004794348,3643.7043933928,0.3360282071,3736.4096503256,14.9734902768,Relatively Low,16.6070545485,Relatively Low,0.0000000000,0.0000653310,912658000.0000000000,8385.0000000000,63726000000.0000000000,147860.5647200878,64638805860.5647201538,0.0089662390,0.0000059784,0.0021079463,Very Low,534.6107152638,0.0000032750,24.8896914625,0.0203625042,559.5207692305,5.8706925202,Very Low,6.7469108145,Very Low,7.0000000000,0.0319693090,198555247.5326173902,978.4678896234,7436355961.1380958557,7634911208.6707124710,0.0015593140,0.0000038734,Very Low,9898.0167648649,0.0001211641,920.8471781755,10818.8639430404,23.6580872265,Relatively High,20.2115884136,Relatively Moderate,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0005411070,0.0000037371,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,,,,,,,,,,Insufficient Data,,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000851,0.0000001057,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,November 2021,0.0000035067,0.0031812618,True,0.0000661520
+1,T06069000802,Hawaii,HI,15,Kauai,County,7,15007,40300,06069000802,8385,992658000.0000000000,147860.5647200878,3.6108521589,18.0705830803,Relatively Low,63.0775787404,63.4969325153,18.6199401875,Relatively Low,59.6420077263,70.5521472393,324935.2155714268,98076.5248682368,0.0296790442,225560.7358958097,1297.9548073803,31.6808724993,Relatively Moderate,48.7278745931,51.8518518519,-0.1330000000,52.5091980000,Relatively Low,23.5125676106,100.0000000000,2.6254599000,,,,,,,,,Not Applicable,,,,,,Not Applicable,,Not Applicable,,0.0699710120,202742385.5800542533,1862.6855876887,14156410466.4337959290,14359152852.0138511658,0.0000357579,0.0000000020,Very Low,507.2650077305,0.0000002606,1.9802850905,509.2452928210,2.6321796000,Very Low,2.5538810410,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000021774,0.0000022062,0.0080465986,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0030589604,No Rating,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,0.0005345855,912658000.0000000000,8385.0000000000,63726000000.0000000000,64638658000.0000076294,0.0167507621,0.0001397988,Very Low,22512.2000000000,0.0001541200,1171.3120000000,23683.5120000000,11.8920653303,Relatively Low,13.0147002820,Relatively Low,0.0000000000,0.0000000000,912658000.0000000000,8385.0000000000,63726000000.0000000000,147860.5647200878,64638805860.5647201538,0.0000180224,0.0000000760,0.0002275779,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000016,0.0000001005,0.0000761839,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,2.0000000000,0.0343605913,912658000.0000000000,8385.0000000000,63726000000.0000000000,147860.5647200878,64638805860.5647201538,0.0000255348,0.0000003276,0.0002460797,Very Low,788.9305592758,0.0000968737,736.2401254130,1.3226671624,1526.4933518512,4.6757862953,Very Low,6.1662913066,Very Low,0.0000000000,0.0148900000,912658000.0000000000,8385.0000000000,63726000000.0000000000,64638658000.0000076294,0.0000058883,0.0000028944,Relatively Low,80.0189118426,0.0003613770,2746.4650635800,2826.4839754226,19.2773661946,Relatively Low,15.4429446232,Relatively Low,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,142.0000000000,5.9166666660,59632790.0585851297,418.9266599156,3183842615.3584799767,51591.3125103788,3243526996.7295761108,0.0001804370,0.0000114831,0.0042466231,Very Low,63663.1136805333,0.0284625391,216315.2971586954,1296.2757495066,281274.6865887354,29.5879096062,Relatively High,26.9708819409,Relatively High,1.0000000000,0.0312500000,912658000.0000000000,8385.0000000000,63726000000.0000000000,147860.5647200878,64638805860.5647201538,0.0000032387,0.0000018297,0.0000727233,Very Low,92.3692287258,0.0004794348,3643.7043933928,0.3360282071,3736.4096503256,14.9734902768,Relatively Low,16.6070545485,Relatively Low,0.0000000000,0.0000653310,912658000.0000000000,8385.0000000000,63726000000.0000000000,147860.5647200878,64638805860.5647201538,0.0089662390,0.0000059784,0.0021079463,Very Low,534.6107152638,0.0000032750,24.8896914625,0.0203625042,559.5207692305,5.8706925202,Very Low,6.7469108145,Very Low,7.0000000000,0.0319693090,198555247.5326173902,978.4678896234,7436355961.1380958557,7634911208.6707124710,0.0015593140,0.0000038734,Very Low,9898.0167648649,0.0001211641,920.8471781755,10818.8639430404,23.6580872265,Relatively High,20.2115884136,Relatively Moderate,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0005411070,0.0000037371,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,,,,,,,,,,Insufficient Data,,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000851,0.0000001057,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,November 2021,0.0000035067,0.0031812618,True,0.0000661520
-2,T06007040300,Hawaii,HI,15,Hawaii,County,1,15001,20100,06007040300,5213,409283000.0000000000,30161527.9142542519,97.0642891247,26.0474557835,Relatively High,89.4815710967,87.4233128834,24.6571275391,Relatively Moderate,83.8106105391,87.4233128834,754552.3595077734,510222.1167381129,0.0320334258,243454.0359926558,876.2067770047,33.3455935266,Relatively Moderate,67.0519519602,65.5270655271,0.9080000000,50.7751980000,Relatively Low,9.3859370029,40.0000000000,2.5387599000,,,,,,,,,Not Applicable,,,,,,Not Applicable,,Not Applicable,,0.0579710120,1082842.5920536572,13.7920666932,104819706.8679994941,105902549.4600531608,0.0000313713,0.0000000025,Very Low,1.9692852387,0.0000000020,0.0151413322,1.9844265710,0.4142077200,Very Low,0.4374499910,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000021774,0.0000022062,0.0080465986,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000541,No Rating,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,0.0099874373,409283000.0000000000,5213.0000000000,39618800000.0000000000,40028083000.0000000000,0.0008505842,0.0000116917,Very Low,509627.8000000000,0.0314233600,238817.5360000000,748445.3360000000,37.5977579168,Very High,44.7882310288,Very High,1.0000000000,0.0312500000,409283000.0000000000,5213.0000000000,39618800000.0000000000,30161527.9142542407,40058244527.9142456055,0.0000180224,0.0000000760,0.0002275779,Very Low,230.5075462219,0.0000123856,94.1304164907,214.5030827638,539.1410454765,5.2311349597,Very Low,5.8932581207,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000016,0.0000001005,0.0000761839,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0100000000,409283000.0000000000,5213.0000000000,39618800000.0000000000,30161527.9142542407,40058244527.9142456055,0.0000255348,0.0000003276,0.0002460797,Very Low,104.5094165573,0.0000170798,129.8064434247,74.2213962963,308.5372562783,2.7440512545,Very Low,3.9390063490,Very Low,0.0000000000,0.0148900000,409283000.0000000000,5213.0000000000,39618800000.0000000000,40028083000.0000000000,0.0000058883,0.0000013610,Very Low,35.8846142757,0.0001056430,802.8864714520,838.7710857277,12.8581949229,Relatively Low,11.2121138672,Relatively Low,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0006245044,0.0000038327,0.0003492485,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,1.0000000000,0.0312500000,409283000.0000000000,5213.0000000000,39618800000.0000000000,30161527.9142542407,40058244527.9142456055,0.0000083601,0.0000003102,0.0006212585,Very Low,106.9261414761,0.0000505411,384.1124266061,585.5657605523,1076.6043286345,9.8898625798,Very Low,11.9394659724,Relatively Low,0.0000000000,0.0006781468,409283000.0000000000,5213.0000000000,39618800000.0000000000,30161527.9142542519,40058244527.9142456055,0.0003985575,0.0000002657,0.0000937001,Very Low,110.6212172132,0.0000009395,7.1398535480,1.9165373923,119.6776081535,3.5109250974,Very Low,4.3917261130,Very Low,4.0000000000,0.0182681760,315888.8587620232,2.2117928286,16809625.4977076985,17125514.3564697206,0.0006654598,0.0000038734,Very Low,3.8401775301,0.0000001565,1.1894532216,5.0296307517,1.8327269938,Very Low,1.7042906890,Very Low,4.0000000000,0.0204021391,407903840.5845158696,5201.9799937840,39535047952.7582778931,39942951793.3427886963,0.0000000070,0.0000040043,Very Low,0.0583395999,0.0004233184,3217.2197865804,3217.2781261802,17.0524727301,Relatively Low,17.9932135371,Relatively Low,,,,,,,,,,,Insufficient Data,,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000851,0.0000001057,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,November 2021,0.0000000358,0.0000290505,True,0.0000014426
+2,T06061021322,Hawaii,HI,15,Hawaii,County,1,15001,20100,06061021322,5213,409283000.0000000000,30161527.9142542519,97.0642891247,26.0474557835,Relatively High,89.4815710967,87.4233128834,24.6571275391,Relatively Moderate,83.8106105391,87.4233128834,754552.3595077734,510222.1167381129,0.0320334258,243454.0359926558,876.2067770047,33.3455935266,Relatively Moderate,67.0519519602,65.5270655271,0.9080000000,50.7751980000,Relatively Low,9.3859370029,40.0000000000,2.5387599000,,,,,,,,,Not Applicable,,,,,,Not Applicable,,Not Applicable,,0.0579710120,1082842.5920536572,13.7920666932,104819706.8679994941,105902549.4600531608,0.0000313713,0.0000000025,Very Low,1.9692852387,0.0000000020,0.0151413322,1.9844265710,0.4142077200,Very Low,0.4374499910,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000021774,0.0000022062,0.0080465986,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000541,No Rating,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,0.0099874373,409283000.0000000000,5213.0000000000,39618800000.0000000000,40028083000.0000000000,0.0008505842,0.0000116917,Very Low,509627.8000000000,0.0314233600,238817.5360000000,748445.3360000000,37.5977579168,Very High,44.7882310288,Very High,1.0000000000,0.0312500000,409283000.0000000000,5213.0000000000,39618800000.0000000000,30161527.9142542407,40058244527.9142456055,0.0000180224,0.0000000760,0.0002275779,Very Low,230.5075462219,0.0000123856,94.1304164907,214.5030827638,539.1410454765,5.2311349597,Very Low,5.8932581207,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000016,0.0000001005,0.0000761839,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0100000000,409283000.0000000000,5213.0000000000,39618800000.0000000000,30161527.9142542407,40058244527.9142456055,0.0000255348,0.0000003276,0.0002460797,Very Low,104.5094165573,0.0000170798,129.8064434247,74.2213962963,308.5372562783,2.7440512545,Very Low,3.9390063490,Very Low,0.0000000000,0.0148900000,409283000.0000000000,5213.0000000000,39618800000.0000000000,40028083000.0000000000,0.0000058883,0.0000013610,Very Low,35.8846142757,0.0001056430,802.8864714520,838.7710857277,12.8581949229,Relatively Low,11.2121138672,Relatively Low,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0006245044,0.0000038327,0.0003492485,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,1.0000000000,0.0312500000,409283000.0000000000,5213.0000000000,39618800000.0000000000,30161527.9142542407,40058244527.9142456055,0.0000083601,0.0000003102,0.0006212585,Very Low,106.9261414761,0.0000505411,384.1124266061,585.5657605523,1076.6043286345,9.8898625798,Very Low,11.9394659724,Relatively Low,0.0000000000,0.0006781468,409283000.0000000000,5213.0000000000,39618800000.0000000000,30161527.9142542519,40058244527.9142456055,0.0003985575,0.0000002657,0.0000937001,Very Low,110.6212172132,0.0000009395,7.1398535480,1.9165373923,119.6776081535,3.5109250974,Very Low,4.3917261130,Very Low,4.0000000000,0.0182681760,315888.8587620232,2.2117928286,16809625.4977076985,17125514.3564697206,0.0006654598,0.0000038734,Very Low,3.8401775301,0.0000001565,1.1894532216,5.0296307517,1.8327269938,Very Low,1.7042906890,Very Low,4.0000000000,0.0204021391,407903840.5845158696,5201.9799937840,39535047952.7582778931,39942951793.3427886963,0.0000000070,0.0000040043,Very Low,0.0583395999,0.0004233184,3217.2197865804,3217.2781261802,17.0524727301,Relatively Low,17.9932135371,Relatively Low,,,,,,,,,,,Insufficient Data,,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000851,0.0000001057,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,November 2021,0.0000000358,0.0000290505,True,0.0000014426
-3,T06007040500,Hawaii,HI,15,Kauai,County,7,15007,40500,06007040500,5943,1030806000.0000000000,459516.6731830848,6.1500338151,19.0467198618,Relatively Moderate,67.4534981234,69.3251533742,18.7719774304,Relatively Low,60.4118835838,72.0858895706,332959.9571449574,167792.7734322688,0.0217301935,165149.4709508616,17.7127618271,33.1217117362,Relatively Moderate,64.7826443794,63.5327635328,0.7680000000,52.5091980000,Relatively Low,23.5125676106,100.0000000000,2.6254599000,,,,,,,,,Not Applicable,,,,,,Not Applicable,,Not Applicable,,0.0699710120,66594737.2848528028,383.9447225607,2917979891.4611377716,2984574628.7459902763,0.0000063169,0.0000000003,Very Low,29.4350693631,0.0000000083,0.0628428106,29.4979121737,1.0184434918,Very Low,1.0330889632,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000021774,0.0000022062,0.0080465986,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,56.0000000000,3.1111111110,0.0000000000,0.0000000000,0.0030589604,No Rating,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,0.0005860614,1030806000.0000000000,5943.0000000000,45166800000.0000000000,46197606000.0000000000,0.0167507621,0.0001397988,Very Low,120075.0000000000,0.0011438300,8693.1080000000,128768.1080000000,20.9111551033,Relatively Moderate,23.9260247408,Relatively Moderate,0.0000000000,0.0000000000,1030806000.0000000000,5943.0000000000,45166800000.0000000000,459516.6731830846,46198065516.6731948853,0.0000180224,0.0000000760,0.0002275779,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000016,0.0000001005,0.0000761839,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,2.0000000000,0.0289855072,1030806000.0000000000,5943.0000000000,45166800000.0000000000,459516.6731830846,46198065516.6731948853,0.0000255348,0.0000003276,0.0002460797,Very Low,762.9385502884,0.0000564393,428.9386307213,3.2776151707,1195.1547961804,4.3095415029,Very Low,5.9417734791,Very Low,0.0000000000,0.0148900000,1030806000.0000000000,5943.0000000000,45166800000.0000000000,46197606000.0000000000,0.0000058883,0.0000028944,Relatively Low,90.3777476786,0.0002561316,1946.6001040973,2036.9778517759,17.2833380202,Relatively Low,14.4752368977,Relatively Low,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,142.0000000000,5.9166666660,40606220.8832914308,293.0385094863,2227092672.0956783295,530.1707312656,2267699423.1497006416,0.0001804370,0.0000114831,0.0042466231,Very Low,43350.6205845832,0.0199094993,151312.1945288390,13.3209920158,194676.1361054380,26.1722849103,Relatively High,24.9423944801,Relatively High,1.0000000000,0.0312500000,1030806000.0000000000,5943.0000000000,45166800000.0000000000,459516.6731830846,46198065516.6731948853,0.0000032387,0.0000018297,0.0000727233,Very Low,104.3268729204,0.0003398069,2582.5325235382,1.0442984855,2687.9036949441,13.4166096589,Relatively Low,15.5570766452,Relatively Low,0.0000000000,0.0001223370,1030806000.0000000000,5943.0000000000,45166800000.0000000000,459516.6731830848,46198065516.6731948853,0.0052856261,0.0000035243,0.0012426410,Very Low,666.5475081608,0.0000025623,19.4736228040,0.0698561550,686.0909871197,6.2836381633,Very Low,7.5500148235,Very Low,9.0000000000,0.0411033970,42337272.9888006300,137.6534442030,1046166175.9429298639,1088503448.9317302704,0.0015593140,0.0000038734,Very Low,2713.5270992744,0.0000219159,166.5606980512,2880.0877973256,15.2190537663,Relatively Moderate,13.5932751503,Relatively Moderate,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0005411070,0.0000037371,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,,,,,,,,,,Insufficient Data,,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000851,0.0000001057,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,November 2021,0.0000034603,0.0000385465,True,0.0000436593
+3,T06027000800,Hawaii,HI,15,Kauai,County,7,15007,40500,06027000800,5943,1030806000.0000000000,459516.6731830848,6.1500338151,19.0467198618,Relatively Moderate,67.4534981234,69.3251533742,18.7719774304,Relatively Low,60.4118835838,72.0858895706,332959.9571449574,167792.7734322688,0.0217301935,165149.4709508616,17.7127618271,33.1217117362,Relatively Moderate,64.7826443794,63.5327635328,0.7680000000,52.5091980000,Relatively Low,23.5125676106,100.0000000000,2.6254599000,,,,,,,,,Not Applicable,,,,,,Not Applicable,,Not Applicable,,0.0699710120,66594737.2848528028,383.9447225607,2917979891.4611377716,2984574628.7459902763,0.0000063169,0.0000000003,Very Low,29.4350693631,0.0000000083,0.0628428106,29.4979121737,1.0184434918,Very Low,1.0330889632,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000021774,0.0000022062,0.0080465986,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,56.0000000000,3.1111111110,0.0000000000,0.0000000000,0.0030589604,No Rating,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,0.0005860614,1030806000.0000000000,5943.0000000000,45166800000.0000000000,46197606000.0000000000,0.0167507621,0.0001397988,Very Low,120075.0000000000,0.0011438300,8693.1080000000,128768.1080000000,20.9111551033,Relatively Moderate,23.9260247408,Relatively Moderate,0.0000000000,0.0000000000,1030806000.0000000000,5943.0000000000,45166800000.0000000000,459516.6731830846,46198065516.6731948853,0.0000180224,0.0000000760,0.0002275779,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000016,0.0000001005,0.0000761839,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,2.0000000000,0.0289855072,1030806000.0000000000,5943.0000000000,45166800000.0000000000,459516.6731830846,46198065516.6731948853,0.0000255348,0.0000003276,0.0002460797,Very Low,762.9385502884,0.0000564393,428.9386307213,3.2776151707,1195.1547961804,4.3095415029,Very Low,5.9417734791,Very Low,0.0000000000,0.0148900000,1030806000.0000000000,5943.0000000000,45166800000.0000000000,46197606000.0000000000,0.0000058883,0.0000028944,Relatively Low,90.3777476786,0.0002561316,1946.6001040973,2036.9778517759,17.2833380202,Relatively Low,14.4752368977,Relatively Low,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,142.0000000000,5.9166666660,40606220.8832914308,293.0385094863,2227092672.0956783295,530.1707312656,2267699423.1497006416,0.0001804370,0.0000114831,0.0042466231,Very Low,43350.6205845832,0.0199094993,151312.1945288390,13.3209920158,194676.1361054380,26.1722849103,Relatively High,24.9423944801,Relatively High,1.0000000000,0.0312500000,1030806000.0000000000,5943.0000000000,45166800000.0000000000,459516.6731830846,46198065516.6731948853,0.0000032387,0.0000018297,0.0000727233,Very Low,104.3268729204,0.0003398069,2582.5325235382,1.0442984855,2687.9036949441,13.4166096589,Relatively Low,15.5570766452,Relatively Low,0.0000000000,0.0001223370,1030806000.0000000000,5943.0000000000,45166800000.0000000000,459516.6731830848,46198065516.6731948853,0.0052856261,0.0000035243,0.0012426410,Very Low,666.5475081608,0.0000025623,19.4736228040,0.0698561550,686.0909871197,6.2836381633,Very Low,7.5500148235,Very Low,9.0000000000,0.0411033970,42337272.9888006300,137.6534442030,1046166175.9429298639,1088503448.9317302704,0.0015593140,0.0000038734,Very Low,2713.5270992744,0.0000219159,166.5606980512,2880.0877973256,15.2190537663,Relatively Moderate,13.5932751503,Relatively Moderate,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0005411070,0.0000037371,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,,,,,,,,,,Insufficient Data,,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000851,0.0000001057,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,November 2021,0.0000034603,0.0000385465,True,0.0000436593
 4,T15001021010,Hawaii,HI,15,Hawaii,County,1,15001,21010,15001021010,7884,737712000.0000000000,8711454.3090733420,58.4401512286,43.1066279987,Very High,99.4459643383,98.1595092025,42.6674572964,Very High,99.2741170486,99.0797546012,3909779.1321200719,2582125.8111252696,0.1746532017,1327364.3330713348,288.9879234675,31.8903618889,Relatively Moderate,51.0956693021,54.4159544160,-0.0020000000,50.7751980000,Relatively Low,9.3859370029,40.0000000000,2.5387599000,,,,,,,,,Not Applicable,,,,,,Not Applicable,,Not Applicable,,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000478451,0.0000000048,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000021774,0.0000022062,0.0080465986,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000541,No Rating,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,0.0099998852,737712000.0000000000,7884.0000000000,59918400000.0000000000,60656112000.0000000000,0.0008505842,0.0000116917,Very Low,2580741.3999999999,0.1736765400,1319941.7039999999,3900683.1039999998,65.1861714882,Very High,74.2640163391,Very High,1.0000000000,0.0312500000,737712000.0000000000,7884.0000000000,59918400000.0000000000,8711454.3090733420,60664823454.3090744019,0.0000180224,0.0000000760,0.0002275779,Very Low,415.4782459486,0.0000187316,142.3602922696,61.9542156517,619.7927538699,5.4799587665,Very Low,5.9041560145,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000016,0.0000001005,0.0000761839,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0269344664,737712000.0000000000,7884.0000000000,59918400000.0000000000,8711454.3090733420,60664823454.3090744019,0.0000255348,0.0000003276,0.0002460797,Very Low,473.5051910310,0.0000651127,494.8567057547,57.2461948490,1025.6080916347,4.0952789981,Very Low,5.6221049906,Very Low,0.0000000000,0.0148900000,737712000.0000000000,7884.0000000000,59918400000.0000000000,60656112000.0000000000,0.0000058883,0.0000013610,Very Low,64.6802104328,0.0001597715,1214.2637523360,1278.9439627688,14.7995789625,Relatively Low,12.3417814165,Relatively Low,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0006245044,0.0000038327,0.0003492485,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,1.0000000000,0.0312500000,737712000.0000000000,7884.0000000000,59918400000.0000000000,8711454.3090733420,60664823454.3090744019,0.0000083601,0.0000003102,0.0006212585,Very Low,192.7289862509,0.0000764370,580.9212298706,169.1270211135,942.7772372349,9.4618177655,Very Low,10.9242145239,Very Low,1.0000000000,0.0004673635,737712000.0000000000,7884.0000000000,59918400000.0000000000,8711454.3090733420,60664823454.3090744019,0.0006900376,0.0000004601,0.0001622266,Very Low,237.9109428670,0.0000016953,12.8843062101,0.6604918534,251.4557409305,4.4968090785,Very Low,5.3796416501,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0006654598,0.0000038734,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,4.0000000000,0.0207448000,737708710.8628113270,7883.9591351862,59918089427.4153594971,60655798138.2781677246,0.0000000070,0.0000040043,Very Low,0.1075487398,0.0006549135,4977.3427848938,4977.4503336337,19.7224171343,Relatively Low,19.9022650650,Relatively Low,,,,,,,,,,,Insufficient Data,,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000851,0.0000001057,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,November 2021,0.0000000408,0.0000331733,True,0.0000018765
 5,T15001021101,Hawaii,HI,15,Hawaii,County,1,15001,21101,15001021101,3531,365469000.0000000000,1115552.9463470120,41.0551206444,39.6369371498,Very High,99.0514029613,96.6257668712,35.4631324234,Relatively High,97.7453635601,94.4785276074,2244880.4514211570,1569603.2441089998,0.0888473124,675239.5743199890,37.6329921689,35.2805718581,Relatively High,83.0000273575,82.3361823362,2.1180000000,50.7751980000,Relatively Low,9.3859370029,40.0000000000,2.5387599000,,,,,,,,,Not Applicable,,,,,,Not Applicable,,Not Applicable,,0.0679710120,53358423.6905883327,515.5255139327,3917993905.8884682655,3971352329.5790553093,0.0000009778,0.0000000001,Very Low,3.5462107144,0.0000000023,0.0178004814,3.5640111958,0.5034846073,Very Low,0.5625920420,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000021774,0.0000022062,0.0080465986,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000541,No Rating,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,0.0099998512,365469000.0000000000,3531.0000000000,26835600000.0000000000,27201069000.0000000000,0.0008505842,0.0000116917,Very Low,1549795.8000000000,0.0875910700,665692.1319999999,2215487.9320000000,53.9839983966,Very High,68.0399795668,Very High,1.0000000000,0.0312500000,365469000.0000000000,3531.0000000000,26835600000.0000000000,1115552.9463470120,27202184552.9463424683,0.0000180224,0.0000000760,0.0002275779,Very Low,205.8315698678,0.0000083893,63.7587762572,7.9336015953,277.5239477203,4.1923926160,Very Low,4.9971070139,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000016,0.0000001005,0.0000761839,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0289855072,365469000.0000000000,3531.0000000000,26835600000.0000000000,1115552.9463470120,27202184552.9463424683,0.0000255348,0.0000003276,0.0002460797,Very Low,270.4974447523,0.0000335331,254.8514731746,7.9569545004,533.3058724274,3.2931774779,Very Low,5.0015747332,Very Low,0.0000000000,0.0148900000,365469000.0000000000,3531.0000000000,26835600000.0000000000,27201069000.0000000000,0.0000058883,0.0000013610,Very Low,32.0431439731,0.0000715567,543.8312163240,575.8743602971,11.3433526973,Very Low,10.4651653429,Relatively Low,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,142.0000000000,5.9166666660,4828130.5279219840,35.1384012388,267051849.4150594473,0.0000000000,271879979.9429814219,0.0006245044,0.0000038327,0.0003492485,Very Low,17839.8663537918,0.0007968309,6055.9146131274,0.0000000000,23895.7809669192,13.0070200492,Relatively Moderate,13.6546608024,Relatively Moderate,1.0000000000,0.0312500000,365469000.0000000000,3531.0000000000,26835600000.0000000000,1115552.9463470120,27202184552.9463424683,0.0000083601,0.0000003102,0.0006212585,Very Low,95.4796314509,0.0000342338,260.1766695466,21.6577094941,377.3140104915,6.9727783560,Very Low,8.9063071715,Very Low,0.0000000000,0.0003634330,365469000.0000000000,3531.0000000000,26835600000.0000000000,1115552.9463470120,27202184552.9463424683,0.0008889061,0.0000005927,0.0002089802,Very Low,118.0676167774,0.0000007606,5.7804922284,0.0847265791,123.9328355849,3.5520526364,Very Low,4.7010550308,Very Low,13.0000000000,0.0593715740,31437177.7921413518,196.0173546829,1489731895.5901708603,1521169073.3823122978,0.0006654598,0.0000038734,Very Low,1242.0638448472,0.0000450783,342.5948426489,1584.6586874961,12.4708959075,Relatively Moderate,12.2698912376,Relatively Moderate,3.0000000000,0.0188028000,365467633.7354047298,3530.9854379618,26835489328.5099411011,27200956962.2453422546,0.0000000070,0.0000040043,Very Low,0.0482928249,0.0002658574,2020.5164362008,2020.5647290257,14.6032241308,Relatively Low,16.3029908639,Relatively Low,,,,,,,,,,,Insufficient Data,,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000851,0.0000001057,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,November 2021,0.0000002677,0.0000337348,True,0.0000507987
 6,T15007040603,Hawaii,HI,15,Kauai,County,7,15007,40603,15007040603,2544,509507000.0000000000,3763051.3782403329,15.9289735326,23.8613675670,Relatively Moderate,84.6148558545,84.9693251534,22.2413255033,Relatively Moderate,75.9028856597,83.7423312883,553788.5026946985,159866.0053362670,0.0465200191,353552.1448416796,40370.3525167520,35.0215086434,Relatively Moderate,81.3161710393,79.7720797721,1.9560000000,52.5091980000,Relatively Low,23.5125676106,100.0000000000,2.6254599000,,,,,,,,,Not Applicable,,,,,,Not Applicable,,Not Applicable,,0.0699710120,59268365.9828897640,295.9306212878,2249072721.7871074677,2308341087.7699966431,0.0000020063,0.0000000001,Very Low,8.3203647759,0.0000000014,0.0109218690,8.3312866448,0.6682062552,Very Low,0.7166933897,Very Low,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000021774,0.0000022062,0.0080465986,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,119.0000000000,6.6111111110,1994468.3763317089,1994468.3763317089,0.0030589604,Relatively Moderate,40334.3876510453,40334.3876510453,9.3173396900,Relatively Moderate,10.0118819196,Relatively Moderate,,0.0006288023,509507000.0000000000,2544.0000000000,19334400000.0000000000,19843907000.0000000000,0.0167507621,0.0001397988,Very Low,29888.8000000000,0.0002046000,1554.9600000000,31443.7600000000,13.0703357152,Relatively Low,15.8125293377,Relatively Low,0.0000000000,0.0000000000,509507000.0000000000,2544.0000000000,19334400000.0000000000,3763051.3782403329,19847670051.3782386780,0.0000180224,0.0000000760,0.0002275779,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000016,0.0000001005,0.0000761839,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,2.0000000000,0.0289855072,509500026.7867159843,2543.9789504995,19334240023.7962799072,3763051.3782403329,19847503101.9612274170,0.0000255348,0.0000003276,0.0002460797,Very Low,377.1002611632,0.0000241596,183.6127961654,26.8408852286,587.5539425572,3.4012529352,Very Low,4.9584510525,Very Low,0.0000000000,0.0148900000,509507000.0000000000,2544.0000000000,19334400000.0000000000,19843907000.0000000000,0.0000058883,0.0000028944,Relatively Low,44.6719315627,0.0001096414,833.2745523849,877.9464839477,13.0553404852,Relatively Low,11.5613443431,Relatively Low,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,,,,,,,,,Insufficient Data,,,,,,Insufficient Data,,Insufficient Data,142.0000000000,5.9166666660,119566421.2469792217,677.5008183296,5149006219.3049850464,0.0000000000,5268572640.5519647598,0.0001804370,0.0000114831,0.0042466231,Very Low,127647.4010480262,0.0460304759,349831.6169989206,0.0000000000,477479.0180469467,35.2957296359,Relatively High,35.5664685650,Very High,1.0000000000,0.0312500000,509507000.0000000000,2544.0000000000,19334400000.0000000000,3763051.3782403329,19847670051.3782386780,0.0000032387,0.0000018297,0.0000727233,Very Low,51.5667080334,0.0001454600,1105.4960019992,8.5519178837,1165.6146279163,10.1552327033,Very Low,12.4507973241,Relatively Low,0.0000000000,0.0002990171,509507000.0000000000,2544.0000000000,19334400000.0000000000,3763051.3782403329,19847670051.3782386780,0.0021625099,0.0000014419,0.0005084021,Very Low,329.4612383326,0.0000010968,8.3360081463,0.5720625944,338.3693090733,4.9645617720,Very Low,6.3071150891,Very Low,3.0000000000,0.0137011320,71084897.0818793178,86.3741073938,656443216.1930950880,727528113.2749742270,0.0015593140,0.0000038734,Relatively Low,1518.6837843730,0.0000045839,34.8375621943,1553.5213465673,12.3886737842,Relatively Moderate,11.6999323670,Relatively Moderate,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0005411070,0.0000037371,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,,,,,,,,,,,Insufficient Data,,,,,,,Insufficient Data,,Insufficient Data,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000851,0.0000001057,0.0000000000,No Rating,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,0.0000000000,No Expected Annual Losses,0.0000000000,No Rating,November 2021,0.0000182039,0.0107280896,True,0.0002521232

data/data-pipeline/data_pipeline/tests/sources/national_risk_index/test_etl.py

@@ -1,7 +1,5 @@
 # pylint: disable=protected-access
-from unittest import mock
 import pathlib
-import requests
 from data_pipeline.etl.base import ValidGeoLevel
 
 from data_pipeline.etl.sources.national_risk_index.etl import (
@@ -36,35 +34,6 @@ class TestNationalRiskIndexETL(TestETL):
         """
         super().setup_method(_method=_method, filename=filename)
 
-    def _setup_etl_instance_and_run_extract(self, mock_etl, mock_paths):
-        with mock.patch("data_pipeline.utils.requests") as requests_mock:
-            zip_file_fixture_src = (
-                self._DATA_DIRECTORY_FOR_TEST / "NRI_Table_CensusTracts.zip"
-            )
-            tmp_path = mock_paths[1]
-
-            # Create mock response.
-            with open(zip_file_fixture_src, mode="rb") as file:
-                file_contents = file.read()
-            response_mock = requests.Response()
-            response_mock.status_code = 200
-            # pylint: disable=protected-access
-            response_mock._content = file_contents
-
-            # Return text fixture:
-            requests_mock.get = mock.MagicMock(return_value=response_mock)
-
-            # Instantiate the ETL class.
-            etl = NationalRiskIndexETL()
-
-            # Monkey-patch the temporary directory to the one used in the test
-            etl.TMP_PATH = tmp_path
-
-            # Run the extract method.
-            etl.extract()
-
-        return etl
-
     def test_init(self, mock_etl, mock_paths):
         """Tests that the mock NationalRiskIndexETL class instance was
         initiliazed correctly.
@@ -87,11 +56,6 @@ class TestNationalRiskIndexETL(TestETL):
         assert etl.GEOID_FIELD_NAME == "GEOID10"
         assert etl.GEOID_TRACT_FIELD_NAME == "GEOID10_TRACT"
         assert etl.NAME == "national_risk_index"
-        assert etl.LAST_UPDATED_YEAR == 2020
-        assert (
-            etl.SOURCE_URL
-            == "https://hazards.fema.gov/nri/Content/StaticDocuments/DataDownload//NRI_Table_CensusTracts/NRI_Table_CensusTracts.zip"
-        )
         assert etl.GEO_LEVEL == ValidGeoLevel.CENSUS_TRACT
         assert etl.COLUMNS_TO_KEEP == [
             etl.GEOID_TRACT_FIELD_NAME,
@@ -109,6 +73,6 @@ class TestNationalRiskIndexETL(TestETL):
 
         output_file_path = etl._get_output_file_path()
         expected_output_file_path = (
-            data_path / "dataset" / "national_risk_index_2020" / "usa.csv"
+            data_path / "dataset" / "national_risk_index" / "usa.csv"
         )
         assert output_file_path == expected_output_file_path

data/data-pipeline/data_pipeline/tile/generate.py

@@ -73,6 +73,10 @@ def generate_tiles(data_path: Path, generate_tribal_layer: bool) -> None:
 
     def _generate_tribal_tiles() -> None:
         """Generates tribal layer tiles"""
+
+        USA_TRIBAL_MIN_ZOOM = 0
+        USA_TRIBAL_MAX_ZOOM = 11
+
         tribal_tiles_path = data_path / "tribal" / "tiles"
         tribal_geojson_dir = data_path / "tribal" / "geojson"
 
@@ -83,6 +87,7 @@ def generate_tiles(data_path: Path, generate_tribal_layer: bool) -> None:
         logger.info("Generating Tribal mbtiles file")
         cmd = "tippecanoe "
         cmd += "--layer=blocks "
+        cmd += f"--minimum-zoom={USA_TRIBAL_MIN_ZOOM} --maximum-zoom={USA_TRIBAL_MAX_ZOOM} "
         cmd += f"--output={tribal_tiles_path}/usa.mbtiles "
         cmd += str(tribal_geojson_dir / "usa.json")
         call(cmd, shell=True)
@@ -92,6 +97,7 @@ def generate_tiles(data_path: Path, generate_tribal_layer: bool) -> None:
         cmd = "tippecanoe "
         cmd += "--no-tile-compression "
         cmd += "--drop-densest-as-needed "
+        cmd += f"--minimum-zoom={USA_TRIBAL_MIN_ZOOM} --maximum-zoom={USA_TRIBAL_MAX_ZOOM} "
         cmd += f"--output-to-directory={tribal_tiles_path} --layer=blocks "
         cmd += str(tribal_geojson_dir / "usa.json")
         call(cmd, shell=True)

data/data-pipeline/data_pipeline/utils.py

@@ -8,6 +8,7 @@ import shutil
 import uuid
 import zipfile
 from pathlib import Path
+from marshmallow import ValidationError
 import urllib3
 import requests
 import yaml
@@ -350,7 +351,13 @@ def load_yaml_dict_from_file(
 
         # validate YAML
         yaml_config_schema = class_schema(schema_class)
-        yaml_config_schema().load(yaml_dict)
+
+        try:
+            yaml_config_schema().load(yaml_dict)
+        except ValidationError as e:
+            logger.error(f"Invalid YAML config file {yaml_file_path}")
+            logger.error(e.normalized_messages())
+            sys.exit()
 
     return yaml_dict
 

mlc_config.json

@@ -1,4 +1,5 @@
 {
+  "_comment": "Markdown Link Checker configuration, see https://github.com/gaurav-nelson/github-action-markdown-link-check and https://github.com/tcort/markdown-link-check",
   "ignorePatterns": [
     {
       "pattern": "^http://localhost"

package.json

@@ -1,7 +0,0 @@
-{
-  "dependencies": {
-    "@turf/turf": "^6.5.0",
-    "@types/d3-ease": "^3.0.0",
-    "d3-ease": "^3.0.1"
-  }
-}