Add tests for all non-census sources (#1899)

* Refactor CDC life-expectancy (1554)

* Update to new tract list (#1554)

* Adjust for tests (#1848)

* Add tests for cdc_places (#1848)

* Add EJScreen tests (#1848)

* Add tests for HUD housing (#1848)

* Add tests for GeoCorr (#1848)

* Add persistent poverty tests (#1848)

* Update for sources without zips, for new validation (#1848)

* Update tests for new multi-CSV but (#1848)

Lucas updated the CDC life expectancy data to handle a bug where two
states are missing from the US Overall download. Since virtually none of
our other ETL classes download multiple CSVs directly like this, it
required a pretty invasive new mocking strategy.

* Add basic tests for nature deprived (#1848)

* Add wildfire tests (#1848)

* Add flood risk tests (#1848)

* Add DOT travel tests (#1848)

* Add historic redlining tests (#1848)

* Add tests for ME and WI (#1848)

* Update now that validation exists (#1848)

* Adjust for validation (#1848)

* Add health insurance back to cdc places (#1848)

Ooops

* Update tests with new field (#1848)

* Test for blank tract removal (#1848)

* Add tracts for clipping behavior

* Test clipping and zfill behavior (#1848)

* Fix bad test assumption (#1848)

* Simplify class, add test for tract padding (#1848)

* Fix percentage inversion, update tests (#1848)

Looking through the transformations, I noticed that we were subtracting
a percentage that is usually between 0-100 from 1 instead of 100, and so
were endind up with some surprising results. Confirmed with lucasmbrown-usds

* Add note about first street data (#1848)

data/data-pipeline/data_pipeline/etl/sources/historic_redlining/etl.py

@@ -1,6 +1,6 @@
 import pandas as pd
 
-from data_pipeline.etl.base import ExtractTransformLoad
+from data_pipeline.etl.base import ExtractTransformLoad, ValidGeoLevel
 from data_pipeline.utils import get_module_logger
 from data_pipeline.config import settings
 
@@ -8,11 +8,28 @@ logger = get_module_logger(__name__)
 
 
 class HistoricRedliningETL(ExtractTransformLoad):
+    NAME = "historic_redlining"
+    GEO_LEVEL: ValidGeoLevel = ValidGeoLevel.CENSUS_TRACT
+    EXPECTED_MISSING_STATES = [
+        "10",
+        "11",
+        "16",
+        "23",
+        "30",
+        "32",
+        "35",
+        "38",
+        "46",
+        "50",
+        "56",
+    ]
+    PUERTO_RICO_EXPECTED_IN_DATA = False
+    ALASKA_AND_HAWAII_EXPECTED_IN_DATA: bool = False
+    SOURCE_URL = settings.AWS_JUSTICE40_DATASOURCES_URL + "/HRS_2010.zip"
+
     def __init__(self):
         self.CSV_PATH = self.DATA_PATH / "dataset" / "historic_redlining"
-        self.HISTORIC_REDLINING_URL = (
-            settings.AWS_JUSTICE40_DATASOURCES_URL + "/HRS_2010.zip"
-        )
+
         self.HISTORIC_REDLINING_FILE_PATH = (
             self.get_tmp_path() / "HRS_2010.xlsx"
         )
@@ -25,13 +42,6 @@ class HistoricRedliningETL(ExtractTransformLoad):
         ]
         self.df: pd.DataFrame
 
-    def extract(self) -> None:
-        logger.info("Downloading Historic Redlining Data")
-        super().extract(
-            self.HISTORIC_REDLINING_URL,
-            self.get_tmp_path(),
-        )
-
     def transform(self) -> None:
         logger.info("Transforming Historic Redlining Data")
         # this is obviously temporary
@@ -57,16 +67,4 @@ class HistoricRedliningETL(ExtractTransformLoad):
                 f"{self.REDLINING_SCALAR} meets or exceeds {round(threshold, 2)}"
             )
 
-        self.df = historic_redlining_data
-
-    def load(self) -> None:
-        logger.info("Saving Historic Redlining CSV")
-        # write selected states csv
-        self.CSV_PATH.mkdir(parents=True, exist_ok=True)
-        self.df[self.COLUMNS_TO_KEEP].to_csv(
-            self.CSV_PATH / "usa.csv", index=False
-        )
-
-    def validate(self) -> None:
-        logger.info("Validating Historic Redlining Data")
-        pass
+        self.output_df = historic_redlining_data