Add tests for all non-census sources (#1899)

* Refactor CDC life-expectancy (1554)

* Update to new tract list (#1554)

* Adjust for tests (#1848)

* Add tests for cdc_places (#1848)

* Add EJScreen tests (#1848)

* Add tests for HUD housing (#1848)

* Add tests for GeoCorr (#1848)

* Add persistent poverty tests (#1848)

* Update for sources without zips, for new validation (#1848)

* Update tests for new multi-CSV but (#1848)

Lucas updated the CDC life expectancy data to handle a bug where two
states are missing from the US Overall download. Since virtually none of
our other ETL classes download multiple CSVs directly like this, it
required a pretty invasive new mocking strategy.

* Add basic tests for nature deprived (#1848)

* Add wildfire tests (#1848)

* Add flood risk tests (#1848)

* Add DOT travel tests (#1848)

* Add historic redlining tests (#1848)

* Add tests for ME and WI (#1848)

* Update now that validation exists (#1848)

* Adjust for validation (#1848)

* Add health insurance back to cdc places (#1848)

Ooops

* Update tests with new field (#1848)

* Test for blank tract removal (#1848)

* Add tracts for clipping behavior

* Test clipping and zfill behavior (#1848)

* Fix bad test assumption (#1848)

* Simplify class, add test for tract padding (#1848)

* Fix percentage inversion, update tests (#1848)

Looking through the transformations, I noticed that we were subtracting
a percentage that is usually between 0-100 from 1 instead of 100, and so
were endind up with some surprising results. Confirmed with lucasmbrown-usds

* Add note about first street data (#1848)

data/data-pipeline/data_pipeline/etl/sources/hud_housing/etl.py

@@ -1,16 +1,18 @@
 import pandas as pd
-from data_pipeline.etl.base import ExtractTransformLoad
+from data_pipeline.etl.base import ExtractTransformLoad, ValidGeoLevel
 from data_pipeline.utils import get_module_logger
 
 logger = get_module_logger(__name__)
 
 
 class HudHousingETL(ExtractTransformLoad):
+    NAME = "hud_housing"
+    GEO_LEVEL: ValidGeoLevel = ValidGeoLevel.CENSUS_TRACT
+
     def __init__(self):
-        self.OUTPUT_PATH = self.DATA_PATH / "dataset" / "hud_housing"
         self.GEOID_TRACT_FIELD_NAME = "GEOID10_TRACT"
         self.HOUSING_FTP_URL = "https://www.huduser.gov/portal/datasets/cp/2014thru2018-140-csv.zip"
-        self.HOUSING_ZIP_FILE_DIR = self.get_tmp_path() / "hud_housing"
+        self.HOUSING_ZIP_FILE_DIR = self.get_tmp_path()
 
         # We measure households earning less than 80% of HUD Area Median Family Income by county
         # and paying greater than 30% of their income to housing costs.
@@ -22,6 +24,14 @@ class HudHousingETL(ExtractTransformLoad):
         self.NO_KITCHEN_OR_INDOOR_PLUMBING_FIELD_NAME = (
             "Share of homes with no kitchen or indoor plumbing (percent)"
         )
+        self.COLUMNS_TO_KEEP = [
+            self.GEOID_TRACT_FIELD_NAME,
+            self.HOUSING_BURDEN_NUMERATOR_FIELD_NAME,
+            self.HOUSING_BURDEN_DENOMINATOR_FIELD_NAME,
+            self.HOUSING_BURDEN_FIELD_NAME,
+            self.NO_KITCHEN_OR_INDOOR_PLUMBING_FIELD_NAME,
+            "DENOM INCL NOT COMPUTED",
+        ]
 
         # Note: some variable definitions.
         # HUD-adjusted median family income (HAMFI).
@@ -234,19 +244,4 @@ class HudHousingETL(ExtractTransformLoad):
             float
         )
 
-    def load(self) -> None:
-        logger.info("Saving HUD Housing Data")
-
-        self.OUTPUT_PATH.mkdir(parents=True, exist_ok=True)
-
-        # Drop unnecessary fields
-        self.df[
-            [
-                self.GEOID_TRACT_FIELD_NAME,
-                self.HOUSING_BURDEN_NUMERATOR_FIELD_NAME,
-                self.HOUSING_BURDEN_DENOMINATOR_FIELD_NAME,
-                self.HOUSING_BURDEN_FIELD_NAME,
-                self.NO_KITCHEN_OR_INDOOR_PLUMBING_FIELD_NAME,
-                "DENOM INCL NOT COMPUTED",
-            ]
-        ].to_csv(path_or_buf=self.OUTPUT_PATH / "usa.csv", index=False)
+        self.output_df = self.df