j40-cejst-2/data/data-pipeline/data_pipeline/etl/sources
Matt Bowen 97e17546cc Refactor DOE Energy Burden and COI to use YAML (#1796)
* added tribalId for Supplemental dataset (#1804)

* Setting zoom levels for tribal map (#1810)

* NRI dataset and initial score YAML configuration (#1534)

* update be staging gha

* NRI dataset and initial score YAML configuration

* checkpoint

* adding data checks for release branch

* passing tests

* adding INPUT_EXTRACTED_FILE_NAME to base class

* lint

* columns to keep and tests

* update be staging gha

* checkpoint

* update be staging gha

* NRI dataset and initial score YAML configuration

* checkpoint

* adding data checks for release branch

* passing tests

* adding INPUT_EXTRACTED_FILE_NAME to base class

* lint

* columns to keep and tests

* checkpoint

* PR Review

* renoving source url

* tests

* stop execution of ETL if there's a YAML schema issue

* update be staging gha

* adding source url as class var again

* clean up

* force cache bust

* gha cache bust

* dynamically set score vars from YAML

* docsctrings

* removing last updated year - optional reverse percentile

* passing tests

* sort order

* column ordening

* PR review

* class level vars

* Updating DatasetsConfig

* fix pylint errors

* moving metadata hint back to code

Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov>

* Correct copy typo (#1809)

* Add basic test suite for COI (#1518)

* Update COI to use new yaml (#1518)

* Add tests for DOE energy budren (1518

* Add dataset config for energy budren (1518)

* Refactor ETL to use datasets.yml (#1518)

* Add fake GEOIDs to COI tests (#1518)

* Refactor _setup_etl_instance_and_run_extract to base (#1518)

For the three classes we've done so far, a generic
_setup_etl_instance_and_run_extract will work fine, for the moment we
can reuse the same setup method until we decide future classes need more
flexibility --- but they can also always subclass so...

* Add output-path tests (#1518)

* Update YAML to match constant (#1518)

* Don't blindly set float format (#1518)

* Add defaults for extract (#1518)

* Run YAML load on all subclasses (#1518)

* Update description fields (#1518)

* Update YAML per final format (#1518)

* Update fixture tract IDs (#1518)

* Update base class refactor (#1518)

Now that NRI is final I needed to make a small number of updates to my
refactored code.

* Remove old comment (#1518)

* Fix type signature and return (#1518)

* Update per code review (#1518)

Co-authored-by: Jorge Escobar <83969469+esfoobar-usds@users.noreply.github.com>
Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov>
Co-authored-by: Vim <86254807+vim-usds@users.noreply.github.com>
2022-08-11 12:38:28 -04:00
..
calenviroscreen Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
cdc_life_expectancy Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
cdc_places Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
cdc_svi_index Issue 1141: Definition M (#1151) 2022-01-18 14:56:55 -05:00
census Starting Tribal Boundaries Work (#1736) 2022-07-30 01:13:10 -04:00
census_acs Imputing income using geographic neighbors (#1559) 2022-08-11 12:33:45 -04:00
census_acs_2010 Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
census_acs_median_income Cleaning up quick code (#1349) 2022-03-02 16:50:04 -05:00
census_decennial Issue 1075: Add refactored ETL tests to NRI (#1088) 2022-02-08 19:05:32 -05:00
child_opportunity_index Refactor DOE Energy Burden and COI to use YAML (#1796) 2022-08-11 12:38:28 -04:00
doe_energy_burden Refactor DOE Energy Burden and COI to use YAML (#1796) 2022-08-11 12:38:28 -04:00
ejscreen updating ejscreen data, try two (#1747) 2022-08-11 12:33:46 -04:00
ejscreen_areas_of_concern Issue 838: Update comparison tool to use tracts (#934) 2021-11-30 18:46:29 -05:00
energy_definition_alternative_draft Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
epa_rsei Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
geocorr Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
historic_redlining Adding HOLC indicator (#1579) 2022-08-11 12:33:46 -04:00
housing_and_transportation Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
hud_housing added indoor plumbing to score housing burden 2022-08-11 12:33:46 -04:00
hud_recap Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
mapping_for_ej Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
mapping_inequality Adding HOLC indicator (#1579) 2022-08-11 12:33:46 -04:00
maryland_ejscreen Add a react component generator (#1745) 2022-07-15 09:54:58 -07:00
michigan_ejscreen Add Michigan EJ Screen into data-pipeline's ETL and provide automated scoring and statistics outputs (#1091) 2021-12-31 15:38:52 -05:00
national_risk_index Refactor DOE Energy Burden and COI to use YAML (#1796) 2022-08-11 12:38:28 -04:00
persistent_poverty Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
tree_equity_score Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
tribal added tribalId for Supplemental dataset (#1804) 2022-08-08 17:42:14 -04:00
__init__.py Data directory should adopt standard Poetry-suggested python package structure (#457) 2021-08-05 15:35:54 -04:00