6 commits

Author	SHA1	Message	Date
Jorge Escobar	1c448a77f9	NRI dataset and initial score YAML configuration (#1534) ... * update be staging gha * NRI dataset and initial score YAML configuration * checkpoint * adding data checks for release branch * passing tests * adding INPUT_EXTRACTED_FILE_NAME to base class * lint * columns to keep and tests * update be staging gha * checkpoint * update be staging gha * NRI dataset and initial score YAML configuration * checkpoint * adding data checks for release branch * passing tests * adding INPUT_EXTRACTED_FILE_NAME to base class * lint * columns to keep and tests * checkpoint * PR Review * renoving source url * tests * stop execution of ETL if there's a YAML schema issue * update be staging gha * adding source url as class var again * clean up * force cache bust * gha cache bust * dynamically set score vars from YAML * docsctrings * removing last updated year - optional reverse percentile * passing tests * sort order * column ordening * PR review * class level vars * Updating DatasetsConfig * fix pylint errors * moving metadata hint back to code Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov>	2022-08-09 16:37:10 -04:00
Emma Nechamkin	2279a04c94	Quick fix: updating snapshots to have more sigfigs (#1409) ... Updated snapshots to include 10 digits after the decimal	2022-03-14 21:44:35 -04:00
Emma Nechamkin	9d920d4db4	Updating testing to include pytest-snapshot (#1355) ... In this commit, we slightly change the testing to use `pytest-snapshot`. This is for `ETL`s only.	2022-03-11 21:34:07 -05:00
Lucas Merrill Brown	3e37d9d1a3	Issue 1075: update snapshots using command-line flag (#1249) ... * Adding skippable tests using command-line flag	2022-02-14 12:16:52 -05:00
Lucas Merrill Brown	a0d6e55f0a	Run ETL processes in parallel (#1253) ... * WIP on parallelizing * switching to get_tmp_path for nri * switching to get_tmp_path everywhere necessary * fixing linter errors * moving heavy ETLs to front of line * add hold * moving cdc places up * removing unnecessary print * moving h&t up * adding parallel to geo post * better census labels * switching to concurrent futures * fixing output	2022-02-11 14:04:53 -05:00
Lucas Merrill Brown	43e005cc10	Issue 1075: Add refactored ETL tests to NRI (#1088) ... * Adds a substantially refactored ETL test to the National Risk Index, to be used as a model for other tests	2022-02-08 19:05:32 -05:00