j40-cejst-2/data/data-pipeline/data_pipeline/tests/test_etl.py
Lucas Merrill Brown a0d6e55f0a
Run ETL processes in parallel (#1253)
* WIP on parallelizing

* switching to get_tmp_path for nri

* switching to get_tmp_path everywhere necessary

* fixing linter errors

* moving heavy ETLs to front of line

* add hold

* moving cdc places up

* removing unnecessary print

* moving h&t up

* adding parallel to geo post

* better census labels

* switching to concurrent futures

* fixing output
2022-02-11 14:04:53 -05:00

10 lines
366 B
Python

# pylint: disable=protected-access
import pytest
from data_pipeline.etl import constants, runner
def test_get_datasets_to_run():
assert runner._get_datasets_to_run(None) == constants.DATASET_LIST
assert runner._get_datasets_to_run("census") == [constants.CENSUS_INFO]
with pytest.raises(ValueError):
runner._get_datasets_to_run("doesnt_exist")