mirror of
https://github.com/DOI-DO/j40-cejst-2.git
synced 2025-02-24 02:24:20 -08:00
* added tribalId for Supplemental dataset (#1804) * Setting zoom levels for tribal map (#1810) * NRI dataset and initial score YAML configuration (#1534) * update be staging gha * NRI dataset and initial score YAML configuration * checkpoint * adding data checks for release branch * passing tests * adding INPUT_EXTRACTED_FILE_NAME to base class * lint * columns to keep and tests * update be staging gha * checkpoint * update be staging gha * NRI dataset and initial score YAML configuration * checkpoint * adding data checks for release branch * passing tests * adding INPUT_EXTRACTED_FILE_NAME to base class * lint * columns to keep and tests * checkpoint * PR Review * renoving source url * tests * stop execution of ETL if there's a YAML schema issue * update be staging gha * adding source url as class var again * clean up * force cache bust * gha cache bust * dynamically set score vars from YAML * docsctrings * removing last updated year - optional reverse percentile * passing tests * sort order * column ordening * PR review * class level vars * Updating DatasetsConfig * fix pylint errors * moving metadata hint back to code Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov> * Correct copy typo (#1809) * Add basic test suite for COI (#1518) * Update COI to use new yaml (#1518) * Add tests for DOE energy budren (1518 * Add dataset config for energy budren (1518) * Refactor ETL to use datasets.yml (#1518) * Add fake GEOIDs to COI tests (#1518) * Refactor _setup_etl_instance_and_run_extract to base (#1518) For the three classes we've done so far, a generic _setup_etl_instance_and_run_extract will work fine, for the moment we can reuse the same setup method until we decide future classes need more flexibility --- but they can also always subclass so... * Add output-path tests (#1518) * Update YAML to match constant (#1518) * Don't blindly set float format (#1518) * Add defaults for extract (#1518) * Run YAML load on all subclasses (#1518) * Update description fields (#1518) * Update YAML per final format (#1518) * Update fixture tract IDs (#1518) * Update base class refactor (#1518) Now that NRI is final I needed to make a small number of updates to my refactored code. * Remove old comment (#1518) * Fix type signature and return (#1518) * Update per code review (#1518) Co-authored-by: Jorge Escobar <83969469+esfoobar-usds@users.noreply.github.com> Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov> Co-authored-by: Vim <86254807+vim-usds@users.noreply.github.com>
44 lines
1.4 KiB
YAML
44 lines
1.4 KiB
YAML
# This runs tox in the two directories under data
|
|
name: Data Checks
|
|
on:
|
|
pull_request:
|
|
branches:
|
|
- main
|
|
- "**/release/**"
|
|
paths:
|
|
- "data/**"
|
|
jobs:
|
|
data-pipeline:
|
|
runs-on: ubuntu-latest
|
|
defaults:
|
|
run:
|
|
working-directory: data/data-pipeline
|
|
strategy:
|
|
matrix:
|
|
# checks all of the versions allowed in pyproject.toml
|
|
python-version: [3.8, 3.9]
|
|
steps:
|
|
# installs Python
|
|
# one execution of the tests per version listed above
|
|
- uses: actions/checkout@v2
|
|
- name: Set up Python ${{ matrix.python-version }}
|
|
uses: actions/setup-python@v2
|
|
with:
|
|
python-version: ${{ matrix.python-version }}
|
|
- name: Print variables to help debug
|
|
uses: hmarr/debug-action@v2
|
|
- name: Load cached Poetry installation
|
|
id: cached-poetry-dependencies
|
|
uses: actions/cache@v2
|
|
with:
|
|
path: ~/.cache/pypoetry/virtualenvs
|
|
key: env-${{ runner.os }}-${{ matrix.python-version }}-${{ hashFiles('**/poetry.lock') }}-${{ hashFiles('.github/workflows/data-checks.yml') }}
|
|
- name: Install poetry
|
|
uses: snok/install-poetry@v1
|
|
- name: Print Poetry settings
|
|
run: poetry show -v
|
|
- name: Install dependencies
|
|
run: poetry install
|
|
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
|
|
- name: Run tox
|
|
run: poetry run tox
|