NRI dataset and initial score YAML configuration (#1534)

* update be staging gha

* NRI dataset and initial score YAML configuration

* checkpoint

* adding data checks for release branch

* passing tests

* adding INPUT_EXTRACTED_FILE_NAME to base class

* lint

* columns to keep and tests

* update be staging gha

* checkpoint

* update be staging gha

* NRI dataset and initial score YAML configuration

* checkpoint

* adding data checks for release branch

* passing tests

* adding INPUT_EXTRACTED_FILE_NAME to base class

* lint

* columns to keep and tests

* checkpoint

* PR Review

* renoving source url

* tests

* stop execution of ETL if there's a YAML schema issue

* update be staging gha

* adding source url as class var again

* clean up

* force cache bust

* gha cache bust

* dynamically set score vars from YAML

* docsctrings

* removing last updated year - optional reverse percentile

* passing tests

* sort order

* column ordening

* PR review

* class level vars

* Updating DatasetsConfig

* fix pylint errors

* moving metadata hint back to code

Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov>
This commit is contained in:
Jorge Escobar 2022-08-09 16:37:10 -04:00 committed by GitHub
commit 1c448a77f9
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
15 changed files with 272 additions and 3485 deletions

View file

@ -2,7 +2,9 @@
name: Data Checks
on:
pull_request:
branches: [main] # runs on any PR against main
branches:
- main
- "**/release/**"
paths:
- "data/**"
jobs:
@ -16,7 +18,7 @@ jobs:
# checks all of the versions allowed in pyproject.toml
python-version: [3.8, 3.9]
steps:
# installs python
# installs Python
# one execution of the tests per version listed above
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}

View file

@ -1,7 +1,9 @@
name: Deploy Backend Staging
on:
pull_request:
branches: [main]
branches:
- main
- "**/release/**"
paths:
- "data/**"
env:
@ -60,7 +62,7 @@ jobs:
- name: Update PR with deployed Score URLs
uses: mshick/add-pr-comment@v1
with:
# Deploy to S3 for the staging URL
# Deploy to S3 for the Staging URL
message: |
** Score Deployed! **
Find it here: