* update be staging gha
* NRI dataset and initial score YAML configuration
* checkpoint
* adding data checks for release branch
* passing tests
* adding INPUT_EXTRACTED_FILE_NAME to base class
* lint
* columns to keep and tests
* update be staging gha
* checkpoint
* update be staging gha
* NRI dataset and initial score YAML configuration
* checkpoint
* adding data checks for release branch
* passing tests
* adding INPUT_EXTRACTED_FILE_NAME to base class
* lint
* columns to keep and tests
* checkpoint
* PR Review
* renoving source url
* tests
* stop execution of ETL if there's a YAML schema issue
* update be staging gha
* adding source url as class var again
* clean up
* force cache bust
* gha cache bust
* dynamically set score vars from YAML
* docsctrings
* removing last updated year - optional reverse percentile
* passing tests
* sort order
* column ordening
* PR review
* class level vars
* Updating DatasetsConfig
* fix pylint errors
* moving metadata hint back to code
Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov>
* starting tribal pr
* further pipeline work
* bia merge working
* alaska villages and tribal geo generate
* tribal folders
* adding data full run
* tile generation
* tribal tile deploy
* Change low to high transition and global zoom
- change the low to high transition from 7 to 5. This can not go any lower as high tiles on AWS only go to zoom level 5
- reduce the zoom level globally on all census tracts
* Remove geolocation from feature flag
- geolocation is now available to all
* Add python notebook that sorts all tracts by area
- add a column of the required zoom level for the tract to be fully contained in the viewport
* Place geolocation back to behind a feature flag
* Differentiate zoom levels b/w shortcuts and tracts
* PyPi Packaging of Data Pipeline
* package rename
* adding python version
* trigger data checks
* print env vars
* python version 2
* trigger data check
* python version 3
* update caching for other GHAs
* updating column E label to "Identified as disadvantaged"
* passing tests
* adding cached poetry flow
* working dir
Co-authored-by: Jorge Escobar <jorge.e.escobar@omb.eop.gov>
* installation step
* trigger action
* installing to home dir
* dry-run
* pyenv
* py 2.8
* trying s4cmd
* removing pyenv
* poetry s4cmd
* num-threads
* public read
* poetry cache
* s4cmd all around
* poetry cache
* poetry cache
* install poetry packages
* poetry echo
* let's do this
* s4cmd install on run
* s4cmd
* ad aws back
* add aws back
* testing census api key and poetry caching
* census api key
* census api
* census api key #3
* 250
* poetry update
* poetry change
* check census api key
* force flag
* update score gen and tilefy; remove cached fips
* small gdal update
* invalidation
* missing cache ids
Summary In this PR, we create a new variable so that the % college students is expressed as % not college students. This means that the front end can display % not college students.
Includes old variables so that this will not break fe.
In order to solve an issue where states with few census tracts appear to have no DACs, we change the low-zoom for states with under some threshold of tracts to be the high-zoom for those states. Thus, WY now has DACs even in low zoom. Yay!
Did some quick, mostly cosmetic changes and updates to the quick launch changes. This mostly entailed changing strings to constants and cleaning up some code to make it neater.
Changes -- PR AMI, updating ag loss, and dropping pr from some threshold counts.
we wanted to implement a slightly different FEMA AG LOSS indicator. Here, we take the 90th percentile only of tracts that have agvalue, and then we also floor the denominator of the rate calculation (loss/total value) at $408k
* Update field name to follow constant standard
* Add table to ETL commands to README
* Update Generate Map Tiles run time
* Add a comma to copy
* Add 3 state UI experience
- PR will only show workforce dev
- IA will only show workforce dev w/o linguistic iso
- update tests to tests 3 states
- change state to territory for Island Areas
* Modify PR and IA threshold counts
* Update tile_data_expected.pkl file
* WIP on parallelizing
* switching to get_tmp_path for nri
* switching to get_tmp_path everywhere necessary
* fixing linter errors
* moving heavy ETLs to front of line
* add hold
* moving cdc places up
* removing unnecessary print
* moving h&t up
* adding parallel to geo post
* better census labels
* switching to concurrent futures
* fixing output
This updates the backend to produce tile data with island indicators / island fields.
Contains:
- new tile codes for island data
- threshold column that specifies number of thresholds to show
- ui experience column that specifies which ui experience to show
TODO: Drop the logger info message from main :)
