Jorge Escobar
5ce4a5c1f7
column ordening
2022-07-07 11:47:19 -04:00
Jorge Escobar
0d81480133
sort order
2022-07-06 16:24:23 -04:00
Jorge Escobar
c9632571f3
passing tests
2022-06-27 13:46:09 -04:00
Jorge Escobar
e5b84dc28d
removing last updated year - optional reverse percentile
2022-06-27 12:59:17 -04:00
Jorge Escobar
7415a15bc0
docsctrings
2022-06-27 11:19:11 -04:00
Jorge Escobar
158c8bffb2
dynamically set score vars from YAML
2022-06-24 15:18:57 -04:00
Jorge Escobar
34f7ef27ce
adding source url as class var again
2022-06-01 17:10:30 -04:00
Jorge Escobar
0460852bf0
Merge branch 'esfoobar-usds/release/yaml-based-score-datasets' into esfoobar-usds/1417-nri-yaml-score
2022-06-01 13:34:58 -04:00
Kameron Kerger
7c808eb2e0
Add files via upload ( #1656 )
...
updated TSD (new - naming convention) and new TSD-es
2022-05-31 13:19:01 -04:00
Jorge Escobar
400746f7fb
stop execution of ETL if there's a YAML schema issue
2022-05-19 12:29:56 -04:00
Jorge Escobar
38304f846f
tests
2022-05-16 15:52:00 -04:00
Jorge Escobar
1db2362426
renoving source url
2022-05-16 15:24:48 -04:00
Jorge Escobar
a127ac2e72
PR Review
2022-05-16 14:47:31 -04:00
Jorge Escobar
105d2cb442
checkpoint
2022-05-16 13:26:07 -04:00
Jorge Escobar
5f2228913b
columns to keep and tests
2022-05-16 13:26:05 -04:00
Jorge Escobar
c18e69ef90
lint
2022-05-16 13:23:52 -04:00
Jorge Escobar
27599f2ebc
adding INPUT_EXTRACTED_FILE_NAME to base class
2022-05-16 13:23:51 -04:00
Jorge Escobar
f0e8ac40e0
passing tests
2022-05-16 13:23:51 -04:00
Jorge Escobar
0a9c04c501
checkpoint
2022-05-16 13:23:50 -04:00
Jorge Escobar
0ee0a4d15f
NRI dataset and initial score YAML configuration
2022-05-16 13:23:49 -04:00
Jorge Escobar
ce89214a60
Adding Technical Training Slides ( #1638 )
...
* Adding Technical Training Slides
* small update on CI/CD map staging URL
2022-05-12 15:01:26 -04:00
Jorge Escobar
2af6fca98d
Columnn headers update ( #1618 )
...
* Columnn headers update
* passing tests
* updated date stamp
* js tests
2022-05-06 14:10:15 -04:00
Kameron Kerger
303c200fbe
Add files via upload ( #1612 )
...
updated pdf
2022-05-04 10:47:54 -04:00
Jorge Escobar
eb1cb8884e
Adding a note about Scipy installation on newer MacOS
2022-05-03 17:26:05 -04:00
Emma Nechamkin
ae725f0a3e
arcgis column name fix ( #1581 )
...
eliminates duplicate column and ensures all column names are unique.
2022-04-22 14:09:12 -04:00
Jorge Escobar
fbd56e3bd5
Put the pdf back in the package and add TSD to pipeline ( #1580 )
...
* Put the pdf back in the package and add TSD to pipeline
* updated pdf with logo
* wrong path
2022-04-21 13:42:04 -04:00
Kameron Kerger
72e6dbc1dd
/1354-update-pdf ( #1568 )
...
updated pdf for the put the pdf back in the package issue
2022-04-19 11:07:31 -04:00
Emma Nechamkin
2ce4cfe80e
updated with codebook ( #1573 )
2022-04-18 18:12:18 -04:00
Jorge Escobar
859177a877
Marshmallow Schemas for YAML files ( #1497 )
...
* Marshmallow Schemas for YAML files
* completed ticket
* passing tests
* lint
* click dep
* staging BE map
* Pr review
2022-03-31 13:56:10 -04:00
Emma Nechamkin
cb963cff5f
Updating comparison tool to be easier for pairwise comparisons ( #1400 )
...
Creating pairwise comparison tool to compare two lists of prioritized tracts to each other.
2022-03-30 14:02:06 -04:00
Emma Nechamkin
2628afacf9
Creating a data dictionary for the download packet ( #1469 )
...
Adding automated codebook creation. Future ticket to refactor.
2022-03-30 11:01:43 -04:00
Emma Nechamkin
dc981919f1
Adding booleans for FE to display ( #1393 )
...
PR adds booleans for each individual threshold category for the front end to display.
2022-03-29 20:17:10 -04:00
Emma Nechamkin
0c07cdac55
Adding category count to BE signals ( #1486 )
...
Added category count to downloadable data and backend signals.
2022-03-29 17:11:57 -04:00
Jorge Escobar
dd723b6c19
PyPi Packaging of Data Pipeline ( #1464 )
...
* PyPi Packaging of Data Pipeline
* package rename
* adding python version
* trigger data checks
* print env vars
* python version 2
* trigger data check
* python version 3
* update caching for other GHAs
2022-03-21 18:55:15 -04:00
Katherine D. Mlika
68c882b3de
updating column E label to "Identified as disadvantaged" ( #1406 )
...
* updating column E label to "Identified as disadvantaged"
* passing tests
* adding cached poetry flow
* working dir
Co-authored-by: Jorge Escobar <jorge.e.escobar@omb.eop.gov>
2022-03-18 14:50:03 -04:00
Jorge Escobar
7b05ee9c76
S3 Parallel Upload and Deletions ( #1410 )
...
* installation step
* trigger action
* installing to home dir
* dry-run
* pyenv
* py 2.8
* trying s4cmd
* removing pyenv
* poetry s4cmd
* num-threads
* public read
* poetry cache
* s4cmd all around
* poetry cache
* poetry cache
* install poetry packages
* poetry echo
* let's do this
* s4cmd install on run
* s4cmd
* ad aws back
* add aws back
* testing census api key and poetry caching
* census api key
* census api
* census api key #3
* 250
* poetry update
* poetry change
* check census api key
* force flag
* update score gen and tilefy; remove cached fips
* small gdal update
* invalidation
* missing cache ids
2022-03-17 23:19:23 -04:00
Emma Nechamkin
e7c7c0abeb
Updating higher education to be reversed ( #1387 )
...
Summary In this PR, we create a new variable so that the % college students is expressed as % not college students. This means that the front end can display % not college students.
Includes old variables so that this will not break fe.
2022-03-15 16:43:32 -04:00
Emma Nechamkin
2279a04c94
Quick fix: updating snapshots to have more sigfigs ( #1409 )
...
Updated snapshots to include 10 digits after the decimal
2022-03-14 21:44:35 -04:00
Emma Nechamkin
9d920d4db4
Updating testing to include pytest-snapshot ( #1355 )
...
In this commit, we slightly change the testing to use `pytest-snapshot`. This is for `ETL`s only.
2022-03-11 21:34:07 -05:00
Jorge Escobar
7f91e2b06b
ArcGIS zipping ( #1391 )
...
* ArcGIS zipping
* lint
* shapefile zip
* removing space in GMT
* adding shapefile to be staging gha
2022-03-09 18:00:20 -05:00
Jorge Escobar
1730572aa6
Reducing Docker start up and adding ArcGIS URL ( #1386 )
...
* Reducing Docker start up and adding ArcGIS URL
* Updating ArcGIS URLs
2022-03-09 08:55:17 -05:00
Emma Nechamkin
917b84dc2e
WY tracts are not showing up until zoom >7 ( #1342 )
...
In order to solve an issue where states with few census tracts appear to have no DACs, we change the low-zoom for states with under some threshold of tracts to be the high-zoom for those states. Thus, WY now has DACs even in low zoom. Yay!
2022-03-08 17:33:11 -05:00
Jorge Escobar
6425beb9f4
YAML Config for Downloadable Assets ( #1252 )
...
* starting yaml config load work
* working version for downloadable file
* yaml file update
* checkpoint
* sort if needed
* refactoring
* moving config
* checkpoint
* old files
* skipping downloadble tests for now
* more modularization
* more refactor, new excel yml
* pylint
* completed tabs
* Update excel.yml
* remvoing obsolete tests
* addressing PR feedback
* addressing changes
* confirmed change in yaml breaks tests
* safety bump
* PR review
* adding tests back
* pylint
* Incorporating latest score fields from Emma
* incorporating newest fields from Emma
* passing tests
* adding shapefile aws sync
* missing test
* passing tests
2022-03-04 15:02:09 -05:00
Emma Nechamkin
1f5633ef74
Adding constants for front end to display booleans ( #1348 )
...
Added constants for the threshold categories and socioeconomic indicators for front end.
2022-03-02 17:12:28 -05:00
Emma Nechamkin
aea49cbb5a
Cleaning up quick code ( #1349 )
...
Did some quick, mostly cosmetic changes and updates to the quick launch changes. This mostly entailed changing strings to constants and cleaning up some code to make it neater.
Changes -- PR AMI, updating ag loss, and dropping pr from some threshold counts.
2022-03-02 16:50:04 -05:00
Emma Nechamkin
f9be97d8c8
This is a quick addition to include PR AMI. To be revised in the "clean up code" pr
2022-03-01 16:31:38 -05:00
Jorge Escobar
dac8ed29d5
Removing PDF from packet ( #1306 )
2022-03-01 13:41:44 -05:00
Emma Nechamkin
fab828dc66
Updating tiles csv to include state code ( #1272 )
...
Adding state codes for island areas and puerto rico to the tiles csv.
2022-02-25 11:10:09 -05:00
Emma Nechamkin
f0a4e40a79
Creating shapefiles for ArcGIS users ( #1275 )
...
Added shapefiles to the files generated when the pipeline is run. Produces both shapefile and a key for column names.
2022-02-24 10:32:49 -05:00
Lucas Merrill Brown
6e64134dc6
1295-college-attendance-field ( #1297 )
...
Lucas' work. Adding college attendance to tiles.
2022-02-17 19:50:52 -05:00
