publicdata/j40-cejst-2
1
0
Fork 0
mirror of https://github.com/DOI-DO/j40-cejst-2.git synced 2025-02-24 10:34:18 -08:00
Code Issues Projects Releases Packages Wiki Activity Actions
262 commits 159 branches 0 tags 330 MiB
Commit graph

27 commits

Author SHA1 Message Date
Lucas Merrill Brown
b1a4d26be8
Adding persistent poverty tracts (#738) 
* persistent poverty working

* fixing left-padding

* running black and adding persistent poverty to comp tool

* fixing bug

* running black and fixing linter

* fixing linter

* fixing linter error
 2021-09-22 17:57:08 -04:00
Shelby Switzer
d3a18352fc
Add pytest to tox run in CI/CD (#713) 
* Add pytest to tox run in CI/CD

* Try fixing tox dependencies for pytest

* update poetry to get ci/cd passing

* Run poetry export with --dev flag to include dev dependencies such as pytest

* WIP updating test fixtures to include PDF

* Remove dev dependencies from reqs and add pytest to envlist to make build faster

* passing score_post tests

* Add pytest tox (#729)

* Fix failing pytest

* Fixes failing tox tests and updates requirements.txt to include dev deps

* pickle protocol 4

Co-authored-by: Shelby Switzer <shelby.switzer@cms.hhs.gov>
Co-authored-by: Jorge Escobar <jorge.e.escobar@omb.eop.gov>
Co-authored-by: Billy Daly <williamdaly422@gmail.com>
Co-authored-by: Jorge Escobar <83969469+esfoobar-usds@users.noreply.github.com>
 2021-09-22 13:47:37 -04:00
Vincent La
7709836a12
Ticket 355: Adding map to Urban vs Rural Census Tracts (#696) 
* Adding urban vs rural notebook

* Adding new code

* Adding settings

* Adding usa.csv

* Adding etl

* Adding etl

* Adding to etl_score

* quick changes to notebook

* Ensuring notebook can run

* Adding urban vs rural notebook

* Adding new code

* Adding settings

* Adding usa.csv

* Adding etl

* Adding etl

* Adding to etl_score

* quick changes to notebook

* Ensuring notebook can run

* adding urban to comparison tool

* renaming file

* adding urban rural to more comp tool outputs

* updating requirements and poetry

* Adding ej screen notebook

* removing ej screen notebook since it's in justice40-tool-iss-719

Co-authored-by: La <ryy0@cdc.gov>
Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov>
 2021-09-22 12:31:03 -04:00
Jorge Escobar
cd33f323c8
Revised Columns on Download File + PDF (#701) 
* Revised Columns on Download File + PDF

* finishing ticket
 2021-09-17 13:11:23 -04:00
Jorge Escobar
487f6a8e04
Score Indicators (#690) 
* Score Indicators

* roudning issue with housing burden column

* switching out score g

* final list of columns

* removing duplicate housing burden percentile fields

* removing duplicate

Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov>
 2021-09-16 10:53:05 -04:00
Lucas Merrill Brown
1c0d87d84b
Add FEMA risk index to score file (#687) 
* Add to score file
 2021-09-15 13:31:32 -05:00
Lucas Merrill Brown
e94d05882c
Issue 675 & 676: Adding life expectancy and DOE energy burden data (#683) 
* Adding two new data sources.
 2021-09-15 09:59:28 -05:00
Lucas Merrill Brown
52e70653f0
Prototype H (#682) 2021-09-14 16:16:41 -05:00
Lucas Merrill Brown
1083e953da
Prototype G (#672) 
* wip

* cleanup

* cleanup 2

* fixing import ordering linter error

* updating backend to use score G

* adding percentile to score output

* update tippeanoe compression

Co-authored-by: Jorge Escobar <jorge.e.escobar@omb.eop.gov>
 2021-09-14 10:48:11 -04:00
Jorge Escobar
879cb7d022
hotfix wrong score tile csv path (#671) 
* hotfix wrong score tile csv path

* updating test

* forcing update

* triggering action
 2021-09-14 07:27:48 -04:00
Lucas Merrill Brown
7d13be7651
Ticket 492: Integrate Area Median Income and Poverty measures into ETL (#660) 
* Loading AMI and poverty data
 2021-09-13 15:36:35 -05:00
Shelby Switzer
d7274888b6
Update downloadable zip file (#659) 
* Update downloadable zip file

* Don't use spaces in the name, as per #620
* Add the score D columns, as per #596

* fix paths and directories in etl_score_post

while the tests seemed to  be passing, I encountered an error when
running poetry run score, which was caused by us creating a directory
called <name>.csv, instead of creating the parent directory.

Co-authored-by: Shelby Switzer <shelby.switzer@cms.hhs.gov>
 2021-09-10 16:06:47 -04:00
Nat Hillard
536a35d6a0
Data Unit Tests (#509) 
* Fixes #341 -
As a J40 developer, I want to write Unit Tests for the ETL files,
so that tests are run on each commit

* Location bug

* Adding Load tests

* Fixing XLSX filename

* Adding downloadable zip test

* updating pickle

* Fixing pylint warnings

* Updte readme to correct some typos and reorganize test content structure

* Removing unused schemas file, adding details to readme around pickles, per PR feedback

* Update test to pass with Score D added to score file; update path in readme

* fix requirements.txt after merge

* fix poetry.lock after merge

Co-authored-by: Shelby Switzer <shelby.switzer@cms.hhs.gov>
 2021-09-10 14:17:34 -04:00
Shelby Switzer
ac62933d16
Initial refactor for Score ETL (#618) 
* WIP refactor

* Exract score calculations into their own methods

* do all initial df prep in single method

* Fix error in docs for running etl for single dataset

* WIP understanding HUD and linguistic iso data

* Add comments from initial group review on PR

Co-authored-by: Shelby Switzer <shelby.switzer@cms.hhs.gov>
 2021-09-10 10:34:34 -04:00
Jorge Escobar
327e27e713
Add Score D to USA Low (#629) 
* added score D

* Adding Score D to usa-low

* rounding score d

* small vscode update

* last couple of vscode changes

* uncommited bscode changes
 2021-09-08 16:44:26 -04:00
Jorge Escobar
94298635c2
Add to decimal rounding (#623) 
* added score D

* forgot to add decimal rounding
 2021-09-07 14:30:45 -04:00
Jorge Escobar
99503a2541
added score D (#621) 2021-09-07 13:37:16 -04:00
Lucas Merrill Brown
65ceb7900f
Score F, testing methodology (#510) 
* fixing dependency issue

* fixing more dependencies

* including fraction of state AMI

* wip

* nitpick whitespace

* etl working now

* wip on scoring

* fix rename error

* reducing metrics

* fixing score f

* fixing readme

* adding dependency

* passing tests;

* linting/black

* removing unnecessary sample

* fixing error

* adding verify flag on etl/base

Co-authored-by: Jorge Escobar <jorge.e.escobar@omb.eop.gov>
 2021-08-24 16:40:54 -04:00
Jorge Escobar
c24e13c930
Update GHA to push only client changes to S3 (#543) 2021-08-16 17:00:43 -04:00
Jorge Escobar
c19cd3ee55
hotfix on float cols (#526) 2021-08-13 15:48:31 -04:00
Vim
1dbb1018d6
sets column as percentiles (#525) 
* sets column as percentiles

* adds trailing comma
 2021-08-13 12:01:34 -07:00
Jorge Escobar
6dc1283ee2 added comment 2021-08-10 15:37:36 -04:00
Jorge Escobar
3d8dbb293c
Tile-baking columns with floating rounds completed (#491) 
* Tile-baking columns with floating rounds completed

* completed

* correction on github workflow

* tiles folder no longer needed

* addressed comments

* updating requirements.txt

* poetry lock update

* adding xlswriter

* final poetrylock

* updated requirements.txt

* checkpoint

* removed matplotlib

* ignoring pylint too many statements

* reinstated too many statements

* converting data sync to generate score GHA UI-driven
 2021-08-10 15:28:50 -04:00
lucasmbrown-usds
ebe6180f7c wip 2021-08-09 22:24:14 -05:00
lucasmbrown-usds
4ae7eff4c4 adding median income field and running black 2021-08-09 20:47:51 -05:00
Nat Hillard
9a9d5fdf7f
Backend change for Zipfile pt. 2 (#469) 
* Fixes #303 : adding downloadable zip archive logic
* linter recommendations
* Pushes data directory to AWS. We'll want to move to use AWS for this ASAP, but this works for now
* updating pattern
 2021-08-09 10:39:59 -04:00
Nat Hillard
c1568e87c0
Data directory should adopt standard Poetry-suggested python package structure (#457) 
* Fixes #456 - Our data directory should adopt standard python package structure
* a few missed references
* updating readme
* updating requirements
* Running Black
* Fixes for flake8
* updating pylint
 2021-08-05 15:35:54 -04:00