Commit graph

66 commits

Author SHA1 Message Date
Emma Nechamkin
d16d0109a4
OOPS!
Old changes persisted
2022-08-25 16:48:42 -04:00
Emma Nechamkin
4a25a28b0e just testing that the boolean is preserved on gha 2022-08-25 10:54:13 -04:00
Emma Nechamkin
637b8c305c
updated to show T/F/null vs T/F for AML and FUDS (#1866) 2022-08-24 20:22:59 -04:00
Emma Nechamkin
6418335219
Updates backend constants to N (#1854) 2022-08-23 16:19:00 -04:00
Lucas Merrill Brown
4bf7773797
Issue 1827: Add demographics to tiles and download files (#1833)
* Adding demographics for use in sidebar and download files
2022-08-22 10:05:23 -04:00
Emma Nechamkin
d892bce6cf
Fast flag update (#1844)
Added additional flags for the front end based on our conversation in stand up this morning.
2022-08-19 13:14:44 -04:00
Emma Nechamkin
1ee26bf30d
Quick fix to kitchen or plumbing indicator
Yikes! I think I messed something up and dropped the pctile field suffix from when the KP score gets calculated. Fixing right quick.
2022-08-18 17:47:34 -04:00
Emma Nechamkin
cb4866b93f
Adding eamlis and fuds data to legacy pollution in score (#1832)
Update to add EAMLIS and FUDS data to score
2022-08-18 13:32:29 -04:00
Matt Bowen
6e41e0d9f0
Add donut hole calculation to score (#1828)
Adds adjacency index to the pipeline. Requires thorough QA
2022-08-18 12:04:46 -04:00
Emma Nechamkin
7d89d41e49
Adding NLCD data (#1826)
Adding NLCD's natural space indicator end to end to the score.
2022-08-17 14:21:28 -04:00
Emma Nechamkin
5e378aea81
Adding first street foundation data (#1823)
Adding FSF flood and wildfire risk datasets to the score.
2022-08-17 10:14:23 -04:00
Emma Nechamkin
ebac552d75
Adding DOT composite to travel score (#1820)
This adds the DOT dataset to the ETL and to the score. Note that currently we take a percentile of an average of percentiles.
2022-08-16 14:44:39 -04:00
Emma Nechamkin
481a2a05f7
updated to fix linting errors (#1818)
Cleans and updates base branch
2022-08-11 16:34:56 -04:00
Emma Nechamkin
15450cf91f added indoor plumbing to score housing burden 2022-08-11 12:33:46 -04:00
Emma Nechamkin
0d90ae563a Changing LHE in tiles to a boolean (#1767)
also includes merging / clean up of the release
2022-08-11 12:33:46 -04:00
Emma Nechamkin
b0a728437c adds UST indicator (#1786)
adds leaky underground storage tanks
2022-08-11 12:33:46 -04:00
Emma Nechamkin
2ab24c60fa updating ejscreen data, try two (#1747) 2022-08-11 12:33:46 -04:00
Emma Nechamkin
7559cf46f6 Emma nechamkin/holc patch (#1742)
Removing HOLC calculation from score narwhal.
2022-08-11 12:33:46 -04:00
Emma Nechamkin
b41a2870f3 updating 2022-08-11 12:33:46 -04:00
Shelby Switzer
05748c9fa2 Update backend for Puerto Rico (#1686)
* Update PR threshold count to 10

We now show 10 indicators for PR. See the discussion on the github issue for more info: https://github.com/usds/justice40-tool/issues/1621

* Do not use linguistic iso for Puerto Rico

Closes 1350.

Co-authored-by: Shelby Switzer <shelbyswitzer@gmail.com>
2022-08-11 12:33:46 -04:00
Emma Nechamkin
1782d022a9 Adding HOLC indicator (#1579)
Added HOLC indicator (Historic Redlining Score) from NCRC work; included 3.25 cutoff and low income as part of the housing burden category.
2022-08-11 12:33:46 -04:00
Emma Nechamkin
f047ca9d83 Imputing income using geographic neighbors (#1559)
Imputes income field with a light refactor. Needs more refactor and more tests (I spotchecked). Next ticket will check and address but a lot of "narwhal" architecture is here.
2022-08-11 12:33:45 -04:00
Jorge Escobar
859177a877
Marshmallow Schemas for YAML files (#1497)
* Marshmallow Schemas for YAML files

* completed ticket

* passing tests

* lint

* click dep

* staging BE map

* Pr review
2022-03-31 13:56:10 -04:00
Emma Nechamkin
dc981919f1
Adding booleans for FE to display (#1393)
PR adds booleans for each individual threshold category for the front end to display.
2022-03-29 20:17:10 -04:00
Emma Nechamkin
0c07cdac55
Adding category count to BE signals (#1486)
Added category count to downloadable data and backend signals.
2022-03-29 17:11:57 -04:00
Emma Nechamkin
e7c7c0abeb
Updating higher education to be reversed (#1387)
Summary In this PR, we create a new variable so that the % college students is expressed as % not college students. This means that the front end can display % not college students.

Includes old variables so that this will not break fe.
2022-03-15 16:43:32 -04:00
Emma Nechamkin
1f5633ef74
Adding constants for front end to display booleans (#1348)
Added constants for the threshold categories and socioeconomic indicators for front end.
2022-03-02 17:12:28 -05:00
Emma Nechamkin
aea49cbb5a
Cleaning up quick code (#1349)
Did some quick, mostly cosmetic changes and updates to the quick launch changes. This mostly entailed changing strings to constants and cleaning up some code to make it neater.

Changes -- PR AMI, updating ag loss, and dropping pr from some threshold counts.
2022-03-02 16:50:04 -05:00
Lucas Merrill Brown
6e64134dc6
1295-college-attendance-field (#1297)
Lucas' work. Adding college attendance to tiles.
2022-02-17 19:50:52 -05:00
Emma Nechamkin
cee13b50cc
Stripping thresholds from PR so the UI matches the count
Add a tuple to skip FIPS 72 when incrementing counter. TODO: clean up so it's a constant.
2022-02-17 16:54:33 -05:00
Emma Nechamkin
1b76a68838
FEMA data check (#1270)
we wanted to implement a slightly different FEMA AG LOSS indicator. Here, we take the 90th percentile only of tracts that have agvalue, and then we also floor the denominator of the rate calculation (loss/total value) at $408k
2022-02-17 16:53:04 -05:00
Emma Nechamkin
389eb59ac4
Adding island area indicators to the tiles (#1213)
This updates the backend to produce tile data with island indicators / island fields. 

Contains:
- new tile codes for island data
- threshold column that specifies number of thresholds to show
- ui experience column that specifies which ui experience to show

TODO: Drop the logger info message from main :)
2022-02-09 20:33:42 -05:00
Emma Nechamkin
6a00b29f5d
Adding VA and CO ETL from mapping for environmental justice (#1177)
Adding the mapping for environmental justice data, which contains information about VA and CO, to the ETL pipeline.
2022-02-04 10:00:41 -05:00
Emma Nechamkin
49868401be
Updating field names to match score M definitions (#1190)
When implementing definition M for the score, the variable names were not yet updated. For example:

This legacy field naming: 
```
UNEMPLOYMENT_LOW_HS_EDUCATION_FIELD = (
    f"Greater than or equal to the {PERCENTILE}th percentile for unemployment"
    " and has low HS education"
)
``` 

Should actually be renamed something like this:
```
UNEMPLOYMENT_LOW_HS_LOW_HIGHER_ED_FIELD = (
    f"Greater than or equal to the {PERCENTILE}th percentile for unemployment"
    " and has low HS education and low higher ed attendance"
)
``` 

This PR is for the backend updates for this -- keeping the old fields, and adding new, Score M specific fields as listed below: 
- [x] `field_names`: add new fields to capture low_higher_ed
- [x] `score_m`: replace old fields with new fields 
- [x] `DOWNLOADABLE_SCORE_COLUMNS`: replace old fields with new fields
- [x] `TILES_SCORE_COLUMNS`: replace old fields with new fields
2022-02-01 18:54:43 -05:00
Jorge Escobar
2b35a8937a
Hot fix for Score M (#1182)
* fixes

* pr feedback

* tuple
2022-01-27 17:22:39 -05:00
Lucas Merrill Brown
18f299c5f8
Issue 1141: Definition M (#1151) 2022-01-18 14:56:55 -05:00
Saran Ahluwalia
87e08f5fe1
CDC SVI Index: Additions to data-pipeline and comparison tool (#1096)
* wip

* working

* working

* rename

* documentation

* add link

* add readme

* update fieldnames

* typo

* add comparison tool

* revise wording

* variable change for FIPS

* workding

* wording in readme

* cleanup wording

* update comparison tool

* final tune up

* grammar and punctuation in the documentation

* period

* cleanup comments

* added revisions

* parallelism

* PR feedback from Lucas

* remove extraneous fields from comparison tool

* style

* updates

* remove themes

* formatting

* remove referenes to percentile rank

* remove referenes to percentile rank

* typo in fieldnames

* updates based on feedback from Lucas

* fieldnames formatting

* fix broken markdown link

Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov>
2022-01-14 14:52:37 -05:00
Saran Ahluwalia
95a14adb35
Added Census Tract Aggregated Micro-data from EPA Risk-Screening Environmental Indicators (RSEI) model (#1101)
* added initial source code - todo is comparison tool

* added values

* rename fields

* check geoid

* added black

* added revisions

* added clean up to comments

* more comments

* formatting

* cleanup and address PR feedback

* fix changes

* final path changes

* style

* PR feedback

* added final PR comment

* fix flake 8

* add revisions
2022-01-14 13:50:49 -05:00
Saran Ahluwalia
a98ea35f74
Maryland EJSCREEN Addition to comparison tool (#1143)
* finalized

* cleanup notebook

* cleanup

* run black
2022-01-14 13:26:48 -05:00
Jorge Escobar
d686bb856e
Download column order completed (#1077)
* Download column order completed

* Kameron changes

* Lucas and Beth column order changes

* cdc_places update

* passing score

* pandas error

* checkpoint

* score passing

* rounding complete - percentages still showing one decimal

* fixing tests

* fixing percentages

* updating comment

* int percentages! 🎉🎉

* forgot to pass back to df

* passing tests

Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov>
2022-01-13 15:04:16 -05:00
Saran Ahluwalia
a4137fdc98
Add Michigan EJ Screen into data-pipeline's ETL and provide automated scoring and statistics outputs (#1091)
* draft wip

* initial commit

* clear output from notebook

* revert to 65ceb7900f

* draft wip

* initial commit

* clear output from notebook

* revert to 65ceb7900f

* make michigan prefix for readable

* standardize Michigan names and move all constants from class into field names module

* standardize Michigan names and move all constants from class into field names module

* include only pertinent columns for scoring comparison tool

* michigan EJSCREEN standardization

* final PR feedback

* added exposition and summary of Michigan EJSCREEN

* added exposition and summary of Michigan EJSCREEN

* fix typo

Co-authored-by: Saran Ahluwalia <ahlusar.ahluwalia@gmail.com>
2021-12-31 15:38:52 -05:00
Lucas Merrill Brown
beb0eea5cc
Alternative definition of DACs for comparison (#1068)
* Alternative energy-related definition of DACs
2021-12-27 12:05:59 -05:00
Lucas Merrill Brown
0d10534725
Issue 1044: Add low HS education fields to tiles and download (#1046) 2021-12-14 15:41:06 -05:00
Jorge Escobar
9709d08ca3
Update Side Panel Tile Data (#866)
* Update Side Panel Tile Data

* Update Side Panel Tile Data

* Correct indicator names to match csv

* Replace Score with Rate

* Comment out FEMA Loss Rate to troubleshoot

* Removes all "FEMA Loss Rate" array elements

* Revert FEMA to Score

* Remove expected loss rate

* Remove RMP and NPL from BASIC array

* Attempt to make shape mismatch align

- update README typo

* Add Score L indicators to TILE_SCORE_FLOAT_COLUMNS

* removing cbg references

* completes the ticket

* Update side panel fields

* Update index file writing to create parent dir

* Updates from linting

* fixing missing field_names for island territories 90th percentile fields

* Update downloadable fields and fix field name

* Update file fields and tests

* Update ordering of fields and leave TODO

* Update pickle after re-ordering of file

* fixing bugs in etl_score_geo

* Repeating index for diesel fix

* passing tests

* adding pytest.ini

Co-authored-by: Vim USDS <vimal.k.shah@omb.eop.gov>
Co-authored-by: Shelby Switzer <shelby.switzer@cms.hhs.gov>
Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov>
2021-12-13 14:53:50 -05:00
Lucas Merrill Brown
7fcecaee42
Issue 970: reverse percentiles for AMI and life expectancy (#1018)
* switching to low

* fixing score-etl-post

* updating comments

* fixing comparison

* create separate field for clarity

* comment fix

* removing healthy food

* fixing bug in score post

* running black and adding comment

* Update pickles and add a helpful notes to README

Co-authored-by: Shelby Switzer <shelby.switzer@cms.hhs.gov>
2021-12-10 10:16:22 -05:00
Lucas Merrill Brown
f91de51a75
Issue 1007 continued: Re-ordering fields for clarity (#1014) 2021-12-09 11:07:37 -05:00
Lucas Merrill Brown
524b822651
Issue 1007: remove some recent additions to Definition L (#1008) 2021-12-08 10:26:52 -05:00
Lucas Merrill Brown
1a61026ecf
Issue 967: Calculate urban/rural percentiles (#1006) 2021-12-07 17:28:36 -05:00
Lucas Merrill Brown
5706837956
Add NATA cancer risk and respiratory hazard to definition L (#1001) 2021-12-07 12:45:45 -05:00
Lucas Merrill Brown
5a6d6d8557
Issue 954: Add various data sources from Child Opportunity Index (#986)
* Adds four fields:
    * Summer days above 90F
    * Percent low access to healthy food
    * Percent impenetrable surface areas
    * Low third grade reading proficiency

* Each of these four gets added into Definition L in various factors.

* Additionally, I add college attendance fields to the ETL for Census ACS.

* This PR also introduces the notion of "reverse percentiles", relevant to ticket #970.
2021-12-07 11:33:49 -05:00