Commit graph

453 commits

Author SHA1 Message Date
Saran Ahluwalia
7dc2b1f45f final PR stuff 2022-01-13 15:05:25 -05:00
Saran Ahluwalia
0176c770c0 updates to PR review 2022-01-13 14:26:24 -05:00
Saran Ahluwalia
a8820e4146 add details and references 2022-01-13 07:00:03 -05:00
Saran Ahluwalia
f619470286 revert commit 2022-01-12 19:21:39 -05:00
Saran Ahluwalia
a8e94ef8de updates 2022-01-12 19:06:06 -05:00
Saran Ahluwalia
f990c98275 temporary revert field names 2022-01-12 19:01:42 -05:00
Saran Ahluwalia
cf07cc29a1 revert decennial 2022-01-12 18:59:43 -05:00
Saran Ahluwalia
d93377a34b remove acs 2010 2022-01-12 18:55:38 -05:00
Saran Ahluwalia
1856034fe1 undo acs 2022-01-12 18:53:13 -05:00
Saran Ahluwalia
f6891f84fa undo cdcplaces 2022-01-12 18:52:10 -05:00
Saran Ahluwalia
321dadf6d5 undo score module 2022-01-12 18:46:07 -05:00
Saran Ahluwalia
159516df80 added PR fixes 2022-01-12 17:31:34 -05:00
Saran Ahluwalia
38990a1449 Merge branch 'esfoobar-usds/1062-implement-changes-export-files' into issue-239-saran-ahluwalia 2022-01-12 13:33:00 -05:00
Saran Ahluwalia
5703ef1b27 updates to comparison tool 2022-01-12 13:10:34 -05:00
Saran Ahluwalia
6e869ed025 PR feedback 2022-01-12 13:10:34 -05:00
Saran Ahluwalia
fccd9d978c clarify fieldnames 2022-01-12 13:10:34 -05:00
Saran Ahluwalia
c1e35fb2f5 details on ranked percentiles 2022-01-12 13:10:34 -05:00
Saran Ahluwalia
d56a084df7 typo 2022-01-12 13:10:34 -05:00
Saran Ahluwalia
c0b8580791 parallelism 2022-01-12 13:10:34 -05:00
Saran Ahluwalia
7e6dd1d3db added revised 2022-01-12 13:10:34 -05:00
Saran Ahluwalia
00590bc086 added revisions and grammar 2022-01-12 13:10:34 -05:00
Saran Ahluwalia
7ffd80d247 added scoring comparison 2022-01-12 13:10:34 -05:00
Saran Ahluwalia
0d464e518b revise merge 2022-01-12 13:10:34 -05:00
Shaun Verch
0e3ca6bf30
Run markdown link checker only on markdown changes (#1117)
* Run markdown link checker only on markdown changes

This makes sure that this action only runs if the pull request is on the
main branch and is changing markdown files.

Fixes: https://github.com/usds/justice40-tool/issues/1114

* Remove push to main trigger

* Ignore eeoc.gov in link checker

Sometimes it seems down from the perspective of github actions.
2022-01-12 13:01:52 -05:00
Lucas Merrill Brown
114e6b765a
Issue 1129: remove deprecated field other_census_tract_fields_to_keep (#1130) 2022-01-12 10:16:09 -05:00
Jorge Escobar
c6ed82fd2e checkpoint 2022-01-11 15:03:18 -05:00
Jorge Escobar
9ab75a156f score passing 2022-01-11 14:07:04 -05:00
Shaun Verch
0abf04d6c2
Remove requirements.txt as a dependency (#1111)
* Remove requirements.txt as a dependency

This converts both docker and tox to use poetry, eliminating usage of
requirements.txt in both flows.

- In tox, uses the tox-poetry package which installs dependencies from
  the lockfile.
- In docker, uses
  https://stackoverflow.com/questions/53835198/integrating-python-poetry-with-docker
  as a reference.

* Don't copy pyproject.toml

* Remove obsoleted docs about requirements.txt

* Add --full-trace option to pytest

* Fix liccheck

liccheck works with requirements.txt, not with poetry, so there needs to
be an extra translation step.

* TEMP: Add WIP fix for pandas issue

This is just to see if the github actions would pass once this fix gets
merged, but it's being reviewed separately.

* Revert "TEMP: Add WIP fix for pandas issue"

This reverts commit 06e38e8cc77f5f3105c6e7a9449901db67aa1c82.
2022-01-10 16:43:56 -05:00
Shaun Verch
f90ec76295
More obviously call out deploy time in quickstart (#1113)
Just to set expectation that this will take a while, so new users don't
think they did something wrong.
2022-01-10 16:34:43 -05:00
Jorge Escobar
a995542e70 checkpoint 2022-01-07 16:47:42 -05:00
Jorge Escobar
7029a0dff5 pandas error 2022-01-06 12:43:25 -05:00
Jorge Escobar
d01bbc7dfa Merge branch 'main' into esfoobar-usds/1062-implement-changes-export-files 2022-01-06 12:18:51 -05:00
Vim
7a623101e2
Add category header and updates from Living Copy (#1084)
* Add category header (Indicator and Percentile)

- add local styling
- override accordion-content styles to push into parent container
- intl copy
- update snapshots

* Add latest Living Copy

- refactor dataset cards to allow for any styling in description, used in and responsible party
- add percent for HS edu indicator
- updates snapshots

* Update copy from Living Copy doc

-update snapshots

* Add methodology to copy

* Remove link from readme file based on md checker

* Comment out certain markdown links from checker

* Comment out markdown links that render in browser
2022-01-05 14:37:47 -08:00
Saran Ahluwalia
56644698ff
Address rounding issue in Pandas series to floor numerically unstable values (#1085)
* wip - added tests - 1 failing

* added check for empty series + added test

* passing tests

* parallelism in variable assingnment choice

* resolve merge conflicts

* variable name changes

* cleanup logic and move comments out of main code execution + add one more test for an extreme example eith -np.inf

* cleanup logic and move comments out of main code execution + add one more test for an extreme example eith -np.inf

* revisions to handle type ambiguity

* fixing tests

* fix pytest

* fix linting

* fix pytest

* reword comments

* cleanup comments

* cleanup comments - fix typo

* added type check and corresponding test

* added type check and corresponding test

* language cleanup

* revert

* update picke fixture

Co-authored-by: Jorge Escobar <jorge.e.escobar@omb.eop.gov>
2022-01-05 17:03:37 -05:00
Shaun Verch
93595b7bb4
Re-export requirements.txt to fix version errors (#1099)
* Re-export requirements.txt to fix version errors

The version of lxml in this file had a known vulnerability that got
caught by the "safety" checker, but it is updated in the poetry files.

Regenerated using:
https://github.com/usds/justice40-tool/tree/main/data/data-pipeline#miscellaneous

* Fix lint error

* Run lint on all envs and add comments

* Ignore testst that fail lint because of dev deps

* Ignore medium.com in link checker

It's returning 403s to github actions...
2022-01-05 15:58:24 -05:00
Shaun Verch
2aa79a334c
Add worst case scenario rollback plan (#1069)
Fixes: https://github.com/usds/justice40-tool/issues/947

This is ensuring we at least have some way to rollback in the worst case
scenario of the data being corrupted somehow, and our upstream data
sources being down in a way that prevents us from regenerating any
datasets. See
https://github.com/usds/justice40-tool/issues/946#issuecomment-989155252
for an analysis of the different ways the pipeline can fail and their
impacts.

This guide is very brittle to pipeline changes, and should be considered
a temporary solution for the worst case. I think that someday the github
actions should run more frequently and write to different paths, perhaps
with a timestamp. That would make rolling back more straightforward, as
a previous version of the data would already exist at some path in s3.
2022-01-05 11:17:43 -05:00
Jorge Escobar
a16bf7cf58 passing score 2022-01-04 16:58:08 -05:00
Jorge Escobar
c9ee6a43c1 cdc_places update 2022-01-03 18:14:36 -05:00
Jorge Escobar
006493ab24 Merge branch 'main' into esfoobar-usds/1062-implement-changes-export-files 2022-01-03 15:53:41 -05:00
Saran Ahluwalia
a4137fdc98
Add Michigan EJ Screen into data-pipeline's ETL and provide automated scoring and statistics outputs (#1091)
* draft wip

* initial commit

* clear output from notebook

* revert to 65ceb7900f

* draft wip

* initial commit

* clear output from notebook

* revert to 65ceb7900f

* make michigan prefix for readable

* standardize Michigan names and move all constants from class into field names module

* standardize Michigan names and move all constants from class into field names module

* include only pertinent columns for scoring comparison tool

* michigan EJSCREEN standardization

* final PR feedback

* added exposition and summary of Michigan EJSCREEN

* added exposition and summary of Michigan EJSCREEN

* fix typo

Co-authored-by: Saran Ahluwalia <ahlusar.ahluwalia@gmail.com>
2021-12-31 15:38:52 -05:00
Saran Ahluwalia
24f8eb93c4
Tree Equity Output: Change output from Geojson to CSV format for easier analysis (#1089)
Added Tree Equity

* draft wip

* revised documentation

* revised documentation

* revised documentation and defer to super

* change word in logger

* fix flake 8

* address nit

Co-authored-by: Saran Ahluwalia <ahlusar.ahluwalia@gmail.com>
2021-12-30 17:17:28 -05:00
Vim
356e16950f
Fix territory shortcuts when census tract is selected (#1082)
* Refactor map click event architecture

- combine territory map clickHandlers
- centers AS on the map

* Center US on the map

- make the east and west coast both viewable
- make clicking on the 48, show the same zoom/lat/long as initial map
- centers Hawaii on map

* Update link to map performance

* Explicitly show links as the links return a 403

* Removes link and spells link out
2021-12-28 15:30:22 -08:00
Lucas Merrill Brown
beb0eea5cc
Alternative definition of DACs for comparison (#1068)
* Alternative energy-related definition of DACs
2021-12-27 12:05:59 -05:00
Jorge Escobar
45b33ea55f Merge branch 'main' into esfoobar-usds/1062-implement-changes-export-files 2021-12-21 16:52:36 -05:00
Jorge Escobar
cec0c42102 Lucas and Beth column order changes 2021-12-21 14:32:24 -05:00
Kameron Kerger
e15bb52bad
548-update-pdf (#1081)
latest pdf copy with links now added for each data source
2021-12-21 14:12:20 -05:00
Vim
409c7238ae
Make latest copy changes from Living Copy (#1055)
* Make latest copy changes

- update snapshots

* Update cypress test on feedback link

- update snapshot

* Update side panel and copy

- update snapshots

* Make 2nd EO link open in new tab

* Add latest changes from Living copy

* Add back HS indicator to map

* Add "X of Y thresholds exceed" to side panel

- update snapshots

* Update with latest copy

* Update to latest copy

- make BETA pill in logo bold
- correct exceed to exceeded
- update snapshots
- update page title to Meth & data

* Update total indicators to 21

* Update snapshot
2021-12-17 13:17:57 -08:00
Jorge Escobar
943f6283f4 Kameron changes 2021-12-17 15:16:43 -05:00
Jorge Escobar
fb8be1a8ce Download column order completed 2021-12-17 13:48:31 -05:00
Lucas Merrill Brown
0d57dd572b
Stop swallowing Census API errors (#1051) 2021-12-16 10:54:41 -05:00