Commit graph

768 commits

Author SHA1 Message Date
Emma Nechamkin
389eb59ac4
Adding island area indicators to the tiles (#1213)
This updates the backend to produce tile data with island indicators / island fields. 

Contains:
- new tile codes for island data
- threshold column that specifies number of thresholds to show
- ui experience column that specifies which ui experience to show

TODO: Drop the logger info message from main :)
2022-02-09 20:33:42 -05:00
Emma Nechamkin
b86450c72b
Remove USVI and Guam territories from data and include/show on map American Samoa and Mariana Islands (#1248)
This updates the tile data so that guam and usvi do not appear in the tiles csv, from issue 1003
2022-02-09 15:23:37 -05:00
Lucas Merrill Brown
43e005cc10
Issue 1075: Add refactored ETL tests to NRI (#1088)
* Adds a substantially refactored ETL test to the National Risk Index, to be used as a model for other tests
2022-02-08 19:05:32 -05:00
Jorge Escobar
f5fe8d90e2
Excel formatting and tract id ordering (#1172)
* excel formatting and tract id ordering

* lint

* lint try $2

* lint 3

* addressed comments

* typo
2022-02-04 18:35:45 -05:00
Emma Nechamkin
6a00b29f5d
Adding VA and CO ETL from mapping for environmental justice (#1177)
Adding the mapping for environmental justice data, which contains information about VA and CO, to the ETL pipeline.
2022-02-04 10:00:41 -05:00
Jorge Escobar
1d399d3ca9
Tox Security Fix (#1242)
* checkpoint

* safety ignore

* update python matrix for data checks

* downloading census once
2022-02-03 17:05:51 -05:00
Vim
fb69a09485
Update FE to Def M (#1235)
* Update to Def M

- update first column of meth page, formula intro, ID,AND, & remove THEN
- remove Low income box under download box
- update id as disadv. in each category
- modify AND clause in each category
- add dataset Higher ed enrollment
- update snapshots

* Fix HS and higher ed DOM id in links

* Update side panel description to def m

* Fix typo on AND part of the formula

* Update snapshots

* Update snapshot

* Resolve rebase error where Send feedback was removed

- add back send feedback button
- update snapshots

* Add back blue indicator boolean
2022-02-03 12:07:44 -08:00
Vim
9b4d45a96c
Add Send feedback button census track feedback (#1232)
* Add send feedback btn to side panel

* Update contact page copy

* remove redundant constant
2022-02-03 09:39:49 -08:00
Vim
4b95583117
Update boolean for HS degree indicator to LHE (#1234) 2022-02-03 08:23:07 -08:00
Vim
fadc875934
Add tooltip to territory buttons (#1220) 2022-02-02 15:23:59 -08:00
Jorge Escobar
722c738a5b re-enabling cypress 2022-02-02 18:21:04 -05:00
Vim
00f522d66c
trivial comment (#1229) 2022-02-02 15:08:44 -08:00
Jorge Escobar
ff2958a8d2 closed PR GHA 2022-02-02 18:01:43 -05:00
Vim
b726ea04ad
Add trivial message (#1228) 2022-02-02 14:48:26 -08:00
Jorge Escobar
8187d028aa suspending cypress for q/a 2022-02-02 17:47:41 -05:00
Vim
0201c6cd68
remove UT banner comment (#1227) 2022-02-02 14:33:17 -08:00
Jorge Escobar
d55dc8c5e3 update PR number 2022-02-02 17:21:10 -05:00
Vim
9dacd66673
remove UT banner comment (#1225) 2022-02-02 13:56:51 -08:00
Vim
1c7b043cdb
change deploy main to be on push (#1224) 2022-02-02 16:44:10 -05:00
Emma Nechamkin
49868401be
Updating field names to match score M definitions (#1190)
When implementing definition M for the score, the variable names were not yet updated. For example:

This legacy field naming: 
```
UNEMPLOYMENT_LOW_HS_EDUCATION_FIELD = (
    f"Greater than or equal to the {PERCENTILE}th percentile for unemployment"
    " and has low HS education"
)
``` 

Should actually be renamed something like this:
```
UNEMPLOYMENT_LOW_HS_LOW_HIGHER_ED_FIELD = (
    f"Greater than or equal to the {PERCENTILE}th percentile for unemployment"
    " and has low HS education and low higher ed attendance"
)
``` 

This PR is for the backend updates for this -- keeping the old fields, and adding new, Score M specific fields as listed below: 
- [x] `field_names`: add new fields to capture low_higher_ed
- [x] `score_m`: replace old fields with new fields 
- [x] `DOWNLOADABLE_SCORE_COLUMNS`: replace old fields with new fields
- [x] `TILES_SCORE_COLUMNS`: replace old fields with new fields
2022-02-01 18:54:43 -05:00
Jorge Escobar
403a490985
Esfoobar usds/488 generate score per commit pr (#1211)
* Score run on every commit to data PR

* testing score run

* source aws
2022-01-31 16:07:21 -05:00
Jorge Escobar
54e9f50625
Score run on every commit to data PR (#1210) 2022-01-31 14:19:49 -05:00
Vim
f791a25a76
Fix territory shortcuts on mobile (#1145)
* Fix territory shortcuts on mobile

- remove zoom controls on mobile
- center search on mobile
- move up the territoriy shortcuts on mobile
- increase height of map on mobile in portrait mode
- update tests

* Reduce map height

- troubleshooting staging map height

* Removes null render in map

* Comment out conditional render of NavControls

* Revert height changes

* Remove MapSearch component

* Remove territory styling

* Console.log the device width

* Add logging width/mobile onLoad()

* Add isMobile to map height prop

* Swap conditional order of map height

* Add isMobileMapState to map height

* Add back all changes and force height to non-100%

- staging mobile seems to have height at 100% regardless of conditional. This will test this.
- this will break desktop on staging and is purposeful

* Remove API key for mapbox

* Add height as 90% to check conditional

* trying isMobile and windowWidth

* is not mobile and width > 1024

* use a function instead of a conditional

* Modify getHeight function

- remove Cypress tests from GHA to speed up build / deplot to staging

* Console.log windowWidth and constants.desktop

* Add boolean console.log

* Place values in DOM itself

* add height to div

* log out types

* add types nicely formatted

* Move height styling on map from inline to parent

- use a media query to detect mobile and set the map parents height via SASS

* Add back cypress tests
2022-01-31 08:14:02 -08:00
dependabot[bot]
8b72f743e3
Bump pillow from 8.4.0 to 9.0.0 in /data/data-pipeline (#1136)
* Bump pillow from 8.4.0 to 9.0.0 in /data/data-pipeline

Bumps [pillow](https://github.com/python-pillow/Pillow) from 8.4.0 to 9.0.0.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](https://github.com/python-pillow/Pillow/compare/8.4.0...9.0.0)

---
updated-dependencies:
- dependency-name: pillow
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* pillow bump

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jorge Escobar <83969469+esfoobar-usds@users.noreply.github.com>
2022-01-27 18:19:49 -05:00
dependabot[bot]
7201304e80
Bump cached-path-relative from 1.0.2 to 1.1.0 in /client (#1191)
Bumps [cached-path-relative](https://github.com/ashaffer/cached-path-relative) from 1.0.2 to 1.1.0.
- [Release notes](https://github.com/ashaffer/cached-path-relative/releases)
- [Commits](https://github.com/ashaffer/cached-path-relative/commits)

---
updated-dependencies:
- dependency-name: cached-path-relative
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-27 15:04:06 -08:00
dependabot[bot]
4a83ae458e
Bump ipython from 7.28.0 to 7.31.1 in /data/data-pipeline (#1169)
Bumps [ipython](https://github.com/ipython/ipython) from 7.28.0 to 7.31.1.
- [Release notes](https://github.com/ipython/ipython/releases)
- [Commits](https://github.com/ipython/ipython/compare/7.28.0...7.31.1)

---
updated-dependencies:
- dependency-name: ipython
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-27 17:36:14 -05:00
Jorge Escobar
2b35a8937a
Hot fix for Score M (#1182)
* fixes

* pr feedback

* tuple
2022-01-27 17:22:39 -05:00
Vim
f5f7a254af
Add sources/layers back to OS map style file (#1194) 2022-01-27 14:13:25 -08:00
Vim
ba3c370c53
Update file with branching naming (#1188) 2022-01-27 07:58:57 -08:00
dependabot[bot]
ceaec7c5fa
Bump nanoid from 3.1.23 to 3.2.0 in /client (#1171)
Bumps [nanoid](https://github.com/ai/nanoid) from 3.1.23 to 3.2.0.
- [Release notes](https://github.com/ai/nanoid/releases)
- [Changelog](https://github.com/ai/nanoid/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ai/nanoid/compare/3.1.23...3.2.0)

---
updated-dependencies:
- dependency-name: nanoid
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-26 15:37:25 -08:00
Vim
e677df794d
Add external link icons & all outstanding changes from Living copy (#1173)
* Refactor Footer component

- Make Footer component align to normal folder structure
- Make links in Footer use LinkTypeWrapper, which will allow icon to be added to all external links in one place

* Add Trussworks link to LinkTypeWrapper

* Add icon to SurveyButton comp

* Add launch icon to About page cards

* Add launch icon to FederalRegister link

* Refactor Methodology page

- Add Source and Available for fields to dataset Cards
- refactor data structure for dataset Cards to handle more than 1 source
- update constants file
- modify SASS for source list items
- update snapshots

* Add copy changes; How you can help and territories

* Update methodology section with copy changes

* Swap out all email links with <LinkTypeWrapper>

- update tests
2022-01-26 13:12:33 -08:00
Emma Nechamkin
4c7d729cf7
Issue 1140 loss rate rounding (#1170)
* updated loss rate rounding

* fixing a typo in variable name

* fixing typo in variable name

* oops, now ready to push

* updated pickle with float for loss rate columns

* updated a typo, now multiplies all loss rates by 100 consistent with other pcts

* updated with final pickles, all tests passing

* updated incorporating lucas pr comments

* changed literal to field name
2022-01-26 13:57:45 -05:00
Vim
c6642ce586
Add methodology change process (#1167) 2022-01-21 09:58:03 -08:00
Jorge Escobar
9712a21f38
Update CODEOWNERS (#1166) 2022-01-21 12:56:40 -05:00
Vim
0c3213875f
Replace SITE_URL with secret URLs (#1165) 2022-01-21 09:24:27 -08:00
Vim
f4926376fa
Allow open source map with new API key (#1155)
* Allow open source map

- create new getOSBaseMap function
- set secret on GitHub actions
- read secret into J40Map.tsx
- add secret to gatsby-config
- remove secret from .env.*

* Fix typo on yml file

* Add placeholder for API TOKEN

- add placeholder in docker compose
- add placeholder in .env.dev
- remove console.log

* Add Mapbox API Token to main deploy
2022-01-18 12:16:33 -08:00
Lucas Merrill Brown
18f299c5f8
Issue 1141: Definition M (#1151) 2022-01-18 14:56:55 -05:00
Saran Ahluwalia
a07bf752b0
Notebook investigating NHPD as a source for providing contemporary foreclosure data (#1012)
Co-authored-by: Saran Ahluwalia <sarahluw@cisco.com>
2022-01-18 13:08:27 -05:00
dependabot[bot]
b7f1483bd2
Bump follow-redirects from 1.14.1 to 1.14.7 in /client (#1138)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.14.1 to 1.14.7.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.14.1...v1.14.7)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-18 09:52:25 -08:00
dependabot[bot]
7cba80a805
Bump engine.io from 4.1.1 to 4.1.2 in /client (#1139)
Bumps [engine.io](https://github.com/socketio/engine.io) from 4.1.1 to 4.1.2.
- [Release notes](https://github.com/socketio/engine.io/releases)
- [Changelog](https://github.com/socketio/engine.io/blob/4.1.2/CHANGELOG.md)
- [Commits](https://github.com/socketio/engine.io/compare/4.1.1...4.1.2)

---
updated-dependencies:
- dependency-name: engine.io
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-18 09:52:03 -08:00
Saran Ahluwalia
87e08f5fe1
CDC SVI Index: Additions to data-pipeline and comparison tool (#1096)
* wip

* working

* working

* rename

* documentation

* add link

* add readme

* update fieldnames

* typo

* add comparison tool

* revise wording

* variable change for FIPS

* workding

* wording in readme

* cleanup wording

* update comparison tool

* final tune up

* grammar and punctuation in the documentation

* period

* cleanup comments

* added revisions

* parallelism

* PR feedback from Lucas

* remove extraneous fields from comparison tool

* style

* updates

* remove themes

* formatting

* remove referenes to percentile rank

* remove referenes to percentile rank

* typo in fieldnames

* updates based on feedback from Lucas

* fieldnames formatting

* fix broken markdown link

Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov>
2022-01-14 14:52:37 -05:00
Saran Ahluwalia
95a14adb35
Added Census Tract Aggregated Micro-data from EPA Risk-Screening Environmental Indicators (RSEI) model (#1101)
* added initial source code - todo is comparison tool

* added values

* rename fields

* check geoid

* added black

* added revisions

* added clean up to comments

* more comments

* formatting

* cleanup and address PR feedback

* fix changes

* final path changes

* style

* PR feedback

* added final PR comment

* fix flake 8

* add revisions
2022-01-14 13:50:49 -05:00
Saran Ahluwalia
a98ea35f74
Maryland EJSCREEN Addition to comparison tool (#1143)
* finalized

* cleanup notebook

* cleanup

* run black
2022-01-14 13:26:48 -05:00
Saran Ahluwalia
2604b66cf7
Fix errors and improve code quality and readability in Health Scores (#1147)
* run black on health_score.py

* to_numpy() versus values - see https://pandas.pydata.org/pandas-docs/version/0.24.0rc1/api/generated/pandas.Series.to_numpy.html
2022-01-14 13:11:47 -05:00
Vim
667678f20e
Add additional base layers behind feature flags (#945)
* Add additional base layers behind feature flags

- add voyager base layer under vy
- add positron base layer under ps

* Add mapbox base layer

- requires API token

* Add mapbox layers with API token in URL

* Add base map layers from mapTiler

- add comments to mapping components
- add mapTiler base maps behind feature flags

* Comment out intermittent cypress test failures

* Add flag to remove label layer

* Add MapBox Raster and Vector tiles

- tilesets are commented out until more information is provided by Mikel

* Remove white layer on non-prioritized features

- removes makePaint function
- adds Todo to renaming constants

* refactor all contants to have standard naming

- renames layers, sources, colors, opacity, and zoom
- Adds a large amount of comments to understand how this map works

* remove some instances of mapbox-gl

- this the first step in having only maplibre-gl being used in app

* Remove chroma.js

- chroma.js  was used in the fill function of makeStyle. This was used to create a gradient between non-prio, threshold and prio. Since these 3 step values are no longer needed this function along with the libraries it used is not removed.

* Add comments on mapbox base layer

- adds apiaccesstoken

* set basemap to mapbox and move all layers to Map

* Add API KEY to .env, adjust opacity of prio'd CBTs

- remove this function as it is no longer being used
- add comments on map
- create a high layer opacity and low layer opacity
- add API KEY to prod and dev .env
- add MapBox API key to deploy_staging

* add logging to troubleshoot API KEY

* Remove temp echo of API KEY

* Add GHA env var to gatsby config

* Remove API KEY from GitHub and GHA
2022-01-13 12:25:43 -08:00
Jorge Escobar
d686bb856e
Download column order completed (#1077)
* Download column order completed

* Kameron changes

* Lucas and Beth column order changes

* cdc_places update

* passing score

* pandas error

* checkpoint

* score passing

* rounding complete - percentages still showing one decimal

* fixing tests

* fixing percentages

* updating comment

* int percentages! 🎉🎉

* forgot to pass back to df

* passing tests

Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov>
2022-01-13 15:04:16 -05:00
Saran Ahluwalia
98ff4bd9d8
Add experimental Jupyter notebook with Health Scoring Methodology Example for Health Scores (#989)
Co-authored-by: Saran Ahluwalia <sarahluw@cisco.com>
2022-01-13 14:43:27 -05:00
Shaun Verch
4cec1bb37e
Install and run pandas-vet (#1119)
* Install and run pandas-vet

This doesn't fix the errors, but it can give us a starting point for the
discussion of which of these errors we care about.

* Ignore the errors for now

* Ignore eeoc.gov in link checker

Sometimes it seems down from the perspective of github actions.
2022-01-13 13:17:30 -05:00
Shaun Verch
73d6aa937d
Add pyproject.toml to fix docker compose build (#1131)
* Add pyproject.toml to fix docker compose build

Even though we want to use locked dependencies, pyproject.toml is still
required.

* update Dockerfile

Co-authored-by: Jorge Escobar <83969469+esfoobar-usds@users.noreply.github.com>
2022-01-13 13:05:32 -05:00
Vim
b042e32ea0
Add methodology, living copy updates, fixes markdown checker failures (#1104)
* Add category header (Indicator and Percentile)

- add local styling
- override accordion-content styles to push into parent container
- intl copy
- update snapshots

* Add latest Living Copy

- refactor dataset cards to allow for any styling in description, used in and responsible party
- add percent for HS edu indicator
- updates snapshots

* Update copy from Living Copy doc

-update snapshots

* Add methodology to copy

* Remove link from readme file based on md checker

* Comment out certain markdown links from checker

* Comment out markdown links that render in browser

* Add missing methodology to Clean Water
2022-01-13 01:36:02 -08:00