When implementing definition M for the score, the variable names were not yet updated. For example:
This legacy field naming:
```
UNEMPLOYMENT_LOW_HS_EDUCATION_FIELD = (
f"Greater than or equal to the {PERCENTILE}th percentile for unemployment"
" and has low HS education"
)
```
Should actually be renamed something like this:
```
UNEMPLOYMENT_LOW_HS_LOW_HIGHER_ED_FIELD = (
f"Greater than or equal to the {PERCENTILE}th percentile for unemployment"
" and has low HS education and low higher ed attendance"
)
```
This PR is for the backend updates for this -- keeping the old fields, and adding new, Score M specific fields as listed below:
- [x] `field_names`: add new fields to capture low_higher_ed
- [x] `score_m`: replace old fields with new fields
- [x] `DOWNLOADABLE_SCORE_COLUMNS`: replace old fields with new fields
- [x] `TILES_SCORE_COLUMNS`: replace old fields with new fields
* Fix territory shortcuts on mobile
- remove zoom controls on mobile
- center search on mobile
- move up the territoriy shortcuts on mobile
- increase height of map on mobile in portrait mode
- update tests
* Reduce map height
- troubleshooting staging map height
* Removes null render in map
* Comment out conditional render of NavControls
* Revert height changes
* Remove MapSearch component
* Remove territory styling
* Console.log the device width
* Add logging width/mobile onLoad()
* Add isMobile to map height prop
* Swap conditional order of map height
* Add isMobileMapState to map height
* Add back all changes and force height to non-100%
- staging mobile seems to have height at 100% regardless of conditional. This will test this.
- this will break desktop on staging and is purposeful
* Remove API key for mapbox
* Add height as 90% to check conditional
* trying isMobile and windowWidth
* is not mobile and width > 1024
* use a function instead of a conditional
* Modify getHeight function
- remove Cypress tests from GHA to speed up build / deplot to staging
* Console.log windowWidth and constants.desktop
* Add boolean console.log
* Place values in DOM itself
* add height to div
* log out types
* add types nicely formatted
* Move height styling on map from inline to parent
- use a media query to detect mobile and set the map parents height via SASS
* Add back cypress tests
* Refactor Footer component
- Make Footer component align to normal folder structure
- Make links in Footer use LinkTypeWrapper, which will allow icon to be added to all external links in one place
* Add Trussworks link to LinkTypeWrapper
* Add icon to SurveyButton comp
* Add launch icon to About page cards
* Add launch icon to FederalRegister link
* Refactor Methodology page
- Add Source and Available for fields to dataset Cards
- refactor data structure for dataset Cards to handle more than 1 source
- update constants file
- modify SASS for source list items
- update snapshots
* Add copy changes; How you can help and territories
* Update methodology section with copy changes
* Swap out all email links with <LinkTypeWrapper>
- update tests
* updated loss rate rounding
* fixing a typo in variable name
* fixing typo in variable name
* oops, now ready to push
* updated pickle with float for loss rate columns
* updated a typo, now multiplies all loss rates by 100 consistent with other pcts
* updated with final pickles, all tests passing
* updated incorporating lucas pr comments
* changed literal to field name
* Allow open source map
- create new getOSBaseMap function
- set secret on GitHub actions
- read secret into J40Map.tsx
- add secret to gatsby-config
- remove secret from .env.*
* Fix typo on yml file
* Add placeholder for API TOKEN
- add placeholder in docker compose
- add placeholder in .env.dev
- remove console.log
* Add Mapbox API Token to main deploy
* Add additional base layers behind feature flags
- add voyager base layer under vy
- add positron base layer under ps
* Add mapbox base layer
- requires API token
* Add mapbox layers with API token in URL
* Add base map layers from mapTiler
- add comments to mapping components
- add mapTiler base maps behind feature flags
* Comment out intermittent cypress test failures
* Add flag to remove label layer
* Add MapBox Raster and Vector tiles
- tilesets are commented out until more information is provided by Mikel
* Remove white layer on non-prioritized features
- removes makePaint function
- adds Todo to renaming constants
* refactor all contants to have standard naming
- renames layers, sources, colors, opacity, and zoom
- Adds a large amount of comments to understand how this map works
* remove some instances of mapbox-gl
- this the first step in having only maplibre-gl being used in app
* Remove chroma.js
- chroma.js was used in the fill function of makeStyle. This was used to create a gradient between non-prio, threshold and prio. Since these 3 step values are no longer needed this function along with the libraries it used is not removed.
* Add comments on mapbox base layer
- adds apiaccesstoken
* set basemap to mapbox and move all layers to Map
* Add API KEY to .env, adjust opacity of prio'd CBTs
- remove this function as it is no longer being used
- add comments on map
- create a high layer opacity and low layer opacity
- add API KEY to prod and dev .env
- add MapBox API key to deploy_staging
* add logging to troubleshoot API KEY
* Remove temp echo of API KEY
* Add GHA env var to gatsby config
* Remove API KEY from GitHub and GHA
* Install and run pandas-vet
This doesn't fix the errors, but it can give us a starting point for the
discussion of which of these errors we care about.
* Ignore the errors for now
* Ignore eeoc.gov in link checker
Sometimes it seems down from the perspective of github actions.
* Add pyproject.toml to fix docker compose build
Even though we want to use locked dependencies, pyproject.toml is still
required.
* update Dockerfile
Co-authored-by: Jorge Escobar <83969469+esfoobar-usds@users.noreply.github.com>
* Add category header (Indicator and Percentile)
- add local styling
- override accordion-content styles to push into parent container
- intl copy
- update snapshots
* Add latest Living Copy
- refactor dataset cards to allow for any styling in description, used in and responsible party
- add percent for HS edu indicator
- updates snapshots
* Update copy from Living Copy doc
-update snapshots
* Add methodology to copy
* Remove link from readme file based on md checker
* Comment out certain markdown links from checker
* Comment out markdown links that render in browser
* Add missing methodology to Clean Water
* Run markdown link checker only on markdown changes
This makes sure that this action only runs if the pull request is on the
main branch and is changing markdown files.
Fixes: https://github.com/usds/justice40-tool/issues/1114
* Remove push to main trigger
* Ignore eeoc.gov in link checker
Sometimes it seems down from the perspective of github actions.
* Remove requirements.txt as a dependency
This converts both docker and tox to use poetry, eliminating usage of
requirements.txt in both flows.
- In tox, uses the tox-poetry package which installs dependencies from
the lockfile.
- In docker, uses
https://stackoverflow.com/questions/53835198/integrating-python-poetry-with-docker
as a reference.
* Don't copy pyproject.toml
* Remove obsoleted docs about requirements.txt
* Add --full-trace option to pytest
* Fix liccheck
liccheck works with requirements.txt, not with poetry, so there needs to
be an extra translation step.
* TEMP: Add WIP fix for pandas issue
This is just to see if the github actions would pass once this fix gets
merged, but it's being reviewed separately.
* Revert "TEMP: Add WIP fix for pandas issue"
This reverts commit 06e38e8cc77f5f3105c6e7a9449901db67aa1c82.
* Add category header (Indicator and Percentile)
- add local styling
- override accordion-content styles to push into parent container
- intl copy
- update snapshots
* Add latest Living Copy
- refactor dataset cards to allow for any styling in description, used in and responsible party
- add percent for HS edu indicator
- updates snapshots
* Update copy from Living Copy doc
-update snapshots
* Add methodology to copy
* Remove link from readme file based on md checker
* Comment out certain markdown links from checker
* Comment out markdown links that render in browser
* wip - added tests - 1 failing
* added check for empty series + added test
* passing tests
* parallelism in variable assingnment choice
* resolve merge conflicts
* variable name changes
* cleanup logic and move comments out of main code execution + add one more test for an extreme example eith -np.inf
* cleanup logic and move comments out of main code execution + add one more test for an extreme example eith -np.inf
* revisions to handle type ambiguity
* fixing tests
* fix pytest
* fix linting
* fix pytest
* reword comments
* cleanup comments
* cleanup comments - fix typo
* added type check and corresponding test
* added type check and corresponding test
* language cleanup
* revert
* update picke fixture
Co-authored-by: Jorge Escobar <jorge.e.escobar@omb.eop.gov>
* Re-export requirements.txt to fix version errors
The version of lxml in this file had a known vulnerability that got
caught by the "safety" checker, but it is updated in the poetry files.
Regenerated using:
https://github.com/usds/justice40-tool/tree/main/data/data-pipeline#miscellaneous
* Fix lint error
* Run lint on all envs and add comments
* Ignore testst that fail lint because of dev deps
* Ignore medium.com in link checker
It's returning 403s to github actions...
Fixes: https://github.com/usds/justice40-tool/issues/947
This is ensuring we at least have some way to rollback in the worst case
scenario of the data being corrupted somehow, and our upstream data
sources being down in a way that prevents us from regenerating any
datasets. See
https://github.com/usds/justice40-tool/issues/946#issuecomment-989155252
for an analysis of the different ways the pipeline can fail and their
impacts.
This guide is very brittle to pipeline changes, and should be considered
a temporary solution for the worst case. I think that someday the github
actions should run more frequently and write to different paths, perhaps
with a timestamp. That would make rolling back more straightforward, as
a previous version of the data would already exist at some path in s3.
* draft wip
* initial commit
* clear output from notebook
* revert to 65ceb7900f
* draft wip
* initial commit
* clear output from notebook
* revert to 65ceb7900f
* make michigan prefix for readable
* standardize Michigan names and move all constants from class into field names module
* standardize Michigan names and move all constants from class into field names module
* include only pertinent columns for scoring comparison tool
* michigan EJSCREEN standardization
* final PR feedback
* added exposition and summary of Michigan EJSCREEN
* added exposition and summary of Michigan EJSCREEN
* fix typo
Co-authored-by: Saran Ahluwalia <ahlusar.ahluwalia@gmail.com>
* Refactor map click event architecture
- combine territory map clickHandlers
- centers AS on the map
* Center US on the map
- make the east and west coast both viewable
- make clicking on the 48, show the same zoom/lat/long as initial map
- centers Hawaii on map
* Update link to map performance
* Explicitly show links as the links return a 403
* Removes link and spells link out
* Make latest copy changes
- update snapshots
* Update cypress test on feedback link
- update snapshot
* Update side panel and copy
- update snapshots
* Make 2nd EO link open in new tab
* Add latest changes from Living copy
* Add back HS indicator to map
* Add "X of Y thresholds exceed" to side panel
- update snapshots
* Update with latest copy
* Update to latest copy
- make BETA pill in logo bold
- correct exceed to exceeded
- update snapshots
- update page title to Meth & data
* Update total indicators to 21
* Update snapshot
* First pass of updating documentation for new users
Trying to look at this from the perspective of someone new to the
project, and create some pathways to make it easier for people to get to
the content they are looking for.
* Make it clear that docker is doing the setup
* Link installation again from the main README
* Add some docs about the github actions
* Add markdown link check
* Move git installation first
* Add config for markdown link checker
* Fix some links
* Correct handling of repo root relative links
* Fix broken links in data roadmap
* Fix more broken links
* Fix more links
* Ignore link that's returning a 403 to the checker
It actually works if you go in a browser.
* Fix another broken link
* Ignore more urls that don't work
* Update the readme under docs
* Add some more dataset links
* More strongly call out the quickstart
* Try to call out even more the quickstart link
* Fix dead links
* Add note about initialization time
* Remove broken link from spanish install guide
These will be updated later with a full translation
