Commit graph

171 commits

Author SHA1 Message Date
Nat Hillard
a2c7b8e1c5 Fixes #467:
If the census download task is run more than once,
us.csv doubles in size and all data is removed from dataframe
2021-08-05 13:53:42 -04:00
Nat Hillard
f75d40dcb0 Merge branch 'main' into nathillardusds/456_project_structure 2021-08-05 13:48:21 -04:00
Jorge Escobar
4d7465c833
Hotfix for fips zip download location + added full-score-run command (#465)
* Hotfix for S3 locations of data sources

* updated README

* lint failures

Co-authored-by: Nat Hillard <Nathaniel.K.Hillard@omb.eop.gov>
2021-08-05 12:55:21 -04:00
Nat Hillard
2c54aee71c updating pylint 2021-08-04 21:15:55 -04:00
Nat Hillard
6c86dd7e4e Fixes for flake8 2021-08-04 21:13:56 -04:00
Nat Hillard
508925618b Running Black 2021-08-04 21:10:45 -04:00
Nat Hillard
dd8c37e06d updating requirements 2021-08-04 21:10:08 -04:00
Nat Hillard
aeb71137e4 updating readme 2021-08-04 12:30:26 -04:00
Nat Hillard
6ec4dafafd a few missed references 2021-08-04 12:30:19 -04:00
Nat Hillard
03df86a368 Fixes #456 - Our data directory should adopt standard python package structure 2021-08-04 11:42:03 -04:00
Jorge Escobar
5cb00ef0ce
Tile Generation Script (#433) 2021-08-03 18:23:57 -04:00
Vim
849437c9d3
adds nvmrc file (#445) 2021-08-02 10:35:18 -07:00
Billy Daly
5504528fdf
Issue 308 python linting (#443)
* Adds flake8, pylint, liccheck, flake8 to dependencies for data-pipeline

* Sets up and runs black autoformatting

* Adds flake8 to tox linting

* Fixes flake8 error F541 f string missing placeholders

* Fixes flake8 E501 line too long

* Fixes flake8 F401 imported but not used

* Adds pylint to tox and disables the following pylint errors:
- C0114: module docstrings
- R0201: method could have been a function
- R0903: too few public methods
- C0103: name case styling
- W0511: fix me
- W1203: f-string interpolation in logging

* Adds utils.py to tox.ini linting, runs black on utils.py

* Fixes import related pylint errors: C0411 and C0412

* Fixes or ignores remaining pylint errors (for discussion later)

* Adds safety and liccheck to tox.ini
2021-08-02 12:16:38 -04:00
Vim
51f7666062
Adds dataset cards to Methodology page (#442)
* intial cards for methodology page

* PR and QA feedback
- adds alert above dataset section
- adds intl
- removes nbsp
- creates directory structure for new components

* revert noUsedLocals flag

* fixed path error

* re-creates scss file to test build failure

* renaming file to troubleshoot build error

* links open in new tabs and removes console.log

* removes units on all scss value that equal 0

* resolving merge conflicts from header merge

* updates snapshots from conflict resolution
2021-08-02 08:49:49 -07:00
TomNUSDS
08e21e5d5b
fix #249 header 2 (#425)
* Header updates
* More pixel alignment for nav
* Update unit test snapshots
I removed a stray `,` from `testHelpers.tsx` which changed the output of several snapshots.
* Fixes from PR review
* Forgot to update snapshots after merge with main
* Removed unused (and outdated) timeline page
* Updated some translation descriptions.
2021-08-02 07:48:19 -07:00
Nat Hillard
8df548f506
Adding ADR PR for associated ADR issue #402 (#437) 2021-08-02 10:17:44 -04:00
Billy Daly
55dabb2b57
Issue 379 tox setup (#405)
* Adds tox as a dev dependency to data/data-pipeline/pyproject.toml: Also updates poetry.lock and requirements.txt

* Adds tox.ini to test build of data/data-pipeline

* Sets up GitHub actions workflow for data/ directory

* Tries to get Data Checks GitHub action to run

* Fixes error with GitHub action

* Migrates data/data-roadmap from setuptools to poetry

* Sets up tox file for data/data-roadmap

* Adds github action for data/data-roadmap

* Fixes syntax error in data-checks.yml

* Second attempt at fixing data-checks.yml

* Export poetry requirements to requirements.txt

* Revert "Migrates data/data-roadmap from setuptools to poetry"

This reverts commit e8367652d43c1c9beee500f792c8f41e1c1fc462.

* Removes pyproject.toml and reverts requirements.txt as well
2021-07-29 14:00:20 -04:00
Shelby Switzer
387ee3a382
Update data documentation and some data steps (#407)
* Minor documentation updates, plus calenvironscreen  S3 URL fix

* Update score comparison docs and code

* Add steps for running the comparison tool
* Update HUD recap ETL to ensure GEOID is imported as a string (if it is
imported as an interger by default it  will strip the beginning "0" from
many IDs)

* Add note about execution time

* Move step from paragraph to list

* Update output dir in README for comp tool

Co-authored-by: Shelby Switzer <shelby.switzer@cms.hhs.gov>
2021-07-29 10:28:52 -04:00
Jorge Escobar
b404fdcc43
Generate Geo-aware scores for all zoom levels (#391)
* generate Geo-aware scores for all zoom levels

* usa high progress

* testing dissolve

* checkpoint

* changing type

* removing breakpoint

* validation notebooks

* quick update

* score validation

* fixes for county merge

* code completed
2021-07-28 16:07:28 -04:00
Nat Hillard
446c8d1f68
Update dataset_request.yaml
Wrapping date with quotes to avoid "Tried to load unspecified class: Date" error
2021-07-28 14:58:17 -04:00
Nat Hillard
7bc494b759 Update issue templates
Adding "task" template for one-off changes
2021-07-28 14:55:28 -04:00
Nat Hillard
84500bc744
Adding template for dataset request (#427) 2021-07-28 14:49:38 -04:00
Nat Hillard
3238e09792
Loading a URL with lat/lng/zoom specified does not work (#414) 2021-07-27 20:51:35 -04:00
Nat Hillard
937770ab3c
Fixes #383 - I want to see the cumulative impact score and associated info for communities that are not prioritized (#386) 2021-07-27 18:23:38 -04:00
Nat Hillard
a2ba9236ab
Clicking territory focus button sometimes focuses on a different CBG, not the actual territory (#412)
* Fixing #410 - Clicking a CBG region, then another, then hitting territory button results in random CBG being selected

* Part of fix for #410 - refactoring selection
logic to remove setFeatureState call.

* Renaming layers for clarity and adding constants

* Using constant for layer identifier

* Fixes #409 - Loading a URL with lat/lng/zoom specified occasionally does not work . We now parse the URL on page before initializing the map. Also adds tests for this
2021-07-27 18:21:47 -04:00
Vim
36f43b2d44
adds map side panel (#406)
* initial map side panel

* componentize MapSidePanel

* remove selection from J40Map

* adds isFeatureSelected to toggle component

* filters data from server for client UI

* styling and refactor

* added TODO

* adds styling to intro and pairing feedback

* add mobile styling

* adds popup back to fs feature flag

* adds tests and aria roles

* makes mobile content same as desktop

* prettier update

* initial e2e mapSidePanel test

* adds cypress tests on desktop and mobile

* adds sass util and updates cypress tests

* cleans up tests

* reverts tsconfig file

* fixes map alignment

* renaming and using constants

* renaming sidePanel to infoPanel

* intl messaging

* adds snapshot testing and utility sass file

* PR feedback
- adds intl messages
- adds data-cy attr to cy tests
- snapshot testing for unit tests
- fixes bug where side panel extends past the map
- moves all wrapper content in MapWrapper

* logs isMobile to troubleshoot deployed PR

* adds react-device-detect for isMobile detection

* adds new instance of map for mobile

* adds instance

* adds isMobile to state

* tests the fix for mobile map view on PR

* PR review feedback
- localize MapIntroduction
- update snapshot tests
- QA feedback
- constants.isMobile points to react-device-detect
2021-07-27 12:05:25 -07:00
Nat Hillard
a787bd71ab
Merges to main don't get posted to main && it should be clearer how branches are deployed (#415)
* fixes #319 - merges to main still don't get posted to /main , and #362 It should be clearer how branches are deployed -- move to using 'pull_request' instead of 'push' (we can't push to master directly anyhow), and run on 'closed'; also add comment on PR itself
* updating to add subsequent pushes to the same PR
2021-07-27 12:37:17 -04:00
Lucas Merrill Brown
67b39475f7
Analysis by region (#385)
* Adding regional comparisons

* Small ETL fixes
2021-07-26 10:02:25 -05:00
Rohit Musti
81290ce672
adding tree equity score to the data pipeline (#398)
* adding tree equity score to the downloading pipeline so it can be easily compared as a reference index!

* removed redundant dependencies
2021-07-26 08:00:57 -04:00
Nat Hillard
7b241795fa
Cypress tests are failing locally (#403)
* adding readme with debug instructions
* updating translation tests to pass
* fixes failing latlng url test
* localizing focus buttons
* updating map tests to pass
* temporarily setting to only flag critical+  accessibility errors. To be addressed with full accessibility audit later
2021-07-23 08:52:34 -04:00
Nat Hillard
ab72dc24cf
Zoom button styling does not match spec (#395)
* Fixes #320, Zoom button styling does not match spec
* Setting explicit size for icon dimension to avoid automatic SVG sizing weirdness for gatsby build static site
2021-07-22 12:53:18 -04:00
Nat Hillard
8eb15bcac0
adding Vim to CODEOWNERS (#396) 2021-07-22 10:53:01 -04:00
Nat Hillard
a7cdf1c021
Adding notebook to create score dissolve (#333) 2021-07-21 16:10:32 -04:00
Vim
14e6a98008
Update README.md
adds when flags are to be used
2021-07-21 08:08:08 -07:00
Jorge Escobar
543d147e61
Data folder restructuring in preparation for 361 (#376)
* initial checkin

* gitignore and docker-compose update

* readme update and error on hud

* encoding issue

* one more small README change

* data roadmap re-strcuture

* pyproject sort

* small update to score output folders

* checkpoint

* couple of last fixes
2021-07-20 14:55:39 -04:00
Billy Daly
3032a8305d
Issue 198 score docs (#358)
* Adds matplotlib to the pyproject.toml for score/ to avoid import error

* Adds document structure for workflow steps and source data dictionaries

* Reverts 793381e because Matplotlib is no longer a dependency

* Adds the workflow steps for the first 2 stages of the workflow

* Updates the README with some TODOs

* Makes changes requested in PR review

* Fixes one more reference to Justice40 score
2021-07-20 14:29:14 -04:00
Shelby Switzer
5600443f1a
Update install steps (#359)
* Update INSTALLATION.md

Update INSTALLATION with more narrative text on how to install and start running parts of the app. We will need to update this soon with more info about the repo organization (once we have a `data` directory), but for now this reflects the current state of the repo.

* Update docs

Move client-specific things to client README, and update client README with steps to get the clientside app running locally.
2021-07-20 14:22:04 -04:00
TomNUSDS
d10f2ae0c9
Fix #353 - site icon (#388)
- site not being at the "topmost level" (e.g. there's a `/path`) makes the approach of having the favicon in a known, static location break (`https://[tld]/favicon.ico`).
Setting the icon via the plugin should make it work. It should be here anyways.
We'll still need to add the SEO plugin as part of the accessibility fixes (setting page meta tags in a i18n way)
2021-07-20 10:46:38 -07:00
TomNUSDS
609506d98d
Fix I18n and testing (#387)
Get snapshots working with Gatsby Intl plugin.
2021-07-20 10:46:13 -07:00
Vim
e5908cf7d7
updates filename and comments (#378) 2021-07-19 08:05:32 -07:00
Shelby Switzer
a1a3a013a8
Create codeql-analysis.yml 2021-07-19 09:21:28 -04:00
TomNUSDS
dca7980195
Fix inlined css in every single page. (#369)
Fix for issue #366
* Create gatsby-ssr.js
Note: SSR means "server side rendering". This fix is from a long github discussion about how broken including all css inline in the file is. (Especially since newer CSP guidelines say all css should be in a separate file.)

* PR code review fixes
* Check el.props['data-href'] is valid
* Add comment for production only.
2021-07-16 22:09:55 -07:00
Vim
4ea93e77ba
Update INSTALLATION-es.md
remove steps of the installation of gatsby-cli
2021-07-16 11:06:52 -07:00
Vim
7225bad804
Update INSTALLATION.md
remove steps to require gatsby-cli installation
2021-07-16 11:05:46 -07:00
TomNUSDS
e066f459fa
Fixed es.json (#354)
* Fixed es.json
- converted remaining english to spanish in file so as to check more locations.
- the `formatjs compile` for Spanish was failing converted back to english for now. build does the correct thing, so it's more just to keep things correct in source.
* add support for eslint-plugin-formatjs
 - eslint rules for formatjs
 - we are mostly protected by typescript checking, but this is a nice sanity check.
2021-07-15 13:00:15 -07:00
Vim
33c3288160
Allows the user to download CBG list (#318)
* adds initial download CBG box

* componentize download packet, adds e2e tests

* changes download file url in test to smaller size

* improve testing functions

* removes underline, colors icon

* replace csv with zip for safari security issue
2021-07-15 11:01:10 -07:00
Jorge Escobar
0316906a69
County Names for Score #188 (#347)
* starting PR

* completed feature

* checkpoint

* adding new fips and updating counties to 2010

* updated sources to 2010 - 2019

* more cleanup

* creating tiles score csv
2021-07-15 13:34:08 -04:00
Vim
f301dfcd80
cypress e2e nightly (#360)
* fixes nightly e2e test

* adds on command
2021-07-15 10:27:22 -07:00
Nat Hillard
dbf1ae2ad8
Zoom button styling does not match spec // General custom controls fix (#357) 2021-07-15 10:28:51 -04:00
TomNUSDS
cfce0dc826
End-to-end working localization (#336)
* End-to-end localization working
* Update package.json
* fix FormattedMessage confusion by using the `FormattedMessage` from `gatsby-plugin-intl` NOT `react-intl`
* TODO: `client/src/intl/es.json` should be generated from a translation tool.
* Update package-lock.json
2021-07-14 12:05:18 -07:00