Commit graph

348 commits

Author SHA1 Message Date
Vim
d449e9c554
update areaDetail with data from d3 CDN (#547)
* update areaDetail with data from d3 CDN

- updated constants with d3 CDN info
- placed AreaDetail component in folder
- fixed J40Alert padding-left console warning
- added tests for both J40Alert rendering

* udpates FE to percentile data

* testing CORS on PR'd URL

* testing PR'd URL
2021-08-18 08:47:34 -07:00
Jorge Escobar
c24e13c930
Update GHA to push only client changes to S3 (#543) 2021-08-16 17:00:43 -04:00
Shelby Switzer
2c79396550
Initial draft for data provenance addition to README (#528)
* Initial draft for data provenance

We want to make the data usable/available at every step of our data
pipeline. This starts te addition to the README that spells out the data
provenance and where each version of the data as it goes through our
pipeline lives.

* Update README with placeholders for next steps in data provenance

* Add coming soon placeholders for remaining data locations

Co-authored-by: Shelby Switzer <shelby.switzer@cms.hhs.gov>
2021-08-16 16:45:54 -04:00
Jorge Escobar
ac746741ea
disabling data checks on non-data folders (#532) 2021-08-16 15:12:58 -04:00
Jorge Escobar
6851c0a38d
Wrong creds name (#531) 2021-08-16 14:47:15 -04:00
Jorge Escobar
86c45ab3f0
Updating Front End Deploy Keys (#530) 2021-08-16 14:25:34 -04:00
TomNUSDS
2af9d6dcd3
fix contact fix footer (#511)
* Fix footer
* Remove one contact
* Fix some spacing issues
* Update J40Footer.spec.tsx.snap
* Fixes suggested in code review
2021-08-16 09:08:54 -07:00
Jorge Escobar
c19cd3ee55
hotfix on float cols (#526) 2021-08-13 15:48:31 -04:00
Vim
1dbb1018d6
sets column as percentiles (#525)
* sets column as percentiles

* adds trailing comma
2021-08-13 12:01:34 -07:00
Jorge Escobar
773c035493
AWS Sync Public Read (#508)
* adding layer to mvts

* small fix for GHA

* AWS Sync Public Read

* removed temp file

* updated state media income ftp
2021-08-12 14:17:25 -04:00
dependabot[bot]
1c5d5de82b
Bump yamale from 3.0.6 to 3.0.8 in /data/data-roadmap (#506)
Bumps [yamale](https://github.com/23andMe/Yamale) from 3.0.6 to 3.0.8.
- [Release notes](https://github.com/23andMe/Yamale/releases)
- [Commits](https://github.com/23andMe/Yamale/compare/3.0.6...3.0.8)

---
updated-dependencies:
- dependency-name: yamale
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-08-12 12:50:26 -04:00
Jorge Escobar
d259d97ba9
adding layer to mvts (#503)
* adding layer to mvts

* small fix for GHA
2021-08-12 10:56:54 -04:00
Jorge Escobar
09b3ba6715 Merge branch 'main' of github-usds:usds/justice40-tool 2021-08-11 09:13:57 -04:00
TomNUSDS
ec4344676d
Add Cumulative Score "steps" section to methodology page (#489)
* Step progress list
* Layout mostly done. Need to do intl() conversion but it's easier to understand the layout like this.
Will do intl in a different PR
* fix: `em` to `rem`
* Add basic snapshot unit test
2021-08-10 15:36:09 -07:00
Jorge Escobar
6dc1283ee2 added comment 2021-08-10 15:37:36 -04:00
Jorge Escobar
3d8dbb293c
Tile-baking columns with floating rounds completed (#491)
* Tile-baking columns with floating rounds completed

* completed

* correction on github workflow

* tiles folder no longer needed

* addressed comments

* updating requirements.txt

* poetry lock update

* adding xlswriter

* final poetrylock

* updated requirements.txt

* checkpoint

* removed matplotlib

* ignoring pylint too many statements

* reinstated too many statements

* converting data sync to generate score GHA UI-driven
2021-08-10 15:28:50 -04:00
Vim
176cb71d9a
updates snapshot (#498) 2021-08-10 11:43:24 -07:00
dependabot[bot]
2812b0c404
Bump jszip from 3.6.0 to 3.7.1 in /infrastructure (#497)
Bumps [jszip](https://github.com/Stuk/jszip) from 3.6.0 to 3.7.1.
- [Release notes](https://github.com/Stuk/jszip/releases)
- [Changelog](https://github.com/Stuk/jszip/blob/master/CHANGES.md)
- [Commits](https://github.com/Stuk/jszip/compare/v3.6.0...v3.7.1)

---
updated-dependencies:
- dependency-name: jszip
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-08-10 13:08:09 -04:00
Vim
174a0e1330
Modifies ExploreTool page to match sprint 4 design (#481)
* initial commit of sprint 4 explore page

* adds styling on HowYouCanHelp module

* troubleshooting li element on deployed URL

removing local bullet styles

* removing unused styles

* recreating HowYouCanHelp

* explicit list el styles

* adds bullets back in

* fixes tooltip style and alert padding

* componentize MapLegend

* fix links

* inital intl and unit tests

* adds trusswork tooltip for comparison

* updates based on various feedback and disucssions:

- removes react-tooltip
- placeholder trussworks tooltip
- removes download packet component
- intl on HowYouCanHelp
- updates MapLegend tests
- add initial cy test on ExploreTool page

* removes bold on alert

* PR feedback:

- removes location from J40Alert
- localizes `COLOR KEY`

* adds intl to constants file

* modifies download zip URL to new S3 location

* removes location depedencies on Alerts

* add localization for HowYouCanHelp
2021-08-10 09:45:45 -07:00
lucasmbrown-usds
ebe6180f7c wip 2021-08-09 22:24:14 -05:00
lucasmbrown-usds
cf13036d20 clearing output 2021-08-09 21:31:07 -05:00
lucasmbrown-usds
ce5e8c5351 including fraction of state AMI 2021-08-09 21:30:41 -05:00
lucasmbrown-usds
4ae7eff4c4 adding median income field and running black 2021-08-09 20:47:51 -05:00
Nat Hillard
6c986adfe4
Check in VSCode config for easier local debug (#487)
* Fixes #466 - Task: Check in VSCode config for easier local debug
2021-08-09 14:55:13 -04:00
TomNUSDS
73a205dc48
Refactor layout (#486)
* Refactor to move `<DatasetContainer>` out of layout
* `<DatasetContainer>` is now in the methodology page. Starting the review there will help understand the motive for this change.
* Fixed 404 page which seems way out of date.
* Use row/col grid syntax
2021-08-09 08:04:24 -07:00
Nat Hillard
9a9d5fdf7f
Backend change for Zipfile pt. 2 (#469)
* Fixes #303 : adding downloadable zip archive logic
* linter recommendations
* Pushes data directory to AWS. We'll want to move to use AWS for this ASAP, but this works for now
* updating pattern
2021-08-09 10:39:59 -04:00
dependabot[bot]
6eb7382d48
Bump tar from 6.1.0 to 6.1.6 in /infrastructure (#468)
Bumps [tar](https://github.com/npm/node-tar) from 6.1.0 to 6.1.6.
- [Release notes](https://github.com/npm/node-tar/releases)
- [Changelog](https://github.com/npm/node-tar/blob/main/CHANGELOG.md)
- [Commits](https://github.com/npm/node-tar/compare/v6.1.0...v6.1.6)

---
updated-dependencies:
- dependency-name: tar
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-08-09 10:37:42 -04:00
Nat Hillard
ec19d86f6f
Adding back census to list of potential datasets, but separating out from standard list (#484)
Error this addresses:
  File "/Users/lucas/Documents/usds/repos/justice40-tool/data/data-pipeline/data_pipeline/etl/runner.py", line 71, in etl_runner
    f"data_pipeline.etl.sources.{dataset['module_dir']}.etl"
TypeError: 'NoneType' object is not subscriptable
2021-08-09 09:52:06 -04:00
Jorge Escobar
f51b0d69d9
Poetry updates for application (#483) 2021-08-06 16:24:30 -04:00
TomNUSDS
9af8aba70b
Added .DS_Store (#482) 2021-08-06 12:13:49 -07:00
Nat Hillard
6fb36ded9c
adding additional missed import (#477) 2021-08-06 11:48:11 -04:00
Nat Hillard
9d962eb5d9
Moving from relative imports to absolute to enable poetry run python data-pipeline/application.py [command] (#476) 2021-08-06 11:41:28 -04:00
Nat Hillard
45a8b1c026
Census ETL should use standard ETL form (#474)
* Fixes #473
Census ETL should use standard ETL form

* linter fixes
2021-08-06 11:01:51 -04:00
TomNUSDS
ef044fa36b
fix footer layout and links (#447)
* fix footer layout and links
* Fix up trusswork's `<Footer>` + `<Logo>` + `<Address>` to match design
2021-08-05 21:34:43 -07:00
Nat Hillard
9f3b2f056b
Fixes #467: (#470)
If the census download task is run more than once,
us.csv doubles in size and all data is removed from dataframe
2021-08-05 16:20:18 -04:00
Nat Hillard
c1568e87c0
Data directory should adopt standard Poetry-suggested python package structure (#457)
* Fixes #456 - Our data directory should adopt standard python package structure
* a few missed references
* updating readme
* updating requirements
* Running Black
* Fixes for flake8
* updating pylint
2021-08-05 15:35:54 -04:00
Jorge Escobar
4d7465c833
Hotfix for fips zip download location + added full-score-run command (#465)
* Hotfix for S3 locations of data sources

* updated README

* lint failures

Co-authored-by: Nat Hillard <Nathaniel.K.Hillard@omb.eop.gov>
2021-08-05 12:55:21 -04:00
Jorge Escobar
5cb00ef0ce
Tile Generation Script (#433) 2021-08-03 18:23:57 -04:00
Vim
849437c9d3
adds nvmrc file (#445) 2021-08-02 10:35:18 -07:00
Billy Daly
5504528fdf
Issue 308 python linting (#443)
* Adds flake8, pylint, liccheck, flake8 to dependencies for data-pipeline

* Sets up and runs black autoformatting

* Adds flake8 to tox linting

* Fixes flake8 error F541 f string missing placeholders

* Fixes flake8 E501 line too long

* Fixes flake8 F401 imported but not used

* Adds pylint to tox and disables the following pylint errors:
- C0114: module docstrings
- R0201: method could have been a function
- R0903: too few public methods
- C0103: name case styling
- W0511: fix me
- W1203: f-string interpolation in logging

* Adds utils.py to tox.ini linting, runs black on utils.py

* Fixes import related pylint errors: C0411 and C0412

* Fixes or ignores remaining pylint errors (for discussion later)

* Adds safety and liccheck to tox.ini
2021-08-02 12:16:38 -04:00
Vim
51f7666062
Adds dataset cards to Methodology page (#442)
* intial cards for methodology page

* PR and QA feedback
- adds alert above dataset section
- adds intl
- removes nbsp
- creates directory structure for new components

* revert noUsedLocals flag

* fixed path error

* re-creates scss file to test build failure

* renaming file to troubleshoot build error

* links open in new tabs and removes console.log

* removes units on all scss value that equal 0

* resolving merge conflicts from header merge

* updates snapshots from conflict resolution
2021-08-02 08:49:49 -07:00
TomNUSDS
08e21e5d5b
fix #249 header 2 (#425)
* Header updates
* More pixel alignment for nav
* Update unit test snapshots
I removed a stray `,` from `testHelpers.tsx` which changed the output of several snapshots.
* Fixes from PR review
* Forgot to update snapshots after merge with main
* Removed unused (and outdated) timeline page
* Updated some translation descriptions.
2021-08-02 07:48:19 -07:00
Nat Hillard
8df548f506
Adding ADR PR for associated ADR issue #402 (#437) 2021-08-02 10:17:44 -04:00
Billy Daly
55dabb2b57
Issue 379 tox setup (#405)
* Adds tox as a dev dependency to data/data-pipeline/pyproject.toml: Also updates poetry.lock and requirements.txt

* Adds tox.ini to test build of data/data-pipeline

* Sets up GitHub actions workflow for data/ directory

* Tries to get Data Checks GitHub action to run

* Fixes error with GitHub action

* Migrates data/data-roadmap from setuptools to poetry

* Sets up tox file for data/data-roadmap

* Adds github action for data/data-roadmap

* Fixes syntax error in data-checks.yml

* Second attempt at fixing data-checks.yml

* Export poetry requirements to requirements.txt

* Revert "Migrates data/data-roadmap from setuptools to poetry"

This reverts commit e8367652d43c1c9beee500f792c8f41e1c1fc462.

* Removes pyproject.toml and reverts requirements.txt as well
2021-07-29 14:00:20 -04:00
Shelby Switzer
387ee3a382
Update data documentation and some data steps (#407)
* Minor documentation updates, plus calenvironscreen  S3 URL fix

* Update score comparison docs and code

* Add steps for running the comparison tool
* Update HUD recap ETL to ensure GEOID is imported as a string (if it is
imported as an interger by default it  will strip the beginning "0" from
many IDs)

* Add note about execution time

* Move step from paragraph to list

* Update output dir in README for comp tool

Co-authored-by: Shelby Switzer <shelby.switzer@cms.hhs.gov>
2021-07-29 10:28:52 -04:00
Jorge Escobar
b404fdcc43
Generate Geo-aware scores for all zoom levels (#391)
* generate Geo-aware scores for all zoom levels

* usa high progress

* testing dissolve

* checkpoint

* changing type

* removing breakpoint

* validation notebooks

* quick update

* score validation

* fixes for county merge

* code completed
2021-07-28 16:07:28 -04:00
Nat Hillard
446c8d1f68
Update dataset_request.yaml
Wrapping date with quotes to avoid "Tried to load unspecified class: Date" error
2021-07-28 14:58:17 -04:00
Nat Hillard
7bc494b759 Update issue templates
Adding "task" template for one-off changes
2021-07-28 14:55:28 -04:00
Nat Hillard
84500bc744
Adding template for dataset request (#427) 2021-07-28 14:49:38 -04:00
Nat Hillard
3238e09792
Loading a URL with lat/lng/zoom specified does not work (#414) 2021-07-27 20:51:35 -04:00