Commit graph

153 commits

Author SHA1 Message Date
Matt Bowen
0d0cf2a5f9
Update score version workflow to also upload tiles (#2042)
* Actually upload the score

* Add back tile generation (#2035)

* Move score GEO up (#2035)

* Copy beta README to 1.0 (#2035)

* Remove dates from file names (#2035)

* Remove unused score ETLs from constants (#2035)

* Put back child opp index (#2035)

* Remove unused import (#2035)
2022-10-28 10:53:19 -04:00
Matt Bowen
d7a0be7ee7
Update namespaces (#2035) (#2041) 2022-10-26 16:02:54 -04:00
Matt Bowen
5f588672a7
Add action for creating new score versions (#2039)
* Add action for creating new score versions (2035)

Add a new action that allows us to manually create and upload score
versions to s3.

* Remove PR-related chat message

* Temportarily remove tile generation (#2035)

We want to see how this works, so we cut the time down for actually
running it by removing tile generation.

* Remove another PR-comment block
2022-10-26 12:01:19 -07:00
Jorge Escobar
dc234de551
Score versioning (#1986)
* Score versioning

* adding new zip assets

* tests passing

* debugging

* debugging 2

* get codebook from downloadable path

* upload version file for codebook and shapefile
2022-10-25 10:42:07 -04:00
Jorge Escobar
1c448a77f9
NRI dataset and initial score YAML configuration (#1534)
* update be staging gha

* NRI dataset and initial score YAML configuration

* checkpoint

* adding data checks for release branch

* passing tests

* adding INPUT_EXTRACTED_FILE_NAME to base class

* lint

* columns to keep and tests

* update be staging gha

* checkpoint

* update be staging gha

* NRI dataset and initial score YAML configuration

* checkpoint

* adding data checks for release branch

* passing tests

* adding INPUT_EXTRACTED_FILE_NAME to base class

* lint

* columns to keep and tests

* checkpoint

* PR Review

* renoving source url

* tests

* stop execution of ETL if there's a YAML schema issue

* update be staging gha

* adding source url as class var again

* clean up

* force cache bust

* gha cache bust

* dynamically set score vars from YAML

* docsctrings

* removing last updated year - optional reverse percentile

* passing tests

* sort order

* column ordening

* PR review

* class level vars

* Updating DatasetsConfig

* fix pylint errors

* moving metadata hint back to code

Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov>
2022-08-09 16:37:10 -04:00
Jorge Escobar
8149ac31c5
Starting Tribal Boundaries Work (#1736)
* starting tribal pr

* further pipeline work

* bia merge working

* alaska villages and tribal geo generate

* tribal folders

* adding data full run

* tile generation

* tribal tile deploy
2022-07-30 01:13:10 -04:00
Vim
746d36d4a6
Update Cypress GHA to v4 (#1768)
* Update cypress gha to v4

* Trigger build
2022-07-26 10:59:12 -07:00
Jorge Escobar
237122b2c5
Update CODEOWNERS (#1769) 2022-07-26 11:49:48 -04:00
Vim
1681a72045
Remove threshold line in side panel (#1666)
* Remove categories

* Remove disadvantaged in sidepanel

* Add comment
2022-06-06 10:52:37 -07:00
Vim
9f0ca1c17d
Add address to search placeholder and add "at or above" to category copy for low income (#1665)
* Add address to search placeholder text

* Adds "at or above" to category copy

* Updates map search placeholder text

- Update es and en.json files
- adds mobile search placeholder text
- udpates desktop search placeholder text

* Update logic to determine mobile for search text

* Update search error message

- update to Spanish text as well

* Adjust spacing between search and territory btns

* Modify mobile detection logic

- remove cypress tests for faster deploys

* Add placeholder text to state variable

- wrap state variable in useEffect()
2022-06-03 09:35:28 -07:00
Jorge Escobar
ce89214a60
Adding Technical Training Slides (#1638)
* Adding Technical Training Slides

* small update on CI/CD map staging URL
2022-05-12 15:01:26 -04:00
Jorge Escobar
2245730cf4
Update generate-score.yml 2022-04-27 10:49:56 -04:00
Jorge Escobar
5d7a97d662
Fix to downloadable assets 2022-04-27 10:49:02 -04:00
Jorge Escobar
fbd56e3bd5
Put the pdf back in the package and add TSD to pipeline (#1580)
* Put the pdf back in the package and add TSD to pipeline

* updated pdf with logo

* wrong path
2022-04-21 13:42:04 -04:00
Vim
ce6f7974fb
Add Spanish to site (#1541)
* Add script to un-nest the translated es.json file

* Update unit tests

* Update i18n IDs to conformity

* Update READMEs

* Make ordinal values in sidepanel i18n compliant

* remove redudant IDs in en.json

* Add oridinal i18n to IndicatorValue sub-text

* Update plurals in sidepanel

- Disadvantaged in 1 category
- Disadvantaged in 5 categories
- At or above 1 threshold
- At or above 10 thresholds

* Add en.json link to GHA

* Enable FormatJS linter rules to catch i18n issues

* Testing if commit hash will render in PR message

* Testing if commit hash will render in PR message

* Enable allow repeats

* Try to add run command

* remove attempts to link to en.json

* Define COMMIT HASH env variable

* Move commit hash to deploy job
2022-04-11 15:51:38 -07:00
Vim
92501e4715
Re-order nav links, update download box to include shapefiles, squish pages (#1538)
* Re-order nav links, update download box to include shapefiles, squishy pages (#1536)

* Add dropdown to navigation links

- add download page
- move main pages tests to their own folder
- add download and public eng snapshot test
- remove public engagement button on each page
- swap index with cejst
- update cypress ENDPOINTS
- upate gatsby-config sitemap
- update snapshots
- cypress tests are failing

* Make all page endpoints constants

- fix all cypress tests
- refactor all test to use LegacyTests/constants/PAGES_ENDPOINTS
- gatsby-config to use PAGES_ENDPOINTS
- remove ScoreStepList component
- update J40Header to use constants.PAGES_ENDPOINTS
- update snapshots
- there are 3 locations to update PAGES_ENDPOINTS, namely

1. LegacyTests/constants.tsx
2. constants.tsx
3. gatsby-config

* Add two placeholder pages

- FAQs
- TSD

* remove dropdown from navigation menu

- remove dropdown from gherkin tests
- update snapshots

* Add back public engagement button to all pages

- add send button in area detail alt tag to i18n
- modify public engagement button to have same CSS as send feedback button
- update snapshots

* Update download box to include shapefile downloads

- refator download packet scss
- add shapefile URL to .env files
- update tests
- index forwarding pattern
- update snapshots

* Add FAQ link to About page

* Update footer

- add PEC
- add RFI
- add github link

* Update the staging link URL and title

* Update download Box to match design sync

- update FAQ link copy

* Add comment for reason of commented code
2022-04-07 10:14:54 -07:00
Jorge Escobar
859177a877
Marshmallow Schemas for YAML files (#1497)
* Marshmallow Schemas for YAML files

* completed ticket

* passing tests

* lint

* click dep

* staging BE map

* Pr review
2022-03-31 13:56:10 -04:00
Vim
27311b11e2
Modify data tile server to static URL to fix CORS (#1512)
* Modify data tile server to static URL to fix CORS

* Remove the cron from the cache invalidate action
2022-03-30 13:44:58 -07:00
Vim
db6b5de24e
Release of improve census tract display info (#1480)
* will replace mapbox logo (#1477)

* Replace maplibre logo with mapbox
* change the logo go to mapbox
* Add CodeQL and modify mapbox logo via useRef()

* Make side panel formula more apparent (#1481)

* Add ExceedBurden and CategorySpacer

- create ExceedBurden component that is used twice
- create a .categorySpacer class for the AND component
- modify indicator styling to work with background higlighting
- add BE stubs for Burden booleans
- remove Indicators header
- add copy to intl

* Add comments on disadv. indicator

- darken bg color
- add a border
- bold the text

* Adds indicator arrow and sub text to sidepanel

- add threshold to IndicatorInfo
- update SASS for indicatorValueCol
- update tests
- add constants to intl

* Make disadv indicators bold

- add 1px margin between indicators

* Add BE signals for new sidepanel

- tested with staging backend

* Add staging hash to URL

* Fix poverty backend signal

- refactor backend signals in constants file

* Make exceed burden value bold

* Refactor indicator values

- remove arrows and subtext when value is N/A
- Show -- when value is N/A
- intl alt tags in indicator
- fix alignment of arrows
- update snapshots

* Revert settings.json file

* revert settings file again

* Refactor what is displayed when data unavailable

- add an unavailable icon
- add data is not available subtext
- modify low income threshold
- update snapshots
- factor out rendered logic to JSX functions
- update image alt tags and intl

* Refactor Indicator component to unit test

- Add unit test for Indicator value icon
- Add unit test for Indicator value sub-text
- update snapshots

* Add de-coupled BE signals

* Rebase hotfix

* Fix indicator value 0 shows N/A icon

- remove coercion of 0 as null
- make components check null / failure case first and default all else
- update unit test to account for this use case
- update snapshots

* Add null check for percentage suffix

- update unit tests
- update snapshots

* remove cypress zoom test

- intermittent failure continue

* revert cypress to 8.3.0

* Revert cypress-cucumber

* Add Chrome to frontend deploy action

* Update logo css classname after update to maplibre (#1482)

* Add new wording to the map panel that appears upon load when no tract is selected and Outstanding CEQ changes to sidepanel (#1483)

* Add new side panel unselected tract

- add new icons
- refactor old component
- follow component folder pattern
- update snapshot tests
- add to intl

* Add bold to text, add spacing and correct typos

* Add tabindex to sidepanel content to pass a11y

* Refactor i18n anti-patterns on explore tool page

- add i18n patterns for nesting
- add i18n pattern for partial strings
- add i18n rich-text functions
- add i18n pattern for minimal context
- add i18n pattern for dates
- add i18n pattern for numbers
- add i18n pattern for centralizing rich text functions
- add i18n patter for description
- add i18n pattern for ids
- see shared drive file J40 Localization Patterns for status on refactor

* Remove links of expired public engagement sessions

- allow cypress tests to pass

* Update snapshots for public enagement page

* Copy updates to non-selected side panel

- update snapshots

* Updates to side panel copy

- ag loss and building loss text
- clean transit
- NPL RMP sites
- proportion to percent
- update snapshots
- updates to es.json

* Updates from QA

- make title smaller
- make margins above icons smaller
- add bottom margin on container
- add census before tracts in copy
- update snapshots

* Update snapshot after rebase

* disable max-len on description fields on i18n copy (#1487)

* Remove color key from Explore the tool page (#1484)

* Remove color key

* Add comment to Language component

* Move tribal note copy to meth page

- adjust responsive sizing props on Grid to allow for proper mobile viewing on Explore page

* Add responsive size to text under map

- reduce z-index of territory focus control so that it doesn't go over the survey button on mobile

* Rewrite the two "notes about" the "low" datasets (#1489)

* Refactor all copy to adhere to recommended patterns

- remove LowIncome component
- add intl README
- update snapshots

* Fix key error missing in datasetCard

- update type in IIndicators
- update snapshots

* Add two notes on low dataset cards

- refactor DatasetCard to standard component pattern
- add a note to the interface
- update snapshots

* Add function comment to force re-build

* Update missing sass module file name

* Update sidepanel non-selected copy (#1495)

* Update sidepanel non-selected copy

- update snapshots

* Update URL in deploy FE for cypress test

* removing trailing slash

* Add wording to UI that calls out improvements to display of census tract information (#1492)

* Adds census tract alert on all pages

- add i18n text
- updates snapshots
- makes public engagement page a fast link

* Style the Alert to have more space around it

* Update copy on Alert

- update snapshots

* Swap gerkhin order

* Set Alert to expire on Apr 15th 2022

* Add WHEJAC meetings (#1501)

* Add WHEJAC meetings

- add expired icons
- automatically load expired icons when event has passed
- update snapshots

* Update public engagement button

* Remove public eng gherkin tests

- need to troubleshoot why these are failing

* Remove the before CEQ in copy

* Make the count of thresholds exceeded on the side panel more clear (#1503)

* Update category / thrsh count in side panel

- connect BE signal of CC to side panel
- i18n functions to call from AreaDetail component
- update snapshots
-

* Update send feedback - color and icon

- update snapshots

* Update both exceed statements in sidepanel

- update snapshots

* Update copy for higher ed and high school (#1502)

* Update copy for higher ed and high school

- update dataset cards
- update taskforce card AND
- update side panel indicator titles
- add i18n for dataset cards title
- update snapshots

* Update threshold values for Higher ed and HS.

- update snapshots

* Update AND clause

- missing 'of' in copy
- update BE signal for non-higher ed enrollment signal
2022-03-29 19:49:57 -07:00
Jorge Escobar
a986674ed9
Refresh Backend CDN Cron (#1471) 2022-03-23 11:49:42 -04:00
Jorge Escobar
dd723b6c19
PyPi Packaging of Data Pipeline (#1464)
* PyPi Packaging of Data Pipeline

* package rename

* adding python version

* trigger data checks

* print env vars

* python version 2

* trigger data check

* python version 3

* update caching for other GHAs
2022-03-21 18:55:15 -04:00
Jorge Escobar
53e35427f2
removing delete step 2022-03-21 14:52:19 -04:00
Jorge Escobar
80ad3c62d2
staging upload fix 2022-03-21 11:54:52 -04:00
Jorge Escobar
0364d3fbc9
removing s4cmd del for errors 2022-03-21 10:59:33 -04:00
Katherine D. Mlika
68c882b3de
updating column E label to "Identified as disadvantaged" (#1406)
* updating column E label to "Identified as disadvantaged"

* passing tests

* adding cached poetry flow

* working dir

Co-authored-by: Jorge Escobar <jorge.e.escobar@omb.eop.gov>
2022-03-18 14:50:03 -04:00
Jorge Escobar
7b05ee9c76
S3 Parallel Upload and Deletions (#1410)
* installation step

* trigger action

* installing to home dir

* dry-run

* pyenv

* py 2.8

* trying s4cmd

* removing pyenv

* poetry s4cmd

* num-threads

* public read

* poetry cache

* s4cmd all around

* poetry cache

* poetry cache

* install poetry packages

* poetry echo

* let's do this

* s4cmd install on run

* s4cmd

* ad aws back

* add aws back

* testing census api key and poetry caching

* census api key

* census api

* census api key #3

* 250

* poetry update

* poetry change

* check census api key

* force flag

* update score gen and tilefy; remove cached fips

* small gdal update

* invalidation

* missing cache ids
2022-03-17 23:19:23 -04:00
Vim
e31a4f3b94
Add release branch to frontend deploy staging (#1455) 2022-03-17 17:01:24 -07:00
Vim
14ad8c2ee4
Add intl pre-commit hook and update GHA (#1416)
* Add intl pre-commit hook and update GHA

- add Husky lib
- add pre-commit script
- detect warning on extraction and abort commit
- update deploy staging yml to show message that en.json was modified

* Test if block of message post

- test if the message is posted even when checking a file with no changes

* Add back in full deploy action

* Add back en.json
2022-03-17 10:03:26 -07:00
Vim
31d783bd1b
Modify CDN IDs (#1429) 2022-03-16 15:03:59 -07:00
Jorge Escobar
7f91e2b06b
ArcGIS zipping (#1391)
* ArcGIS zipping

* lint

* shapefile zip

* removing space in GMT

* adding shapefile to be staging gha
2022-03-09 18:00:20 -05:00
Jorge Escobar
6425beb9f4
YAML Config for Downloadable Assets (#1252)
* starting yaml config load work

* working version for downloadable file

* yaml file update

* checkpoint

* sort if needed

* refactoring

* moving config

* checkpoint

* old files

* skipping downloadble tests for now

* more modularization

* more refactor, new excel yml

* pylint

* completed tabs

* Update excel.yml

* remvoing obsolete tests

* addressing PR feedback

* addressing changes

* confirmed change in yaml breaks tests

* safety bump

* PR review

* adding tests back

* pylint

* Incorporating latest score fields from Emma

* incorporating newest fields from Emma

* passing tests

* adding shapefile aws sync

* missing test

* passing tests
2022-03-04 15:02:09 -05:00
Vim
eb03d6cb5f
Fix broken YML links and add PR template (#1340)
* Fix MD files with broken links

- fixing all broken links
- all YML files are now in capital case

* Add PR template

* remove comments
2022-02-25 11:39:38 -08:00
Jorge Escobar
44a8b04d3b
rm backend staging folder 2022-02-17 17:36:48 -05:00
Jorge Escobar
8b5cef21d2
update aws creds 2022-02-17 15:30:26 -05:00
Jorge Escobar
c6a83d1e63
close pr be gha 2022-02-17 13:59:46 -05:00
Vim
d5a15f1053
Update side panel copy to Living copy (#1285)
* Update side panel copy to Living copy

* Update lead paint short description

* Rebase main and update areaDetail snapshots

* Update cypress test on dataset used in beta header

* QA review updates

* Update deploy-fe-main to point to new CDN (d3r)
2022-02-17 10:59:23 -08:00
Jorge Escobar
8d2c8f77cc
zip package update 2022-02-17 11:10:15 -05:00
Jorge Escobar
73583dfae0
updating URLs 2022-02-17 08:49:38 -05:00
Jorge Escobar
b310405a67
staging be gha update 2022-02-16 17:39:49 -05:00
Jorge Escobar
f0c1ea9b5f
adding generate score back 2022-02-16 17:37:36 -05:00
Jorge Escobar
59862a098e
Test Staging Data Backend (#1282)
* Test Staging Data Backend

* action updates
2022-02-16 16:45:59 -05:00
Jorge Escobar
82809a5123
Github Actions for Staging Backend (#1281)
* Github Actions for Staging Backend

* trigger run
2022-02-16 16:40:25 -05:00
Jorge Escobar
1d399d3ca9
Tox Security Fix (#1242)
* checkpoint

* safety ignore

* update python matrix for data checks

* downloading census once
2022-02-03 17:05:51 -05:00
Jorge Escobar
722c738a5b re-enabling cypress 2022-02-02 18:21:04 -05:00
Jorge Escobar
ff2958a8d2 closed PR GHA 2022-02-02 18:01:43 -05:00
Jorge Escobar
8187d028aa suspending cypress for q/a 2022-02-02 17:47:41 -05:00
Jorge Escobar
d55dc8c5e3 update PR number 2022-02-02 17:21:10 -05:00
Vim
1c7b043cdb
change deploy main to be on push (#1224) 2022-02-02 16:44:10 -05:00
Jorge Escobar
403a490985
Esfoobar usds/488 generate score per commit pr (#1211)
* Score run on every commit to data PR

* testing score run

* source aws
2022-01-31 16:07:21 -05:00
Jorge Escobar
54e9f50625
Score run on every commit to data PR (#1210) 2022-01-31 14:19:49 -05:00