publicdata/j40-cejst-2
1
0
Fork 0
mirror of https://github.com/DOI-DO/j40-cejst-2.git synced 2025-02-23 10:04:18 -08:00
Code Issues Projects Releases Packages Wiki Activity Actions
643 commits 159 branches 0 tags 330 MiB
Commit graph

216 commits

Author SHA1 Message Date
Jorge Escobar
3a84fadaf6 Merge branch 'main' into esfoobar-usds/1417-nri-yaml-score 2022-08-02 11:24:21 -04:00
Jorge Escobar
8149ac31c5
Starting Tribal Boundaries Work (#1736) 
* starting tribal pr

* further pipeline work

* bia merge working

* alaska villages and tribal geo generate

* tribal folders

* adding data full run

* tile generation

* tribal tile deploy
 2022-07-30 01:13:10 -04:00
Vim
e1a61faf5d
Add a react component generator (#1745) 
* Add a react component generator

* Update markdown links

* Change commented code to block comment
 2022-07-15 09:54:58 -07:00
Vim
eb3004c0d5
Fix on large AK tracts that are off screen (#1740) 
* Change low to high transition and global zoom

- change the low to high transition from 7 to 5. This can not go any lower as high tiles on AWS only go to zoom level 5
- reduce the zoom level globally on all census tracts

* Remove geolocation from feature flag

- geolocation is now available to all

* Add python notebook that sorts all tracts by area

- add a column of the required zoom level for the tract to be fully contained in the viewport

* Place geolocation back to behind a feature flag

* Differentiate zoom levels b/w shortcuts and tracts
 2022-07-13 19:01:43 -07:00
dependabot[bot]
2992f8df0b
Bump notebook from 6.4.10 to 6.4.12 in /data/data-pipeline (#1685) 
Bumps [notebook](http://jupyter.org) from 6.4.10 to 6.4.12.

---
updated-dependencies:
- dependency-name: notebook
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-07-07 17:10:03 -04:00
dependabot[bot]
0555d896fd
Bump lxml from 4.8.0 to 4.9.1 in /data/data-pipeline (#1719) 
Bumps [lxml](https://github.com/lxml/lxml) from 4.8.0 to 4.9.1.
- [Release notes](https://github.com/lxml/lxml/releases)
- [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt)
- [Commits](https://github.com/lxml/lxml/compare/lxml-4.8.0...lxml-4.9.1)

---
updated-dependencies:
- dependency-name: lxml
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-07-07 17:09:49 -04:00
Jorge Escobar
5ce4a5c1f7 column ordening 2022-07-07 11:47:19 -04:00
Jorge Escobar
0d81480133 sort order 2022-07-06 16:24:23 -04:00
Jorge Escobar
c9632571f3 passing tests 2022-06-27 13:46:09 -04:00
Jorge Escobar
e5b84dc28d removing last updated year - optional reverse percentile 2022-06-27 12:59:17 -04:00
Jorge Escobar
7415a15bc0 docsctrings 2022-06-27 11:19:11 -04:00
Jorge Escobar
158c8bffb2 dynamically set score vars from YAML 2022-06-24 15:18:57 -04:00
Jorge Escobar
34f7ef27ce adding source url as class var again 2022-06-01 17:10:30 -04:00
Jorge Escobar
0460852bf0 Merge branch 'esfoobar-usds/release/yaml-based-score-datasets' into esfoobar-usds/1417-nri-yaml-score 2022-06-01 13:34:58 -04:00
Kameron Kerger
7c808eb2e0
Add files via upload (#1656) 
updated TSD (new - naming convention) and new TSD-es
 2022-05-31 13:19:01 -04:00
Jorge Escobar
400746f7fb stop execution of ETL if there's a YAML schema issue 2022-05-19 12:29:56 -04:00
Jorge Escobar
38304f846f tests 2022-05-16 15:52:00 -04:00
Jorge Escobar
1db2362426 renoving source url 2022-05-16 15:24:48 -04:00
Jorge Escobar
a127ac2e72 PR Review 2022-05-16 14:47:31 -04:00
Jorge Escobar
105d2cb442 checkpoint 2022-05-16 13:26:07 -04:00
Jorge Escobar
5f2228913b columns to keep and tests 2022-05-16 13:26:05 -04:00
Jorge Escobar
c18e69ef90 lint 2022-05-16 13:23:52 -04:00
Jorge Escobar
27599f2ebc adding INPUT_EXTRACTED_FILE_NAME to base class 2022-05-16 13:23:51 -04:00
Jorge Escobar
f0e8ac40e0 passing tests 2022-05-16 13:23:51 -04:00
Jorge Escobar
0a9c04c501 checkpoint 2022-05-16 13:23:50 -04:00
Jorge Escobar
0ee0a4d15f NRI dataset and initial score YAML configuration 2022-05-16 13:23:49 -04:00
Jorge Escobar
ce89214a60
Adding Technical Training Slides (#1638) 
* Adding Technical Training Slides

* small update on CI/CD map staging URL
 2022-05-12 15:01:26 -04:00
Jorge Escobar
2af6fca98d
Columnn headers update (#1618) 
* Columnn headers update

* passing tests

* updated date stamp

* js tests
 2022-05-06 14:10:15 -04:00
Kameron Kerger
303c200fbe
Add files via upload (#1612) 
updated pdf
 2022-05-04 10:47:54 -04:00
Jorge Escobar
eb1cb8884e
Adding a note about Scipy installation on newer MacOS 2022-05-03 17:26:05 -04:00
Emma Nechamkin
ae725f0a3e
arcgis column name fix (#1581) 
eliminates duplicate column and ensures all column names are unique.
 2022-04-22 14:09:12 -04:00
Jorge Escobar
fbd56e3bd5
Put the pdf back in the package and add TSD to pipeline (#1580) 
* Put the pdf back in the package and add TSD to pipeline

* updated pdf with logo

* wrong path
 2022-04-21 13:42:04 -04:00
Kameron Kerger
72e6dbc1dd
/1354-update-pdf (#1568) 
updated pdf for the put the pdf back in the package issue
 2022-04-19 11:07:31 -04:00
Emma Nechamkin
2ce4cfe80e
updated with codebook (#1573) 2022-04-18 18:12:18 -04:00
Jorge Escobar
859177a877
Marshmallow Schemas for YAML files (#1497) 
* Marshmallow Schemas for YAML files

* completed ticket

* passing tests

* lint

* click dep

* staging BE map

* Pr review
 2022-03-31 13:56:10 -04:00
Emma Nechamkin
cb963cff5f
Updating comparison tool to be easier for pairwise comparisons (#1400) 
Creating pairwise comparison tool to compare two lists of prioritized tracts to each other.
 2022-03-30 14:02:06 -04:00
Emma Nechamkin
2628afacf9
Creating a data dictionary for the download packet (#1469) 
Adding automated codebook creation. Future ticket to refactor.
 2022-03-30 11:01:43 -04:00
Emma Nechamkin
dc981919f1
Adding booleans for FE to display (#1393) 
PR adds booleans for each individual threshold category for the front end to display.
 2022-03-29 20:17:10 -04:00
Emma Nechamkin
0c07cdac55
Adding category count to BE signals (#1486) 
Added category count to downloadable data and backend signals.
 2022-03-29 17:11:57 -04:00
Jorge Escobar
dd723b6c19
PyPi Packaging of Data Pipeline (#1464) 
* PyPi Packaging of Data Pipeline

* package rename

* adding python version

* trigger data checks

* print env vars

* python version 2

* trigger data check

* python version 3

* update caching for other GHAs
 2022-03-21 18:55:15 -04:00
Katherine D. Mlika
68c882b3de
updating column E label to "Identified as disadvantaged" (#1406) 
* updating column E label to "Identified as disadvantaged"

* passing tests

* adding cached poetry flow

* working dir

Co-authored-by: Jorge Escobar <jorge.e.escobar@omb.eop.gov>
 2022-03-18 14:50:03 -04:00
Jorge Escobar
7b05ee9c76
S3 Parallel Upload and Deletions (#1410) 
* installation step

* trigger action

* installing to home dir

* dry-run

* pyenv

* py 2.8

* trying s4cmd

* removing pyenv

* poetry s4cmd

* num-threads

* public read

* poetry cache

* s4cmd all around

* poetry cache

* poetry cache

* install poetry packages

* poetry echo

* let's do this

* s4cmd install on run

* s4cmd

* ad aws back

* add aws back

* testing census api key and poetry caching

* census api key

* census api

* census api key #3

* 250

* poetry update

* poetry change

* check census api key

* force flag

* update score gen and tilefy; remove cached fips

* small gdal update

* invalidation

* missing cache ids
 2022-03-17 23:19:23 -04:00
Emma Nechamkin
e7c7c0abeb
Updating higher education to be reversed (#1387) 
Summary In this PR, we create a new variable so that the % college students is expressed as % not college students. This means that the front end can display % not college students.

Includes old variables so that this will not break fe.
 2022-03-15 16:43:32 -04:00
Emma Nechamkin
2279a04c94
Quick fix: updating snapshots to have more sigfigs (#1409) 
Updated snapshots to include 10 digits after the decimal
 2022-03-14 21:44:35 -04:00
Emma Nechamkin
9d920d4db4
Updating testing to include pytest-snapshot (#1355) 
In this commit, we slightly change the testing to use `pytest-snapshot`. This is for `ETL`s only.
 2022-03-11 21:34:07 -05:00
Jorge Escobar
7f91e2b06b
ArcGIS zipping (#1391) 
* ArcGIS zipping

* lint

* shapefile zip

* removing space in GMT

* adding shapefile to be staging gha
 2022-03-09 18:00:20 -05:00
Jorge Escobar
1730572aa6
Reducing Docker start up and adding ArcGIS URL (#1386) 
* Reducing Docker start up and adding ArcGIS URL

* Updating ArcGIS URLs
 2022-03-09 08:55:17 -05:00
Emma Nechamkin
917b84dc2e
WY tracts are not showing up until zoom >7 (#1342) 
In order to solve an issue where states with few census tracts appear to have no DACs, we change the low-zoom for states with under some threshold of tracts to be the high-zoom for those states. Thus, WY now has DACs even in low zoom. Yay!
 2022-03-08 17:33:11 -05:00
Jorge Escobar
6425beb9f4
YAML Config for Downloadable Assets (#1252) 
* starting yaml config load work

* working version for downloadable file

* yaml file update

* checkpoint

* sort if needed

* refactoring

* moving config

* checkpoint

* old files

* skipping downloadble tests for now

* more modularization

* more refactor, new excel yml

* pylint

* completed tabs

* Update excel.yml

* remvoing obsolete tests

* addressing PR feedback

* addressing changes

* confirmed change in yaml breaks tests

* safety bump

* PR review

* adding tests back

* pylint

* Incorporating latest score fields from Emma

* incorporating newest fields from Emma

* passing tests

* adding shapefile aws sync

* missing test

* passing tests
 2022-03-04 15:02:09 -05:00
Emma Nechamkin
1f5633ef74
Adding constants for front end to display booleans (#1348) 
Added constants for the threshold categories and socioeconomic indicators for front end.
 2022-03-02 17:12:28 -05:00