From 2aa79a334c72e90f39ce614519fe599909a796b7 Mon Sep 17 00:00:00 2001
From: Shaun Verch
Date: Wed, 5 Jan 2022 11:17:43 -0500
Subject: [PATCH 1/4] Add worst case scenario rollback plan (#1069)
Fixes: https://github.com/usds/justice40-tool/issues/947
This is ensuring we at least have some way to rollback in the worst case
scenario of the data being corrupted somehow, and our upstream data
sources being down in a way that prevents us from regenerating any
datasets. See
https://github.com/usds/justice40-tool/issues/946#issuecomment-989155252
for an analysis of the different ways the pipeline can fail and their
impacts.
This guide is very brittle to pipeline changes, and should be considered
a temporary solution for the worst case. I think that someday the github
actions should run more frequently and write to different paths, perhaps
with a timestamp. That would make rolling back more straightforward, as
a previous version of the data would already exist at some path in s3.
---
docs/operations/README.md | 3 +
docs/operations/rollback-plan.md | 194 +++++++++++++++++++++++++++++++
2 files changed, 197 insertions(+)
create mode 100644 docs/operations/README.md
create mode 100644 docs/operations/rollback-plan.md
diff --git a/docs/operations/README.md b/docs/operations/README.md
new file mode 100644
index 00000000..b79d0fce
--- /dev/null
+++ b/docs/operations/README.md
@@ -0,0 +1,3 @@
+# Operations Guides
+
+This directory should have guides for operating any services that are being supported by the Justice40 team.
diff --git a/docs/operations/rollback-plan.md b/docs/operations/rollback-plan.md
new file mode 100644
index 00000000..ad260b52
--- /dev/null
+++ b/docs/operations/rollback-plan.md
@@ -0,0 +1,194 @@
+# Rollback Plan
+
+> Note: This guide is up to date as of [this commit](https://github.com/usds/justice40-tool/commit/0d57dd572be027a2fc8b1625958ed68c4b900653), 12/16/2021. If you want to sanity check that this guide is still relevant, go to [Rollback Details](#rollback-details).
+
+## When To Rollback?
+
+If for some reason the final map data that has been generated by the pipeline
+has become incorrect or is missing, this page documents the emergency steps to
+get the data back to a known good state.
+
+## Rollback Theory
+
+The theory of rollback depends on two things:
+
+1. The s3 bucket containing our data uses [s3 bucket versioning](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.html), allowing us to revert specific files back to a previous version.
+2. The [Combine and Tileify step](https://github.com/usds/justice40-tool/blob/main/.github/workflows/combine-tilefy.yml) consumes only two files in s3 as input, making the strategy of reverting those files to a previous version a feasible way to run this job against a previous version.
+
+If you feel confident in this and want to do the rollback now, proceed to [Rollback Steps](#rollback-steps).
+
+If you want to understand more deeply what's going on, and sanity check that the
+code hasn't changed since this guide was written, go to [Rollback Details](#rollback-details).
+
+## Rollback Steps
+
+### 1. List Object Versions
+
+```shell
+aws s3api list-object-versions \
+ --bucket justice40-data \
+ --prefix data-sources/census.zip \
+ --query "Versions[*].{Key:Key,VersionId:VersionId,LastModified:LastModified}"
+```
+
+You should get something like:
+
+```
+[
+...
+{
+ "Key": "data-sources/census.zip",
+ "VersionId": "",
+ "LastModified": "2021-11-29T18:57:40+00:00"
+}
+...
+]
+```
+
+Do the same thing with the score file:
+
+```shell
+aws s3api list-object-versions \
+ --bucket justice40-data \
+ --prefix data-pipeline/data/score/csv/tiles/usa.csv \
+ --query "Versions[*].{Key:Key,VersionId:VersionId,LastModified:LastModified}"
+```
+
+### 2. Download Previous Version
+
+Based on the output from the commands above, select the `` of the known good version of that file, and run the following command to download it:
+
+```shell
+aws s3api get-object \
+ --bucket justice40-data \
+ --key data-sources/census.zip \
+ --version-id \
+ census.zip
+```
+
+Do the same for the score file:
+
+```shell
+aws s3api get-object \
+ --bucket justice40-data \
+ --key data-pipeline/data/score/csv/tiles/usa.csv \
+ --version-id \
+ usa.csv
+```
+
+> Note: This command doesn't give you feedback like `curl` does on download progress, it just sits there. To verify the download is happening, open another terminal and check the output file size.
+
+### 3. Upload New Version
+
+After you've verified that the local files are correct, you can overwrite the
+live versions by running a normal s3 copy:
+
+```shell
+aws s3 cp census.zip s3://justice40-data/data-sources/census.zip
+aws s3 cp usa.csv s3://justice40-data/data-pipeline/data/score/csv/tiles/usa.csv
+```
+
+### 4. Rerun Combine and Tileify
+
+Run the [Combine and Tileify Github Action](https://github.com/usds/justice40-tool/actions/workflows/combine-tilefy.yml) to regenerate the map tiles from the data you just rolled back.
+
+## Rollback Details
+
+> Note: The links to the relevant code are included alongside the relevant snippets as a defense against this page becoming outdated. Make sure the code matches what the links point to, and verify that things still work as this guide assumes.
+
+The relevant step that consumes the files from s3 should be in the [Combine and Tileify Job](https://github.com/usds/justice40-tool/blob/main/.github/workflows/combine-tilefy.yml#L56-L59):
+
+```yaml
+- name: Run Scripts
+ run: |
+ poetry run python3 data_pipeline/application.py geo-score -s aws
+ poetry run python3 data_pipeline/application.py generate-map-tiles
+```
+
+This runs the [`score_geo` task](https://github.com/usds/justice40-tool/blob/main/data/data-pipeline/data_pipeline/application.py#L166):
+
+```python
+score_geo(data_source=data_source)
+```
+
+This uses the [`GeoScoreETL` object to run the etl](https://github.com/usds/justice40-tool/blob/main/data/data-pipeline/data_pipeline/etl/runner.py#L121-L124):
+
+```python
+score_geo = GeoScoreETL(data_source=data_source)
+score_geo.extract()
+score_geo.transform()
+score_geo.load()
+```
+
+In the extract step, `GeoScoreETL` calls [`check_census_data` and `check_score_data`](https://github.com/usds/justice40-tool/blob/main/data/data-pipeline/data_pipeline/etl/score/etl_score_geo.py#L57-L67):
+
+```python
+# check census data
+check_census_data_source(
+ census_data_path=self.DATA_PATH / "census",
+ census_data_source=self.DATA_SOURCE,
+)
+
+# check score data
+check_score_data_source(
+ score_csv_data_path=self.SCORE_CSV_PATH,
+ score_data_source=self.DATA_SOURCE,
+)
+```
+
+The `check_score_data_source` function downloads [one file from s3](https://github.com/usds/justice40-tool/blob/main/data/data-pipeline/data_pipeline/etl/score/etl_utils.py#L32-L43):
+
+```python
+TILE_SCORE_CSV_S3_URL = (
+ settings.AWS_JUSTICE40_DATAPIPELINE_URL
+ + "/data/score/csv/tiles/usa.csv"
+)
+TILE_SCORE_CSV = score_csv_data_path / "tiles" / "usa.csv"
+
+# download from s3 if census_data_source is aws
+if score_data_source == "aws":
+ logger.info("Fetching Score Tile data from AWS S3")
+ download_file_from_url(
+ file_url=TILE_SCORE_CSV_S3_URL, download_file_name=TILE_SCORE_CSV
+ )
+```
+
+This can be found here:
+
+```
+% aws s3 ls s3://justice40-data/data-pipeline/data/score/csv/tiles/usa.csv
+2021-12-13 15:23:49 27845542 usa.csv
+% curl --head https://justice40-data.s3.amazonaws.com/data-pipeline/data/score/csv/tiles/usa.csv
+HTTP/1.1 200 OK
+...
+```
+
+The `check_census_data_source` function downloads [one file from s3](https://github.com/usds/justice40-tool/blob/main/data/data-pipeline/data_pipeline/etl/sources/census/etl_utils.py#L99-L109):
+
+```python
+CENSUS_DATA_S3_URL = settings.AWS_JUSTICE40_DATASOURCES_URL + "/census.zip"
+DATA_PATH = settings.APP_ROOT / "data"
+
+# download from s3 if census_data_source is aws
+if census_data_source == "aws":
+ logger.info("Fetching Census data from AWS S3")
+ unzip_file_from_url(
+ CENSUS_DATA_S3_URL,
+ DATA_PATH / "tmp",
+ DATA_PATH,
+ )
+```
+
+This can be found here:
+
+```
+% aws s3 ls s3://justice40-data/data-sources/census.zip
+2021-11-29 13:57:40 845390373 census.zip
+% curl --head https://justice40-data.s3.amazonaws.com/data-sources/census.zip
+HTTP/1.1 200 OK
+...
+```
+
+So this is how to see that the pipeline needs those two files. You can also confirm this in the [Combine and Tileify Github Action logs](https://github.com/usds/justice40-tool/actions/workflows/combine-tilefy.yml).
+
+If you feel confident in this and want to do the rollback now, proceed to [Rollback Steps](#rollback-steps).
From 93595b7bb45e6b173889b9a02ff1ec82d864e0d3 Mon Sep 17 00:00:00 2001
From: Shaun Verch
Date: Wed, 5 Jan 2022 15:58:24 -0500
Subject: [PATCH 2/4] Re-export requirements.txt to fix version errors (#1099)
* Re-export requirements.txt to fix version errors
The version of lxml in this file had a known vulnerability that got
caught by the "safety" checker, but it is updated in the poetry files.
Regenerated using:
https://github.com/usds/justice40-tool/tree/main/data/data-pipeline#miscellaneous
* Fix lint error
* Run lint on all envs and add comments
* Ignore testst that fail lint because of dev deps
* Ignore medium.com in link checker
It's returning 403s to github actions...
---
.../etl.py | 10 +-
data/data-pipeline/requirements.txt | 104 +++++++-----------
data/data-pipeline/tox.ini | 19 +++-
mlc_config.json | 3 +
4 files changed, 66 insertions(+), 70 deletions(-)
diff --git a/data/data-pipeline/data_pipeline/etl/sources/energy_definition_alternative_draft/etl.py b/data/data-pipeline/data_pipeline/etl/sources/energy_definition_alternative_draft/etl.py
index c8a95fe1..6023f71d 100644
--- a/data/data-pipeline/data_pipeline/etl/sources/energy_definition_alternative_draft/etl.py
+++ b/data/data-pipeline/data_pipeline/etl/sources/energy_definition_alternative_draft/etl.py
@@ -95,9 +95,13 @@ class EnergyDefinitionAlternativeDraft(ExtractTransformLoad):
)
# Convert to boolean:
- self.df[field_names.ENERGY_RELATED_COMMUNITIES_DEFINITION_ALTERNATIVE] = \
- self.df[field_names.ENERGY_RELATED_COMMUNITIES_DEFINITION_ALTERNATIVE
- ].astype('bool')
+ self.df[
+ field_names.ENERGY_RELATED_COMMUNITIES_DEFINITION_ALTERNATIVE
+ ] = self.df[
+ field_names.ENERGY_RELATED_COMMUNITIES_DEFINITION_ALTERNATIVE
+ ].astype(
+ "bool"
+ )
def validate(self) -> None:
logger.info("Validating data")
diff --git a/data/data-pipeline/requirements.txt b/data/data-pipeline/requirements.txt
index a12d0449..3eacb5cf 100644
--- a/data/data-pipeline/requirements.txt
+++ b/data/data-pipeline/requirements.txt
@@ -1,48 +1,39 @@
appnope==0.1.2; sys_platform == "darwin" and python_version >= "3.7" and platform_system == "Darwin"
argcomplete==1.12.3; python_version < "3.8.0" and python_version >= "3.7"
argon2-cffi==21.1.0; python_version >= "3.6"
-astroid==2.8.0; python_version >= "3.6" and python_version < "4.0"
-atomicwrites==1.4.0; python_version >= "3.6" and python_full_version < "3.0.0" and sys_platform == "win32" or sys_platform == "win32" and python_version >= "3.6" and python_full_version >= "3.4.0"
-attrs==21.2.0; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.6"
+astroid==2.8.3; python_version >= "3.6" and python_version < "4.0"
+attrs==21.2.0; python_version >= "3.7" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.7"
backcall==0.2.0; python_version >= "3.7"
-backports.entry-points-selectable==1.1.0; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "2.7"
-black==21.9b0; python_full_version >= "3.6.2"
bleach==4.1.0; python_version >= "3.7"
censusdata==1.15; python_version >= "2.7"
-certifi==2021.5.30; python_version >= "3.7" and python_full_version < "3.0.0" or python_full_version >= "3.6.0" and python_version >= "3.7"
-cffi==1.14.6; implementation_name == "pypy" and python_version >= "3.6"
-charset-normalizer==2.0.6; python_full_version >= "3.6.0" and python_version >= "3"
+certifi==2021.10.8; python_version >= "3.7" and python_full_version < "3.0.0" or python_full_version >= "3.6.0" and python_version >= "3.7"
+cffi==1.15.0; implementation_name == "pypy" and python_version >= "3.7" and python_full_version >= "3.6.1"
+charset-normalizer==2.0.7; python_full_version >= "3.6.0" and python_version >= "3"
click-plugins==1.1.1; python_version >= "3.6"
-click==8.0.1; python_version >= "3.6"
+click==8.0.3; python_version >= "3.6"
cligj==0.7.2; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.3.0" and python_version < "4" and python_version >= "3.6"
-colorama==0.4.4; platform_system == "Windows" and python_version >= "3.7" and python_full_version >= "3.6.2" and sys_platform == "win32" and python_version < "4.0" and (python_version >= "3.6" and python_full_version < "3.0.0" and sys_platform == "win32" or sys_platform == "win32" and python_version >= "3.6" and python_full_version >= "3.5.0") and (python_version >= "3.7" and python_full_version < "3.0.0" and sys_platform == "win32" or sys_platform == "win32" and python_version >= "3.7" and python_full_version >= "3.5.0")
-configparser==5.0.2; python_version >= "3.6"
+colorama==0.4.4; python_version >= "3.7" and python_full_version < "3.0.0" and platform_system == "Windows" and sys_platform == "win32" and python_version < "4.0" or platform_system == "Windows" and python_version >= "3.7" and python_full_version >= "3.5.0" and sys_platform == "win32" and python_version < "4.0"
cycler==0.10.0; python_version >= "3.7"
-debugpy==1.4.3; python_version >= "3.7" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.7"
+debugpy==1.5.1; python_version >= "3.7" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.7"
decorator==5.1.0; python_version >= "3.7"
defusedxml==0.7.1; python_version >= "3.7" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.7"
-distlib==0.3.2; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.5.0"
-dparse==0.5.1; python_version >= "3.5"
dynaconf==3.1.7; python_version >= "3.7"
entrypoints==0.3; python_full_version >= "3.6.1" and python_version >= "3.7"
-et-xmlfile==1.1.0; python_version >= "3.6"
-filelock==3.0.12; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.5.0"
fiona==1.8.20; python_version >= "3.6"
-flake8==3.9.2; (python_version >= "2.7" and python_full_version < "3.0.0") or (python_full_version >= "3.5.0")
geopandas==0.9.0; python_version >= "3.6"
-idna==3.2; python_version >= "3.5" and python_full_version < "3.0.0" or python_full_version >= "3.6.0" and python_version >= "3.5"
-importlib-metadata==4.8.1; python_version == "3.7" and (python_version >= "3.6" and python_full_version < "3.0.0" and python_version < "3.8" or python_full_version >= "3.5.0" and python_version < "3.8" and python_version >= "3.6") and python_full_version >= "3.6.2"
-iniconfig==1.1.1; python_version >= "3.6"
-ipykernel==6.4.1; python_version >= "3.7"
+idna==3.3; python_version >= "3.5" and python_full_version < "3.0.0" or python_full_version >= "3.6.0" and python_version >= "3.5"
+importlib-metadata==4.8.1; python_version == "3.7"
+ipdb==0.13.9; python_version >= "2.7"
+ipykernel==6.4.2; python_version >= "3.7"
ipython-genutils==0.2.0; python_version >= "3.7"
-ipython==7.27.0; python_version >= "3.7"
+ipython==7.28.0; python_version >= "3.7"
ipywidgets==7.6.5
isort==5.9.3; python_full_version >= "3.6.1" and python_version < "4.0" and python_version >= "3.6"
jedi==0.18.0; python_version >= "3.7"
jellyfish==0.6.1
-jinja2==3.0.1; python_version >= "3.7"
-jsonschema==3.2.0; python_version >= "3.5"
-jupyter-client==7.0.3; python_full_version >= "3.6.1" and python_version >= "3.7"
+jinja2==3.0.2; python_version >= "3.7"
+jsonschema==4.1.2; python_version >= "3.7"
+jupyter-client==7.0.6; python_full_version >= "3.6.1" and python_version >= "3.7"
jupyter-console==6.4.0; python_version >= "3.6"
jupyter-contrib-core==0.3.3
jupyter-contrib-nbextensions==0.5.1
@@ -55,79 +46,64 @@ jupyterlab-pygments==0.1.2; python_version >= "3.7"
jupyterlab-widgets==1.0.2; python_version >= "3.6"
kiwisolver==1.3.2; python_version >= "3.7"
lazy-object-proxy==1.6.0; python_version >= "3.6" and python_full_version < "3.0.0" and python_version < "4.0" or python_version >= "3.6" and python_version < "4.0" and python_full_version >= "3.6.0"
-liccheck==0.6.2; python_version >= "2.7"
-lxml==4.6.3; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.5.0"
+lxml==4.6.5; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.5.0"
markupsafe==2.0.1; python_version >= "3.7"
matplotlib-inline==0.1.3; python_version >= "3.7"
matplotlib==3.4.3; python_version >= "3.7"
-mccabe==0.6.1; python_version >= "3.6" and python_full_version < "3.0.0" and python_version < "4.0" or python_version >= "3.6" and python_version < "4.0" and python_full_version >= "3.5.0"
+mccabe==0.6.1; python_version >= "3.6" and python_version < "4.0"
mistune==0.8.4; python_version >= "3.7"
munch==2.5.0; python_version >= "3.6"
-mypy-extensions==0.4.3; python_full_version >= "3.6.2" and python_version >= "3.5"
-mypy==0.910; python_version >= "3.5"
nbclient==0.5.4; python_full_version >= "3.6.1" and python_version >= "3.7"
-nbconvert==6.1.0; python_version >= "3.7"
+nbconvert==6.2.0; python_version >= "3.7"
nbformat==5.1.3; python_full_version >= "3.6.1" and python_version >= "3.7"
nest-asyncio==1.5.1; python_full_version >= "3.6.1" and python_version >= "3.7"
-notebook==6.4.4; python_version >= "3.6"
+notebook==6.4.5; python_version >= "3.6"
numpy==1.21.1; python_version >= "3.7"
-openpyxl==3.0.7; python_version >= "3.6"
-packaging==21.0; python_version >= "3.7" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.7"
-pandas==1.3.3; python_full_version >= "3.7.1"
+packaging==21.0; python_version >= "3.7"
+pandas==1.3.4; python_full_version >= "3.7.1"
pandocfilters==1.5.0; python_version >= "3.7" and python_full_version < "3.0.0" or python_full_version >= "3.4.0" and python_version >= "3.7"
parso==0.8.2; python_version >= "3.7"
-pathspec==0.9.0; python_full_version >= "3.6.2"
pexpect==4.8.0; sys_platform != "win32" and python_version >= "3.7"
pickleshare==0.7.5; python_version >= "3.7"
-pillow==8.3.2; python_version >= "3.7"
-platformdirs==2.3.0; python_version >= "3.6" and python_full_version >= "3.6.2" and python_version < "4.0" and (python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.6")
-pluggy==1.0.0; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.6"
+pillow==8.4.0; python_version >= "3.7"
+platformdirs==2.4.0; python_version >= "3.6" and python_version < "4.0"
prometheus-client==0.11.0; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.4.0" and python_version >= "3.6"
-prompt-toolkit==3.0.20; python_full_version >= "3.6.2" and python_version >= "3.7"
+prompt-toolkit==3.0.21; python_full_version >= "3.6.2" and python_version >= "3.7"
ptyprocess==0.7.0; sys_platform != "win32" and python_version >= "3.7" and os_name != "nt"
-py==1.10.0; python_version >= "3.6" and python_full_version < "3.0.0" and implementation_name == "pypy" or python_full_version >= "3.5.0" and python_version >= "3.6" and implementation_name == "pypy"
-pycodestyle==2.7.0; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.5.0"
-pycparser==2.20; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.4.0" and python_version >= "3.6"
-pyflakes==2.3.1; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.5.0"
+py==1.10.0; implementation_name == "pypy" and python_version >= "3.7" and python_full_version >= "3.6.1"
+pycparser==2.20; implementation_name == "pypy" and python_version >= "3.7" and python_full_version >= "3.6.1"
pygments==2.10.0; python_version >= "3.7"
pylint==2.11.1; python_version >= "3.6" and python_version < "4.0"
pypandoc==1.6.4
pyparsing==2.4.7; python_version >= "3.7" and python_full_version < "3.0.0" or python_full_version >= "3.3.0" and python_version >= "3.7"
pyproj==3.2.1; python_version >= "3.7"
-pyrsistent==0.18.0; python_version >= "3.6"
-pytest-mock==3.6.1; python_version >= "3.6"
-pytest==6.2.5; python_version >= "3.6"
+pyrsistent==0.18.0; python_version >= "3.7"
python-dateutil==2.8.2; python_full_version >= "3.7.1" and python_version >= "3.7"
-pytz==2021.1; python_full_version >= "3.7.1" and python_version >= "2.7"
-pywin32==301; sys_platform == "win32" and platform_python_implementation != "PyPy" and python_version >= "3.6"
+pytz==2021.3; python_full_version >= "3.7.1" and python_version >= "3.6"
+pywin32==302; sys_platform == "win32" and platform_python_implementation != "PyPy" and python_version >= "3.7"
pywinpty==1.1.4; os_name == "nt" and python_version >= "3.6"
-pyyaml==5.4.1; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.6.0"
+pyyaml==6.0; python_version >= "3.6"
pyzmq==22.3.0; python_full_version >= "3.6.1" and python_version >= "3.7"
qtconsole==5.1.1; python_version >= "3.6"
-qtpy==1.11.1; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.6.0" and python_version >= "3.6"
-regex==2021.8.28; python_full_version >= "3.6.2"
+qtpy==1.11.2; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.6.0" and python_version >= "3.6"
requests==2.26.0; (python_version >= "2.7" and python_full_version < "3.0.0") or (python_full_version >= "3.6.0")
-safety==1.10.3; python_version >= "3.5"
-semantic-version==2.8.5; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.4.0" and python_version >= "2.7"
send2trash==1.8.0; python_version >= "3.6"
shapely==1.7.1; python_version >= "3.6"
-six==1.16.0; python_version >= "3.7" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.7"
+six==1.16.0; python_full_version >= "3.7.1" and python_version >= "3.7" and (python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.3.0" and python_version >= "3.6") and (python_version >= "3.7" and python_full_version < "3.0.0" or python_full_version >= "3.3.0" and python_version >= "3.7")
terminado==0.12.1; python_version >= "3.6"
testpath==0.5.0; python_version >= "3.7"
-toml==0.10.2; python_version >= "3.6" and python_full_version < "3.0.0" and python_version < "4.0" or python_full_version >= "3.5.0" and python_version >= "3.6" and python_version < "4.0"
-tomli==1.2.1; python_version >= "3.6" and python_full_version >= "3.6.2"
+toml==0.10.2; python_version > "3.6" and python_full_version < "3.0.0" and python_version < "4.0" or python_full_version >= "3.3.0" and python_version > "3.6" and python_version < "4.0"
tornado==6.1; python_full_version >= "3.6.1" and python_version >= "3.7"
-tox==3.24.4; (python_version >= "2.7" and python_full_version < "3.0.0") or (python_full_version >= "3.5.0")
tqdm==4.62.0; (python_version >= "2.7" and python_full_version < "3.0.0") or (python_full_version >= "3.4.0")
traitlets==5.1.0; python_full_version >= "3.6.1" and python_version >= "3.7"
-types-requests==2.25.8
-typing-extensions==3.10.0.2; python_version < "3.8" and python_version >= "3.6"
-urllib3==1.26.6; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.6.0" and python_version < "4" and python_version >= "2.7"
+typed-ast==1.4.3; implementation_name == "cpython" and python_version < "3.8" and python_version >= "3.6"
+types-requests==2.25.11
+typing-extensions==3.10.0.2; python_version >= "3.6" and python_version < "3.8"
+urllib3==1.26.7; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.6.0" and python_version < "4" and python_version >= "2.7"
us==2.0.2
-virtualenv==20.8.0; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.5.0"
wcwidth==0.2.5; python_full_version >= "3.6.2" and python_version >= "3.7"
webencodings==0.5.1; python_version >= "3.7"
widgetsnbextension==3.5.1
-wrapt==1.12.1; python_version >= "3.6" and python_version < "4.0"
+wrapt==1.13.2; python_version >= "3.6" and python_full_version < "3.0.0" and python_version < "4.0" or python_version >= "3.6" and python_version < "4.0" and python_full_version >= "3.5.0"
xlsxwriter==2.0.0
-zipp==3.5.0; python_version < "3.8" and python_version >= "3.6"
+zipp==3.6.0; python_version < "3.8" and python_version >= "3.6"
diff --git a/data/data-pipeline/tox.ini b/data/data-pipeline/tox.ini
index 381139bb..059520a4 100644
--- a/data/data-pipeline/tox.ini
+++ b/data/data-pipeline/tox.ini
@@ -7,19 +7,32 @@ skip_missing_interpreters = true
[testenv:lint]
# lints python code in src and tests
-basepython = python3.9
deps = -rrequirements.txt
+# These are "external" because they are dev dependencies that not in
+# "requirements.txt" and therefore not installed in this env, but they will be
+# available where this is being run. See:
+# https://stackoverflow.com/questions/47642747/tox-warningtest-command-found-but-not-installed-in-testenv
+allowlist_externals = black
+ flake8
+ pylint
commands = black data_pipeline
flake8 data_pipeline
- pylint data_pipeline
+# Ignore tests this lint check because test dependencies are not installed here.
+ pylint data_pipeline --ignore tests
[testenv:checkdeps]
# checks the dependencies for security vulnerabilities and open source licenses
deps = -rrequirements.txt
+# These are "external" because they are dev dependencies that not in
+# "requirements.txt" and therefore not installed in this env, but they will be
+# available where this is being run. See:
+# https://stackoverflow.com/questions/47642747/tox-warningtest-command-found-but-not-installed-in-testenv
+allowlist_externals = safety
+ liccheck
commands = safety check
liccheck
[testenv:pytest]
# Run tests
deps = pytest
-commands = pytest
\ No newline at end of file
+commands = pytest
diff --git a/mlc_config.json b/mlc_config.json
index ebc7c01e..53f78d89 100644
--- a/mlc_config.json
+++ b/mlc_config.json
@@ -14,6 +14,9 @@
},
{
"pattern": "^https://github.com/usds/environmental-justice/"
+ },
+ {
+ "pattern": "^https://.*medium.com"
}
],
"replacementPatterns": [
From 56644698ff40b0a155ef1f872cd798bb3462f25f Mon Sep 17 00:00:00 2001
From: Saran Ahluwalia <94847739+saran-ahluwalia@users.noreply.github.com>
Date: Wed, 5 Jan 2022 17:03:37 -0500
Subject: [PATCH 3/4] Address rounding issue in Pandas series to floor
numerically unstable values (#1085)
* wip - added tests - 1 failing
* added check for empty series + added test
* passing tests
* parallelism in variable assingnment choice
* resolve merge conflicts
* variable name changes
* cleanup logic and move comments out of main code execution + add one more test for an extreme example eith -np.inf
* cleanup logic and move comments out of main code execution + add one more test for an extreme example eith -np.inf
* revisions to handle type ambiguity
* fixing tests
* fix pytest
* fix linting
* fix pytest
* reword comments
* cleanup comments
* cleanup comments - fix typo
* added type check and corresponding test
* added type check and corresponding test
* language cleanup
* revert
* update picke fixture
Co-authored-by: Jorge Escobar
---
.../data_pipeline/etl/score/etl_score_post.py | 15 ++--
.../data_pipeline/etl/score/etl_utils.py | 60 +++++++++++++++
.../tests/snapshots/tile_data_expected.pkl | Bin 3034 -> 3085 bytes
.../etl/score/tests/test_etl_utils.py | 72 ++++++++++++++++++
4 files changed, 141 insertions(+), 6 deletions(-)
create mode 100644 data/data-pipeline/data_pipeline/etl/score/tests/test_etl_utils.py
diff --git a/data/data-pipeline/data_pipeline/etl/score/etl_score_post.py b/data/data-pipeline/data_pipeline/etl/score/etl_score_post.py
index 8f527c7a..244cccbe 100644
--- a/data/data-pipeline/data_pipeline/etl/score/etl_score_post.py
+++ b/data/data-pipeline/data_pipeline/etl/score/etl_score_post.py
@@ -3,6 +3,7 @@ import json
import pandas as pd
from data_pipeline.etl.base import ExtractTransformLoad
+from data_pipeline.etl.score.etl_utils import floor_series
from data_pipeline.utils import get_module_logger, zip_files
from data_pipeline.score import field_names
@@ -207,13 +208,15 @@ class PostScoreETL(ExtractTransformLoad):
# filter the columns on full score
score_tiles = score_county_state_merged_df[tiles_score_column_titles]
- # round decimals
- decimals = pd.Series(
- [constants.TILES_ROUND_NUM_DECIMALS]
- * len(constants.TILES_SCORE_FLOAT_COLUMNS),
- index=constants.TILES_SCORE_FLOAT_COLUMNS,
+ score_tiles[constants.TILES_SCORE_FLOAT_COLUMNS] = score_tiles[
+ constants.TILES_SCORE_FLOAT_COLUMNS
+ ].apply(
+ func=lambda series: floor_series(
+ series=series,
+ number_of_decimals=constants.TILES_ROUND_NUM_DECIMALS,
+ ),
+ axis=0,
)
- score_tiles = score_tiles.round(decimals)
# create indexes
score_tiles = score_tiles.rename(
diff --git a/data/data-pipeline/data_pipeline/etl/score/etl_utils.py b/data/data-pipeline/data_pipeline/etl/score/etl_utils.py
index c3fec035..2219ab18 100644
--- a/data/data-pipeline/data_pipeline/etl/score/etl_utils.py
+++ b/data/data-pipeline/data_pipeline/etl/score/etl_utils.py
@@ -1,6 +1,9 @@
import os
import sys
from pathlib import Path
+import numpy as np
+import pandas as pd
+
from data_pipeline.config import settings
from data_pipeline.utils import (
@@ -48,3 +51,60 @@ def check_score_data_source(
"No local score tiles data found. Please use '-d aws` to fetch from AWS"
)
sys.exit()
+
+
+def floor_series(series: pd.Series, number_of_decimals: int) -> pd.Series:
+ """Floors all non-null numerical values to a specific number of decimal points
+
+ Args:
+ series (pd.Series): Input pandas series
+ number_of_decimals (int): Number of decimal points to floor all numerical values to
+ Returns:
+ floored_series (pd.Series): A Pandas Series of numerical values with appropriate number of decimal points
+ """
+
+ # we perform many operations using the division operator
+ # as well as elementwise multiplication. The result of such
+ # operations can introduce such values, below, due to numerical
+ # instability. This results in unsafe type inference for numpy
+ # float types - exacerbated by panda's type inference engine.
+ # Hence, to handle such offending values we default to None
+ # Please see the reference, below, on nullable integer types for more details
+ unacceptable_values = [-np.inf, np.inf, "None", np.nan]
+ mapping = {
+ unacceptable_value: None for unacceptable_value in unacceptable_values
+ }
+
+ # ensure we are working with a numpy array (which is really what a pandas series is)
+ if not isinstance(series, pd.Series):
+ raise TypeError(
+ f"Argument series must be of type pandas series, not of type {type(series).__name__}."
+ )
+
+ # raise exception for handling empty series
+ if series.empty:
+ raise ValueError("Empty series provided.")
+
+ # if we have any values, just replace them with None
+ if series.isin(unacceptable_values).any():
+ series.replace(mapping, regex=False, inplace=True)
+
+ multiplication_factor = 10 ** number_of_decimals
+
+ # In order to safely cast NaNs
+ # First coerce series to float type: series.astype(float)
+ # Please see here:
+ # https://pandas.pydata.org/pandas-docs/stable/user_guide/integer_na.html#nullable-integer-data-type
+ product_for_numerator = np.floor(
+ series.astype(float) * multiplication_factor
+ )
+
+ floored_series = np.where(
+ series.isnull(),
+ # For all null values default to null
+ None,
+ # The other default condition - floor non-null values
+ product_for_numerator / multiplication_factor,
+ )
+
+ return floored_series
diff --git a/data/data-pipeline/data_pipeline/etl/score/tests/snapshots/tile_data_expected.pkl b/data/data-pipeline/data_pipeline/etl/score/tests/snapshots/tile_data_expected.pkl
index 8c47d6538afd0dfd2b146bce3b936c243ade83eb..d066a170c285542f993355c2d18925d057ef3394 100644
GIT binary patch
delta 755
zcmca5-Ydb{z%rGIXCrF}6Qk1PC?+jNm&vV6_G}CwFlF+2c2S`46QHobWPaw)let(F
zCU0gpWMrJ2!lEhTEU+SAp;If@R{I^&(>1>P9W@GwWv5hPr*6AG_Ry`
zN=Bc%{kt83uUqAIxZA&v(fE3(KgQkuwJ{jD+rR1u3%T1r=dvwWe4WeP{w;{IEp)ek
z0oD}cZvW;C2%H7#0vZAo1eya=`~arr5m0wMP#;hm$OesXK!d=F<=yQc!?Zn|2?Dd+
z?VnUVbYCY@?QVY+r1tA!Bop7Ifq}dIKA59HMuSa$1!uq#LsQm-fRy-mwsZ>xJP{~mDW@w#~q1iqqXv*XV>?{&8KwfC4
z^8-exIwmMR*?_HJ@(MN&8C9r&HI#OQ(x5ao*@?r1an|H=4lTy6&5JmKnHc9x{>Ej<
QxL~pdcOc`-$yMA50Le=j-~a#s
delta 698
zcmeB`xFycoz%unb_eRzbCPtOXQA}El9+O*{>=hUppkPYLl%x!m40UgY)+rg9?NfrL
zXm~Svt4w|XRH`tUoB1=N_2lo&>Wq$)#aS#EB_;>3Xx2M50JU>nUtD0z_1=D+`@<@c
zb?@zs!Qiz$i0A(Bm3>VAp|2V-&+Q9sK`P(c2knq+eI5A1{_GhLcw-ObfJE$J=0M~h
z*n_q8KeGR(0W#qQTt7tpWBWIO^&ou@p$0v%KMb^{|Ik&qL124!yt7{dbl&yF2jB*R
z-C_F*?m$hj3-{R1nh63cq2|1W(<=fNI<<0bwcjBfW7<@>YzJN3t!noLihz#<7>FjA^P4x9dZk<4iZv#?LpyJCGyT5;*ojw@?fvO
zg@=S%HCV+Pd+c#10gTAdPUi!Y7qS;H3Qp$Y@SiNlV#vbm&D1tIpIcqpKv5XtvLbd^
zB7o4043qz{NNZsv24;wX5)x45s!-Y%oFteg2eF!~P3;lzN=-{kEh^5>Q*h2N%`2
zk})k6l(=kw>518!5tOthmvWde&YQf7LyK|J=BpgROw0>
Date: Wed, 5 Jan 2022 17:37:47 -0500
Subject: [PATCH 4/4] Add category header and updates from Living Copy (#1084)
* Add category header (Indicator and Percentile)
- add local styling
- override accordion-content styles to push into parent container
- intl copy
- update snapshots
* Add latest Living Copy
- refactor dataset cards to allow for any styling in description, used in and responsible party
- add percent for HS edu indicator
- updates snapshots
* Update copy from Living Copy doc
-update snapshots
* Add methodology to copy
* Remove link from readme file based on md checker
* Comment out certain markdown links from checker
* Comment out markdown links that render in browser
---
INSTALLATION-es.md | 2 +-
client/README.md | 2 +
.../AreaDetail/areaDetail.module.scss | 18 +
.../AreaDetail/areaDetail.module.scss.d.ts | 1 +
client/src/components/AreaDetail/index.tsx | 125 +--
.../__snapshots__/areaDetail.test.tsx.snap | 73 +-
.../__snapshots__/BetaBanner.test.tsx.snap | 4 +-
.../__snapshots__/Categories.test.tsx.snap | 2 +-
client/src/components/DatasetCard/index.tsx | 4 +-
.../__snapshots__/datasetCard.test.tsx.snap | 8 +-
.../datasetContainer.test.tsx.snap | 253 +++---
client/src/components/Indicator/Indicator.tsx | 9 +-
.../__snapshots__/J40Header.test.tsx.snap | 4 +-
client/src/data/copy/common.tsx | 4 +-
client/src/data/copy/explore.tsx | 15 +-
client/src/data/copy/methodology.tsx | 741 +++++++++++++-----
.../pages/__snapshots__/contact.test.tsx.snap | 4 +-
.../pages/__snapshots__/index.test.tsx.snap | 4 +-
.../__snapshots__/methodology.test.tsx.snap | 259 +++---
client/src/styles/global.scss | 6 +
.../0002-mapping-visualization-library.md | 1 +
docs/decisions/0004-client-side-framework.md | 6 +-
22 files changed, 1059 insertions(+), 486 deletions(-)
diff --git a/INSTALLATION-es.md b/INSTALLATION-es.md
index b4365aa3..9403ea2b 100644
--- a/INSTALLATION-es.md
+++ b/INSTALLATION-es.md
@@ -25,7 +25,7 @@ Descargar desde [sitio web](https://git-scm.com/download/win)
2. Valide la instalación escribiendo `brew -v` en la terminal y asegúrese de que se muestre un número de versión.
### Instalar Node usando NVM
-
+
Esto funcionará tanto para MacOS como para Win10. Siga las instrucciones de este [enlace](https://medium.com/@nodesource/installing-node-js-tutorial-using-nvm-5c6ff5925dd8)
### Configuración de IDE
diff --git a/client/README.md b/client/README.md
index 84eee920..2f7497bd 100644
--- a/client/README.md
+++ b/client/README.md
@@ -19,6 +19,7 @@ This README contains the following content:
#### Install Node using NVM
+
This will work for both MacOS and Win10. Follow instructions on this [link](https://medium.com/@nodesource/installing-node-js-tutorial-using-nvm-5c6ff5925dd8). Be sure to read through the whole doc to find the sections within each step relevant to you (e.g. if you're using Homebrew, when you get to Step 2 look for the section, "Install NVM with Homebrew").
If you install NVM using Homebrew, make sure to read the output in terminal after you run `brew install nvm`. You will need to add a few lines to your ~/.bash_profile and perhaps complete a couple other tasks.
@@ -70,6 +71,7 @@ DATA_SOURCE env variable in the docker-compose.yml. See [environment variables](
#### Troubleshooting docker
+
- If an error is thrown about [running out of space](https://medium.com/@wlarch/no-space-left-on-device-when-using-docker-compose-why-c4a2c783c6f6) on device see this for ways to reclaim space.
diff --git a/client/src/components/AreaDetail/areaDetail.module.scss b/client/src/components/AreaDetail/areaDetail.module.scss
index 9e319643..03f644da 100644
--- a/client/src/components/AreaDetail/areaDetail.module.scss
+++ b/client/src/components/AreaDetail/areaDetail.module.scss
@@ -82,3 +82,21 @@ $sidePanelLabelFontColor: #171716;
font-size: medium;
}
}
+
+// The following class is used in the AccordionItems in the AreaDetail component.
+// The Accordion component (parent of AccordionItems) requires some CSS overrides.
+// Local styling is not allowing the override.
+// The override is needed to push into the bounds of the Accordion component's styles.
+// To override this, in globals.scss, we set the this .categoryHeader's vertical alignment by
+// setting styles in:
+// .usa-accordion__content > *:first-child
+// This first child of the accordion content is the category header:
+.categoryHeader {
+ display: flex;
+ justify-content: space-between;
+ text-transform: uppercase;
+ font-size: small;
+ @include u-bg('gray-cool-5');
+ @include u-padding-left(2.5);
+ @include u-padding-right(2);
+}
\ No newline at end of file
diff --git a/client/src/components/AreaDetail/areaDetail.module.scss.d.ts b/client/src/components/AreaDetail/areaDetail.module.scss.d.ts
index ad2fc967..03e161a5 100644
--- a/client/src/components/AreaDetail/areaDetail.module.scss.d.ts
+++ b/client/src/components/AreaDetail/areaDetail.module.scss.d.ts
@@ -11,6 +11,7 @@ declare namespace MapModuleScssNamespace {
versionInfo: string;
showThresholdExceed:string;
hideThresholdExceed:string;
+ categoryHeader:string;
}
}
diff --git a/client/src/components/AreaDetail/index.tsx b/client/src/components/AreaDetail/index.tsx
index b017b6bd..9a91c02d 100644
--- a/client/src/components/AreaDetail/index.tsx
+++ b/client/src/components/AreaDetail/index.tsx
@@ -18,11 +18,20 @@ interface IAreaDetailProps {
properties: constants.J40Properties,
}
+/**
+ * This interface is used as define the various fields for each indicator in the side panel
+ * label: the indicator label or title
+ * description: the description of the indicator used in the side panel
+ * value: the number from the geoJSON tile
+ * isDisadvagtaged: the flag from the geoJSON tile
+ * isPercent: is the value a percent or percentile
+ * */
export interface indicatorInfo {
label: string,
description: string,
value: number,
isDisadvagtaged: boolean,
+ isPercent?: boolean,
}
const AreaDetail = ({properties}:IAreaDetailProps) => {
@@ -45,84 +54,84 @@ const AreaDetail = ({properties}:IAreaDetailProps) => {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.EXP_AG_LOSS),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.EXP_AG_LOSS),
value: properties[constants.EXP_AGRICULTURE_LOSS_PERCENTILE] ?
- properties[constants.EXP_AGRICULTURE_LOSS_PERCENTILE] : null,
+ properties[constants.EXP_AGRICULTURE_LOSS_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_EXP_AGR_LOSS_AND_IS_LOW_INCOME] ?
- properties[constants.IS_GTE_90_EXP_AGR_LOSS_AND_IS_LOW_INCOME] : null,
+ properties[constants.IS_GTE_90_EXP_AGR_LOSS_AND_IS_LOW_INCOME] : null,
};
const expBldLoss:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.EXP_BLD_LOSS),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.EXP_BLD_LOSS),
value: properties[constants.EXP_BUILDING_LOSS_PERCENTILE] ?
- properties[constants.EXP_BUILDING_LOSS_PERCENTILE] : null,
+ properties[constants.EXP_BUILDING_LOSS_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_EXP_BLD_LOSS_AND_IS_LOW_INCOME] ?
- properties[constants.IS_GTE_90_EXP_BLD_LOSS_AND_IS_LOW_INCOME] : null,
+ properties[constants.IS_GTE_90_EXP_BLD_LOSS_AND_IS_LOW_INCOME] : null,
};
const expPopLoss:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.EXP_POP_LOSS),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.EXP_POP_LOSS),
value: properties[constants.EXP_POPULATION_LOSS_PERCENTILE] ?
- properties[constants.EXP_POPULATION_LOSS_PERCENTILE] : null,
+ properties[constants.EXP_POPULATION_LOSS_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_EXP_POP_LOSS_AND_IS_LOW_INCOME] ?
- properties[constants.IS_GTE_90_EXP_POP_LOSS_AND_IS_LOW_INCOME] : null,
+ properties[constants.IS_GTE_90_EXP_POP_LOSS_AND_IS_LOW_INCOME] : null,
};
const lowInc:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.LOW_INCOME),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.LOW_INCOME),
value: properties[constants.POVERTY_BELOW_200_PERCENTILE] ?
- properties[constants.POVERTY_BELOW_200_PERCENTILE] : null,
+ properties[constants.POVERTY_BELOW_200_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_FEDERAL_POVERTY_LEVEL_200] ?
- properties[constants.IS_FEDERAL_POVERTY_LEVEL_200] : null,
+ properties[constants.IS_FEDERAL_POVERTY_LEVEL_200] : null,
};
const energyBurden:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.ENERGY_BURDEN),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.ENERGY_BURDEN),
value: properties[constants.ENERGY_PERCENTILE] ?
- properties[constants.ENERGY_PERCENTILE] : null,
+ properties[constants.ENERGY_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_ENERGY_BURDEN_AND_IS_LOW_INCOME] ?
- properties[constants.IS_GTE_90_ENERGY_BURDEN_AND_IS_LOW_INCOME] : null,
+ properties[constants.IS_GTE_90_ENERGY_BURDEN_AND_IS_LOW_INCOME] : null,
};
const pm25:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.PM_2_5),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.PM_2_5),
value: properties[constants.PM25_PERCENTILE] ?
- properties[constants.PM25_PERCENTILE] : null,
+ properties[constants.PM25_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_PM25_AND_IS_LOW_INCOME] ?
- properties[constants.IS_GTE_90_PM25_AND_IS_LOW_INCOME] : null,
+ properties[constants.IS_GTE_90_PM25_AND_IS_LOW_INCOME] : null,
};
const dieselPartMatter:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.DIESEL_PARTICULATE_MATTER),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.DIESEL_PARTICULATE_MATTER),
value: properties[constants.DIESEL_MATTER_PERCENTILE] ?
- properties[constants.DIESEL_MATTER_PERCENTILE] : null,
+ properties[constants.DIESEL_MATTER_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_DIESEL_PM_AND_IS_LOW_INCOME] ?
- properties[constants.IS_GTE_90_DIESEL_PM_AND_IS_LOW_INCOME] : null,
+ properties[constants.IS_GTE_90_DIESEL_PM_AND_IS_LOW_INCOME] : null,
};
const trafficVolume:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.TRAFFIC_VOLUME),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.TRAFFIC_VOLUME),
value: properties[constants.TRAFFIC_PERCENTILE] ?
- properties[constants.TRAFFIC_PERCENTILE] : null,
+ properties[constants.TRAFFIC_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_TRAFFIC_PROX_AND_IS_LOW_INCOME] ?
- properties[constants.IS_GTE_90_TRAFFIC_PROX_AND_IS_LOW_INCOME] : null,
+ properties[constants.IS_GTE_90_TRAFFIC_PROX_AND_IS_LOW_INCOME] : null,
};
const houseBurden:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.HOUSE_BURDEN),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.HOUSE_BURDEN),
value: properties[constants.HOUSING_BURDEN_PROPERTY_PERCENTILE] ?
- properties[constants.HOUSING_BURDEN_PROPERTY_PERCENTILE] : null,
+ properties[constants.HOUSING_BURDEN_PROPERTY_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_HOUSE_BURDEN_AND_IS_LOW_INCOME] ?
- properties[constants.IS_GTE_90_HOUSE_BURDEN_AND_IS_LOW_INCOME] : null,
+ properties[constants.IS_GTE_90_HOUSE_BURDEN_AND_IS_LOW_INCOME] : null,
};
const leadPaint:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.LEAD_PAINT),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.LEAD_PAINT),
value: properties[constants.LEAD_PAINT_PERCENTILE] ?
- properties[constants.LEAD_PAINT_PERCENTILE] : null,
+ properties[constants.LEAD_PAINT_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_LEAD_PAINT_AND_MEDIAN_HOME_VAL_AND_IS_LOW_INCOME] ?
- properties[constants.IS_GTE_90_LEAD_PAINT_AND_MEDIAN_HOME_VAL_AND_IS_LOW_INCOME] : null,
+ properties[constants.IS_GTE_90_LEAD_PAINT_AND_MEDIAN_HOME_VAL_AND_IS_LOW_INCOME] : null,
};
// const medHomeVal:indicatorInfo = {
// label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.MED_HOME_VAL),
@@ -136,108 +145,109 @@ const AreaDetail = ({properties}:IAreaDetailProps) => {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.PROX_HAZ),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.PROX_HAZ),
value: properties[constants.PROXIMITY_TSDF_SITES_PERCENTILE] ?
- properties[constants.PROXIMITY_TSDF_SITES_PERCENTILE] : null,
+ properties[constants.PROXIMITY_TSDF_SITES_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_HAZARD_WASTE_AND_IS_LOW_INCOME] ?
- properties[constants.IS_GTE_90_HAZARD_WASTE_AND_IS_LOW_INCOME] : null,
+ properties[constants.IS_GTE_90_HAZARD_WASTE_AND_IS_LOW_INCOME] : null,
};
const proxNPL:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.PROX_NPL),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.PROX_NPL),
value: properties[constants.PROXIMITY_NPL_SITES_PERCENTILE] ?
- properties[constants.PROXIMITY_NPL_SITES_PERCENTILE] : null,
+ properties[constants.PROXIMITY_NPL_SITES_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_SUPERFUND_AND_IS_LOW_INCOME] ?
- properties[constants.IS_GTE_90_SUPERFUND_AND_IS_LOW_INCOME] : null,
+ properties[constants.IS_GTE_90_SUPERFUND_AND_IS_LOW_INCOME] : null,
};
const proxRMP:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.PROX_RMP),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.PROX_RMP),
value: properties[constants.PROXIMITY_RMP_SITES_PERCENTILE] ?
- properties[constants.PROXIMITY_RMP_SITES_PERCENTILE] : null,
+ properties[constants.PROXIMITY_RMP_SITES_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_RMP_AND_IS_LOW_INCOME] ?
- properties[constants.IS_GTE_90_RMP_AND_IS_LOW_INCOME] : null,
+ properties[constants.IS_GTE_90_RMP_AND_IS_LOW_INCOME] : null,
};
const wasteWater:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.WASTE_WATER),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.WASTE_WATER),
value: properties[constants.WASTEWATER_PERCENTILE] ?
- properties[constants.WASTEWATER_PERCENTILE] : null,
+ properties[constants.WASTEWATER_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_WASTEWATER_AND_IS_LOW_INCOME] ?
- properties[constants.IS_GTE_90_WASTEWATER_AND_IS_LOW_INCOME] : null,
+ properties[constants.IS_GTE_90_WASTEWATER_AND_IS_LOW_INCOME] : null,
};
const asthma:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.ASTHMA),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.ASTHMA),
value: properties[constants.ASTHMA_PERCENTILE] ?
- properties[constants.ASTHMA_PERCENTILE] : null,
+ properties[constants.ASTHMA_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_ASTHMA_AND_IS_LOW_INCOME] ?
- properties[constants.IS_GTE_90_ASTHMA_AND_IS_LOW_INCOME] : null,
+ properties[constants.IS_GTE_90_ASTHMA_AND_IS_LOW_INCOME] : null,
};
const diabetes:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.DIABETES),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.DIABETES),
value: properties[constants.DIABETES_PERCENTILE] ?
- properties[constants.DIABETES_PERCENTILE] : null,
+ properties[constants.DIABETES_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_DIABETES_AND_IS_LOW_INCOME] ?
- properties[constants.IS_GTE_90_DIABETES_AND_IS_LOW_INCOME] : null,
+ properties[constants.IS_GTE_90_DIABETES_AND_IS_LOW_INCOME] : null,
};
const heartDisease:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.HEART_DISEASE),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.HEART_DISEASE),
value: properties[constants.HEART_PERCENTILE] ?
- properties[constants.HEART_PERCENTILE] : null,
+ properties[constants.HEART_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_HEART_DISEASE_AND_IS_LOW_INCOME] ?
- properties[constants.IS_GTE_90_HEART_DISEASE_AND_IS_LOW_INCOME] : null,
+ properties[constants.IS_GTE_90_HEART_DISEASE_AND_IS_LOW_INCOME] : null,
};
const lifeExpect:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.LIFE_EXPECT),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.LOW_LIFE_EXPECT),
value: properties[constants.LIFE_PERCENTILE] ?
- properties[constants.LIFE_PERCENTILE] : null,
+ properties[constants.LIFE_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_LOW_LIFE_EXP_AND_IS_LOW_INCOME] ?
- properties[constants.IS_GTE_90_LOW_LIFE_EXP_AND_IS_LOW_INCOME] : null,
+ properties[constants.IS_GTE_90_LOW_LIFE_EXP_AND_IS_LOW_INCOME] : null,
};
const lowMedInc:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.LOW_MED_INC),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.LOW_MED_INCOME),
value: properties[constants.LOW_MEDIAN_INCOME_PERCENTILE] ?
- properties[constants.LOW_MEDIAN_INCOME_PERCENTILE] : null,
+ properties[constants.LOW_MEDIAN_INCOME_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_LOW_MEDIAN_INCOME_AND_LOW_HIGH_SCHOOL_EDU] ?
- properties[constants.IS_GTE_90_LOW_MEDIAN_INCOME_AND_LOW_HIGH_SCHOOL_EDU] : null,
+ properties[constants.IS_GTE_90_LOW_MEDIAN_INCOME_AND_LOW_HIGH_SCHOOL_EDU] : null,
};
const lingIso:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.LING_ISO),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.LING_ISO),
value: properties[constants.LINGUISTIC_ISOLATION_PROPERTY_PERCENTILE] ?
- properties[constants.LINGUISTIC_ISOLATION_PROPERTY_PERCENTILE] : null,
+ properties[constants.LINGUISTIC_ISOLATION_PROPERTY_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_LINGUISITIC_ISO_AND_IS_LOW_INCOME] ?
- properties[constants.IS_GTE_90_LINGUISITIC_ISO_AND_IS_LOW_INCOME] : null,
+ properties[constants.IS_GTE_90_LINGUISITIC_ISO_AND_IS_LOW_INCOME] : null,
};
const unemploy:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.UNEMPLOY),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.UNEMPLOY),
value: properties[constants.UNEMPLOYMENT_PROPERTY_PERCENTILE] ?
- properties[constants.UNEMPLOYMENT_PROPERTY_PERCENTILE] : null,
+ properties[constants.UNEMPLOYMENT_PROPERTY_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_UNEMPLOYMENT_AND_LOW_HIGH_SCHOOL_EDU] ?
- properties[constants.IS_GTE_90_UNEMPLOYMENT_AND_LOW_HIGH_SCHOOL_EDU] : null,
+ properties[constants.IS_GTE_90_UNEMPLOYMENT_AND_LOW_HIGH_SCHOOL_EDU] : null,
};
const poverty:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.POVERTY),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.POVERTY),
value: properties[constants.POVERTY_PROPERTY_PERCENTILE] ?
- properties[constants.POVERTY_PROPERTY_PERCENTILE] : null,
+ properties[constants.POVERTY_PROPERTY_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_BELOW_100_POVERTY_AND_LOW_HIGH_SCHOOL_EDU] ?
- properties[constants.IS_GTE_90_BELOW_100_POVERTY_AND_LOW_HIGH_SCHOOL_EDU] : null,
+ properties[constants.IS_GTE_90_BELOW_100_POVERTY_AND_LOW_HIGH_SCHOOL_EDU] : null,
};
const highSchool:indicatorInfo = {
label: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATORS.HIGH_SCL),
description: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_INDICATOR_DESCRIPTION.HIGH_SKL),
value: properties[constants.HIGH_SCHOOL_PROPERTY_PERCENTILE] ?
- properties[constants.HIGH_SCHOOL_PROPERTY_PERCENTILE] : null,
+ properties[constants.HIGH_SCHOOL_PROPERTY_PERCENTILE] : null,
isDisadvagtaged: properties[constants.IS_GTE_90_UNEMPLOYMENT_AND_LOW_HIGH_SCHOOL_EDU] ?
- properties[constants.IS_GTE_90_UNEMPLOYMENT_AND_LOW_HIGH_SCHOOL_EDU] : null,
+ properties[constants.IS_GTE_90_UNEMPLOYMENT_AND_LOW_HIGH_SCHOOL_EDU] : null,
+ isPercent: true,
};
// Aggregate indicators based on categories
@@ -247,56 +257,56 @@ const AreaDetail = ({properties}:IAreaDetailProps) => {
titleText: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_CATEGORY.CLIMATE),
indicators: [expAgLoss, expBldLoss, expPopLoss, lowInc],
isDisadvagtaged: properties[constants.IS_CLIMATE_FACTOR_DISADVANTAGED_L] ?
- properties[constants.IS_CLIMATE_FACTOR_DISADVANTAGED_L] : null,
+ properties[constants.IS_CLIMATE_FACTOR_DISADVANTAGED_L] : null,
},
{
id: 'clean-energy',
titleText: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_CATEGORY.CLEAN_ENERGY),
indicators: [energyBurden, pm25, lowInc],
isDisadvagtaged: properties[constants.IS_ENERGY_FACTOR_DISADVANTAGED_L] ?
- properties[constants.IS_ENERGY_FACTOR_DISADVANTAGED_L] : null,
+ properties[constants.IS_ENERGY_FACTOR_DISADVANTAGED_L] : null,
},
{
id: 'clean-transport',
titleText: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_CATEGORY.CLEAN_TRANSPORT),
indicators: [dieselPartMatter, trafficVolume, lowInc],
isDisadvagtaged: properties[constants.IS_TRANSPORT_FACTOR_DISADVANTAGED_L] ?
- properties[constants.IS_TRANSPORT_FACTOR_DISADVANTAGED_L] : null,
+ properties[constants.IS_TRANSPORT_FACTOR_DISADVANTAGED_L] : null,
},
{
id: 'sustain-house',
titleText: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_CATEGORY.SUSTAIN_HOUSE),
indicators: [houseBurden, leadPaint, lowInc],
isDisadvagtaged: properties[constants.IS_HOUSING_FACTOR_DISADVANTAGED_L] ?
- properties[constants.IS_HOUSING_FACTOR_DISADVANTAGED_L] : null,
+ properties[constants.IS_HOUSING_FACTOR_DISADVANTAGED_L] : null,
},
{
id: 'leg-pollute',
titleText: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_CATEGORY.LEG_POLLUTE),
indicators: [proxHaz, proxNPL, proxRMP, lowInc],
isDisadvagtaged: properties[constants.IS_POLLUTION_FACTOR_DISADVANTAGED_L] ?
- properties[constants.IS_POLLUTION_FACTOR_DISADVANTAGED_L] : null,
+ properties[constants.IS_POLLUTION_FACTOR_DISADVANTAGED_L] : null,
},
{
id: 'clean-water',
titleText: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_CATEGORY.CLEAN_WATER),
indicators: [wasteWater, lowInc],
isDisadvagtaged: properties[constants.IS_WATER_FACTOR_DISADVANTAGED_L] ?
- properties[constants.IS_WATER_FACTOR_DISADVANTAGED_L] : null,
+ properties[constants.IS_WATER_FACTOR_DISADVANTAGED_L] : null,
},
{
id: 'health-burdens',
titleText: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_CATEGORY.HEALTH_BURDEN),
indicators: [asthma, diabetes, heartDisease, lifeExpect, lowInc],
isDisadvagtaged: properties[constants.IS_HEALTH_FACTOR_DISADVANTAGED_L] ?
- properties[constants.IS_HEALTH_FACTOR_DISADVANTAGED_L] : null,
+ properties[constants.IS_HEALTH_FACTOR_DISADVANTAGED_L] : null,
},
{
id: 'work-dev',
titleText: intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_CATEGORY.WORK_DEV),
indicators: [lowMedInc, lingIso, unemploy, poverty, highSchool],
isDisadvagtaged: properties[constants.IS_WORKFORCE_FACTOR_DISADVANTAGED_L] ?
- properties[constants.IS_WORKFORCE_FACTOR_DISADVANTAGED_L] : null,
+ properties[constants.IS_WORKFORCE_FACTOR_DISADVANTAGED_L] : null,
},
];
@@ -308,6 +318,13 @@ const AreaDetail = ({properties}:IAreaDetailProps) => {
title: ,
content: (
<>
+ {/* Category Header */}
+
+
{intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_CATEGORY.INDICATOR)}
+
{intl.formatMessage(EXPLORE_COPY.SIDE_PANEL_CATEGORY.PERCENTILE)}
+
+
+ {/* Category Indicators */}
{category.indicators.map((indicator:any, index:number) => {
return ;
})}
diff --git a/client/src/components/AreaDetail/tests/__snapshots__/areaDetail.test.tsx.snap b/client/src/components/AreaDetail/tests/__snapshots__/areaDetail.test.tsx.snap
index 40d5ac81..55b7b218 100644
--- a/client/src/components/AreaDetail/tests/__snapshots__/areaDetail.test.tsx.snap
+++ b/client/src/components/AreaDetail/tests/__snapshots__/areaDetail.test.tsx.snap
@@ -87,6 +87,14 @@ exports[`rendering of the AreaDetail checks if various text fields are visible 1
hidden=""
id="climate-change"
>
+
+
+ Indicator
+
+
+ Percentile (0-100)
+
+
@@ -190,6 +198,14 @@ exports[`rendering of the AreaDetail checks if various text fields are visible 1
hidden=""
id="clean-energy"
>
+
+
+ Indicator
+
+
+ Percentile (0-100)
+
+
@@ -273,6 +289,14 @@ exports[`rendering of the AreaDetail checks if various text fields are visible 1
hidden=""
id="clean-transport"
>
+
+
+ Indicator
+
+
+ Percentile (0-100)
+
+
@@ -356,6 +380,14 @@ exports[`rendering of the AreaDetail checks if various text fields are visible 1
hidden=""
id="sustain-house"
>
+
+
+ Indicator
+
+
+ Percentile (0-100)
+
+
@@ -441,6 +473,14 @@ exports[`rendering of the AreaDetail checks if various text fields are visible 1
hidden=""
id="leg-pollute"
>
+
+
+ Indicator
+
+
+ Percentile (0-100)
+
+
@@ -542,6 +582,14 @@ exports[`rendering of the AreaDetail checks if various text fields are visible 1
hidden=""
id="clean-water"
>
+
+
+ Indicator
+
+
+ Percentile (0-100)
+
+
@@ -607,6 +655,14 @@ exports[`rendering of the AreaDetail checks if various text fields are visible 1
hidden=""
id="health-burdens"
>
+
+
+ Indicator
+
+
+ Percentile (0-100)
+
+
@@ -731,6 +787,14 @@ exports[`rendering of the AreaDetail checks if various text fields are visible 1
hidden=""
id="work-dev"
>
+
+
+ Indicator
+
+
+ Percentile (0-100)
+
+
@@ -822,15 +886,16 @@ exports[`rendering of the AreaDetail checks if various text fields are visible 1
High school degree achievement rate
- Percent of people ages 25 years or older whose education level is less than a high school diploma
+ Percent (not a percentile) of people ages 25 years or older whose education level is less than a
+ high school diploma
N/A
-
-
-
+
+ %
+
diff --git a/client/src/components/BetaBanner/__snapshots__/BetaBanner.test.tsx.snap b/client/src/components/BetaBanner/__snapshots__/BetaBanner.test.tsx.snap
index a688be38..cbc3f79f 100644
--- a/client/src/components/BetaBanner/__snapshots__/BetaBanner.test.tsx.snap
+++ b/client/src/components/BetaBanner/__snapshots__/BetaBanner.test.tsx.snap
@@ -10,8 +10,8 @@ exports[`rendering of the BetaBanner checks if component renders 1`] = `
This is a beta site.
- It is an early, in-progress version of the tool with limited datasets
- that will be continuously updated.
+ It is an early, in-progress version of the tool with limited datasets that will
+ be regularly updated.
diff --git a/client/src/components/Categories/__snapshots__/Categories.test.tsx.snap b/client/src/components/Categories/__snapshots__/Categories.test.tsx.snap
index 50a9511f..167f5aa7 100644
--- a/client/src/components/Categories/__snapshots__/Categories.test.tsx.snap
+++ b/client/src/components/Categories/__snapshots__/Categories.test.tsx.snap
@@ -176,7 +176,7 @@ exports[`rendering of the Categories checks if component renders 1`] = `
low median home value
is at or less than
- 90th percentile OR at or above the 10th percentile for the
+ 90th percentile OR at or above the 90th percentile for the
diff --git a/client/src/components/DatasetCard/index.tsx b/client/src/components/DatasetCard/index.tsx
index f20223a5..52c14dfa 100644
--- a/client/src/components/DatasetCard/index.tsx
+++ b/client/src/components/DatasetCard/index.tsx
@@ -25,9 +25,7 @@ const DatasetCard = ({datasetCardProps}:IDatasetCardProps) => {
{intl.formatMessage(METHODOLOGY_COPY.DATASET_CARD_LABELS.RESP_PARTY)}
-
- {datasetCardProps.respPartyLabel}
-
+ {datasetCardProps.responsibleParty}
diff --git a/client/src/components/DatasetCard/tests/__snapshots__/datasetCard.test.tsx.snap b/client/src/components/DatasetCard/tests/__snapshots__/datasetCard.test.tsx.snap
index 2006c358..cc95578d 100644
--- a/client/src/components/DatasetCard/tests/__snapshots__/datasetCard.test.tsx.snap
+++ b/client/src/components/DatasetCard/tests/__snapshots__/datasetCard.test.tsx.snap
@@ -10,9 +10,9 @@ exports[`rendering of indicator dataset card checks if component renders 1`] = `
- Percent of a block group's population in households where household income is at or below
- 200% of the federal poverty level.
-
+ Percent of a census tract's population in households where household income is at or below
+ 200% of the federal poverty level.
+
,
+ }}
+ />,
+ usedIn: CATEGORIES.HEALTH_BURDENS.METHODOLOGY,
+ responsibleParty: RESPONSIBLE_PARTIES.CDC_SLEEP,
+ dateRange: DATE_RANGE.TEN_PLUS_5,
},
{
domID: 'low-med-inc',
indicator: 'Low median Income',
- description: `Median income of the census tract calculated as a percent of the area’s median income.`,
- usedIn: `Training and workforce development`,
- respPartyLabel: `Census's American Community Survey`,
- dataSourceURL: `https://www.census.gov/programs-surveys/acs`,
- dateRange: `2015-2019`,
+ description: ,
+ usedIn: CATEGORIES.WORKFORCE_DEV.METHODOLOGY,
+ responsibleParty: RESPONSIBLE_PARTIES.CENSUS_ACS,
+ dateRange: DATE_RANGE.FIFETEEN_PLUS_4,
},
{
domID: 'ling-iso',
indicator: 'Linguistic Isolation',
- description: `
- The percent of limited speaking households, which are households where no one over age 14 speaks English well.
- `,
- usedIn: `Training and workforce development`,
- respPartyLabel: `Census's American Community Survey`,
- dataSourceURL: `https://www.census.gov/programs-surveys/acs`,
- dateRange: `2015-2019`,
+ description: ,
+ usedIn: CATEGORIES.WORKFORCE_DEV.METHODOLOGY,
+ responsibleParty: RESPONSIBLE_PARTIES.CENSUS_ACS,
+ dateRange: DATE_RANGE.FIFETEEN_PLUS_4,
},
{
domID: 'unemploy',
indicator: 'Unemployment',
- description: `Number of unemployed people as a percentage of the civilian labor force`,
- usedIn: `Training and workforce development`,
- respPartyLabel: `Census's American Community Survey`,
- dataSourceURL: `https://www.census.gov/programs-surveys/acs`,
- dateRange: `2015-2019`,
+ description: ,
+ usedIn: CATEGORIES.WORKFORCE_DEV.METHODOLOGY,
+ responsibleParty: RESPONSIBLE_PARTIES.CENSUS_ACS,
+ dateRange: DATE_RANGE.FIFETEEN_PLUS_4,
},
{
domID: 'poverty',
indicator: 'Poverty',
- description: `Percent of a tract's population in households where the household income` +
- ` is at or below 100% of the federal poverty level.`,
- usedIn: `Training and workforce development`,
- respPartyLabel: `Census's American Community Survey`,
- dataSourceURL: `https://www.census.gov/programs-surveys/acs`,
- dateRange: `2015-2019`,
+ description: ,
+ usedIn: CATEGORIES.WORKFORCE_DEV.METHODOLOGY,
+ responsibleParty: RESPONSIBLE_PARTIES.CENSUS_ACS,
+ dateRange: DATE_RANGE.FIFETEEN_PLUS_4,
},
{
domID: 'high-school',
indicator: 'High school degree achievement rate',
- description: `Percent of people ages 25 years or older in a census tract whose
- education level is less than a high school diploma.`,
- usedIn: `Training and workforce development`,
- respPartyLabel: `Census's American Community Survey`,
- dataSourceURL: `https://www.census.gov/programs-surveys/acs`,
- dateRange: `2015-2019`,
+ description: ,
+ usedIn: CATEGORIES.WORKFORCE_DEV.METHODOLOGY,
+ responsibleParty: RESPONSIBLE_PARTIES.CENSUS_ACS,
+ dateRange: DATE_RANGE.FIFETEEN_PLUS_4,
+ isPercent: true,
},
];
diff --git a/client/src/pages/__snapshots__/contact.test.tsx.snap b/client/src/pages/__snapshots__/contact.test.tsx.snap
index ca06f9bb..0150d5ea 100644
--- a/client/src/pages/__snapshots__/contact.test.tsx.snap
+++ b/client/src/pages/__snapshots__/contact.test.tsx.snap
@@ -150,8 +150,8 @@ exports[`rendering of the DatasetContainer checks if various text fields are vis
This is a beta site.
- It is an early, in-progress version of the tool with limited datasets
- that will be continuously updated.
+ It is an early, in-progress version of the tool with limited datasets that will
+ be regularly updated.
diff --git a/client/src/pages/__snapshots__/index.test.tsx.snap b/client/src/pages/__snapshots__/index.test.tsx.snap
index b1fc8bf6..1e1368c7 100644
--- a/client/src/pages/__snapshots__/index.test.tsx.snap
+++ b/client/src/pages/__snapshots__/index.test.tsx.snap
@@ -150,8 +150,8 @@ exports[`rendering of the DatasetContainer checks if various text fields are vis
This is a beta site.
- It is an early, in-progress version of the tool with limited datasets
- that will be continuously updated.
+ It is an early, in-progress version of the tool with limited datasets that will
+ be regularly updated.
diff --git a/client/src/pages/__snapshots__/methodology.test.tsx.snap b/client/src/pages/__snapshots__/methodology.test.tsx.snap
index 44fb8798..5a098225 100644
--- a/client/src/pages/__snapshots__/methodology.test.tsx.snap
+++ b/client/src/pages/__snapshots__/methodology.test.tsx.snap
@@ -150,8 +150,8 @@ exports[`rendering of the DatasetContainer checks if various text fields are vis
This is a beta site.
- It is an early, in-progress version of the tool with limited datasets
- that will be continuously updated.
+ It is an early, in-progress version of the tool with limited datasets that will
+ be regularly updated.
@@ -575,7 +575,7 @@ exports[`rendering of the DatasetContainer checks if various text fields are vis
low median home value
is at or less than
- 90th percentile OR at or above the 10th percentile for the
+ 90th percentile OR at or above the 90th percentile for the
@@ -836,9 +836,9 @@ exports[`rendering of the DatasetContainer checks if various text fields are vis
- Percent of a block group's population in households where household income is at or below
- 200% of the federal poverty level.
-
+ Percent of a census tract's population in households where household income is at or below
+ 200% of the federal poverty level.
+
-
@@ -850,7 +850,7 @@ exports[`rendering of the DatasetContainer checks if various text fields are vis
rel="noreferrer"
target="_blank"
>
- Census's American Community Survey.
+ Census's American Community Survey
@@ -875,12 +875,12 @@ exports[`rendering of the DatasetContainer checks if various text fields are vis
- Percent of agriculture value at risk from losses due to natural hazards. Calculated by dividing
- the agriculture value at risk in a census tract by the total agriculture value in that census
- tract. Fourteen natural hazards that have some link to climate change include: avalanche,
- coastal flooding, cold wave, drought, hail, heat wave, hurricane, ice storm, landslide,
- riverine flooding, strong wind, tornado, wildfire, and winter weather.
-
+ Percent of agriculture value at risk from losses due to natural hazards. Calculated by dividing
+ the agriculture value at risk in a census tract by the total agriculture value in that census
+ tract. Fourteen natural hazards that have some link to climate change include: avalanche,
+ coastal flooding, cold wave, drought, hail, heat wave, hurricane, ice storm, landslide,
+ riverine flooding, strong wind, tornado, wildfire, and winter weather.
+
-
@@ -917,12 +917,12 @@ exports[`rendering of the DatasetContainer checks if various text fields are vis
- Percent of building value at risk from losses due to natural hazards. Calculated by dividing the
- building value at risk in a census tract by the total building value in that census tract.
- Fourteen natural hazards that have some link to climate change include: avalanche, coastal flooding,
- cold wave, drought, hail, heat wave, hurricane, ice storm, landslide, riverine flooding, strong
- wind, tornado, wildfire, and winter weather.
-
+ Percent of building value at risk from losses due to natural hazards. Calculated by dividing the
+ building value at risk in a census tract by the total building value in that census tract.
+ Fourteen natural hazards that have some link to climate change include: avalanche,
+ coastal flooding, cold wave, drought, hail, heat wave, hurricane, ice storm, landslide,
+ riverine flooding, strong wind, tornado, wildfire, and winter weather.
+
-
@@ -959,19 +959,18 @@ exports[`rendering of the DatasetContainer checks if various text fields are vis
- Rate relative to the population in fatalities and injuries due to natural hazards each year.
- Fourteen natural hazards that have some link to climate change include: avalanche, coastal
- flooding, cold wave, drought, hail, heat wave, hurricane, ice storm, landslide, riverine
- flooding, strong wind, tornado, wildfire, and winter weather.
- Population loss is defined as the Spatial Hazard Events and Losses or National Centers
- for Environmental Information’s (NCEI) reported number of fatalities and injuries caused by the
- hazard occurrence. To combine fatalities and injuries for the computation of population loss value,
- an injury is counted as one-tenth (1/10) of a fatality. The NCEI Storm Events Database
- classifies injuries and fatalities as direct or indirect. Both direct and indirect injuries
- and fatalities are counted as population loss. This total number of injuries and fatalities
- is then divided by the population in the census tract to get a per-capita rate of population risk.
-
-
+ Rate relative to the population in fatalities and injuries due to natural hazards each year.
+ Fourteen natural hazards that have some link to climate change include: avalanche, coastal
+ flooding, cold wave, drought, hail, heat wave, hurricane, ice storm, landslide, riverine
+ flooding, strong wind, tornado, wildfire, and winter weather.
+ Population loss is defined as the Spatial Hazard Events and Losses or National Centers
+ for Environmental Information’s (NCEI) reported number of fatalities and injuries caused by the
+ hazard occurrence. To combine fatalities and injuries for the computation of population loss value,
+ an injury is counted as one-tenth (1/10) of a fatality. The NCEI Storm Events Database
+ classifies injuries and fatalities as direct or indirect. Both direct and indirect injuries
+ and fatalities are counted as population loss. This total number of injuries and fatalities
+ is then divided by the population in the census tract to get a per-capita rate of population risk.
+