publicdata/j40-cejst-2
1
0
Fork 0
mirror of https://github.com/DOI-DO/j40-cejst-2.git synced 2025-02-23 01:54:18 -08:00
Code Issues Projects Releases Packages Wiki Activity Actions

Put the pdf back in the package and add TSD to pipeline (#1580)

Browse source 
* Put the pdf back in the package and add TSD to pipeline

* updated pdf with logo

* wrong path
This commit is contained in:
Jorge Escobar 2022-04-21 13:42:04 -04:00 • committed by GitHub
parent c07d790b77
commit fbd56e3bd5
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
6 changed files with 2837 additions and 6735 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
.github/workflows/deploy_be_staging.yml vendored
View file

@ -55,6 +55,8 @@ jobs:
run: | run: |
poetry run s4cmd put ./data_pipeline/data/score/csv/ s3://justice40-data/data-pipeline-staging/${{env.PR_NUMBER}}/${{env.SHA_NUMBER}}/data/score/csv --recursive --force --API-ACL=public-read poetry run s4cmd put ./data_pipeline/data/score/csv/ s3://justice40-data/data-pipeline-staging/${{env.PR_NUMBER}}/${{env.SHA_NUMBER}}/data/score/csv --recursive --force --API-ACL=public-read
poetry run s4cmd put ./data_pipeline/data/score/downloadable/ s3://justice40-data/data-pipeline-staging/${{env.PR_NUMBER}}/${{env.SHA_NUMBER}}/data/score/downloadable --recursive --force --API-ACL=public-read poetry run s4cmd put ./data_pipeline/data/score/downloadable/ s3://justice40-data/data-pipeline-staging/${{env.PR_NUMBER}}/${{env.SHA_NUMBER}}/data/score/downloadable --recursive --force --API-ACL=public-read
poetry run s4cmd put ./data_pipeline/files/ s3://justice40-data/data-pipeline-staging/${{env.PR_NUMBER}}/${{env.SHA_NUMBER}}/data/score/downloadable --recursive --force --API-ACL=public-read
- name: Update PR with deployed Score URLs - name: Update PR with deployed Score URLs
uses: mshick/add-pr-comment@v1 uses: mshick/add-pr-comment@v1
with: with:

1
.github/workflows/generate-score.yml vendored
View file

@ -57,3 +57,4 @@ jobs:
run: | run: |
aws s3 sync ./data_pipeline/data/score/csv/ s3://justice40-data/data-pipeline/data/score/csv --acl public-read --delete aws s3 sync ./data_pipeline/data/score/csv/ s3://justice40-data/data-pipeline/data/score/csv --acl public-read --delete
aws s3 sync ./data_pipeline/data/score/downloadable/ s3://justice40-data/data-pipeline/data/score/downloadable --acl public-read --delete aws s3 sync ./data_pipeline/data/score/downloadable/ s3://justice40-data/data-pipeline/data/score/downloadable --acl public-read --delete
aws s3 sync ./data_pipeline/files/ s3://justice40-data/data-pipeline/data/score/downloadable --acl public-read --delete

4188
data/data-pipeline/Draft_Communities_List.pdf
View file

File diff suppressed because one or more lines are too long

6
data/data-pipeline/data_pipeline/etl/score/etl_score_post.py
View file

@ -528,8 +528,7 @@ class PostScoreETL(ExtractTransformLoad):
excel_path = constants.SCORE_DOWNLOADABLE_EXCEL_FILE_PATH excel_path = constants.SCORE_DOWNLOADABLE_EXCEL_FILE_PATH
codebook_path = constants.SCORE_DOWNLOADABLE_CODEBOOK_FILE_PATH codebook_path = constants.SCORE_DOWNLOADABLE_CODEBOOK_FILE_PATH
zip_path = constants.SCORE_DOWNLOADABLE_ZIP_FILE_PATH zip_path = constants.SCORE_DOWNLOADABLE_ZIP_FILE_PATH
# TODO: reinstate when PDF is added back pdf_path = constants.SCORE_DOWNLOADABLE_PDF_FILE_PATH
# pdf_path = constants.SCORE_DOWNLOADABLE_PDF_FILE_PATH
logger.info("Writing downloadable excel") logger.info("Writing downloadable excel")
excel_config = self._load_excel_from_df( excel_config = self._load_excel_from_df(
@ -583,7 +582,8 @@ class PostScoreETL(ExtractTransformLoad):
csv_path, csv_path,
excel_path, excel_path,
codebook_path, codebook_path,
] # add pdf_path here to include PDF pdf_path,
]
zip_files(zip_path, files_to_compress) zip_files(zip_path, files_to_compress)
def load(self) -> None: def load(self) -> None:

5375
data/data-pipeline/data_pipeline/files/Draft_Communities_List.pdf
View file

File diff suppressed because one or more lines are too long

BIN
data/data-pipeline/data_pipeline/files/cejst_technical_support_document.pdf Normal file
View file

Binary file not shown.