j40-cejst-2/data/data-pipeline/data_pipeline/etl/sources
Jorge Escobar 7b05ee9c76
S3 Parallel Upload and Deletions (#1410)
* installation step

* trigger action

* installing to home dir

* dry-run

* pyenv

* py 2.8

* trying s4cmd

* removing pyenv

* poetry s4cmd

* num-threads

* public read

* poetry cache

* s4cmd all around

* poetry cache

* poetry cache

* install poetry packages

* poetry echo

* let's do this

* s4cmd install on run

* s4cmd

* ad aws back

* add aws back

* testing census api key and poetry caching

* census api key

* census api

* census api key #3

* 250

* poetry update

* poetry change

* check census api key

* force flag

* update score gen and tilefy; remove cached fips

* small gdal update

* invalidation

* missing cache ids
2022-03-17 23:19:23 -04:00
..
calenviroscreen Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
cdc_life_expectancy Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
cdc_places Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
cdc_svi_index Issue 1141: Definition M (#1151) 2022-01-18 14:56:55 -05:00
census S3 Parallel Upload and Deletions (#1410) 2022-03-17 23:19:23 -04:00
census_acs S3 Parallel Upload and Deletions (#1410) 2022-03-17 23:19:23 -04:00
census_acs_2010 Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
census_acs_median_income Cleaning up quick code (#1349) 2022-03-02 16:50:04 -05:00
census_decennial Issue 1075: Add refactored ETL tests to NRI (#1088) 2022-02-08 19:05:32 -05:00
child_opportunity_index Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
doe_energy_burden Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
ejscreen Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
ejscreen_areas_of_concern Issue 838: Update comparison tool to use tracts (#934) 2021-11-30 18:46:29 -05:00
energy_definition_alternative_draft Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
epa_rsei Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
geocorr Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
housing_and_transportation Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
hud_housing Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
hud_recap Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
mapping_for_ej Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
mapping_inequality Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
maryland_ejscreen Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
michigan_ejscreen Add Michigan EJ Screen into data-pipeline's ETL and provide automated scoring and statistics outputs (#1091) 2021-12-31 15:38:52 -05:00
national_risk_index FEMA data check (#1270) 2022-02-17 16:53:04 -05:00
persistent_poverty Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
tree_equity_score Run ETL processes in parallel (#1253) 2022-02-11 14:04:53 -05:00
__init__.py Data directory should adopt standard Poetry-suggested python package structure (#457) 2021-08-05 15:35:54 -04:00