publicdata/j40-cejst-2
2
0
Fork 0
mirror of https://github.com/DOI-DO/j40-cejst-2.git synced 2025-09-30 05:53:17 -07:00
Code Issues Projects Releases Packages Wiki Activity Actions

Data sources from S3 (#769)

Browse source 
* Started 535

* Data sources from S3

* lint

* renove breakpoints

* PR comments

* lint

* census data completed

* lint

* renaming data source
This commit is contained in:
Jorge Escobar 2021-10-13 16:00:33 -04:00 • committed by GitHub
commit 3b04356fb3
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
10 changed files with 317 additions and 67 deletions
Download patch file Download diff file Expand all files Collapse all files

9
data/data-pipeline/data_pipeline/etl/runner.py
View file

@ -104,18 +104,21 @@ def score_post() -> None:
score_post.cleanup()
def score_geo() -> None:
def score_geo(data_source: str = "local") -> None:
"""Generates the geojson files with score data baked in
Args:
None
census_data_source (str): Source for the census data (optional)
Options:
- local (default): fetch census data from the local data directory
- aws: fetch census from AWS S3 J40 data repository
Returns:
None
"""
# Score Geo
score_geo = GeoScoreETL()
score_geo = GeoScoreETL(data_source=data_source)
score_geo.extract()
score_geo.transform()
score_geo.load()