Add diagrams detailing our geo data pipeline (#170)

* Detailed flow for data inputs, census data, score creation, and geo
data generation (geoJSON and MVT)
* Include diagram showing vision for geoplatform ownership

Co-authored-by: Shelby Switzer <shelby.switzer@cms.hhs.gov>
This commit is contained in:
Shelby Switzer 2021-06-18 10:21:23 -04:00 committed by GitHub
parent c8a7f81f7c
commit b61d971f15
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
5 changed files with 59 additions and 0 deletions

View file

@ -4,6 +4,14 @@ The below is a general architecture of our proposed system:
![Architecture](architecture-mmd.svg)
The following is a more detailed diagram of the geo data pipeline architecture (the Data Pipeline and Server boxes in the general architecture diagram above).
![Geo Data Pipeline](geodata-pipeline-arch-mmd.svg)
We are partnering with Geoplatform to turn some of these pieces into open source shared services that they would own. The following is a modified diagram showing which pieces would tentatively be owned by Geoplatform.
![Geo Data Pipeline](geodata-pipeline-arch-geoplatform-mmd.svg)
## Updating the Diagram
**Note: Do Not directly modify the svg file, it is generated automatically!**

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 35 KiB

View file

@ -0,0 +1,28 @@
graph TD
Dataset1["Dataset 1"]-->Score
Dataset2["Dataset 2"]-->Score
Census["Census TIGER Data"]-->CGTiger
subgraph "Owned by Geoplatform"
CGTiger["Create GeoJSON from Shapefile with osgeo/gdal"]-->TS3
end
TS3("TIGER GeoJSON (S3)")-->CGJ
Score["Create Score CSV"]--Event Notification-->CSV
CSV("CSV (S3)")--"Event Notification (Geoplatform)"-->CGJ
subgraph "Owned by Geoplatform"
CGJ["Combine (ogr2ogr)+ Create GeoJSON"]--Event Notification-->GeoJSON
end
GeoJSON("GeoJSON (S3)")-->Tip
GeoJSON--"Access non-geo data"-->Client
subgraph "Owned by Geoplatform"
Tip[/Tippecanoe/]-->CreateMVT["Create and Send MVT"]
end
subgraph production
CreateMVT-->MBTiles
MBTiles-->Uncompressed("Uncompressed MVT (Geoplatform S3)")
end
subgraph development
CreateMVT-->Compressed("MBTiles (GeoPlatform S3)")-->TS[/Tileserver-GL/]
end
TS--"XYZ URL"-->Client
Uncompressed--"XYZ URL"-->Client["Gatsby+OpenLayers Client"]

Binary file not shown.

After

Width:  |  Height:  |  Size: 69 KiB

View file

@ -0,0 +1,22 @@
graph TD
Dataset1["Dataset 1"]-->Score
Dataset2["Dataset 2"]-->Score
Census["Census TIGER Data"]-->CGTiger
CGTiger["Create GeoJSON from Shapefile with osgeo/gdal"]-->TS3
TS3("TIGER GeoJSON (S3)")-->CGJ
Score["Create Score CSV"]-->CSV
CSV("CSV (S3)")-->CGJ
CGJ["Combine (ogr2ogr)+ Create GeoJSON"]-->GeoJSON
GeoJSON("GeoJSON (S3)")-->Tip
GeoJSON--"Access non-geo data"-->Client
subgraph "Generate MVT"
Tip[/Tippecanoe/]-->CreateMVT["Create and Send MVT"]
end
subgraph production
CreateMVT-->Uncompressed("Uncompressed MVT (S3)")
end
subgraph development
CreateMVT-->Compressed("MBTiles (S3)")-->TS[/Tileserver-GL/]
end
TS--"XYZ URL"-->Client
Uncompressed--"XYZ URL"-->Client["Gatsby+OpenLayers"]