mirror of
https://github.com/DOI-DO/j40-cejst-2.git
synced 2025-02-23 01:54:18 -08:00
Add diagrams detailing our geo data pipeline (#170)
* Detailed flow for data inputs, census data, score creation, and geo data generation (geoJSON and MVT) * Include diagram showing vision for geoplatform ownership Co-authored-by: Shelby Switzer <shelby.switzer@cms.hhs.gov>
This commit is contained in:
parent
c8a7f81f7c
commit
b61d971f15
5 changed files with 59 additions and 0 deletions
|
@ -4,6 +4,14 @@ The below is a general architecture of our proposed system:
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
|
The following is a more detailed diagram of the geo data pipeline architecture (the Data Pipeline and Server boxes in the general architecture diagram above).
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
We are partnering with Geoplatform to turn some of these pieces into open source shared services that they would own. The following is a modified diagram showing which pieces would tentatively be owned by Geoplatform.
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
## Updating the Diagram
|
## Updating the Diagram
|
||||||
|
|
||||||
**Note: Do Not directly modify the svg file, it is generated automatically!**
|
**Note: Do Not directly modify the svg file, it is generated automatically!**
|
||||||
|
|
File diff suppressed because one or more lines are too long
After Width: | Height: | Size: 35 KiB |
28
docs/architecture/geodata-pipeline-arch-geoplatform.mmd
Normal file
28
docs/architecture/geodata-pipeline-arch-geoplatform.mmd
Normal file
|
@ -0,0 +1,28 @@
|
||||||
|
graph TD
|
||||||
|
Dataset1["Dataset 1"]-->Score
|
||||||
|
Dataset2["Dataset 2"]-->Score
|
||||||
|
Census["Census TIGER Data"]-->CGTiger
|
||||||
|
subgraph "Owned by Geoplatform"
|
||||||
|
CGTiger["Create GeoJSON from Shapefile with osgeo/gdal"]-->TS3
|
||||||
|
|
||||||
|
end
|
||||||
|
TS3("TIGER GeoJSON (S3)")-->CGJ
|
||||||
|
Score["Create Score CSV"]--Event Notification-->CSV
|
||||||
|
CSV("CSV (S3)")--"Event Notification (Geoplatform)"-->CGJ
|
||||||
|
subgraph "Owned by Geoplatform"
|
||||||
|
CGJ["Combine (ogr2ogr)+ Create GeoJSON"]--Event Notification-->GeoJSON
|
||||||
|
end
|
||||||
|
GeoJSON("GeoJSON (S3)")-->Tip
|
||||||
|
GeoJSON--"Access non-geo data"-->Client
|
||||||
|
subgraph "Owned by Geoplatform"
|
||||||
|
Tip[/Tippecanoe/]-->CreateMVT["Create and Send MVT"]
|
||||||
|
end
|
||||||
|
subgraph production
|
||||||
|
CreateMVT-->MBTiles
|
||||||
|
MBTiles-->Uncompressed("Uncompressed MVT (Geoplatform S3)")
|
||||||
|
end
|
||||||
|
subgraph development
|
||||||
|
CreateMVT-->Compressed("MBTiles (GeoPlatform S3)")-->TS[/Tileserver-GL/]
|
||||||
|
end
|
||||||
|
TS--"XYZ URL"-->Client
|
||||||
|
Uncompressed--"XYZ URL"-->Client["Gatsby+OpenLayers Client"]
|
BIN
docs/architecture/geodata-pipeline-arch-mmd.png
Normal file
BIN
docs/architecture/geodata-pipeline-arch-mmd.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 69 KiB |
22
docs/architecture/geodata-pipeline-arch.mmd
Normal file
22
docs/architecture/geodata-pipeline-arch.mmd
Normal file
|
@ -0,0 +1,22 @@
|
||||||
|
graph TD
|
||||||
|
Dataset1["Dataset 1"]-->Score
|
||||||
|
Dataset2["Dataset 2"]-->Score
|
||||||
|
Census["Census TIGER Data"]-->CGTiger
|
||||||
|
CGTiger["Create GeoJSON from Shapefile with osgeo/gdal"]-->TS3
|
||||||
|
TS3("TIGER GeoJSON (S3)")-->CGJ
|
||||||
|
Score["Create Score CSV"]-->CSV
|
||||||
|
CSV("CSV (S3)")-->CGJ
|
||||||
|
CGJ["Combine (ogr2ogr)+ Create GeoJSON"]-->GeoJSON
|
||||||
|
GeoJSON("GeoJSON (S3)")-->Tip
|
||||||
|
GeoJSON--"Access non-geo data"-->Client
|
||||||
|
subgraph "Generate MVT"
|
||||||
|
Tip[/Tippecanoe/]-->CreateMVT["Create and Send MVT"]
|
||||||
|
end
|
||||||
|
subgraph production
|
||||||
|
CreateMVT-->Uncompressed("Uncompressed MVT (S3)")
|
||||||
|
end
|
||||||
|
subgraph development
|
||||||
|
CreateMVT-->Compressed("MBTiles (S3)")-->TS[/Tileserver-GL/]
|
||||||
|
end
|
||||||
|
TS--"XYZ URL"-->Client
|
||||||
|
Uncompressed--"XYZ URL"-->Client["Gatsby+OpenLayers"]
|
Loading…
Add table
Reference in a new issue