mirror of
https://github.com/DOI-DO/j40-cejst-2.git
synced 2025-02-22 17:44:20 -08:00
Add diagrams detailing our geo data pipeline (#170)
* Detailed flow for data inputs, census data, score creation, and geo data generation (geoJSON and MVT) * Include diagram showing vision for geoplatform ownership Co-authored-by: Shelby Switzer <shelby.switzer@cms.hhs.gov>
This commit is contained in:
parent
c8a7f81f7c
commit
b61d971f15
5 changed files with 59 additions and 0 deletions
|
@ -4,6 +4,14 @@ The below is a general architecture of our proposed system:
|
|||
|
||||

|
||||
|
||||
The following is a more detailed diagram of the geo data pipeline architecture (the Data Pipeline and Server boxes in the general architecture diagram above).
|
||||
|
||||

|
||||
|
||||
We are partnering with Geoplatform to turn some of these pieces into open source shared services that they would own. The following is a modified diagram showing which pieces would tentatively be owned by Geoplatform.
|
||||
|
||||

|
||||
|
||||
## Updating the Diagram
|
||||
|
||||
**Note: Do Not directly modify the svg file, it is generated automatically!**
|
||||
|
|
File diff suppressed because one or more lines are too long
After Width: | Height: | Size: 35 KiB |
28
docs/architecture/geodata-pipeline-arch-geoplatform.mmd
Normal file
28
docs/architecture/geodata-pipeline-arch-geoplatform.mmd
Normal file
|
@ -0,0 +1,28 @@
|
|||
graph TD
|
||||
Dataset1["Dataset 1"]-->Score
|
||||
Dataset2["Dataset 2"]-->Score
|
||||
Census["Census TIGER Data"]-->CGTiger
|
||||
subgraph "Owned by Geoplatform"
|
||||
CGTiger["Create GeoJSON from Shapefile with osgeo/gdal"]-->TS3
|
||||
|
||||
end
|
||||
TS3("TIGER GeoJSON (S3)")-->CGJ
|
||||
Score["Create Score CSV"]--Event Notification-->CSV
|
||||
CSV("CSV (S3)")--"Event Notification (Geoplatform)"-->CGJ
|
||||
subgraph "Owned by Geoplatform"
|
||||
CGJ["Combine (ogr2ogr)+ Create GeoJSON"]--Event Notification-->GeoJSON
|
||||
end
|
||||
GeoJSON("GeoJSON (S3)")-->Tip
|
||||
GeoJSON--"Access non-geo data"-->Client
|
||||
subgraph "Owned by Geoplatform"
|
||||
Tip[/Tippecanoe/]-->CreateMVT["Create and Send MVT"]
|
||||
end
|
||||
subgraph production
|
||||
CreateMVT-->MBTiles
|
||||
MBTiles-->Uncompressed("Uncompressed MVT (Geoplatform S3)")
|
||||
end
|
||||
subgraph development
|
||||
CreateMVT-->Compressed("MBTiles (GeoPlatform S3)")-->TS[/Tileserver-GL/]
|
||||
end
|
||||
TS--"XYZ URL"-->Client
|
||||
Uncompressed--"XYZ URL"-->Client["Gatsby+OpenLayers Client"]
|
BIN
docs/architecture/geodata-pipeline-arch-mmd.png
Normal file
BIN
docs/architecture/geodata-pipeline-arch-mmd.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 69 KiB |
22
docs/architecture/geodata-pipeline-arch.mmd
Normal file
22
docs/architecture/geodata-pipeline-arch.mmd
Normal file
|
@ -0,0 +1,22 @@
|
|||
graph TD
|
||||
Dataset1["Dataset 1"]-->Score
|
||||
Dataset2["Dataset 2"]-->Score
|
||||
Census["Census TIGER Data"]-->CGTiger
|
||||
CGTiger["Create GeoJSON from Shapefile with osgeo/gdal"]-->TS3
|
||||
TS3("TIGER GeoJSON (S3)")-->CGJ
|
||||
Score["Create Score CSV"]-->CSV
|
||||
CSV("CSV (S3)")-->CGJ
|
||||
CGJ["Combine (ogr2ogr)+ Create GeoJSON"]-->GeoJSON
|
||||
GeoJSON("GeoJSON (S3)")-->Tip
|
||||
GeoJSON--"Access non-geo data"-->Client
|
||||
subgraph "Generate MVT"
|
||||
Tip[/Tippecanoe/]-->CreateMVT["Create and Send MVT"]
|
||||
end
|
||||
subgraph production
|
||||
CreateMVT-->Uncompressed("Uncompressed MVT (S3)")
|
||||
end
|
||||
subgraph development
|
||||
CreateMVT-->Compressed("MBTiles (S3)")-->TS[/Tileserver-GL/]
|
||||
end
|
||||
TS--"XYZ URL"-->Client
|
||||
Uncompressed--"XYZ URL"-->Client["Gatsby+OpenLayers"]
|
Loading…
Add table
Reference in a new issue