diff --git a/docs/architecture/README.md b/docs/architecture/README.md index 2e642520..d173a664 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -4,6 +4,14 @@ The below is a general architecture of our proposed system: ![Architecture](architecture-mmd.svg) +The following is a more detailed diagram of the geo data pipeline architecture (the Data Pipeline and Server boxes in the general architecture diagram above). + +![Geo Data Pipeline](geodata-pipeline-arch-mmd.svg) + +We are partnering with Geoplatform to turn some of these pieces into open source shared services that they would own. The following is a modified diagram showing which pieces would tentatively be owned by Geoplatform. + +![Geo Data Pipeline](geodata-pipeline-arch-geoplatform-mmd.svg) + ## Updating the Diagram **Note: Do Not directly modify the svg file, it is generated automatically!** diff --git a/docs/architecture/geodata-pipeline-arch-geoplatform-mmd.svg b/docs/architecture/geodata-pipeline-arch-geoplatform-mmd.svg new file mode 100644 index 00000000..b9c77dc4 --- /dev/null +++ b/docs/architecture/geodata-pipeline-arch-geoplatform-mmd.svg @@ -0,0 +1 @@ +
development
production
Owned by Geoplatform
Owned by Geoplatform
Owned by Geoplatform
Event Notification
Event Notification (Geoplatform)
Event Notification
Access non-geo data
XYZ URL
XYZ URL
Tileserver-GL
MBTiles (GeoPlatform S3)
MBTiles
Uncompressed MVT (Geoplatform S3)
Create and Send MVT
Tippecanoe
GeoJSON (S3)
Combine (ogr2ogr)+ Create GeoJSON
TIGER GeoJSON (S3)
Create GeoJSON from Shapefile with osgeo/gdal
Dataset 1
Create Score CSV
Dataset 2
Census TIGER Data
CSV (S3)
Gatsby+OpenLayers Client
 \ No newline at end of file diff --git a/docs/architecture/geodata-pipeline-arch-geoplatform.mmd b/docs/architecture/geodata-pipeline-arch-geoplatform.mmd new file mode 100644 index 00000000..e64be0a2 --- /dev/null +++ b/docs/architecture/geodata-pipeline-arch-geoplatform.mmd @@ -0,0 +1,28 @@ +graph TD + Dataset1["Dataset 1"]-->Score + Dataset2["Dataset 2"]-->Score + Census["Census TIGER Data"]-->CGTiger + subgraph "Owned by Geoplatform" + CGTiger["Create GeoJSON from Shapefile with osgeo/gdal"]-->TS3 + + end + TS3("TIGER GeoJSON (S3)")-->CGJ + Score["Create Score CSV"]--Event Notification-->CSV + CSV("CSV (S3)")--"Event Notification (Geoplatform)"-->CGJ + subgraph "Owned by Geoplatform" + CGJ["Combine (ogr2ogr)+ Create GeoJSON"]--Event Notification-->GeoJSON + end + GeoJSON("GeoJSON (S3)")-->Tip + GeoJSON--"Access non-geo data"-->Client + subgraph "Owned by Geoplatform" + Tip[/Tippecanoe/]-->CreateMVT["Create and Send MVT"] + end + subgraph production + CreateMVT-->MBTiles + MBTiles-->Uncompressed("Uncompressed MVT (Geoplatform S3)") + end + subgraph development + CreateMVT-->Compressed("MBTiles (GeoPlatform S3)")-->TS[/Tileserver-GL/] + end + TS--"XYZ URL"-->Client + Uncompressed--"XYZ URL"-->Client["Gatsby+OpenLayers Client"] diff --git a/docs/architecture/geodata-pipeline-arch-mmd.png b/docs/architecture/geodata-pipeline-arch-mmd.png new file mode 100644 index 00000000..bef89b16 Binary files /dev/null and b/docs/architecture/geodata-pipeline-arch-mmd.png differ diff --git a/docs/architecture/geodata-pipeline-arch.mmd b/docs/architecture/geodata-pipeline-arch.mmd new file mode 100644 index 00000000..dbfb0f72 --- /dev/null +++ b/docs/architecture/geodata-pipeline-arch.mmd @@ -0,0 +1,22 @@ +graph TD + Dataset1["Dataset 1"]-->Score + Dataset2["Dataset 2"]-->Score + Census["Census TIGER Data"]-->CGTiger + CGTiger["Create GeoJSON from Shapefile with osgeo/gdal"]-->TS3 + TS3("TIGER GeoJSON (S3)")-->CGJ + Score["Create Score CSV"]-->CSV + CSV("CSV (S3)")-->CGJ + CGJ["Combine (ogr2ogr)+ Create GeoJSON"]-->GeoJSON + GeoJSON("GeoJSON (S3)")-->Tip + GeoJSON--"Access non-geo data"-->Client + subgraph "Generate MVT" + Tip[/Tippecanoe/]-->CreateMVT["Create and Send MVT"] + end + subgraph production + CreateMVT-->Uncompressed("Uncompressed MVT (S3)") + end + subgraph development + CreateMVT-->Compressed("MBTiles (S3)")-->TS[/Tileserver-GL/] + end + TS--"XYZ URL"-->Client + Uncompressed--"XYZ URL"-->Client["Gatsby+OpenLayers"]