j40-cejst-2/infrastructure/serverless.yml
Lucas Scharenbroich 38fff9cea8
Fargate Serverless Workers for Census Data Enrichment and Tile Generation (#230)
* add basic infrastructure

* add cloudfront distribution

* WIP checkpoint

* add ecs cluster

* add conditions and route53 dns entry to cloudfront

* WIP checkin

* Added a raw execution mode for demo/testing

* Add pre-defined Task for ogr2ogr

* Tweak Task Definition name

* Mostly working except for logging error

* Add additional logging permissions

* Succesfully executed ogr2ogr in fargate.  S3 permissions needs to be addresses

* Add multipart permissions

* Add a few more actions

* Put IAM Policy on the correct resource

* Deploy lambda and update events

* fix iam permissions 🤦🏻‍♂️

* Add reference to Tippecanoe container

* Clean up to only use named actions

* Refactor resources to include support for tippecanoe

* Make a more interesting GDAL command

* Pull all ECS variables into environment file; successful test of running tippecanoe container

* Support pre/post commands

* Refactor codebase and enable linting

* Implement many-to-many enrichment between USDS CSV files and Census zipped shapefiles

* Change the GDAL image to one with the built-in drivers

* Add some additional fixes to support the enrichment use case

* Clean up old hello-world example

* Expand the README to include ways to execute the lambdas

* Validate scheduled lambda execution and then comment out

Co-authored-by: Tim Zwolak <timothypage@gmail.com>
2021-06-30 09:29:01 -04:00

87 lines
2.6 KiB
YAML

service: justice40-data-harvester
configValidationMode: error
frameworkVersion: ">=2.48.0"
provider:
name: aws
runtime: nodejs12.x
stage: ${opt:stage, 'sit'}
region: ${opt:region, 'us-east-1'}
profile: ${self:provider.stage}
lambdaHashingVersion: "20201221"
deploymentBucket:
name: ${self:custom.environment.DEPLOYMENT_BUCKET_PREFIX}-${self:provider.stage}-${self:provider.region}-${self:service}
blockPublicAccess: true
maxPreviousDeploymentArtifacts: 5
stackName: ${self:custom.environment.STACK_NAME_PREFIX}${self:provider.stage}-${self:service}
iam:
role:
statements:
- Effect: "Allow"
# Condition:
# ArnEquals:
# ecs:cluster:
# Fn::GetAtt: [ ECSCluster, Arn ]
Action: "ecs:RunTask"
Resource: "*"
- Effect: "Allow"
# Condition:
# ArnEquals:
# ecs:cluster:
# Fn::GetAtt: [ ECSCluster, Arn ]
Action:
- "iam:ListInstanceProfiles"
- "iam:ListRoles"
- "iam:PassRole"
Resource: "*"
- Effect: Allow
Action:
- "s3:ListBucket"
Resource:
- Fn::Join:
- ""
- - "arn:aws:s3:::"
- Ref: DataBucket
- "/*"
- Effect: Allow
Action:
- "s3:DeleteObject"
- "s3:GetObject"
- "s3:PutObject"
- "s3:PutObjectAcl"
Resource:
- Fn::Join:
- ""
- - "arn:aws:s3:::"
- Ref: DataBucket
plugins:
- serverless-certificate-creator
- serverless-pseudo-parameters
custom:
environment: ${file(./environment.yml):${self:provider.stage}}
namespace: justice40 # Used to tag resources with a "Namespace".
namespaceShort: j40 # Used to prefix stack name, deployment bucket, resource "Name" tags, etc.
customCertificate:
certificateName: ${self:provider.stage}-${self:service}.${self:custom.environment.HOSTED_ZONE_DOMAIN}
hostedZoneIds: ${self:custom.environment.HOSTED_ZONE_ID_DOMAIN}
region: ${self:provider.region}
tags:
Name: ${self:provider.stage}-${self:service}.${self:custom.environment.HOSTED_ZONE_DOMAIN}
Environment: ${self:provider.stage}
rewriteRecords: true
enabled: ${self:custom.environment.SHOULD_CREATE_SSL_CERTIFICATE}
functions: ${file(./functions.yml)}
resources:
- ${file(./conditions.yml)}
- ${file(./resources-s3.yml)}
- ${file(./resources-cloudfront.yml)}
- ${file(./resources-ecs.yml)}
- ${file(./resources-route53.yml)}