38fff9cea8
* add basic infrastructure
* add cloudfront distribution
* WIP checkpoint
* add ecs cluster
* add conditions and route53 dns entry to cloudfront
* WIP checkin
* Added a raw execution mode for demo/testing
* Add pre-defined Task for ogr2ogr
* Tweak Task Definition name
* Mostly working except for logging error
* Add additional logging permissions
* Succesfully executed ogr2ogr in fargate. S3 permissions needs to be addresses
* Add multipart permissions
* Add a few more actions
* Put IAM Policy on the correct resource
* Deploy lambda and update events
* fix iam permissions 🤦🏻♂️
* Add reference to Tippecanoe container
* Clean up to only use named actions
* Refactor resources to include support for tippecanoe
* Make a more interesting GDAL command
* Pull all ECS variables into environment file; successful test of running tippecanoe container
* Support pre/post commands
* Refactor codebase and enable linting
* Implement many-to-many enrichment between USDS CSV files and Census zipped shapefiles
* Change the GDAL image to one with the built-in drivers
* Add some additional fixes to support the enrichment use case
* Clean up old hello-world example
* Expand the README to include ways to execute the lambdas
* Validate scheduled lambda execution and then comment out
Co-authored-by: Tim Zwolak <timothypage@gmail.com>
|
||
|---|---|---|
| .. | ||
| functions/detect-changes-for-worker | ||
| conditions.yml | ||
| environment.yml | ||
| functions.yml | ||
| package-lock.json | ||
| package.json | ||
| README.md | ||
| resources-cloudfront.yml | ||
| resources-ecs.yml | ||
| resources-route53.yml | ||
| resources-s3.yml | ||
| serverless.yml | ||
create acm certificate
This only needs to be run once for the sit environment. stg and prd, we're assuming some other certificate arn will be used
npx serverless create-cert
you'll have to grab the arn of the certificate from the log output or go into the console to get it, looks like the plugin doesn't work any more. Set CLOUDFRONT_CERTIFICATE_ARN in sit to that value
deploy
sls deploy --aws-profile geoplatform --stage sit --verbose
If it's the first time deploying, you'll have to create a dns entry that points to the cloudfront distribution.
testing
The examples can be run several different ways
local
The package.json file incluses several examples to run against the local source code. The actual
tasks will execute within AWS, so an AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY must be set in
the test.env file.
$ cd ./functions/detect-changes-for-worker
$ npm run test:gdal
lambda invoke
The deployed lambda functions can be directly invoked with the serverless invoke function.
$ cat ./functions/detect-changes-for-worker/events/gdal.json | sls invoke -s sit -f DetectChangesForWorker
New event files can be created to perform one-off data processes.