Quick helpful article to get you to setup an API gateway which acts as a S3 proxy using Cloudformation script.

This is a Tidbit, basically whenever you see Tidbit: in the title,

it means I am going to simply throw some helpful code without

explaining too much around it.

What you can use this for?

This is a cheap solution for your application to upload files via a REST API leveraging API gateways authorization options while optionally making the uploaded object available to public, you can even pull private objects via this API by passing the correct Authorization header .

Here is a simple depiction of a possible flow:



The code

I have uploaded the entire code on GitHub here.

Make sure you go through the serverless.yml file and replace stuff.

In case you don’t know what serverless framework is , I have written an article to explicitly remedy that here.

Here is the excerpt of the Cloudformation script that is available in the repository:

Resources: s3filesbucket: Type: 'AWS::S3::Bucket' Properties: AccessControl: PublicReadWrite BucketName: ${env:bucketNamePrefix}<your bucket name> VersioningConfiguration: Status: Suspended Tags: - Key: Environment Value: ${opt:stage} WebsiteConfiguration: IndexDocument: index.html route53BucketDNS: #this adds a DNS recordset which can be used for public viewing of objects Type: "AWS::Route53::RecordSet" Properties: HostedZoneId: <hosted zone id> Name: ${env:bucketNamePrefix}<your bucket name> ResourceRecords: - ${env:bucketNamePrefix}<your bucket name>.s3-website-us-east-1.amazonaws.com TTL: 300 Type: CNAME s3Proxy: Type: "AWS::ApiGateway::RestApi" Properties: BinaryMediaTypes: #add or substract MIME types here - image/png - image/jpg - image/gif - image/x-icon - application/octet-stream Description: Storage service for ${opt:stage} environment FailOnWarnings: false Name: ${env:apiGatewayPrefix}fileStorage #api gateway name s3ProxyAuthorizer: #custom authorizer for the API gateway which adds stuff to the bucket Type: "AWS::ApiGateway::Authorizer" Properties: AuthorizerResultTtlInSeconds: 300 AuthorizerUri: arn:aws:apigateway:us-east-1:lambda:path//2015-03-31/functions/arn:aws:lambda:us-east-1:<your acc id>:function:${opt:stage}<auth fn name>/invocations IdentitySource: method.request.header.Authorization IdentityValidationExpression: ^Bearer.+ Name: CommonAuthorizer RestApiId: Ref: "s3Proxy" Type: TOKEN s3ProxyAnyMethod: Type: "AWS::ApiGateway::Method" Properties: ApiKeyRequired: false AuthorizationType: CUSTOM AuthorizerId: Ref: "s3ProxyAuthorizer" HttpMethod: ANY Integration: Credentials: arn:aws:iam::<your acc id>:role/s3ProxyRole IntegrationHttpMethod: ANY IntegrationResponses: - StatusCode: 200 PassthroughBehavior: WHEN_NO_MATCH RequestParameters: integration.request.header.Content-Disposition: method.request.header.Content-Disposition integration.request.header.Content-Type: method.request.header.Content-Type integration.request.header.x-amz-acl: method.request.header.x-amz-acl integration.request.path.key: method.request.querystring.key Type: AWS Uri: arn:aws:apigateway:us-east-1:s3:path/${env:bucketNamePrefix}<your bucket name>/{key} MethodResponses: - StatusCode: 200 RequestParameters: method.request.header.Content-Disposition: false method.request.header.Content-Type: false method.request.header.x-amz-acl: false method.request.querystring.key: false ResourceId: Fn::GetAtt: - "s3Proxy" - "RootResourceId" RestApiId: Ref: "s3Proxy" ApiGatewayDeploymentthisiswhatiwillreplace: Type: 'AWS::ApiGateway::Deployment' Properties: RestApiId: Ref: "s3Proxy" StageName: ${opt:stage} DependsOn: - s3ProxyAnyMethod

Points to note:

The last part of the YAML declares an API Gateway deployment which will make the deployed changes available to public, the predicament is that unless you change something in that section (and there is nothing to change there ever), Cloudformation will not detect that as a change and will never deploy your changeset even though it will update every change, this is solved by replacing the aptly named thisiswhatiwillreplace part of the name with epoch time during the build process. The build is via AWS Codebuild (Get started with it). Go through the entire script, edit as required, I have replaced some account specific stuff with placeholders like or etc.

The script will deploy the following:

The API gateway acting as the proxy and deploy it. An S3 bucket corresponding to the ‘stage’ that you deploy in. Create a DNS recordset in Route 53 to point to your bucket to have a custom domain name to access your public objects.

Calling the service

Put an object

PUT https://<your api id here>execute-api.us-east-1.amazonaws.com/prod?key=blah.png HTTP/1.1 Content-Type: image/png User-Agent: Fiddler Host: <your api id here>.execute-api.us-east-1.amazonaws.com Authorization: Bearer <some auth token> Content-Disposition: inline x-amz-acl: public-read Content-Length: 205523 <binary data here>

The header x-amz-acl: public-read will make the object available to public, to keep it private, simply omit the header.

Get a private object

GET https://<your api id here>.execute-api.us-east-1.amazonaws.com/prod?key=blah.png HTTP/1.1 Content-Type: image/png User-Agent: Fiddler Host: <your api id here>.execute-api.us-east-1.amazonaws.com Authorization: Bearer <some auth token> Content-Disposition: inline

Delete an object

DELETE https://<your api id here>.execute-api.us-east-1.amazonaws.com/prod?key=blah.png HTTP/1.1 Content-Type: image/png User-Agent: Fiddler Host: <your api id here>.execute-api.us-east-1.amazonaws.com Authorization: Bearer <some auth token> Content-Disposition: inline

If you liked this article, you can choose to follow this blog/subscribe to email alerts (floating follow button {bottom-right} or below comments in mobile) so that you know when any future posts come about.