We use S3 for online storage of files. To reduce costs, many of those files can be generated on demand with a path known ahead of time.
Is it possible to get S3 bucket to redirect missing object requests to my pre-configured endpoint (which can then generate and serve the file on demand)?
For example, a request to http://bucket1.s3.amazonaws.com/path2/file3.jpg would temporarily redirect (307) to http://mydomain.com/missing_s3_obj/bucket1/path2/file3.jpg.
You can use RoutingRules
to have S3 redirect to your domain as described in the Example 3 of Configure a Bucket for Website Hosting from the S3 documentation. I believe it describes your situation accurately.
<RoutingRules>
<RoutingRule>
<Condition>
<HttpErrorCodeReturnedEquals>404</HttpErrorCodeReturnedEquals>
</Condition>
<Redirect>
<HostName>mydomain.com</HostName>
<ReplaceKeyPrefixWith>missing_s3_obj/bucket1/</ReplaceKeyPrefixWith>
<HttpRedirectCode>307</HttpRedirectCode>
</Redirect>
</RoutingRule>
</RoutingRules>
At the moment this isn't possible, although it is a very good idea. (Perhaps you should list it as a feature request on the S3 forum, where feature requests are taken actually very seriously).
What you want can be acheived using a different archiectecture set up though.
For example, rather than using S3, you would use Cloudfront instead. Your edge origin would be your own application serving up requests. You would have to store a hash of all available files on S3, but if the file is available on S3 you would proxy to S3. If it is not, then you would generate the file dynamically and serve that to Cloudfront.
You would then get all of the benefits of cachine for subsequent requests.