How do I cache my images which are stored in Amazon S3

Question

I have some of my web services running on Amazon EC2. As my application needs to deal with a large number of photos, I plan to board them on my Amazon S3. Is there any way to cache the images on EC2? I'm using the EC2 micro instance and I feel it'd be better if I do not cache the Image on EC2. But the scale might grow fast and eventually will need an image cache. If a cache is needed, is it better to do it on EC2, or on S3? (Is there a way for caching for S3. Also, when the client uploads an image should it be uploaded to EC2 or S3 directly?

Answer 1

Why use EC2 here? 

I'd strongly suggest you should use CloudFront for this case as when you use CloudFront with S3 as an origin, your content will get distributed to 49 different locations worldwide directly working out as a cache globally and the content which is being fetched from the nearest location based on the latency to the end users.You don't need to worry about the scale and performance of Cache, EC2 can straightforward offload this to CloudFront and S3.

However, if you're  very keen to work on EC2 only then I am sharing few scenarios for EC2 caching instance:

• your upload/download ratio is far from 50/50
• you hit S3 limit 100req/sec
• you need URL Masking
• you want to optimize kernel, TCP/IP settings, cache SSL session for clients
• you want proper cache invalidating mechanism for all geo locations
• you need 100% control where data is stored
• you need to count the number of requests
• you have the custom authentication mechanism

Answer 2

proper use of the Cache-Control header will make life better. With that header you can tell a browser how long the content is good for. If it is something that never changes, you can instruct a browser to keep it for a year. If it frequently changes, you can instruct a browser to keep it for an hour, or a minute, or revalidate every time. You can give similar instructions to a CDN.

Here's a good guide, and here are some examples:

# keep for one year
Cache-Control: max-age=2592000

# keep for a day on a CDN, but a minute on client browsers
Cache-Control: s-maxage=86400, maxage=60

You can add this to pages served from your EC2 instance (no matter if it's nginx, Tornado, Tomcat, IIS), you can add it to the headers on S3 files, and CloudFront will use these values.

Answer 3

When you use CloudFront in conjunction with S3 as origin; the content gets distributed to 49 different locations worldwide ( as of count of edge locations worldwide today ) directly working out as a cache globally and the content being fetched from nearest location based on the latency to your end users.

Answer 4

when caching ec2 instance these can be the scenario:

• your upload/download ratio is far from 50/50

• you hit S3 limit 100req/sec

• you need URL masking

• you want to optimise kernel, TCP/IP settings, cache SSL session for clients

• you want proper cache invalidating mechanism for all geo locations

• you need 100% control where data is stored

• you need to count number of requests

• you have custom authentication mechanism