Caching strategies for websites

Every remotely popular website has to cope with higher volume traffic at some point. It does not have to be the /. effect, being dugg or StephenFried, but making sure your website might sustain a certain amount of traffic is never a bad idea.

This is where caching becomes useful. Caching describes the process of saving the result of a computation to be able to return the result without invoking the computation a second time. This obviously has some implications:

• the result must not depend on the current user / usersession
• the result should not get stale predictably
• how to invalidate the cache?

To work around the first point is hard and restricts you to more granular caching, to cache only items that are not depended. Djangos Caching Documentation describes all the available ways to cache using Django very well. This allows for various levels of granularity and provides some cache-header decorators as well. If you are using Django and have not had a look at it, I can highly recommend it.

The second point is a little bit trickier. If we really think about it, there are a lot of things we don’t want to cache then, because–maybe–something will change. Yes maybe something will change and you will run into a stale cache. If we have a page with a lot of components that draw their content from different sources, it might be hard to find a perfect cache-invalidation solution and we cannot prevent a stale cache. The question is: does the user of our application really know if he sees a (slightly) stale cache? No, if the cache is bound to have a timeout, it will only be stale for a controllable time frame. Blogs usually have an index page with their latest articles, a category list, latest posts list, comment counts, etc. Not caching this page incurs a lot of computation. If we cache parts it will get better. Assume we agree on a one minute caching strategy, most users won’t notice it. Yes, your latest post might show up one minute late, still not a big problem if you can reduce your load from 40 computation requests to 39 cache hits and one computation. (Assuming 40 request/min)

For the third point, assume we are running a forum like application with a moderate amount of concurrent users. If user A posts a message to the thread, user B won’t notice if he or she is reading the thread already or preparing a post. The frequency of request for reading purpose are higher then those for writing/posting purpose. If we assume we have approx 25 read and 5 write requests per minute, a pretty simple strategy would be to cache the whole thread (pagewise) and invalidate only the affected pages on a write. Thus 3 out of 4 request would hit our cache and not consume too much computational power.

If you are running your application on App Engine, your awareness of limited resources is likely higher then when you run an application on your dedicated VPS. App Engines Dashboard is very frank about your available resources. Even though you can buy additional resources now does not mean you have to.

But why bother at all if you got plenty of computing power and not a lot of requests? I think everyone can judge by their own experience that they are more likely to spend time on fast loading websites then on those that take longer to load. Caching can reduce load times to a great extend.