Reduce Your Database Bill AND Improve Performance
We know databases are expensive. It’s very easy to rack up a 1000$ tab on a database. However expensive they may be, they might still not provide the best service to your users. I know. Shocking. Money doesn’t solve everything. During peaks, the number of times you hit a database can explode. Queues start building up, leading to longer wait times for your users. Make that wait a couple of seconds longer than usual and they could walk out the door (or close the webpage, whichever is more dramatic). You just paid to lose users!
Fixing the problem
Let me introduce you to a little thing called “Cache”. You’re probably familiar with it in the browser context. That is called “client-side cache”, but there is another type of cache called “server-side cache”.
Client-side cache
The most popular scenario is a user that sends a request to a server and the browser proceeds to store the response. Then if the user makes a request again the browser will not send a request to the application server, it will simply serve the response already stored.
Client-side cache isn’t exclusive to browsers. If your application server makes a request to some API, you can apply the concept of client-side cache by storing its response.
This is achieved by using appropriate HTTP Headers, so the client knows when to cache a response and for how long.
Server-side caching
If the API that you’re calling is storing already computed responses, then that is server-side cache. This requires significantly more effort than client-side Cache because you need to define the logic used for caching, setup various servers, etc. With client-side cache you simply tell the user that it can store the response for X amount of time and he handles everything.
In distributed systems there can be tons of cached responses moving around.
The concept
When you make a request to a website you send the request over the network, it arrives at the application which then makes all the processing needed to produce a response. This usually involves accessing databases, etc. This can be a very slow and expensive process.
So instead of doing the computations needed to produce a response for each request, we store the response in a faster server (caching server). When the application sees 2 equal requests it will not go to the database and do all the processing for the 2nd request. It will simply ask the cache server for the already computed response from the 1st request.
Cache Server vs Database
With Cache you are simply trading off capacity and persistence for speed. While a database will have all of your data stored in a way that will guarantee its existence after a system failure, a caching server will usually have a subset of your data and will lose all of its data after a system failure. However, a caching server will be much faster in serving data.
This is because we are using the RAM in a caching server to store our data, which means limited storage capacity and loss of data in case of a shutdown. A database will use non-volatile memory, thus reducing speed dramatically.
Database scaling vs Cache
Concerning the initial high-traffic problem, some of you might think immediately of scaling out your database (add more instances as usage increases) to handle higher workloads. A caching server will be a better solution than that most of the time in terms of performance. For sure it will be cheaper. So if your application can take advantage of cache, do give it a test at least.
Also, you can set up clusters of servers just to handle cache. For larger applications that utilize scaling in and out mechanisms, having a separate layer just for caching is recommended so that the cache isn’t affected by the destruction of machines running your application. Basically, don’t run cache servers in the same machines that are running your application.
Persistence
Often you have a choice between having no persistence in your cache or some persistence (lose only information from the last hour in case of failure, for example). When you make this choice you need to take into account how much your application relies on Cache.
Can you handle the higher workload required to build up the Cache in case of failure, and reap the rewards of faster performance from having no persistence? With websites relying heavily on cache the answer could very well be no, so some degree of persistence might be needed.
Practical example
Let’s say that you have a website that generates its homepage based on user preferences, location, etc. That page is generated by some large computation that requires you to fetch data from multiple sources and combine them into a coherent webpage. The idea is to avoid doing the large computation for users that have the same preferences and instead store the result of the 1st time we do a computation in an in-memory data store so it is served faster.
Popular tools used for Caching
Redis
Redis is an in-memory data store. It supports a variety of data structure (strings, lists, sets, etc.) You can scale-out and set it up for high-availability natively.
It also offers different degrees of persistence. You can choose from no persistence to a level of persistence similar to a Postgres database.
This is the most popular tool used for cache.
Memcached
Memcached is an in-memory key-value store. It only supports strings. It is multi-threaded so you can handle higher loads by scaling up. In higher load cases a single server can sometimes provide better performance than a Redis server (100k+ keys stored). It is more limited in what the server can do (when compared to Redis), so sometimes you might have to fetch data, process it in the application server and send it back.
Conclusion
Cache is an essential component of any modern application. It not only reduces costs but also improves performance. The most complex and widely used applications in the world use it extensively, so it is important to feel comfortable with the topic.
