Caching is an idea that developers learn about early in their careers, and at first, it sounds simple. But when you sit down to really think about it, caching turns out to be a complex and fascinating topic that’s worth discussing in some detail. So before my cache gets busted, let’s get started.
What is Caching?
According to our old flame Wikipedia, a cache may be a hardware or software component that stores data in order that future requests for that data are often served faster. the info stored during a cache could be the results of an earlier computation or a replica of knowledge stored elsewhere.
As an example, when you log into Twitter, you’re greeted with a long list of tweets from the users you follow. The list you see has been created beforehand and cached, rather than gathered and assembled on the spot. This ensures a great user experience. Essentially, when your back-end systems aren’t as fast as you’d like them to be, a cache gives you a holding pen for quick retrieval and thus, a faster response time. Response time refers to the amount of time, usually in milliseconds, that an application takes to respond to user interaction.
What are the benefits of using caching system?
When caching is part of your architecture, you can often reduce response times from hundreds of milliseconds to less than a millisecond. At the same time, you’re reducing the load on your back-end services and getting more from your existing hardware. And most importantly, your users will benefit from faster interactions with your application. So now that you know what caching is and why it’s important, let’s look at how a cache fits in your system architecture, and let’s also review a basic caching algorithm.
How does Caching Works?
The back end receives a request and then queries a disk-based data store to retrieve the necessary records. Let’s say that each request takes around 75 milliseconds to fulfill. This might not seem like a long time, but remember, this is just one back-end request. It’s often necessary to make multiple requests to render a given view in an application. 75 milliseconds doesn’t give us much wiggle room, especially if we’re under load. If we ever need to scale, or if we’re lucky enough to go viral, the extra load will add up, creating bottlenecks, blocking, and high latency. And remember, latency is the new outage. Enter the cache.
What is Redis?
In this case, our cache is a Redis database. Redis stores its entire data set in memory, which makes it an ideal choice for caching. Imagine taking that initial 75 milliseconds response time and reducing it to less than a millisecond. That’s quite an improvement. Now when our application receives a request, it first quickly checks the cache to see if that data is already at hand. If so, great. We call this a cache hit. We can send the data back to the user right away, rather than accessing the original data store. If the data is not in the cache, we call this a cache miss.
We then process the request through the original back-end data store as normal. This is a great opportunity, though. After adding the result of this request to our cache, subsequent requests result in a cache hit and a much faster response time. So our example covered an application requesting data from a database, but caches are used for all kinds of services, not just legacy relational databases, but also microservices and third-party APIs.
Why is that?
Well, it comes down to speed and cost. Disk reads are comparatively slow when compared to RAM. Even though SSDs are fast, they’re still an order of magnitude slower than RAM. So if you perform a query, and the data you’re looking for isn’t in the disk or page cache, then you’re going to incur the hit of going to disk. Remember, Redis stores all data in memory, so you’ll never incur the extra cost of a lookup having to go to disk.
Complex computations, such as those used in machine learning inferences and full-text searches, take a lot of time and compute resources, making them good candidates for caching. Certain types of relational database queries, notably JOINS, are computationally expensive and benefit from caching as well.
What creates lagging?
Let’s also not forget slow network connections between your back-end services. This can add up to a considerable delay in response. Lastly, caching responses from third-party APIs reduces costs where API usage is billed on a per request basis. So caching is useful in a number of situations, and it dramatically decreases your application response times. What’s the downside? Well, one challenge of caching is staleness. At some point, your cache is no longer going to represent the system of record. This is known as cache staleness. The strategies to prevent a stale cache depend on your use case, but in simple terms, they boil down to passive versus active cache expiration.
We can set a TTL on each cache entry to passively clear out the cache. TTL means Time to Live, and it usually is measured in seconds, minutes, or hours. It’s like setting an expiration date. After a defined amount of time, the data will be automatically deleted, ensuring we do not store outdated information forever. We could also actively update the cache by replacing existing entries with newer, fresh versions taken from the system of record. This is known as cache-busting. It’s worth considering whether you really need to add a cache. For certain services, it might make sense to make Redis itself the system of record. This way, the data is always served from memory, and if you take advantage of Redis’ persistence and high availability features, you can ensure that your data is still safe and available.
Quick Review of Cache
We’ve covered quite a lot of ground here, so let’s do a quick review. A cache is a data store that’s faster to access than the system of record, whether that’s a relational database, an array of microservices, or a third-party API. Caches store a copy of frequently requested data in memory, saving your application from repeating the same calls to your back end. Caches become stale when they get out of sync with a system of record, so you need to make sure that you employ an expiration strategy to ensure that relevant, fresh information is always in the cache. Thanks for joining me in a quick exploration of caching. Stay tuned for future segments where we cover different caching strategies, geo-distribution, replication, and persistence. Keep learning, and stay fast.