A Cache is a temporary storage, designed to store the result of resource-intensive tasks and frequently accessed information. By enabling subsequent operations to retrieve data from the cache, it mitigate the risk of overloading database servers.

 

Without cache

When there's no cache, every time the application needs data from the database, it asks the database directly. This puts a lot of stress on the database. When the queries are complicated/resource intensive, it takes a long time to get the response, and the database gets used up quickly. The picture below shows what happens when there's no cache.

 

 

With Cache

With a cache layer, the application first looks in the cache to see if the data it needs is already there before asking the database. If the data is in the cache, it's retrieved from there. If not, the data is fetched from the database, stored in the cache, and then sent to the client. This process is shown in the image below.

 

Pseudo code

function getData(key):
    if cache.contains(key):
        // If data is in cache, retrieve it
        return cache.get(key)
    else:
        // If data is not in cache, fetch from database
        data = queryDatabase(key)
        // Store data in cache for future use
        cache.put(key, data)
        return data

// Application logic
data = getData("some_key")
// Use the data retrieved from either cache or database

In the above code, the getData function first checks if the data is present in the cache. If it is, it retrieves the data from the cache. If not, it queries the database to fetch the data, stores it in the cache, and then returns the data. This way, subsequent requests for the same data can be served from the cache, reducing the load on the database.

 

Points to consider while using the cache

1. Decide the areas that benefitted from cache layer

Frequently accessed data that undergoes infrequent modifications can be efficiently stored in the cache. The results of resource-intensive operations can be cached to reduce the computational load on the system.

 

2. Expiration Policy

It's advisable to establish an expiration policy for cached data. Without one, data remains in the cache indefinitely, potentially causing discrepancies between the database and cache over time.

 

Time based expiration is the most common cache expiration policy, where the data is removed from the cache after a specified period of time has elapsed since it was stored or last accessed.

 

3. Eviction Policy

Eviction refers to the selection of items for removal from the cache when a new item needs to be added but the cache has reached its maximum capacity (Capacity can be defined either by size or by total number of items etc.,).

 

Some common cache eviction policies include:

 

a.   LRU (Least Recently Used): Data that has not been accessed for a certain period of time is evicted from the cache, prioritizing the retention of more recently accessed data.

b.   LFU (Least Frequently Used): Data that has been accessed the fewest number of times is removed from the cache when space is needed.

c.    FIFO (First In First Out): Data that has been added to the cache times is removed from the cache when space is needed.

 

4. Distributed vs in-memory Cache

Distributed cache and in-memory cache do similar jobs but work differently.

 

a.   Distributed Cache: It spreads across many servers, each storing a part of the data. They're good for handling lots of traffic because they split the work between servers. Examples are Redis Cluster and Memcached.

b.   In-Memory Cache: It lives in the server's memory itself. It's easier to use but might not handle as much traffic. Examples include local caches like Java Caching System, Caffeine.

 

Both help things work faster by saving time and reducing how much the database does. Which one you choose depends on how much traffic you have, how well you want things to handle mistakes, and how complicated your system is.

 

5. Cache size

Here are some tips for finding the right cache size:

 

a.   Start with Small and Watch: Begin with a modest cache size and keep an eye on how well it's working. Look at things like how often data is served from the cache instead of the database.

b.   Look at Cache Hit Rate: If most requests (over 90%) are coming from the cache, that's great! But if it's lower, your cache might need to be bigger or hold onto data for longer.

c.    Watch for Old Data: If users are seeing old information because your cache is too big, it might be time to shrink it or make items expire faster.

d.   Use Monitoring Tools: There are lots of tools out there to help you keep an eye on how your cache is doing. They can show you what's happening and where you might need to make changes.

 

Remember, finding the perfect cache size is all about keeping an eye on things and adjusting as needed.

 

6. Security considerations

Caching poses risks if not adequately secured. Here are key considerations:

 

a.   Avoid storing sensitive data such as passwords in the cache, or employ encryption if necessary.

b.   Restrict access to the cache to authorized users to prevent unauthorized access to sensitive information.

c.    Validate data before caching to prevent the storage of potentially harmful content.

d.   Implement encryption protocols like TLS to safeguard data during transit between your application and the cache.

e.   Monitor cache access and modifications to identify any suspicious activity.

f.     Implement expiration policies for cached data to prevent it from persisting indefinitely.

g.   Conduct regular security assessments of your cache infrastructure to address any vulnerabilities.

 

If you want to take a look at popular caching systems, refer below tutorials.

 

Caffeine Cache

Ehcache

ExpiringMap

Guava Cache

JCS (Java Caching System)

Spymemcached

Redis

Caching

                                                  System Design Questions