Part- 1 : Cache : (a.k.a. [kaSH] /kæʃ/Cash/ka-shay)

Caching is a technique to speed up data lookups (data reading). Instead of reading the data directly from it source, which could be a database or another remote system, the data is read directly from a cache on the computer that needs the data. Here is an illustration of the caching principle:

A cache is a storage area that is closer to the entity needing it than the original source. Accessing this cache is typically faster than accessing the data from its original source. A cache is typically stored in memory or on disk. A memory cache is normally faster to read from than a disk cache, but a memory cache typically does not survive system restarts.

Cache population:

Cache Writing Techniques:

• Write through:

In write through, data is simultaneously updated to cache and memory. This process is simpler and more reliable.This is used when there are no frequent writes to the cache(Number of write operation is less).

It helps in data recovery (In case of power outage or system failure). A data write will experience latency (delay) as we have to write to two locations (both Memory and Cache). It Solves the inconsistency problem. But it questions the advantage of having a cache in write operation (As the whole point of using a cache was to avoid multiple accessing to the main memory).

• Write back:

The data is updated only in the cache and updated into the memory in later time. Data is updated in the memory only when the cache line is ready to replaced (cache line replacement is done using Belady’s Anomaly, Least Recently Used Algorithm, FIFO, LIFO and others depending on the application).
Write Back is also known as Write Deferred.

• Dirty bit:

Each Block in the cache needs a bit to indicate if the data present in the cache was modified(Dirty) or not modified(Clean).If it is clean there is no need to write it into the memory. It designed to reduce write operation to a memory. If Cache fails or if System fails or power outage the modified data will be lost. Because its nearly impossible to restore data from cache if lost.

If write occurs to a location that is not present in the Cache(Write Miss), we use two options, Write Allocation and Write Around.

In Write Allocation data is loaded from the memory into cache and then updated. Write allocation works with both Write back and Write through.But it is generally used with Write Back because it is unnecessary to bring data from the memory to cache and then updating the data in both cache and main memory. Thus Write Through is often used with No write Allocate.

Here data is Directly written/updated to main memory without disturbing cache.It is better to use this when the data is not immediately used again.

• Read Through:

Read-through cache sits in-line with the database. When there is a cache miss, it loads missing data from database, populates the cache and returns it to the application.

Different ways to handle incoming requests:

1. Thread per request: If you haven’t a lot of requests, you can use this variant. It means you create new thread for every new request. It’s easy but really wasteful in relation to server resources.
2. Thread pool: Like the previous approach you assign thread to every request, but don’t create new one every time, just take existing one from the pool. In this way you save a lot of resources. But if your system has got a huge amount of requests, you will be forced to increase the number of threads in pool, and the server will be discouraged.
3. Event Driven: Have an event loop, one thread spins forever in event loop. it takes new one request from the queue and pass it to perform by any free thread from the thread pool, then “event loop thread” comes back to the loop for getting new events from the queue, “threads from the thread pool” perform the received task and get the data, put the value to callback function, some thread executes this function and the response is sent back.

Design Considerations:

• Decide when to use cache. Consider using cache when data is read frequently but modified infrequently. Since cached data is stored in volatile memory, a cache server is not ideal for persisting data. For instance, if a cache server restarts, all the data in memory is lost. Thus, important data should be saved in persistent data stores.
• Expiration policy. It is a good practice to implement an expiration policy. Once cached data is expired, it is removed from the cache. When there is no expiration policy, cached data will be stored in the memory permanently. It is advisable not to make the expiration date too short as this will cause the system to reload data from the database too frequently. Meanwhile, it is advisable not to make the expiration date too long as the data can become stale.
• Consistency: This involves keeping the data store and the cache in sync. Inconsistency can happen because data-modifying operations on the data store and cache are not in a single transaction. When scaling across multiple regions, maintaining consistency between the data store and cache is challenging.
• Eviction Policy: Once the cache is full, any requests to add items to the cache might cause existing items to be removed. This is called cache eviction. Least-recently-used (LRU) is the most popular cache eviction policy. Other eviction policies, such as the Least Frequently Used (LFU) or First in First Out (FIFO), can be adopted to satisfy different use cases