Case Study: An Auto-Updating Cache Using Actors

We recently needed to build a caching system in front of a slow backend system with the following requirements:

  • The data in the backend system is constantly being updated so the caches need to be updated every N minutes.
  • Requests to the backend system need to be throttled.

The caching system we built used Akka actors and Scala’s support for functions as first class objects.

Creating Caches and Actors

The first piece of the puzzle is a CacheSystem which creates and queries EhCache caches:

The createCacheActor method creates a cache and an actor and schedules tasks to periodically update the cache. We decided to use actors for this because the actor system’s thread pool is a good way to meet the throttling requirement. In addition using the Akka API it is easy to have scheduled tasks send messages to actors.  createCacheActor takes a function of  type (Cache, CacheSystem) => Actor to create the actor. The findCachedObject and findObjectForCache methods take a finder parameter of type () => T that handles the lookup of objects from the backend system.  

Caching the Business Service

Here is an example of the CacheSystem being used by the business logic layer:

The CachingBusinessService is a caching implementation of the BusinessService interface. It passes a curried CacheActor constructor to createCacheActor() along with the cache parameters to create CacheActors and their corresponding cache. The caching implementation of the BusinessService methods send a FindValue message to the CacheActor, using the ? (ask) method which returns an Akka Future.  Then they wait for the result of the future and return it to the caller. Using Await.result should raise a red flag. You don’t want to block if you don’t have to (especially inside of an actor). However in this case the BusinessServiceis a blocking API. Before the caching layer was introduced it would block waiting for the back end system to respond.

The FindValue Message and Throttling

The CacheActor object defines the FindValue message and the Params that it contains.  Params are the parameters for the backend query. Each unique Params object corresponds to a cache entry so each Params subclass is responsible for generating the appropriate cache key. The CacheActor object also defines the implicit findValueExecutionContext that’s used by the Future call in CacheActor.findValueReceive to throttle the number of concurrent calls to the backend system. This ExecutionContexthas a thread pool with only 25 threads.  

Finding Backend Values

CacheActor, the base class of our actor hierarchy, handles FindValue messages and defines an abstract finder method for querying the backend system.

findValueReceive either returns a value from the cache or makes a call to the backend (via CacheSystem.findObjectForCache) to get the value.

Auto-updating the Caches

UpdateCacheForNow and UpdateCacheForPreviousBusinessDay, messages sent by tasks scheduled in CacheSystem.createCacheActor, are handled by DateCacheActor:

During non-business hours values UpdateCacheForNow messages are ignored and values from the previous business day are returned from the cache.  If the app is started during non-business hours an UpdateCacheForPreviousBusinessDay message is scheduled to populate cache values for the previous business day. A separate thread pool is used to perform the backend system queries for the scheduled UpdateCacheFor* tasks.   We don’t want them to interfere with user requests handled by the findValueExecutionContext.  

The Concrete CacheActor

Finally here is what a concrete DateCacheActor would look like. This is the Service1CacheActor created by CachingBusinessService for use by the service1 method:

Service1CacheActor's implementation of finder calls the uncached service1 method.  updateCacheForDate caches the results of the true and false variations of service1.

Summary

In this post we talked about how to implement an Akka-based auto-updating cache.   In the process of building the cache we’ve explored Akka’s Actors, Futures and Scheduler and Scala’s support for functions as first class objects.  We also talked about how to use thread pools to throttle requests to a backend system.

About the Author

Eric Pederson (@sourcedelica) is the CTO of CPEX Technology, a technology consultancy based in New York City.  Eric has been working in the JVM world since the dot com days and has been working with Scala for most of the last 3 years.  He’s currently working for an investment bank on a team building an Akka-based compute grid.

Recent comments

Blog comments powered by Disqus