We recently needed to build a caching system in front of a slow backend system with the following requirements:
- The data in the backend system is constantly being updated so the caches need to be updated every N minutes.
- Requests to the backend system need to be throttled.
The caching system we built used Akka actors and Scala’s support for functions as first class objects.
Creating Caches and Actors
The first piece of the puzzle is a
CacheSystem which creates and queries EhCache caches:
createCacheActor method creates a cache and an actor and schedules tasks to periodically update the cache. We decided to use actors for this because the actor system’s thread pool is a good way to meet the throttling requirement. In addition using the Akka API it is easy to have scheduled tasks send messages to actors.
createCacheActor takes a function of type
(Cache, CacheSystem) => Actor to create the actor. The
findObjectForCache methods take a
finder parameter of type
() => T that handles the lookup of objects from the backend system.
Caching the Business Service
Here is an example of the
CacheSystem being used by the business logic layer:
CachingBusinessService is a caching implementation of the
BusinessService interface. It passes a curried
CacheActor constructor to
createCacheActor() along with the cache parameters to create
CacheActors and their corresponding cache. The caching implementation of the
BusinessService methods send a
FindValue message to the
CacheActor, using the
ask) method which returns an Akka
Future. Then they wait for the result of the future and return it to the caller. Using
Await.result should raise a red flag. You don’t want to block if you don’t have to (especially inside of an actor). However in this case the
BusinessServiceis a blocking API. Before the caching layer was introduced it would block waiting for the back end system to respond.
The FindValue Message and Throttling
CacheActor object defines the
FindValue message and the
Params that it contains.
Params are the parameters for the backend query. Each unique
Params object corresponds to a cache entry so each
Params subclass is responsible for generating the appropriate cache key. The
CacheActor object also defines the implicit
findValueExecutionContext that’s used by the
Future call in
CacheActor.findValueReceive to throttle the number of concurrent calls to the backend system. This
ExecutionContexthas a thread pool with only 25 threads.
Finding Backend Values
CacheActor, the base class of our actor hierarchy, handles
FindValue messages and defines an abstract
finder method for querying the backend system.
findValueReceive either returns a value from the cache or makes a call to the backend (via
CacheSystem.findObjectForCache) to get the value.
Auto-updating the Caches
UpdateCacheForPreviousBusinessDay, messages sent by tasks scheduled in
CacheSystem.createCacheActor, are handled by
During non-business hours values
UpdateCacheForNow messages are ignored and values from the previous business day are returned from the cache. If the app is started during non-business hours an
UpdateCacheForPreviousBusinessDay message is scheduled to populate cache values for the previous business day. A separate thread pool is used to perform the backend system queries for the scheduled
UpdateCacheFor* tasks. We don’t want them to interfere with user requests handled by the
The Concrete CacheActor
Finally here is what a concrete
DateCacheActor would look like. This is the
Service1CacheActor created by
CachingBusinessService for use by the
Service1CacheActor’s implementation of
finder calls the uncached
updateCacheForDate caches the results of the
false variations of
In this post we talked about how to implement an Akka-based auto-updating cache. In the process of building the cache we’ve explored Akka’s Actors, Futures and Scheduler and Scala’s support for functions as first class objects. We also talked about how to use thread pools to throttle requests to a backend system.
About the Author
Eric Pederson (@sourcedelica) is the CTO of CPEX Technology, a technology consultancy based in New York City. Eric has been working in the JVM world since the dot com days and has been working with Scala for most of the last 3 years. He’s currently working for an investment bank on a team building an Akka-based compute grid.