Problems brought to you by @Cacheable x RxKotlin on Spring

There are some times when programming is really hard. When you’re debugging code for hours, but still can’t seem to find a solution, or don’t even know where the error is coming from in the first place.

A few hours ago, this evening, I’ve found another such nice problem - and I didn’t even look for it. I was refactoring some of the backend code, which we’re using to distribute routing requests (think: opening google maps and looking up how to get from A to B) to a multitude of different routers. For our corporate mobility analysis use-case we’re often analysing hundreds to thousands of employees in one company, and often sending up to ten requests per person, so naturally this is where two things are important: Asynchronous processing and caching, which we utilise in order to minimize the number of calculations that need to be done by our servers.

The context

The code that’s used for requesting a route from some of our servers of course depends on the routing service used, but generally always looks something like the following piece of code (note that the examples given here are in Kotlin, but are not specific to the programming language):

@Service
class RoutingService(
	val restTemplate: RestTemplate
) {
  @Cacheable(value = ["router-1-single-route"])
  fun getRoute(params: RoutingParams): Single<Route> {
    // build request
    val (from, to, date) = params
    val dateString = date.format(DateTimeFormatter.ofPattern("yyyy-MM-dd"))
    val url = "https://server/route/${from}/${to}?date=${dateString}"
    
    return Single.defer {
      // send request
      Single.just(restTemplate.getForObject(url))
    }
  }
}

Note that getRoute function first synchronously builds a request, that is then asynchronously sent to the server. In a practical scenario, the synchronous part - building the request - takes ussually less than a millisecond. On the other hand, the asynchronous part - sending a request and receiving the calculation response from the router - takes something in the order of a tenth of a second to a second - hence why this part is coded asynchronously.

Aside from the asynchronous calculation, we’re also using the cache abstraction provided by spring¹ in order to cache responses and further decrease calculation time. In the aforementioned code piece, the @Cacheable annotation tells spring that all invocations of this function should be first checked and later loaded from a cache that’s named router-1-single-route. This means, that whenever a request is sent, it’s response (the Route) will be saved in the cache, and if we redo the same request, the route will be loaded from the cache instead, and no request will be sent to the server.

That’s at least what we thought…

What went wrong

We’ve discovered that the code above does, in fact, not return the Route when cached. To be precise, the caching mechanism returns exactly what the method returns - an RxJava (or RxKotlin) Single, that is lazily evaluated.

In less technical terms, what we actually cache is not the response from the server, but a block of code that is responsible for calling the server. Needless to say, this doesn’t speed up anything. Our caching mechanism therefore currently caches the part that is already synchronously executed and takes only very little time, but does not interfere at all with our long-running part of the calculation.

How we can solve this

My quick and dirty solution to solve the problem would have looked something like this:

• Make our existing function getRoute return a Route instead of a Single
• Create another function getAsyncRoute that wraps the call to getRoute in a Single.defer{} call

Unfortunately, the cache abstraction works by creating a proxy class based on the original class, which leads to correct caching behaviour when methods are called from outside the class, not when they are called from the same class. Thus, we have to move getAsyncRoute into its own class and end up with something like this:

@Service
class RoutingService(
	val restTemplate: RestTemplate
) {
  @Cacheable(value = ["router-1-single-route"])
  fun getRoute(params: RoutingParams): Route {
    // build request
    ...
    // send request
    return restTemplate.getForObject(url)
  }
}

@Service
class AsyncRoutingService(
	val routingService: RoutingService
) {
  fun getAsyncRoute(params: RoutingParams): Single<Route> = Single.defer {
    return Single.just(routingService.getRoute(params))
  }
}
    

At least we ended up with a workable solution that only means a minimal overhead of code and can be implemented with some quick code refactoring. My main takeaway from this is, however, that functionality that is seemingly provided by a framework still needs to be tested thoroughly. It seems even though framework code might be running without problems, there’s still a lot of room for getting stuff wrong when reading the docs.

References