5 Tips on Concurrency

1. Never Swallow InterruptedException

Let's check the following code snippet:

public class Task implements Runnable {
  private final BlockingQueue<String> queue = ...;

  @Override
  public void run() {
    while (!Thread.currentThread().isInterrupted()) {
      String result = getOrDefault(() -> queue.poll(1L, TimeUnit.MINUTES), "default");
      //do smth with the result
    }
  }

  <T> T getOrDefault(Callable<T> supplier, T defaultValue) {
    try {
      return supplier.call();
    } catch (Exception e) {
      logger.error("Got exception while retrieving value.", e);
      return defaultValue;
    }
  }
}

The problem with the code is that it is impossible to terminate the thread, while it is waiting for a new element in the queue because the interrupted flag is never restored: 

1. The thread, which is running the code, is interrupted.

2.  BlockingQueue#poll() throws InterruptedException and clears the interrupted flag.

3.  The while loop condition (!Thread.currentThread().isInterrupted()) is true, as the flag was cleared.

To prevent this behavior, always catch  InterruptedException  and restore the interrupted flag when a method throws it either explicitly (by declaring throwing InterruptedException) or implicitly (by declaring/throwing a raw Exception):

<T> T getOrDefault(Callable<T> supplier, T defaultValue) {
  try {
    return supplier.call();
  } catch (InterruptedException e) {
    logger.error("Got interrupted while retrieving value.", e);
    Thread.currentThread().interrupt();
    return defaultValue;
  } catch (Exception e) {
    logger.error("Got exception while retrieving value.", e);
    return defaultValue;
  }
}

2. Use Dedicated Executors for Blocking Operations

Making the whole server irresponsive because of one slow operation is not what developers usually want. Unfortunately, when it comes to RPC, the response time is usually unpredictable.

Let's say that a server has 100 worker threads and there is an endpoint, which is called with 100 RPS. Internally it makes an RPC call, which usually takes 10 milliseconds. At some point in time, the response time of this RPC becomes 2 seconds and the only thing the server is able to do during the spike is to wait for these calls, while other endpoints can not be accessed at all.

@GET
@Path("/genre/{name}")
@Produces(MediaType.APPLICATION_JSON)
public Response getGenre(@PathParam("name") String genreName) {
  Genre genre = potentiallyVerySlowSynchronousCall(genreName);
  return Response.ok(genre).build();
}

The easiest way to solve the problem is to submit the code, which makes blocking calls, to a thread pool:

@GET
@Path("/genre/{name}")
@Produces(MediaType.APPLICATION_JSON)
public void getGenre(@PathParam("name") String genreName, @Suspended AsyncResponse response) {
  response.setTimeout(1L, TimeUnit.SECONDS);
  executorService.submit(() -> {
    Genre genre = potentiallyVerySlowSynchronousCall(genreName);
    return response.resume(Response.ok(genre).build());
  });
}

3. Propagate MDC Values

MDC (Mapped Diagnostic Context) is typically used for storing a single task's specific values. For example, in a web application it might store a request id and a user id for each request, therefore MDC makes it much easier to find the log entries related to a single request or the whole user activity.

2017-08-27 14:38:30,893 INFO [server-thread-0] [requestId=060d8c7f, userId=2928ea66] c.g.s.web.Controller - Message.

Unfortunately, if some part of the code is executed in a dedicated thread pool, the MDC's values from the thread, which submits the task, are not propagated. In the following example, the log entry on line 7 contains 'requestId,' whereas the one on line 9 does not:

@GET
@Path("/genre/{name}")
@Produces(MediaType.APPLICATION_JSON)
public void getGenre(@PathParam("name") String genreName, @Suspended AsyncResponse response) {
  try (MDC.MDCCloseable ignored = MDC.putCloseable("requestId", UUID.randomUUID().toString())) {
    String genreId = getGenreIdbyName(genreName); //Sync call
    logger.trace("Submitting task to find genre with id '{}'.", genreId); //'requestId' is logged
    executorService.submit(() -> {
      logger.trace("Starting task to find genre with id '{}'.", genreId); //'requestId' is not logged
      Response result = getGenre(genreId) //Async call
          .map(artist -> Response.ok(artist).build())
          .orElseGet(() -> Response.status(Response.Status.NOT_FOUND).build());
      response.resume(result);
    });
  }
}

This could be fix by using MDC#getCopyOfContextMap():

...
public void getGenre(@PathParam("name") String genreName, @Suspended AsyncResponse response) {
  try (MDC.MDCCloseable ignored = MDC.putCloseable("requestId", UUID.randomUUID().toString())) {
    ...
    logger.trace("Submitting task to find genre with id '{}'.", genreId); //'requestId' is logged
    withCopyingMdc(executorService, () -> {
      logger.trace("Starting task to find genre with id '{}'.", genreId); //'requestId' is logged
      ...
    });
  }
}

private void withCopyingMdc(ExecutorService executorService, Runnable function) {
  Map<String, String> mdcCopy = MDC.getCopyOfContextMap();
  executorService.submit(() -> {
    MDC.setContextMap(mdcCopy);
    try {
      function.run();
    } finally {
      MDC.clear();
    }
  });
}

4. Change the Name of Threads

Customize the name of threads to simplify reading logs and thread dumps. This could be done by passing a ThreadFactory during the creation of ExecutorService. There are a lot of implementations of the ThreadFactory interface in popular utility libraries:

• com.google.common.util.concurrent.ThreadFactoryBuilder in Guava.
• org.springframework.scheduling.concurrent.CustomizableThreadFactory in Spring.
• org.apache.commons.lang3.concurrent.BasicThreadFactory in Apache Commons Lang 3.

ThreadFactory threadFactory = new BasicThreadFactory.Builder()
  .namingPattern("computation-thread-%d")
  .build();
ExecutorService executorService = Executors.newFixedThreadPool(numberOfThreads, threadFactory);

Although ForkJoinPool does not use ThreadFactory interface, the renaming of threads is also supported:

ForkJoinPool.ForkJoinWorkerThreadFactory forkJoinThreadFactory = pool -> {  
  ForkJoinWorkerThread thread = ForkJoinPool.defaultForkJoinWorkerThreadFactory.newThread(pool);  
  thread.setName("computation-thread-" + thread.getPoolIndex());  
  return thread;
};
ForkJoinPool forkJoinPool = new ForkJoinPool(numberOfThreads, forkJoinThreadFactory, null, false);

Just compare the thread dump with default names:

"pool-1-thread-3" #14 prio=5 os_prio=31 tid=0x00007fc06b19f000 nid=0x5703 runnable [0x0000700001ff9000]
   java.lang.Thread.State: RUNNABLE
at com.github.sorokinigor.article.tipsaboutconcurrency.setthreadsname.TaskHandler.compute(TaskHandler.java:16)
...
"pool-2-thread-3" #15 prio=5 os_prio=31 tid=0x00007fc06aa10800 nid=0x5903 runnable [0x00007000020fc000]
   java.lang.Thread.State: RUNNABLE
at com.github.sorokinigor.article.tipsaboutconcurrency.setthreadsname.HealthCheckCallback.recordFailure(HealthChecker.java:21)
at com.github.sorokinigor.article.tipsaboutconcurrency.setthreadsname.HealthChecker.check(HealthChecker.java:9)
...
"pool-1-thread-2" #12 prio=5 os_prio=31 tid=0x00007fc06aa10000 nid=0x5303 runnable [0x0000700001df3000]
   java.lang.Thread.State: RUNNABLE
at com.github.sorokinigor.article.tipsaboutconcurrency.setthreadsname.TaskHandler.compute(TaskHandler.java:16)
    ...

to the one with meaningful names:

"task-handler-thread-1" #14 prio=5 os_prio=31 tid=0x00007fb49c9df000 nid=0x5703 runnable [0x000070000334a000]
   java.lang.Thread.State: RUNNABLE
at com.github.sorokinigor.article.tipsaboutconcurrency.setthreadsname.TaskHandler.compute(TaskHandler.java:16)
...
"authentication-service-ping-thread-0" #15 prio=5 os_prio=31 tid=0x00007fb49c9de000 nid=0x5903 runnable [0x0000700003247000]
   java.lang.Thread.State: RUNNABLE
at com.github.sorokinigor.article.tipsaboutconcurrency.setthreadsname.HealthCheckCallback.recordFailure(HealthChecker.java:21)
at com.github.sorokinigor.article.tipsaboutconcurrency.setthreadsname.HealthChecker.check(HealthChecker.java:9)
...
"task-handler-thread-0" #12 prio=5 os_prio=31 tid=0x00007fb49b9b5000 nid=0x5303 runnable [0x0000700003144000]
   java.lang.Thread.State: RUNNABLE
at com.github.sorokinigor.article.tipsaboutconcurrency.setthreadsname.TaskHandler.compute(TaskHandler.java:16)
    ...

and imagine that there could be much more than 3 threads.

5. Use LongAdder for Counters

Consider using  java.util.concurrent.atomic.LongAdder instead of AtomicLong/AtomicInteger for counters under high contention. LongAdder maintains the value across several cells and grows their number if it's needed, which leads to higher throughput, but also to higher memory consumption compared to AtomicXX family classes.

LongAdder counter = new LongAdder();
counter.increment();
...
long currentValue = counter.sum();