thread model

thread model

You can see Jooby as an event loop server thanks to the supported web servers: Netty, Jetty and Undertow. Being Netty the default web server.

Jooby isn’t a traditional thread server where a HTTP request is bound to a thread.

In Jooby all the HTTP IO operations are performed in async & non blocking fashion. HTTP IO operations run in an IO thread (a.k.a event loop) while the application logic (your code) always run in a worker thread.

worker threads

The worker thread pool is provided by the web server: Netty, Jetty and Undertow. To simplify application programming you can block a worker thread, for example you can safely run a jdbc query in a worker thread:

{
  get("/search-db", () -> {
    try(Connection db = require(Connection.class)) {
      try(Statement stt = db.createStatement()) {
        ...
      } 
    }
  });
}

Here the web server can accept as many connection it can (as its on non blocking) while the worker thread might blocks.

Default worker thread pool is 20/100. The correct/right size depends on your business and work load your application is suppose to handle. We suggest you to start with default setup and see how it goes, later you can reduce or increase the thread pool.

In Jooby we favor simplicity over complexity that is why your code can block, still there are more advanced setup that allow you to build async and reactive applications.

deferred

Async processing is achieved via: deferred result, with a deferred result an application can produces a result from a thread of its choice:

Script API:

{
  get("/async", deferred(() -> {
    return ...;
  });
}

MVC API:

  @GET
  @Path("/async")
  public Deferred async() {
    return Deferred.deferred(() -> {
      return ...;
    });
  }

Previous examples are just syntax sugar for:

  return new Deferred(deferred -> {
     try {
       deferred.resolve(...);
     } catch (Throwable x) {
       deferred.reject(x);
     }
  });

There is more syntax sugar if you add the AsyncMapper to your application:

Script API:

{
  map(new AsyncMapper());
  
   get("/async", () -> {
     Callable<String> callable = () -> {
       return ...;
     }; 
    return callable;
  });
}

MVC API:

@GET
  @Path("/async")
  public Callable<String> async() {
    return () -> {
      return ...;
    };
  }

The AsyncMapper convert java.util.concurrent.Callable and java.util.concurrent.CompletableFuture objects to deferred objects.

Another important thing to notice is that the deferred run in the caller thread (i.e. worker thread), so by default there is no context switch involve while running a deferred result:

{
  get("/async", () -> {
    String callerThread = Thread.current().getName();
    return Deferred.deferred(() -> {
      assertEquals(callerThread, Thread.current().getName());
      return ...;
    });
  });
}

You might first see this as a bad thing, but is actually a good decision, because:

  • It is super easy to setup a default executor (we will see how soon)

  • Provides better integration with async & reactive libraries. A direct executor avoid the need of switching to a new thread and then probably dispatch (again) to a different thread provided by a library.

executor

As we said before, the default executor run in the caller thread (a.k.a direct executor). Let’s see how to override the default executor:

{
  executor(new ForkJoinPool());

  get("/async", deferred(() -> {
    return ...;
  });
}

Done! Now all our deferred result run in a ForkJoinPool. It also possible to specify an alternative executor:

Script API:

{
  executor(new ForkJoinPool());

  executor("jdbc", Executors.newFixedThreadPool(10));

  get("/", deferred(() -> {
    return ...;
  });

  get("/db", deferred("jdbc", () -> {
    return ...;
  });
}

MVC API:


import static org.jooby.Deferred.deferred;
...

  @GET
  @Path("/")
  public Deferred home() {
    return deferred(() -> {
      return ...;
    });
  }

  @GET
  @Path("/db")
  public Deferred db() {
    return deferred("jdbc", () -> {
      return ...;
    });
  }

Worth mention the executor(ExecutorService) methods automatically shutdown at application shutdown time.

promise

The deferred contains two useful methods:

These two methods allow you to use a deferred object as a promise:

Script API:

{
  get("/", promise(deferred -> {
    try {
      deferred.resolve(...);
    } catch (Throwable x) {
      deferred.reject(x);
    }
  });
}

MVC API:

  @Path("/")
  @GET
  public Deferred promise() {
    return new Deferred(deferred -> {
      try {
        deferred.resolve(...);
      } catch (Throwable x) {
        deferred.reject(x);
      }
    });
  }

The “promise” version of deferred object is a key concept for integrating with external libraries.

advanced configuration

Suppose you want to build a truly async application and after a deep analysis of your business you realize your application need to:

  • Access a database
  • Call a remote service
  • Make a CPU intensive computation

These are the 3 points where your application is suppose to block and wait for a result.

Let’s start by reducing the worker thread pool to the number of available processors:

server.threads.Min = ${runtime.processors}
server.threads.Max = ${runtime.processors}

With this change, you need to be careful and don’t run blocking code on routes anymore. Otherwise performance will be affected.

Let’s create a custom thread pool for each blocking access:

{
  executor("db", Executors.newCachedThreadPool());
  executor("remote", Executors.newFixedThreadPool(32));
  executor("intensive", Executors.newSingleThreadExecutor());
}

For database access, we use a cached executor that will grow without a limit but free and release thread that are idle after 60s.

For remote service, we use a fixed executor of 32 thread. The number here: 32 is just a random number for the purpose of the example.

For intensive computation, we use a single thread executor. Computation is too expensive and we want one and only one running at any time.

{
  executor("db", Executors.newCachedThreadPool());
  executor("remote", Executors.newFixedThreadPool(32));
  executor("intensive", Executors.newSingleThreadExecutor());

  get("/nonblocking", () -> "I'm nonblocking");

  get("/list", deferred("db", () -> {
    Database db = require(Database.class);
    return db.fetch();
  });
  
  get("/remote", deferred("remote", () -> {
    RemoteService rs = require(RemoteService.class);
    return rs.call();
  });

  get("/compute", deferred("intensive", () -> {
    return someCPUIntensiveTask();
  });
}

Here is the same example with rx java:

{
  get("/nonblocking", () -> "I'm nonblocking");

  get("/list", deferred(() -> {
    Database db = require(Database.class);
    Observable.<List<String>> create(s -> {
      s.onNext(db.fetch());
      s.onCompleted();
    }).subscribeOn(Schedulers.io())
      .subscribe(deferred::resolve, deferred::reject);
    }));
  });

  get("/remote", deferred(() -> {
    RemoteService rs = require(RemoteService.class);
    Observable.<List<String>> create(s -> {
      s.onNext(rs.call());
      s.onCompleted();
    }).subscribeOn(Schedulers.io())
      .subscribe(deferred::resolve, deferred::reject);
    }));
  });
  
  get("/compute", deferred(() -> {
    Observable.<List<String>> create(s -> {
      s.onNext(someCPUIntensiveTask());
      s.onCompleted();
    }).subscribeOn(Schedulers.computation())
      .subscribe(deferred::resolve, deferred::reject);
    }));
  });
}

Main difference are:

  • we keep the default executor: direct. So we don’t create a new thread and avoid context switching.
  • we use deferred object as promise and integrate with rx java.
  • different thread pool semantic is done via rx schedulers.

This is just one more example to demonstrate the value of the deferred object, because we provide an rxjava module which takes care of binding deferred object into Observables.

That’s all about deferred object, it allows you to build async and reactive applications and at the same time: keep it simple (Jooby design goal).

Also, we invite you to checkout the available async/reactive modules.

modules

akka

  • akka: build concurrent & distributed applications via akka.

executor

  • executor: async processing via Java Executors

reactor

rxjava

  • rx: build reactive web applications via rxjava

  • rxjdbc: efficient execution, concise code, and functional composition of database calls using JDBC and RxJava Observable

scheduling

  • quartz: advanced job scheduling via {{quartz}}.

Share