Use historical data to predict whether performance would be better at smaller and larger pool sizes. If there is no historical data for the smaller/larger pool size, flip a coin to predict performance.

Update the historical dataset with the performance (number of tasks completed) in the latest controller interval. Performance is recorded as a weighted moving average for each pool size, so it reflects historical results but adjusts quickly to changing workload characteristics.

Wake up and check whether the threads in the pool seem to be hung (tasks in queue and no tasks completed in preceding interval). If so, increase the pool size (up to the smaller of maxThreads or MAX_THREADS_TO_BREAK_HANG ) and skip to Step 5.

The finer points of the Open Liberty auto-tuning algorithm

Open Liberty’s thread pool controller maintains a set of data about the thread pool performance since the server was started, recording the throughput (tasks completed by the threadpool per controller cycle) at the various pool sizes which have been previously tried. The historical throughput data is then compared to the current cycle’s throughput to decide what the pool size should be going forward. At each cycle the pool size may be increased or decreased incrementally, or left unchanged.

What if there is no historical data to guide the decision? For example, maybe the pool has been growing and is at the largest size tried so far, so there is no data about throughput for larger pool sizes. In that case the controller will 'flip a coin' to decide whether moving in the no-data direction is a good idea. This is analagous to how a human threadpool tuner will try various thread pool sizes to see how they perform, before settling on an optimal value for the configuration and workload.

The number of threads by which the pool size may be changed in a controller cycle is one-half coreThreads or the number of CPUs, whichever is smaller. For example, if Liberty is running on a platform that has 12 CPUs, by default coreThreads is 24 and the controller adjusts the pool size in 12-thread increments. On the same 12-CPU platform, if coreThreads is configured to four threads, the controller adjusts the pool size by two threads at a time. So the coreThreads setting not only determines the minimum number of threads in the pool, but also affects how dynamically the pool size is changed as the thread pool controller runs.

There are a variety of factors besides the default executor pool size which may have transient effects on throughput in the Liberty server, such as variations in workload offered, garbage collection pauses, and perturbations in adjacent systems e.g. database response variability. Due to these factors the relationship between pool size and observed throughput tends to not be perfectly smooth or continuous. Therefore, to improve the 'signal quality' derived from the historical throughput data, the controller considers not just the closest larger/smaller pool size performance, but several increments in each direction.

For example, if the current pool size is 24 and the increment/decrement is two threads, the controller looks at data for 22, 20, 18, and 16 threads to calculate a 'shrink score' (the probability that shrinking the pool is likely to improve performance), and looks at data for 26, 28, 30, and 32 threads to calculate a 'grow score'. This broader range of input to the shrink/grow calculations results in better decisions about pool size in environments where the transaction flow might tend to be lumpy or transaction latency is long or highly variable, as might be common in cloud service scenarios.

In addition to the basic larger/smaller pool throughput evaluation, there are a few heuristics, or rules-of-thumb, applied by the threadpool controller: