HyperConfig: intelligent configuration search

If you have used the cloud, deployed docker containers with Kubernetes or Mesos, then you know that one of the first problem you face is to figure out what’s the best resource configuration of each component you chose. For example, what VM instance type should I use? How many nodes should I deploy to? What container cpu and memory request/limit should I configure? All of these questions imply different trade offs between cost and performance. Using VM size as an concrete example, picking a large VM instance type will cost a lot more, but may give you a better application performance. Choosing a VM that is too small, leads to performance and SLA problems. And it is also not obvious what the correct choice is, since if you take a MySQL tpcc benchmark and run through every AWS instance type, the best performance and cost ratio choice doesn’t follow a linear and predictable pattern:

Also doing an exhaustive search is very time and cost prohibitive. Luckily this isn’t a brand new problem and there are quite a few research solutions out there, but there isn’t a generic open source solution that we can find that supports a generic load test output.

Therefore, we created hyperconfig that is inspired by the work from CherryPick, and we made it to suggest a set of AWS instance types for different criteria based on a generic load test result.

Instead of exhaustively searching through every instance type, HyperConfig uses a well known optimization technique called Bayesian Optimization, to find a near optimal results with only running a lot fewer sample points. And since the samples can also be ran in parallel, it greatly reduces the time and cost to take to find a near optimal instance type. Note that HyperConfig won’t be able to guarantee find the most optimal one, but in practice we found it to find a close enough selection.

For more information about how to run our demo and details about the code, please refer to sizing section of the analyzer.