NUBO - an intelligent machine learning framework for rapid optimisation

NUBO, short for Newcastle University Bayesian Optimisation, is a new intelligent machine learning framework for the optimisation of expensive-to-evaluate black-box functions. NUBO optimises complex problems by intelligently searching over the inputs to the problem, approximating the global optimum cost-effectively. NUBO is directed at technologists and researchers from all disciplines and makes it easy to optimise products, physical experiments and computer simulations by focusing primarily on transparency and user experience. Transparency is ensured by clean and comprehensible code, precise references, and thorough documentation. User experience is guaranteed by a modular and flexible design, easy-to-write syntax, and careful selection of Bayesian optimisation algorithms. The package is written in Python but does not require expert knowledge to optimise your products, experiments or simulations. NUBO is developed and maintained by the Fluid Dynamics Lab at Newcastle University and is distributed as an open-source software under the BSD 3-Clause licence. NUBO is developed as part of the EPSRC-funded EnAble project, a collaboration between the Fluid Dynamics Lab at Newcastle University and the Turbulence Simulation Group at Imperial College London and is also supported by AIRBUS, the Aerospace Technology Institute (ATI), and the Department for Transport. Our work is also supported by the EPSRC-funded CDT in Cloud Computing for Big Data. 

Download NUBO and become a user today: www.nubopy.com

​What problems can NUBO solve? ​

​NUBO is best suited for the optimisation of expensive-to-evaluate black box functions. Common examples include computer simulations and physical experiments from a wide range of disciplines, such as engineering, chemistry, biology, and computer science. Expensive-to-evaluate means that performing a simulation or experiment with one set of parameter values generates high costs. This could be material costs, computing resources, human labour, or simply time. A black box function presents a simulation or experiment that cannot be described by mathematical equations and thus no information of its derivation or gradients can be leveraged. The only method of inferring any information about the objective function is to provide it with a set of parameter values and observing its outcome. Many problems from various disciplines can be formulated to fit this description. For example, Bayesian optimisation was used in the field of computational fluid dynamics to maximise the drag reduction via active control of blowing actuators, in chemical engineering for molecular design, drug discovery, molecular modelling, electrolyte design, and additive manufacturing, and in computer science to fine-tune hyper-parameters of machine learning models or in architecture search for neural networks. NUBO supports sequential, parallel, and asynchronous optimisation of parameter spaces that are bounded, constrained, and mixed (continuous and discrete).
​

How does NUBO work? 

Bayesian optimisation is an optimisation algorithm that aims to maximise an objective function, such as computer simulations or physical experiments, in a minimum number of function evaluations. The underlying mathematical expression of the objective function is usually unknown or non-existent and every function evaluation is expensive. Thus, solving such a function requires a cost-effective and sample-efficient optimisation strategy. Bayesian optimisation fulfils these criteria via an optimisation loop that consists of three steps: First, the objective function is represented through a surrogate model based on some available data. Second, this representation is used to find the next point, also called a candidate point, that should be evaluated by maximising a criterion specified through an acquisition function. These functions guide the algorithm and ensure an effective exploration of the parameter space. Third, the new point is evaluated by the objective function, that is the experiment or simulation in question is run again with the new set of parameter values. The loop then restarts using all available data including the newly evaluated data point. With each iteration Bayesian optimisation learns and gathers more information about the objective function. The algorithm is run for as many iterations as the evaluation budget allows, until a satisfying solution is found, or until a predefined stopping criterion is met. ​​

Above is an illustration of how NUBO works: here we have a simple problem with one parameter represented by the dashed white line. In reality, we would not know what this white line looks like, we would be searching over parameter space trying to find its maximum value (i.e. the optimum of the problem). Initially NUBO would take some observation data (blue dots) to have an initial idea on how the function behaves (i.e. some initial data to train the machine learning framework). From these initial observation points, NUBO generates a prediction (red line) on how it thinks the function behaves over the entire parameter space. The blue shaded region represents the uncertainty that NUBO has on its prediction. NUBO then generates an acquisition function (orange shaded region) which trades off exploring the parameter space and exploiting what it already knows to home in on the optimum solution. The acquisition function guides the optimisation and decides where to acquire the next observation point. The new observation point is added to the existing data set to re-evaluate NUBO's prediction, and the associated uncertainty of how the underlying function behaves. NUBO repeats this process until the global optimum is found. ​​

​Why use NUBO?

​With NUBO, our main objective is to make Bayesian optimisation readily accessible to technologists and researchers from all disciplines, both within academia and industry, whose expertise does not necessarily lie in statistics or computer science. That is why NUBO is available for completely free as an open-source framework in Python and focuses on transparency and user experience. This gives users the certainty of knowing exactly what the algorithm they are using does but also makes it easy to implement Bayesian optimisation for their specific experiment or simulation. Transparency is ensured by clean and comprehensible, and well-commented code within the package and precise references and thorough documentation. User experience is achieved by a modular and flexible design, easy-to-write syntax, and careful selection of Bayesian optimisation algorithms. NUBO allows you to tailor Bayesian optimisation to your specific problem by writing the optimisation loop yourself using the provided building blocks. Only algorithms and methods that are sufficiently tested and validated to perform well are included in NUBO. This ensures that the package remains compact and does not overwhelm the user with an unnecessary large number of options.

If you are an industrial user and would like support in using NUBO - Get in touch!

Get in touch

Dr Richard Whalley
School of Engineering, Newcastle University, UK