Skip to main content
en

Surrogate models: how many runs do you really need?

If every run is expensive, trial and error is the costliest method of all. A good surrogate gives the same information with an order of magnitude fewer runs.

Changing one design parameter by hand and running again each time is the most common and least efficient optimisation method there is. Past three parameters the number of trials explodes, and which parameter actually matters is never established.

Design of experiments first, optimisation second

The right order is:

1. Screening. Which parameters genuinely affect the result? A screening study with a small number of runs usually eliminates half the parameters. Each parameter eliminated reduces the cost of the next step exponentially.

2. Surrogate model. For the remaining parameters, a response surface is fitted to CFD results run at selected points. From then on a new solution is not needed for every query.

3. Intelligent sampling. Running at the point the surrogate predicts to be best is not enough; you also have to run where the model is most uncertain. The EGO method of Jones, Schonlau and Welch strikes that balance with an "expected improvement" criterion and markedly reduces the number of trials on expensive functions [1].

A surrogate is a prediction, not a calculation

The most common mistake is treating the surrogate's best point as the answer. That point is a prediction and must be confirmed with a real CFD run. The difference between prediction and confirmation is the measure of how much the surrogate can be trusted in that region.

The second mistake is using the surrogate outside the sampled range. Response surfaces behave unreliably on extrapolation; if the optimum comes out on a boundary, the right answer is to widen the range and resample, not to accept the boundary value.

Every run has to be of the same quality

Because optimisation measures the difference between runs, their consistency matters more than absolute accuracy. That requires:

  • The same meshing strategy and comparable cell quality — if the mesh is regenerated as the geometry changes, variation in mesh error mixes with the design effect
  • The same convergence criterion; a half-converged run enters the surrogate as noise. See the convergence note
  • The same discretisation order. See the solver settings note

Grid independence need not be demonstrated separately for every design; showing it once for the most critical design and using the same strategy throughout is an acceptable approach — but it is an assumption and should be stated. See the grid independence note.

In a multi-objective problem there is no single "best"

Pressure drop against heat transfer, efficiency against noise, weight against strength — these compete. The result is then not a single design but a set of designs, none of which is better than the others in every respect. The choice belongs to the decision, not to the engineering; the job of the analysis is to lay out the set and the trade-off rates clearly.

References

  1. Jones, D. R., Schonlau, M. & Welch, W. J. (1998). Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13(4), 455–492. DOI: 10.1023/A:1008306431147
  2. Roache, P. J. (1994). Perspective: A method for uniform reporting of grid refinement studies. Journal of Fluids Engineering, 116(3), 405–413. DOI: 10.1115/1.2910291

Let's discuss your project

Let us pin down together the system to be analysed and the question that needs answering.

Let's discuss your project