Alexander Forrester & András Sóbester 
Engineering Design via Surrogate Modelling [PDF ebook] 
A Practical Guide

Preface ix

About the Authors xi

Foreword xiii

Prologue xv

Part I Fundamentals 1

1 Sampling Plans 3

1.1 The ‘Curse of Dimensionality’ and How to Avoid It 4

1.2 Physical versus Computational Experiments 4

1.3 Designing Preliminary Experiments (Screening) 6

1.3.1 Estimating the Distribution of Elementary Effects 6

1.4 Designing a Sampling Plan 13

1.4.1 Stratification 13

1.4.2 Latin Squares and Random Latin Hypercubes 15

1.4.3 Space-filling Latin Hypercubes 17

1.4.4 Space-filling Subsets 28

1.5 A Note on Harmonic Responses 29

1.6 Some Pointers for Further Reading 30

References 31

2 Constructing a Surrogate 33

2.1 The Modelling Process 33

2.1.1 Stage One: Preparing the Data and Choosing a Modelling Approach 33

2.1.2 Stage Two: Parameter Estimation and Training 35

2.1.3 Stage Three: Model Testing 36

2.2 Polynomial Models 40

2.2.1 Example One: Aerofoil Drag 42

2.2.2 Example Two: A Multimodal Testcase 44

2.2.3 What About the k-variable Case? 45

2.3 Radial Basis Function Models 45

2.3.1 Fitting Noise-Free Data 45

2.3.2 Radial Basis Function Models of Noisy Data 49

2.4 Kriging 49

2.4.1 Building the Kriging Model 51

2.4.2 Kriging Prediction 59

2.5 Support Vector Regression 63

2.5.1 The Support Vector Predictor 64

2.5.2 The Kernel Trick 67

2.5.3 Finding the Support Vectors 68

2.5.4 Finding μ 70

2.5.5 Choosing C and ε 71

2.5.6 Computing ε: v-SVR 73

2.6 The Big(ger) Picture 75

References 76

3 Exploring and Exploiting a Surrogate 77

3.1 Searching the Surrogate 78

3.2 Infill Criteria 79

3.2.1 Prediction Based Exploitation 79

3.2.2 Error Based Exploration 84

3.2.3 Balanced Exploitation and Exploration 85

3.2.4 Conditional Likelihood Approaches 91

3.2.5 Other Methods 101

3.3 Managing a Surrogate Based Optimization Process 102

3.3.1 Which Surrogate for What Use? 102

3.3.2 How Many Sample Plan and Infill Points? 102

3.3.3 Convergence Criteria 103

3.4 Search of the Vibration Isolator Geometry Feasibility Using Kriging Goal Seeking 104

References 106

Part II Advanced Concepts 109

4 Visualization 111

4.1 Matrices of Contour Plots 112

4.2 Nested Dimensions 114

Reference 116

5 Constraints 117

5.1 Satisfaction of Constraints by Construction 117

5.2 Penalty Functions 118

5.3 Example Constrained Problem 121

5.3.1 Using a Kriging Model of the Constraint Function 121

5.3.2 Using a Kriging Model of the Objective Function 123

5.4 Expected Improvement Based Approaches 125

5.4.1 Expected Improvement with Simple Penalty Function 126

5.4.2 Constrained Expected Improvement 126

5.5 Missing Data 131

5.5.1 Imputing Data for Infeasible Designs 133

5.6 Design of a Helical Compression Spring Using Constrained Expected Improvement 136

5.7 Summary 139

References 139

6 Infill Criteria with Noisy Data 141

6.1 Regressing Kriging 143

6.2 Searching the Regression Model 144

6.2.1 Re-Interpolation 146

6.2.2 Re-Interpolation with Conditional Likelihood Approaches 149

6.3 A Note on Matrix Ill-Conditioning 152

6.4 Summary 152

References 153

7 Exploiting Gradient Information 155

7.1 Obtaining Gradients 155

7.1.1 Finite Differencing 155

7.1.2 Complex Step Approximation 156

7.1.3 Adjoint Methods and Algorithmic Differentiation 156

7.2 Gradient-enhanced Modelling 157

7.3 Hessian-enhanced Modelling 162

7.4 Summary 165

References 165

8 Multi-fidelity Analysis 167

8.1 Co-Kriging 167

8.2 One-variable Demonstration 173

8.3 Choosing Xc and Xe 176

8.4 Summary 177

References 177

9 Multiple Design Objectives 179

9.1 Pareto Optimization 179

9.2 Multi-objective Expected Improvement 182

9.3 Design of the Nowacki Cantilever Beam Using Multi-objective, Constrained Expected Improvement 186

9.4 Design of a Helical Compression Spring Using Multi-objective, Constrained Expected Improvement 191

9.5 Summary 192

References 192

Appendix: Example Problems 195

A.1 One-Variable Test Function 195

A.2 Branin Test Function 196

A.3 Aerofoil Design 197

A.4 The Nowacki Beam 198

A.5 Multi-objective, Constrained Optimal Design of a Helical Compression Spring 200

A.6 Novel Passive Vibration Isolator Feasibility 202

References 203

Index 205

Dr. Alexander I. J. Forrester is Lecturer in Engineering Design at the University of Southampton. His main area of research focuses on improving the efficiency with which expensive analysis (particularly computational fluid dynamics) is used in design. His techniques have been applied to wing aerodynamics, satellite structures, sports equipment design and Formula One.
Dr Andras Sobester is a Lecturer and EPSRC/ Royal Academy of Engineering research Fellow in the School of Engineering Sciences at the University of Southampton. His research interests include aircraft design, aerodynamic shape parameterization and optimization, as well as engineering design technology in general.
Professor Andy J. Keane currently holds the Chair of Computational Engineering at the University of Southampton. He leads the University’s Computational Engineering at the Design Research Group and directs the rolls-Royce University Technology centre for Computational Engineering. His interests lie primarily in the aerospace sciences, with a focus on the design of aerospace systems using computational methods. He has published over two hundred papers and three books in this area, many of which deal with surrogate modelling concepts.