Overview page

Table showing the ERT in number of evaluations divided by the best ERT measured during BBOB-2009

Last dimension | Dimension = 2 | Next dimension

Δ fopt
f1, 2-D		 1e1
2.0		 1e0
6.0		 1e-1
6.0		 1e-2
6.0		 1e-3
6.0		 1e-5
6.0		 1e-7
6.0		 #succ
15/15
CMA-ES 2019-gecco-surr 2.2 (3)2.6 (3)6.4 (4)12 (5)17 (4)27 (9)35 (5)15/15
lq-CMA-ES 2019-gecco-surr 1.2 (1)1.4 (0.8)1.9 (0.3)2.7 (0.8)3.3 (0.8)4.6 (0.8)5.5 (1)15/15
SLSQP+CMA 2019-gecco-surr 2.1 (2)1.1 (0)1.2 (0)1.2 (0)1.2 (0)1.2 (0)1.2 (0)15/15
SLSQP-11 2019-gecco-surr 2.1 (2)1.1 (0)1.2 (0)1.2 (0)1.2 (0)1.2 (0)1.2 (0)15/15

Δ fopt
f2, 2-D		 1e1
16		 1e0
19		 1e-1
25		 1e-2
25		 1e-3
26		 1e-5
28		 1e-7
29		 #succ
15/15
CMA-ES 2019-gecco-surr 9.5 (6)12 (3)11 (2)12 (2)13 (2)14 (3)15 (3)15/15
lq-CMA-ES 2019-gecco-surr 1.7 (0.6)1.9 (0.7)1.8 (0.4)2.0 (0.3)2.2 (0.2)2.4 (0.3)2.7 (0.4)15/15
SLSQP+CMA 2019-gecco-surr 1.4 (0.5)1.4 (0.5)1.2 (0.4)1.4 (0.5)1.5 (0.5)1.6 (0.4)1.8 (0.6)15/15
SLSQP-11 2019-gecco-surr 1.4 (0.5)1.4 (0.5)1.2 (0.4)1.4 (0.5)1.5 (0.5)1.6 (0.4)1.8 (0.7)15/15

Δ fopt
f3, 2-D		 1e1
15		 1e0
271		 1e-1
445		 1e-2
446		 1e-3
450		 1e-5
454		 1e-7
464		 #succ
15/15
CMA-ES 2019-gecco-surr 2.5 (1)3.7 (4)6.3 (6)6.5 (6)6.6 (6)6.9 (6)7.0 (6)15/15
lq-CMA-ES 2019-gecco-surr 1.5 (0.9)2.0 (2)3.5 (4)3.9 (5)4.0 (6)4.1 (6)4.0 (6)15/15
SLSQP+CMA 2019-gecco-surr 2.5 (2)2.3 (2)5.4 (4)5.7 (4)5.8 (4)6.0 (5)6.2 (5)15/15
SLSQP-11 2019-gecco-surr 3.9 (7)2.6 (2)10 (8)10 (8)10 (7)10 (7)10 (7)15/15

Δ fopt
f4, 2-D		 1e1
22		 1e0
344		 1e-1
459		 1e-2
496		 1e-3
523		 1e-5
544		 1e-7
566		 #succ
15/15
CMA-ES 2019-gecco-surr 3.2 (2)5.3 (10)42 (49)96 (123)108 (125)112 (129)114 (130)15/15
lq-CMA-ES 2019-gecco-surr 1.7 (2)2.0 (2)30 (37)123 (195)126 (200)134 (226)137 (239)14/15
SLSQP+CMA 2019-gecco-surr 2.0 (2)5.6 (9)43 (48)64 (101)69 (116)72 (114)75 (123)15/15
SLSQP-11 2019-gecco-surr 4.4 (7)3.1 (3)10 (8)9.4 (7)9.0 (7)8.6 (7)8.3 (6)15/15

Δ fopt
f5, 2-D		 1e1
4.0		 1e0
4.0		 1e-1
4.0		 1e-2
4.0		 1e-3
4.0		 1e-5
4.0		 1e-7
4.0		 #succ
15/15
CMA-ES 2019-gecco-surr 3.4 (2)6.9 (6)7.7 (6)7.7 (6)7.7 (6)7.7 (6)7.7 (6)15/15
lq-CMA-ES 2019-gecco-surr 1.8 (0.5)2.9 (2)★3.0 (2)★3.0 (2)★3.0 (2)★3.0 (2)★3.0 (2)★15/15
SLSQP+CMA 2019-gecco-surr 1.5 (0)3.5 (0)3.5 (0)3.5 (0)3.5 (0)3.5 (0)3.5 (0)15/15
SLSQP-11 2019-gecco-surr 1.5 (0)3.5 (0)3.5 (0)3.5 (0)3.5 (0)3.5 (0)3.5 (0)15/15

Δ fopt
f6, 2-D		 1e1
13		 1e0
23		 1e-1
41		 1e-2
54		 1e-3
67		 1e-5
95		 1e-7
124		 #succ
15/15
CMA-ES 2019-gecco-surr 2.0 (2)3.3 (2)3.3 (1)3.5 (1.0)3.7 (1.0)3.8 (0.6)4.0 (0.5)15/15
lq-CMA-ES 2019-gecco-surr 1.7 (2)2.1 (2)2.4 (0.9)2.8 (0.7)3.0 (0.8)3.0 (0.6)3.2 (0.4)15/15
SLSQP+CMA 2019-gecco-surr 5.1 (5)3.9 (3)2.5 (2)2.1 (1)2.1 (2)1.9 (1)2.0 (1)15/15
SLSQP-11 2019-gecco-surr 5.2 (5)4.1 (3)2.6 (2)2.2 (1)2.1 (2)1.8 (1)2.0 (1)15/15

Δ fopt
f7, 2-D		 1e1
3.0		 1e0
21		 1e-1
60		 1e-2
193		 1e-3
217		 1e-5
217		 1e-7
241		 #succ
15/15
CMA-ES 2019-gecco-surr 3.0 (4)1.0 (0.9)2.5 (3)0.95 (0.7)0.93 (0.6)0.93 (0.6)0.95 (0.7)15/15
lq-CMA-ES 2019-gecco-surr 1.6 (2)0.74 (0.5)1.2 (2)0.58 (1.0)0.68 (1.0)0.68 (1.0)0.77 (0.9)15/15
SLSQP+CMA 2019-gecco-surr 2.8 (3)2.0 (1)1.8 (0.7)0.76 (0.4)0.76 (0.4)0.76 (0.4)0.80 (0.4)15/15
SLSQP-11 2019-gecco-surr 17 (32)24 (26)89 (110)241 (296)571 (588)571 (486)514 (518)4/15

Δ fopt
f8, 2-D		 1e1
5.0		 1e0
12		 1e-1
37		 1e-2
46		 1e-3
86		 1e-5
94		 1e-7
112		 #succ
15/15
CMA-ES 2019-gecco-surr 3.8 (4)5.6 (5)5.2 (4)6.2 (3)3.9 (2)4.2 (2)4.1 (1)15/15
lq-CMA-ES 2019-gecco-surr 2.4 (3)3.5 (3)2.2 (2)2.6 (1)1.5 (0.6)1.5 (0.6)1.4 (0.5)15/15
SLSQP+CMA 2019-gecco-surr 3.2 (3)3.2 (4)2.0 (2)2.1 (1)1.2 (0.8)1.2 (0.7)1.1 (0.6)15/15
SLSQP-11 2019-gecco-surr 3.2 (3)3.2 (4)2.0 (2)2.1 (1)1.2 (0.8)1.2 (0.7)1.1 (0.6)15/15

Δ fopt
f9, 2-D		 1e1
1		 1e0
18		 1e-1
30		 1e-2
44		 1e-3
68		 1e-5
81		 1e-7
92		 #succ
15/15
CMA-ES 2019-gecco-surr 1 (0)2.9 (3)6.2 (5)6.1 (4)4.6 (2)4.8 (2)4.8 (2)15/15
lq-CMA-ES 2019-gecco-surr 1 (0)2.1 (2)2.7 (3)2.9 (2)2.1 (1)1.9 (1)1.8 (1)15/15
SLSQP+CMA 2019-gecco-surr 1 (0)0.50 (0)↑00.63 (0)↑00.98 (0)↑00.78 (0)↑00.80 (0)↑00.75 (0)↑015/15
SLSQP-11 2019-gecco-surr 1 (0)0.50 (0)↑00.63 (0)↑00.98 (0)↑00.78 (0)↑00.80 (0)↑00.75 (0)↑015/15

Δ fopt
f10, 2-D		 1e1
30		 1e0
46		 1e-1
54		 1e-2
61		 1e-3
68		 1e-5
82		 1e-7
98		 #succ
15/15
CMA-ES 2019-gecco-surr 4.5 (3)4.5 (2)4.6 (1)4.8 (0.9)4.6 (0.9)4.6 (0.8)4.4 (0.5)15/15
lq-CMA-ES 2019-gecco-surr 0.98 (0.6)0.97 (0.4)0.96 (0.3)1.00 (0.3)0.97 (0.2)0.95 (0.3)1.1 (0.3)15/15
SLSQP+CMA 2019-gecco-surr 0.82 (0.3)0.65 (0.2)↑10.61 (0.2)↑00.60 (0.2)↑00.60 (0.2)↑01.5 (3)8.2 (11)15/15
SLSQP-11 2019-gecco-surr 0.82 (0.3)0.65 (0.2)↑10.61 (0.2)↑00.59 (0.2)↑00.57 (0.2)↑038 (3)166 (228)8/15

Δ fopt
f11, 2-D		 1e1
35		 1e0
45		 1e-1
50		 1e-2
62		 1e-3
67		 1e-5
81		 1e-7
97		 #succ
15/15
CMA-ES 2019-gecco-surr 3.4 (3)4.5 (2)5.0 (2)4.4 (1)4.8 (0.9)4.6 (0.9)4.4 (0.7)15/15
lq-CMA-ES 2019-gecco-surr 0.78 (0.3)0.97 (0.4)1.0 (0.3)0.96 (0.2)0.94 (0.2)0.91 (0.2)1.1 (0.3)15/15
SLSQP+CMA 2019-gecco-surr 0.67 (0.3)0.61 (0.3)↑10.63 (0.2)↑00.59 (0.2)↑00.63 (0.2)↑08.6 (27)19 (23)15/15
SLSQP-11 2019-gecco-surr 0.67 (0.3)0.61 (0.3)↑10.63 (0.2)↑00.59 (0.2)↑00.63 (0.2)↑083 (237)288 (434)8/15

Δ fopt
f12, 2-D		 1e1
35		 1e0
46		 1e-1
75		 1e-2
94		 1e-3
105		 1e-5
153		 1e-7
195		 #succ
15/15
CMA-ES 2019-gecco-surr 4.2 (2)4.3 (2)3.7 (1)4.0 (1)4.0 (1)3.8 (1)3.8 (3)15/15
lq-CMA-ES 2019-gecco-surr 1.0 (0.7)1.9 (3)2.0 (4)2.3 (4)2.3 (4)1.9 (3)1.8 (2)15/15
SLSQP+CMA 2019-gecco-surr 1.1 (0.8)1.0 (0.6)0.64 (0.3)↑00.52 (0.3)↑00.48 (0.2)↑00.37 (0.2)↑01.2 (2)15/15
SLSQP-11 2019-gecco-surr 1.1 (0.8)1.0 (0.6)0.64 (0.3)↑00.52 (0.3)↑00.48 (0.2)↑00.37 (0.2)↑01.5 (3)15/15

Δ fopt
f13, 2-D		 1e1
23		 1e0
35		 1e-1
46		 1e-2
60		 1e-3
71		 1e-5
95		 1e-7
122		 #succ
15/15
CMA-ES 2019-gecco-surr 2.9 (2)3.4 (1)3.8 (2)4.1 (0.9)4.3 (0.8)4.4 (0.5)4.3 (0.5)15/15
lq-CMA-ES 2019-gecco-surr 1.0 (0.4)1.1 (0.4)1.1 (0.5)1.1 (0.3)1.1 (0.3)1.3 (0.4)2.1 (0.5)15/15
SLSQP+CMA 2019-gecco-surr 0.90 (0.3)0.79 (0.2)↑10.82 (0.2)0.76 (0.1)↑10.75 (0.1)↑01.4 (3)5.0 (3)15/15
SLSQP-11 2019-gecco-surr 0.90 (0.3)0.79 (0.2)↑10.82 (0.2)0.76 (0.1)↑10.75 (0.1)↑00.87 (0.6)56 (172)9/15

Δ fopt
f14, 2-D		 1e1
1		 1e0
7.0		 1e-1
16		 1e-2
24		 1e-3
38		 1e-5
67		 1e-7
90		 #succ
15/15
CMA-ES 2019-gecco-surr 3.3 (4)2.1 (2)3.2 (3)4.6 (2)4.1 (1)4.2 (0.9)4.6 (1)15/15
lq-CMA-ES 2019-gecco-surr 2.3 (2)1.6 (1)1.5 (0.5)1.6 (0.7)1.5 (0.5)1.3 (0.3)1.6 (0.4)15/15
SLSQP+CMA 2019-gecco-surr 3.8 (3)1.6 (1)1.3 (0.6)1.4 (0.4)1.1 (0.4)0.97 (0.2)3.8 (2)15/15
SLSQP-11 2019-gecco-surr 3.8 (3)1.6 (1)1.3 (0.6)1.4 (0.4)1.1 (0.4)0.97 (0.2)3.5 (4)14/15

Δ fopt
f15, 2-D		 1e1
37		 1e0
291		 1e-1
1033		 1e-2
1066		 1e-3
1113		 1e-5
1231		 1e-7
1412		 #succ
15/15
CMA-ES 2019-gecco-surr 1.1 (1)1.5 (1)2.5 (3)3.2 (3)3.2 (4)3.0 (3)2.7 (3)15/15
lq-CMA-ES 2019-gecco-surr 0.71 (0.5)1.4 (2)1.6 (1)1.7 (2)1.7 (2)1.6 (1)1.4 (1)15/15
SLSQP+CMA 2019-gecco-surr 1.5 (1)2.5 (2)2.0 (1)2.2 (1)2.1 (1)2.0 (1)1.8 (1)15/15
SLSQP-11 2019-gecco-surr 2.9 (4)2.7 (3)2.4 (2)2.4 (2)2.3 (2)2.1 (2)1.8 (1)15/15

Δ fopt
f16, 2-D		 1e1
9.0		 1e0
50		 1e-1
174		 1e-2
326		 1e-3
358		 1e-5
409		 1e-7
538		 #succ
15/15
CMA-ES 2019-gecco-surr 1.7 (2)4.8 (11)2.3 (3)1.4 (2)1.4 (2)1.4 (1)1.2 (1)15/15
lq-CMA-ES 2019-gecco-surr 3.1 (3)4.0 (6)4.3 (6)2.7 (3)2.9 (3)2.6 (3)2.1 (2)15/15
SLSQP+CMA 2019-gecco-surr 13 (10)11 (10)4.8 (4)2.7 (2)2.6 (2)2.9 (3)2.4 (3)15/15
SLSQP-11 2019-gecco-surr 430 (1470)513 (690)457 (437)599 (1042)1850 (1871)∞∞4e40/15

Δ fopt
f17, 2-D		 1e1
3.0		 1e0
61		 1e-1
133		 1e-2
275		 1e-3
396		 1e-5
1086		 1e-7
1657		 #succ
15/15
CMA-ES 2019-gecco-surr 1.6 (2)0.62 (0.3)↑11.0 (0.5)0.90 (0.4)1.2 (1)1.1 (1.0)1.0 (0.8)15/15
lq-CMA-ES 2019-gecco-surr 2.3 (3)0.82 (0.5)1.3 (0.7)1.3 (1)1.2 (1)1.6 (1)1.4 (0.5)15/15
SLSQP+CMA 2019-gecco-surr 6.6 (7)2.5 (4)1.7 (2)1.1 (1.0)1.2 (0.7)1.5 (1)1.3 (0.8)15/15
SLSQP-11 2019-gecco-surr 11 (16)14 (12)72 (61)2121 (2867)∞∞∞4e40/15

Δ fopt
f18, 2-D		 1e1
19		 1e0
134		 1e-1
666		 1e-2
1249		 1e-3
1708		 1e-5
2438		 1e-7
2858		 #succ
15/15
CMA-ES 2019-gecco-surr 1.6 (1)2.4 (3)1.1 (1)0.78 (0.7)0.75 (0.6)1.00 (0.5)0.95 (0.4)15/15
lq-CMA-ES 2019-gecco-surr 1.2 (0.7)1.4 (2)0.75 (0.9)0.61 (0.6)0.62 (0.5)0.96 (0.8)1.0 (0.7)15/15
SLSQP+CMA 2019-gecco-surr 5.2 (5)4.9 (3)1.3 (1)0.94 (0.8)0.80 (0.6)0.99 (0.5)1.1 (0.8)15/15
SLSQP-11 2019-gecco-surr 10 (12)59 (80)103 (87)501 (326)366 (319)∞∞4e40/15

Δ fopt
f19, 2-D		 1e1
1		 1e0
1		 1e-1
26		 1e-2
216		 1e-3
227		 1e-5
252		 1e-7
276		 #succ
15/15
CMA-ES 2019-gecco-surr 1 (0)1 (0)12 (25)17 (20)18 (19)17 (19)16 (18)15/15
lq-CMA-ES 2019-gecco-surr 1 (0)1 (0)6.2 (5)10 (9)10 (9)11 (14)10 (13)15/15
SLSQP+CMA 2019-gecco-surr 1 (0)1 (0)0.54 (0)↑015 (20)17 (19)18 (23)17 (21)15/15
SLSQP-11 2019-gecco-surr 1 (0)1 (0)0.54 (0)↑0∞∞∞∞4e40/15

Δ fopt
f20, 2-D		 1e1
4.0		 1e0
61		 1e-1
365		 1e-2
366		 1e-3
366		 1e-5
370		 1e-7
375		 #succ
15/15
CMA-ES 2019-gecco-surr 2.3 (2)13 (13)5.0 (5)7.3 (9)7.6 (9)8.0 (10)8.2 (10)15/15
lq-CMA-ES 2019-gecco-surr 1.1 (0.6)3.2 (4)6.5 (11)8.3 (14)9.3 (14)9.3 (14)9.3 (14)15/15
SLSQP+CMA 2019-gecco-surr 2.5 (0)17 (13)6.1 (5)7.4 (6)7.7 (6)8.0 (6)8.2 (6)15/15
SLSQP-11 2019-gecco-surr 2.5 (0)4.7 (7)21 (35)21 (34)21 (34)21 (34)21 (34)15/15

Δ fopt
f21, 2-D		 1e1
2.0		 1e0
51		 1e-1
174		 1e-2
276		 1e-3
290		 1e-5
324		 1e-7
330		 #succ
15/15
CMA-ES 2019-gecco-surr 1.1 (0.8)6.4 (11)2.8 (3)2.0 (2)2.0 (2)1.9 (2)2.0 (2)15/15
lq-CMA-ES 2019-gecco-surr 1.3 (0.5)3.5 (6)3.4 (6)2.3 (4)2.9 (4)2.7 (4)2.8 (4)15/15
SLSQP+CMA 2019-gecco-surr 2.5 (2)8.1 (8)4.4 (4)2.9 (3)2.9 (2)2.7 (2)2.8 (2)15/15
SLSQP-11 2019-gecco-surr 2.5 (2)3.2 (3)1.3 (1)0.88 (0.9)0.86 (0.8)0.83 (0.7)0.89 (0.7)15/15

Δ fopt
f22, 2-D		 1e1
5.0		 1e0
27		 1e-1
168		 1e-2
218		 1e-3
249		 1e-5
289		 1e-7
306		 #succ
15/15
CMA-ES 2019-gecco-surr 1.4 (2)6.8 (12)8.9 (14)16 (19)33 (17)107 (299)157 (296)12/15
lq-CMA-ES 2019-gecco-surr 1.0 (1)5.6 (15)2.8 (4)4.8 (6)6.1 (10)31 (54)39 (65)13/15
SLSQP+CMA 2019-gecco-surr 2.1 (2)3.8 (3)6.3 (13)13 (18)12 (16)49 (15)46 (16)14/15
SLSQP-11 2019-gecco-surr 2.1 (2)4.2 (7)1.5 (1)1.2 (1)1.1 (1)1.0 (0.9)1.0 (0.9)15/15

Δ fopt
f23, 2-D		 1e1
8.0		 1e0
193		 1e-1
234		 1e-2
263		 1e-3
299		 1e-5
348		 1e-7
379		 #succ
15/15
CMA-ES 2019-gecco-surr 1.9 (2)4.3 (3)6.1 (4)5.8 (4)5.3 (3)5.0 (3)5.0 (3)15/15
lq-CMA-ES 2019-gecco-surr 1.5 (1)5.0 (6)9.3 (7)8.7 (6)8.3 (6)7.5 (5)7.4 (5)15/15
SLSQP+CMA 2019-gecco-surr 16 (16)6.3 (8)10 (12)10 (10)9.2 (9)8.2 (8)8.1 (7)15/15
SLSQP-11 2019-gecco-surr 9.2 (13)10 (13)16 (22)23 (33)31 (46)111 (155)∞4e40/15

Δ fopt
f24, 2-D		 1e1
18		 1e0
857		 1e-1
8515		 1e-2
23399		 1e-3
24113		 1e-5
24721		 1e-7
24721		 #succ
5/15
CMA-ES 2019-gecco-surr 2.0 (2)6.5 (8)189 (181)∞∞∞∞6e50/15
lq-CMA-ES 2019-gecco-surr 1.3 (0.9)64 (80)77 (148)99 (100)96 (79)93 (92)93 (145)3/15
SLSQP+CMA 2019-gecco-surr 2.8 (2)99 (155)133 (197)67 (38)65 (77)63 (83)63 (81)4/15
SLSQP-11 2019-gecco-surr 4.2 (4)1.7 (2)1.5 (1)3.4 (4)3.3 (4)3.3 (4)3.3 (3)6/15

Previous dimension | Dimension = 3 | Next dimension

Δ fopt
f1, 3-D		 1e1
4.0		 1e0
8.0		 1e-1
8.0		 1e-2
8.0		 1e-3
8.0		 1e-5
8.0		 1e-7
8.0		 #succ
15/15
CMA-ES 2019-gecco-surr 2.1 (2)4.9 (4)11 (3)16 (4)21 (5)32 (6)43 (7)15/15
lq-CMA-ES 2019-gecco-surr 1.2 (1)1.8 (0.3)2.1 (0.4)3.0 (0.7)3.7 (0.8)4.9 (0.9)6.2 (0.6)15/15
SLSQP+CMA 2019-gecco-surr 1.6 (0.9)1 (0)1 (0)1 (0)1 (0)1 (0)1 (0)15/15
SLSQP-11 2019-gecco-surr 1.6 (0.9)1 (0)1 (0)1 (0)1 (0)1 (0)1 (0)15/15

Δ fopt
f2, 3-D		 1e1
38		 1e0
42		 1e-1
43		 1e-2
44		 1e-3
45		 1e-5
47		 1e-7
48		 #succ
15/15
CMA-ES 2019-gecco-surr 7.0 (2)9.0 (2)10 (2)11 (2)12 (2)13 (2)15 (2)15/15
lq-CMA-ES 2019-gecco-surr 1.1 (0.3)1.3 (0.4)1.4 (0.3)1.6 (0.3)1.6 (0.3)1.8 (0.3)2.0 (0.3)15/15
SLSQP+CMA 2019-gecco-surr 0.98 (0.2)1.1 (0.3)1.2 (0.3)1.3 (0.3)1.4 (0.4)1.6 (0.6)1.7 (0.5)15/15
SLSQP-11 2019-gecco-surr 0.98 (0.2)1.1 (0.3)1.2 (0.3)1.3 (0.3)1.4 (0.4)1.6 (0.6)1.8 (0.6)15/15

Δ fopt
f3, 3-D		 1e1
38		 1e0
822		 1e-1
830		 1e-2
835		 1e-3
842		 1e-5
847		 1e-7
853		 #succ
15/15
CMA-ES 2019-gecco-surr 3.7 (3)3.7 (3)23 (23)66 (118)69 (126)71 (131)73 (133)15/15
lq-CMA-ES 2019-gecco-surr 2.4 (2)2.6 (3)13 (8)22 (9)23 (9)23 (9)23 (9)15/15
SLSQP+CMA 2019-gecco-surr 4.0 (3)2.1 (2)25 (38)27 (43)47 (42)50 (52)51 (54)15/15
SLSQP-11 2019-gecco-surr 15 (26)6.8 (10)47 (38)47 (44)47 (44)46 (40)46 (43)12/15

Δ fopt
f4, 3-D		 1e1
40		 1e0
808		 1e-1
866		 1e-2
921		 1e-3
952		 1e-5
1015		 1e-7
1044		 #succ
15/15
CMA-ES 2019-gecco-surr 9.0 (14)576 (710)4402 (5731)4140 (4747)4005 (4382)3757 (5337)3652 (3524)2/15
lq-CMA-ES 2019-gecco-surr 4.4 (2)232 (611)3730 (2453)3507 (5259)3393 (3318)3182 (3926)3094 (2634)2/15
SLSQP+CMA 2019-gecco-surr 5.6 (4)775 (594)9766 (9161)∞∞∞∞6e50/15
SLSQP-11 2019-gecco-surr 21 (24)39 (38)328 (299)309 (429)299 (383)280 (225)272 (336)3/15

Δ fopt
f5, 3-D		 1e1
7.0		 1e0
7.0		 1e-1
7.0		 1e-2
7.0		 1e-3
7.0		 1e-5
7.0		 1e-7
7.0		 #succ
15/15
CMA-ES 2019-gecco-surr 3.4 (1)5.9 (2)6.2 (2)6.4 (3)6.4 (3)6.4 (3)6.4 (3)15/15
lq-CMA-ES 2019-gecco-surr 1.4 (0.6)2.4 (0.8)★2.6 (1)2.6 (1)2.6 (1)2.6 (1)2.6 (1)15/15
SLSQP+CMA 2019-gecco-surr 1 (0)3.1 (0)3.1 (0)3.1 (0)3.1 (0)3.1 (0)3.1 (0)15/15
SLSQP-11 2019-gecco-surr 1 (0)3.1 (0)3.1 (0)3.1 (0)3.1 (0)3.1 (0)3.1 (0)15/15

Δ fopt
f6, 3-D		 1e1
34		 1e0
56		 1e-1
90		 1e-2
117		 1e-3
149		 1e-5
215		 1e-7
265		 #succ
15/15
CMA-ES 2019-gecco-surr 1.8 (1)2.9 (1)3.0 (0.8)3.1 (0.7)3.0 (0.7)2.9 (0.7)3.1 (0.5)15/15
lq-CMA-ES 2019-gecco-surr 1.6 (0.8)2.1 (0.7)2.1 (0.7)2.2 (0.4)2.3 (0.4)2.3 (0.3)2.5 (0.3)15/15
SLSQP+CMA 2019-gecco-surr 4.7 (4)3.5 (2)2.6 (1)2.3 (1)2.2 (1)2.1 (1.0)2.7 (2)15/15
SLSQP-11 2019-gecco-surr 4.4 (3)3.3 (2)2.6 (2)2.4 (1)2.3 (1)2.0 (0.9)2.4 (0.7)15/15

Δ fopt
f7, 3-D		 1e1
11		 1e0
65		 1e-1
342		 1e-2
464		 1e-3
482		 1e-5
482		 1e-7
535		 #succ
15/15
CMA-ES 2019-gecco-surr 2.4 (2)2.0 (2)0.69 (0.9)0.65 (0.6)0.72 (0.6)0.72 (0.6)0.69 (0.5)15/15
lq-CMA-ES 2019-gecco-surr 1.2 (0.7)0.38 (0.2)★2↑00.38 (0.5)0.34 (0.4)↑10.40 (0.4)↑10.40 (0.4)↑10.39 (0.4)↑115/15
SLSQP+CMA 2019-gecco-surr 3.6 (3)1.7 (0.7)0.86 (0.6)0.92 (0.6)0.94 (0.5)0.94 (0.5)0.88 (0.5)15/15
SLSQP-11 2019-gecco-surr 16 (18)190 (306)324 (387)1846 (1829)1777 (2012)1777 (1604)1601 (2210)1/15

Δ fopt
f8, 3-D		 1e1
27		 1e0
45		 1e-1
152		 1e-2
179		 1e-3
188		 1e-5
198		 1e-7
208		 #succ
15/15
CMA-ES 2019-gecco-surr 2.3 (2)5.3 (4)3.0 (1)3.1 (1)3.3 (1)3.6 (1)3.9 (1.0)15/15
lq-CMA-ES 2019-gecco-surr 1.4 (1)2.3 (2)1.00 (0.5)0.96 (0.4)0.98 (0.4)1.00 (0.4)1.00 (0.4)15/15
SLSQP+CMA 2019-gecco-surr 1.3 (0.4)2.2 (2)0.95 (0.6)0.92 (0.5)0.94 (0.5)0.95 (0.5)0.94 (0.4)15/15
SLSQP-11 2019-gecco-surr 1.3 (0.4)2.2 (2)0.95 (0.6)0.94 (0.6)0.96 (0.5)0.96 (0.5)0.95 (0.5)15/15

Δ fopt
f9, 3-D		 1e1
21		 1e0
65		 1e-1
127		 1e-2
149		 1e-3
159		 1e-5
169		 1e-7
178		 #succ
15/15
CMA-ES 2019-gecco-surr 2.8 (2)4.0 (3)3.6 (2)3.9 (2)4.0 (1)4.3 (1)4.6 (1)15/15
lq-CMA-ES 2019-gecco-surr 1.6 (1)1.3 (0.7)1.1 (0.3)1.1 (0.3)1.0 (0.3)1.1 (0.3)1.1 (0.2)15/15
SLSQP+CMA 2019-gecco-surr 0.48 (0)↑00.15 (0)↑00.44 (0)↑00.48 (0)↑00.55 (0)↑00.57 (0)↑00.57 (0)↑015/15
SLSQP-11 2019-gecco-surr 0.48 (0)↑00.15 (0)↑00.44 (0)↑00.48 (0)↑00.55 (0)↑00.57 (0)↑00.57 (0)↑015/15

Δ fopt
f10, 3-D		 1e1
114		 1e0
152		 1e-1
168		 1e-2
180		 1e-3
194		 1e-5
218		 1e-7
242		 #succ
15/15
CMA-ES 2019-gecco-surr 2.2 (0.8)2.3 (0.5)2.5 (0.5)2.5 (0.4)2.6 (0.3)2.7 (0.2)2.8 (0.3)15/15
lq-CMA-ES 2019-gecco-surr 0.41 (0.1)↑00.40 (0.1)↑00.40 (0.1)↑00.41 (0.1)↑00.41 (0.1)↑00.41 (0.1)↑00.53 (0.1)↑015/15
SLSQP+CMA 2019-gecco-surr 0.33 (0.1)↑00.28 (0.1)↑00.29 (0.1)↑00.31 (0.1)↑00.31 (0.1)↑00.93 (2)5.3 (10)15/15
SLSQP-11 2019-gecco-surr 0.33 (0.1)↑00.29 (0.1)↑00.30 (0.1)↑00.31 (0.1)↑00.32 (0.1)↑00.41 (0.2)↑1108 (187)9/15

Δ fopt
f11, 3-D		 1e1
67		 1e0
105		 1e-1
227		 1e-2
263		 1e-3
277		 1e-5
302		 1e-7
327		 #succ
15/15
CMA-ES 2019-gecco-surr 4.0 (2)3.4 (0.9)1.8 (0.4)1.7 (0.3)1.8 (0.3)1.9 (0.3)2.1 (0.3)15/15
lq-CMA-ES 2019-gecco-surr 0.68 (0.2)↑00.54 (0.1)↑00.28 (0.0)↑00.27 (0.0)↑00.27 (0.0)↑00.29 (0.0)↑00.40 (0.1)★↑015/15
SLSQP+CMA 2019-gecco-surr 0.44 (0.2)↑00.34 (0.1)↑00.19 (0.1)↑00.19 (0.1)↑00.21 (0.1)↑01.5 (1)6.3 (8)15/15
SLSQP-11 2019-gecco-surr 0.44 (0.2)↑00.34 (0.1)↑00.19 (0.1)↑00.19 (0.1)↑00.21 (0.1)↑033 (74)169 (361)5/15

Δ fopt
f12, 3-D		 1e1
65		 1e0
168		 1e-1
338		 1e-2
401		 1e-3
445		 1e-5
696		 1e-7
790		 #succ
15/15
CMA-ES 2019-gecco-surr 6.8 (8)5.1 (7)4.0 (5)4.0 (5)4.0 (5)3.2 (4)4.1 (6)15/15
lq-CMA-ES 2019-gecco-surr 1.2 (0.3)0.81 (0.6)0.61 (0.4)↑10.66 (0.4)↑10.71 (0.4)0.74 (0.8)1.6 (1)15/15
SLSQP+CMA 2019-gecco-surr 2.3 (3)1.5 (2)1.0 (1)1.0 (1)1.0 (1)2.2 (4)3.9 (5)15/15
SLSQP-11 2019-gecco-surr 2.3 (3)1.5 (2)1.0 (1)1.0 (1)1.0 (1)1.2 (1)4.6 (6)9/15

Δ fopt
f13, 3-D		 1e1
49		 1e0
85		 1e-1
108		 1e-2
136		 1e-3
215		 1e-5
281		 1e-7
365		 #succ
15/15
CMA-ES 2019-gecco-surr 2.8 (0.8)2.9 (0.9)3.3 (0.6)3.4 (0.5)2.7 (0.5)2.8 (0.3)2.6 (0.2)15/15
lq-CMA-ES 2019-gecco-surr 0.71 (0.1)★↑10.64 (0.2)★↑00.64 (0.1)★2↑00.64 (0.1)★2↑00.47 (0.1)★3↑00.67 (0.2)★3↑01.2 (0.3)★415/15
SLSQP+CMA 2019-gecco-surr 0.89 (0.2)0.77 (0.1)↑00.79 (0.1)↑00.77 (0.1)↑03.6 (11)12 (9)12 (5)15/15
SLSQP-11 2019-gecco-surr 0.89 (0.2)0.77 (0.1)↑00.79 (0.1)↑00.77 (0.1)↑041 (104)287 (548)∞9e40/15

Δ fopt
f14, 3-D		 1e1
2.0		 1e0
17		 1e-1
28		 1e-2
43		 1e-3
71		 1e-5
110		 1e-7
194		 #succ
15/15
CMA-ES 2019-gecco-surr 1.7 (2)2.6 (1)3.5 (1)4.1 (0.8)3.6 (0.7)3.9 (0.7)3.2 (0.5)15/15
lq-CMA-ES 2019-gecco-surr 2.2 (2)1.2 (0.7)1.2 (0.5)1.1 (0.3)0.90 (0.2)0.90 (0.1)1.0 (0.4)★215/15
SLSQP+CMA 2019-gecco-surr 2.6 (2)0.90 (0.6)0.83 (0.4)0.85 (0.3)0.74 (0.3)0.75 (0.2)↑07.4 (12)15/15
SLSQP-11 2019-gecco-surr 2.6 (2)0.90 (0.6)0.83 (0.4)0.85 (0.3)0.74 (0.3)0.75 (0.2)↑038 (44)11/15

Δ fopt
f15, 3-D		 1e1
121		 1e0
1372		 1e-1
6285		 1e-2
8282		 1e-3
8429		 1e-5
8787		 1e-7
9041		 #succ
15/15
CMA-ES 2019-gecco-surr 1.3 (0.6)1.2 (1)0.99 (1)0.78 (0.9)0.80 (1)0.79 (1.0)0.81 (1)15/15
lq-CMA-ES 2019-gecco-surr 1.2 (0.7)0.87 (0.8)0.58 (0.5)0.45 (0.4)↑10.45 (0.4)↑10.44 (0.4)↑10.43 (0.4)↑115/15
SLSQP+CMA 2019-gecco-surr 1.5 (0.9)1.1 (1.0)1.1 (0.8)0.84 (0.7)0.84 (0.7)0.84 (0.6)0.85 (0.7)15/15
SLSQP-11 2019-gecco-surr 5.0 (5)6.4 (7)9.2 (12)7.0 (6)6.8 (7)6.6 (7)6.4 (8)10/15

Δ fopt
f16, 3-D		 1e1
41		 1e0
319		 1e-1
582		 1e-2
789		 1e-3
1864		 1e-5
3204		 1e-7
3361		 #succ
15/15
CMA-ES 2019-gecco-surr 1.6 (0.9)2.4 (3)4.1 (6)3.9 (5)1.7 (2)1.1 (1)1.1 (1)15/15
lq-CMA-ES 2019-gecco-surr 1.1 (1)1.3 (2)2.6 (4)2.4 (3)1.0 (1)0.68 (0.8)0.67 (0.7)15/15
SLSQP+CMA 2019-gecco-surr 19 (28)3.6 (6)4.1 (4)4.0 (3)1.8 (1)1.1 (0.9)1.1 (0.8)15/15
SLSQP-11 2019-gecco-surr 66 (85)367 (384)1750 (1943)∞∞∞∞7e40/15

Δ fopt
f17, 3-D		 1e1
4.0		 1e0
78		 1e-1
282		 1e-2
491		 1e-3
1134		 1e-5
2347		 1e-7
3469		 #succ
15/15
CMA-ES 2019-gecco-surr 0.95 (1)1.5 (0.7)1.2 (0.4)1.3 (1)1.1 (0.8)1.0 (0.9)0.96 (0.4)15/15
lq-CMA-ES 2019-gecco-surr 1.3 (2)1.2 (0.5)0.96 (1)0.82 (0.7)0.75 (0.5)0.74 (0.4)0.60 (0.3)↑115/15
SLSQP+CMA 2019-gecco-surr 2.7 (2)3.1 (0.8)1.5 (2)1.3 (2)0.95 (0.8)0.95 (0.5)0.86 (0.7)15/15
SLSQP-11 2019-gecco-surr 2.1 (2)46 (72)∞∞∞∞∞6e40/15

Δ fopt
f18, 3-D		 1e1
40		 1e0
145		 1e-1
1289		 1e-2
3084		 1e-3
3523		 1e-5
4738		 1e-7
5527		 #succ
15/15
CMA-ES 2019-gecco-surr 2.6 (2)2.3 (3)0.90 (0.8)0.47 (0.4)↑10.46 (0.3)↑10.57 (0.4)0.66 (0.4)15/15
lq-CMA-ES 2019-gecco-surr 0.87 (1)3.4 (5)0.94 (0.6)0.56 (0.3)↑10.55 (0.3)↑10.58 (0.3)↑10.66 (0.3)15/15
SLSQP+CMA 2019-gecco-surr 4.8 (5)3.8 (5)0.65 (0.6)0.65 (0.7)0.68 (0.6)0.74 (0.4)0.81 (0.3)15/15
SLSQP-11 2019-gecco-surr 18 (22)582 (375)∞∞∞∞∞6e40/15

Δ fopt
f19, 3-D		 1e1
1		 1e0
1		 1e-1
109		 1e-2
6764		 1e-3
7367		 1e-5
7399		 1e-7
7441		 #succ
15/15
CMA-ES 2019-gecco-surr 1 (0)1 (0)42 (34)2.7 (3)2.6 (3)2.6 (3)2.6 (3)15/15
lq-CMA-ES 2019-gecco-surr 1 (0)1 (0)36 (48)2.2 (3)2.6 (3)2.6 (3)2.6 (3)15/15
SLSQP+CMA 2019-gecco-surr 1 (0)1 (0)0.14 (0)↑01.7 (2)2.6 (3)2.6 (4)2.6 (4)15/15
SLSQP-11 2019-gecco-surr 1 (0)1 (0)0.14 (0)↑0∞∞∞∞9e40/15

Δ fopt
f20, 3-D		 1e1
8.0		 1e0
385		 1e-1
2291		 1e-2
2398		 1e-3
2481		 1e-5
2573		 1e-7
2776		 #succ
15/15
CMA-ES 2019-gecco-surr 0.72 (0.6)3.8 (4)4.0 (4)4.1 (4)4.0 (4)4.0 (4)3.9 (4)15/15
lq-CMA-ES 2019-gecco-surr 1.2 (0.8)3.1 (4)3.5 (3)3.6 (3)3.7 (3)3.6 (3)3.3 (3)15/15
SLSQP+CMA 2019-gecco-surr 1.4 (0)5.0 (4)5.5 (4)5.8 (4)5.8 (4)5.8 (4)5.5 (4)15/15
SLSQP-11 2019-gecco-surr 1.4 (0)4.1 (4)11 (11)10 (10)10 (10)10 (10)8.9 (9)14/15

Δ fopt
f21, 3-D		 1e1
6.0		 1e0
184		 1e-1
425		 1e-2
439		 1e-3
458		 1e-5
469		 1e-7
482		 #succ
14/15
CMA-ES 2019-gecco-surr 1.5 (2)7.1 (6)12 (14)12 (14)12 (14)13 (14)13 (15)15/15
lq-CMA-ES 2019-gecco-surr 1.7 (2)2.0 (3)1.9 (3)2.0 (3)2.0 (3)2.1 (3)2.1 (3)15/15
SLSQP+CMA 2019-gecco-surr 2.7 (3)4.9 (10)10 (16)27 (23)27 (25)34 (31)34 (32)15/15
SLSQP-11 2019-gecco-surr 2.7 (3)1.8 (2)1.7 (2)1.6 (1)1.6 (1)1.6 (1)2.3 (2)15/15

Δ fopt
f22, 3-D		 1e1
18		 1e0
170		 1e-1
354		 1e-2
362		 1e-3
384		 1e-5
401		 1e-7
414		 #succ
15/15
CMA-ES 2019-gecco-surr 1.4 (2)5.0 (6)24 (27)87 (193)160 (361)259 (356)252 (379)11/15
lq-CMA-ES 2019-gecco-surr 2.5 (1)2.1 (2)8.0 (10)34 (55)33 (52)50 (149)48 (139)13/15
SLSQP+CMA 2019-gecco-surr 1.9 (2)4.5 (6)10 (9)13 (18)56 (38)54 (37)53 (38)14/15
SLSQP-11 2019-gecco-surr 2.2 (2)2.4 (3)2.1 (3)2.1 (3)2.0 (2)2.0 (2)2.1 (2)15/15

Δ fopt
f23, 3-D		 1e1
3.0		 1e0
407		 1e-1
906		 1e-2
1215		 1e-3
2214		 1e-5
2293		 1e-7
2393		 #succ
15/15
CMA-ES 2019-gecco-surr 3.2 (3)6.8 (9)5.7 (7)4.5 (5)3.6 (4)3.7 (4)3.6 (4)15/15
lq-CMA-ES 2019-gecco-surr 3.2 (3)4.6 (4)4.5 (3)3.5 (3)1.9 (1)2.0 (1)2.0 (1)15/15
SLSQP+CMA 2019-gecco-surr 19 (42)5.3 (5)10 (9)8.1 (8)4.6 (5)4.6 (5)4.5 (5)15/15
SLSQP-11 2019-gecco-surr 21 (30)6.6 (10)21 (33)22 (29)15 (16)83 (103)∞6e40/15

Δ fopt
f24, 3-D		 1e1
97		 1e0
10391		 1e-1
1.0e5		 1e-2
3.6e5		 1e-3
3.6e5		 1e-5
3.6e5		 1e-7
3.6e5		 #succ
2/15
CMA-ES 2019-gecco-surr 1.6 (2)37 (37)106 (154)31 (31)31 (25)31 (37)31 (41)1/15
lq-CMA-ES 2019-gecco-surr 1.5 (2)986 (958)∞∞∞∞∞7e50/15
SLSQP+CMA 2019-gecco-surr 2.6 (2)63 (88)100 (137)29 (37)29 (35)29 (22)29 (55)1/15
SLSQP-11 2019-gecco-surr 8.2 (8)3.2 (5)8.4 (10)∞∞∞∞6e40/15

Previous dimension | Dimension = 5 | Next dimension

Δ fopt
f1, 5-D		 1e1
11		 1e0
12		 1e-1
12		 1e-2
12		 1e-3
12		 1e-5
12		 1e-7
12		 #succ
15/15
CMA-ES 2019-gecco-surr 2.1 (2)7.3 (3)14 (3)19 (4)26 (3)37 (4)51 (5)15/15
lq-CMA-ES 2019-gecco-surr 1.2 (0.6)1.8 (0.2)2.3 (0.5)2.8 (0.6)3.5 (0.6)4.6 (0.5)5.5 (0.8)15/15
SLSQP+CMA 2019-gecco-surr 0.91 (0)↑00.83 (0)↑00.83 (0)↑00.83 (0)↑00.83 (0)↑00.83 (0)↑00.83 (0)↑015/15
SLSQP-11 2019-gecco-surr 0.91 (0)↑00.83 (0)↑00.83 (0)↑00.83 (0)↑00.83 (0)↑00.83 (0)↑00.83 (0)↑015/15

Δ fopt
f2, 5-D		 1e1
83		 1e0
87		 1e-1
88		 1e-2
89		 1e-3
90		 1e-5
92		 1e-7
94		 #succ
15/15
CMA-ES 2019-gecco-surr 7.5 (2)9.2 (2)10 (2)11 (2)12 (2)13 (2)15 (2)15/15
lq-CMA-ES 2019-gecco-surr 1.1 (0.4)1.2 (0.3)1.4 (0.3)1.5 (0.3)1.6 (0.2)1.8 (0.2)2.0 (0.3)15/15
SLSQP+CMA 2019-gecco-surr 1.1 (0.4)1.2 (0.3)1.3 (0.4)1.4 (0.6)1.6 (0.8)1.8 (0.8)2.1 (0.8)15/15
SLSQP-11 2019-gecco-surr 1.1 (0.4)1.2 (0.3)1.3 (0.4)1.4 (0.6)1.5 (0.7)1.8 (0.7)1.9 (0.7)15/15

Δ fopt
f3, 5-D		 1e1
716		 1e0
1622		 1e-1
1637		 1e-2
1642		 1e-3
1646		 1e-5
1650		 1e-7
1654		 #succ
15/15
CMA-ES 2019-gecco-surr 1.5 (1)67 (106)721 (953)719 (1278)717 (943)716 (1555)714 (1056)6/15
lq-CMA-ES 2019-gecco-surr 0.50 (0.2)↑154 (116)765 (863)763 (722)761 (679)759 (1040)757 (1557)5/15
SLSQP+CMA 2019-gecco-surr 1.7 (1)9.2 (5)942 (1307)940 (1710)938 (1376)936 (1405)934 (1149)5/15
SLSQP-11 2019-gecco-surr 17 (25)879 (998)∞∞∞∞∞1e50/15

Δ fopt
f4, 5-D		 1e1
809		 1e0
1633		 1e-1
1688		 1e-2
1758		 1e-3
1817		 1e-5
1886		 1e-7
1903		 #succ
15/15
CMA-ES 2019-gecco-surr 0.77 (1)7196 (5488)∞∞∞∞∞8e50/15
lq-CMA-ES 2019-gecco-surr 1.4 (2)∞∞∞∞∞∞8e50/15
SLSQP+CMA 2019-gecco-surr 1.8 (1)∞∞∞∞∞∞8e50/15
SLSQP-11 2019-gecco-surr 59 (48)∞∞∞∞∞∞1e50/15

Δ fopt
f5, 5-D		 1e1
10		 1e0
10		 1e-1
10		 1e-2
10		 1e-3
10		 1e-5
10		 1e-7
10		 #succ
15/15
CMA-ES 2019-gecco-surr 4.6 (2)6.2 (2)6.4 (2)6.5 (2)6.5 (2)6.5 (2)6.5 (2)15/15
lq-CMA-ES 2019-gecco-surr 1.5 (0.6)2.2 (0.8)2.3 (0.8)★22.3 (0.8)★22.3 (0.8)★22.3 (0.8)★22.3 (0.8)★215/15
SLSQP+CMA 2019-gecco-surr 1.6 (0)2.3 (0)3.0 (0)3.0 (0)3.0 (0)3.0 (0)3.0 (0)15/15
SLSQP-11 2019-gecco-surr 1.6 (0)2.3 (0)3.0 (0)3.0 (0)3.0 (0)3.0 (0)3.0 (0)15/15

Δ fopt
f6, 5-D		 1e1
114		 1e0
214		 1e-1
281		 1e-2
404		 1e-3
580		 1e-5
1038		 1e-7
1332		 #succ
15/15
CMA-ES 2019-gecco-surr 1.8 (0.7)1.7 (0.4)1.8 (0.4)1.7 (0.3)1.5 (0.3)1.1 (0.1)1.1 (0.1)15/15
lq-CMA-ES 2019-gecco-surr 1.5 (0.7)1.5 (0.3)1.7 (0.2)1.5 (0.2)1.4 (0.2)1.1 (0.1)1.1 (0.1)15/15
SLSQP+CMA 2019-gecco-surr 2.9 (2)1.9 (0.7)2.0 (0.9)1.6 (0.6)1.4 (0.7)1.0 (0.5)1.4 (1)15/15
SLSQP-11 2019-gecco-surr 2.9 (1)2.1 (1)2.2 (1)1.7 (0.9)1.4 (0.6)0.99 (0.5)2.2 (2)15/15

Δ fopt
f7, 5-D		 1e1
24		 1e0
324		 1e-1
1171		 1e-2
1451		 1e-3
1572		 1e-5
1572		 1e-7
1597		 #succ
15/15
CMA-ES 2019-gecco-surr 2.5 (1)0.69 (0.1)↑10.47 (0.4)↑00.49 (0.3)↑00.58 (0.3)↑00.58 (0.3)↑00.67 (0.3)↑115/15
lq-CMA-ES 2019-gecco-surr 1.5 (0.5)0.50 (0.7)0.45 (0.4)↑00.48 (0.3)↑00.47 (0.3)↑00.47 (0.3)↑00.51 (0.2)↑015/15
SLSQP+CMA 2019-gecco-surr 4.1 (3)0.86 (0.3)0.74 (0.6)0.68 (0.5)0.65 (0.4)0.65 (0.4)0.67 (0.4)15/15
SLSQP-11 2019-gecco-surr 286 (599)1064 (1651)∞∞∞∞∞1e50/15

Δ fopt
f8, 5-D		 1e1
73		 1e0
273		 1e-1
336		 1e-2
372		 1e-3
391		 1e-5
410		 1e-7
422		 #succ
15/15
CMA-ES 2019-gecco-surr 2.5 (0.7)3.2 (2)3.7 (2)3.9 (1)4.0 (1)4.3 (1)4.6 (1)15/15
lq-CMA-ES 2019-gecco-surr 1.3 (0.6)1.2 (2)1.2 (1)1.2 (1)1.1 (1)1.1 (1)1.1 (1)15/15
SLSQP+CMA 2019-gecco-surr 1.3 (0.5)1.9 (2)2.0 (3)2.0 (3)2.0 (3)2.1 (3)2.1 (3)15/15
SLSQP-11 2019-gecco-surr 1.3 (0.5)1.1 (0.9)1.0 (0.8)1.0 (0.6)1.0 (0.6)1.0 (0.6)1.0 (0.6)15/15

Δ fopt
f9, 5-D		 1e1
35		 1e0
127		 1e-1
214		 1e-2
263		 1e-3
300		 1e-5
335		 1e-7
369		 #succ
15/15
CMA-ES 2019-gecco-surr 4.1 (1.0)7.4 (8)6.1 (5)5.7 (4)5.4 (4)5.3 (3)5.2 (3)15/15
lq-CMA-ES 2019-gecco-surr 2.5 (1.0)2.3 (3)1.7 (2)1.5 (2)1.4 (1)1.3 (1)1.2 (1)15/15
SLSQP+CMA 2019-gecco-surr 0.63 (0)↑00.46 (0)↑00.54 (0)↑00.55 (0)↑00.53 (0)↑00.51 (0)↑00.50 (0)↑015/15
SLSQP-11 2019-gecco-surr 0.63 (0)↑00.46 (0)↑00.54 (0)↑00.55 (0)↑00.53 (0)↑00.51 (0)↑00.50 (0)↑015/15

Δ fopt
f10, 5-D		 1e1
349		 1e0
500		 1e-1
574		 1e-2
607		 1e-3
626		 1e-5
829		 1e-7
880		 #succ
15/15
CMA-ES 2019-gecco-surr 2.0 (0.6)1.6 (0.4)1.6 (0.3)1.7 (0.3)1.7 (0.2)1.5 (0.2)1.6 (0.2)15/15
lq-CMA-ES 2019-gecco-surr 0.34 (0.1)↑00.27 (0.0)↑00.25 (0.0)↑00.25 (0.0)↑00.26 (0.0)↑00.22 (0.0)↑00.28 (0.1)↑015/15
SLSQP+CMA 2019-gecco-surr 0.25 (0.1)↑00.20 (0.0)↑00.19 (0.1)↑00.20 (0.1)↑00.22 (0.1)↑00.95 (1)2.9 (3)15/15
SLSQP-11 2019-gecco-surr 0.25 (0.1)↑00.20 (0.0)↑00.19 (0.0)↑00.19 (0.0)↑00.20 (0.0)↑00.22 (0.1)↑023 (31)10/15

Δ fopt
f11, 5-D		 1e1
143		 1e0
202		 1e-1
763		 1e-2
977		 1e-3
1177		 1e-5
1467		 1e-7
1673		 #succ
15/15
CMA-ES 2019-gecco-surr 3.8 (0.5)3.2 (0.4)0.99 (0.1)0.85 (0.1)↑00.77 (0.1)↑00.72 (0.1)↑00.71 (0.0)↑015/15
lq-CMA-ES 2019-gecco-surr 0.68 (0.2)↑00.63 (0.1)↑00.19 (0.0)↑00.16 (0.0)↑00.14 (0.0)↑00.13 (0.0)↑00.15 (0.0)★4↑015/15
SLSQP+CMA 2019-gecco-surr 0.28 (0.1)↑00.28 (0.1)↑00.08 (0.0)↑00.08 (0.0)↑00.07 (0.0)↑01.1 (2)1.9 (1)15/15
SLSQP-11 2019-gecco-surr 0.28 (0.1)↑00.27 (0.1)↑00.08 (0.0)↑00.07 (0.0)↑00.07 (0.0)↑07.2 (17)195 (147)0/15

Δ fopt
f12, 5-D		 1e1
108		 1e0
268		 1e-1
371		 1e-2
413		 1e-3
461		 1e-5
1303		 1e-7
1494		 #succ
15/15
CMA-ES 2019-gecco-surr 6.4 (3)4.5 (2)5.2 (3)5.6 (3)5.9 (3)2.7 (1)2.8 (1)15/15
lq-CMA-ES 2019-gecco-surr 1.2 (0.6)0.90 (1)1.0 (1)1.1 (1)1.2 (1)0.76 (0.6)1.1 (0.8)15/15
SLSQP+CMA 2019-gecco-surr 2.8 (3)1.7 (2)1.6 (2)1.6 (2)1.5 (2)1.1 (0.7)3.3 (4)15/15
SLSQP-11 2019-gecco-surr 2.8 (3)1.7 (2)1.5 (2)1.5 (2)1.5 (2)0.88 (0.7)4.2 (6)11/15

Δ fopt
f13, 5-D		 1e1
132		 1e0
195		 1e-1
250		 1e-2
319		 1e-3
1310		 1e-5
1752		 1e-7
2255		 #succ
15/15
CMA-ES 2019-gecco-surr 2.3 (0.8)3.1 (1)3.3 (0.8)3.3 (0.6)1.0 (0.3)1.0 (0.2)0.97 (0.1)15/15
lq-CMA-ES 2019-gecco-surr 0.64 (0.0)★↑00.63 (0.1)★↑00.67 (0.1)★↑00.65 (0.1)★↑00.19 (0.0)★3↑00.30 (0.1)★4↑00.45 (0.1)★4↑015/15
SLSQP+CMA 2019-gecco-surr 0.76 (0.1)↑00.74 (0.1)↑00.76 (0.1)↑00.73 (0.0)↑02.6 (2)3.5 (0.2)3.0 (0.2)15/15
SLSQP-11 2019-gecco-surr 0.76 (0.1)↑00.74 (0.1)↑00.76 (0.1)↑00.73 (0.0)↑035 (61)437 (598)∞1e50/15

Δ fopt
f14, 5-D		 1e1
10		 1e0
41		 1e-1
58		 1e-2
90		 1e-3
139		 1e-5
251		 1e-7
476		 #succ
15/15
CMA-ES 2019-gecco-surr 1.5 (2)2.2 (1)3.3 (1)3.6 (0.9)3.4 (0.6)3.2 (0.4)2.5 (0.2)15/15
lq-CMA-ES 2019-gecco-surr 1.1 (1)1.1 (0.2)1.2 (0.2)1.1 (0.3)1.0 (0.2)0.85 (0.1)↑11.0 (0.1)★215/15
SLSQP+CMA 2019-gecco-surr 0.89 (0.8)0.58 (0.3)↑00.74 (0.2)↑00.77 (0.2)↑10.70 (0.1)↑00.59 (0.1)↑05.7 (5)15/15
SLSQP-11 2019-gecco-surr 0.89 (0.8)0.58 (0.3)↑00.74 (0.2)↑00.77 (0.2)↑10.70 (0.1)↑00.59 (0.1)↑0512 (466)3/15

Δ fopt
f15, 5-D		 1e1
511		 1e0
9310		 1e-1
19369		 1e-2
19743		 1e-3
20073		 1e-5
20769		 1e-7
21359		 #succ
14/15
CMA-ES 2019-gecco-surr 1.3 (2)0.68 (0.4)0.76 (0.8)0.76 (0.8)0.76 (0.8)0.77 (0.8)0.78 (0.8)15/15
lq-CMA-ES 2019-gecco-surr 1.3 (1)0.62 (0.3)↑10.54 (0.5)0.54 (0.5)0.53 (0.5)0.52 (0.5)0.51 (0.5)15/15
SLSQP+CMA 2019-gecco-surr 1.7 (2)0.99 (0.4)1.1 (1)1.1 (1)1.1 (1)1.1 (1)1.2 (1)15/15
SLSQP-11 2019-gecco-surr 30 (33)153 (133)∞∞∞∞∞1e50/15

Δ fopt
f16, 5-D		 1e1
120		 1e0
612		 1e-1
2662		 1e-2
10163		 1e-3
10449		 1e-5
11644		 1e-7
12095		 #succ
15/15
CMA-ES 2019-gecco-surr 2.3 (1)1.8 (2)1.7 (1)0.61 (0.7)0.69 (0.7)0.65 (0.6)0.65 (0.6)15/15
lq-CMA-ES 2019-gecco-surr 2.0 (2)1.6 (2)1.2 (1)0.43 (0.4)↑10.47 (0.3)↑10.47 (0.3)↑10.47 (0.3)↑115/15
SLSQP+CMA 2019-gecco-surr 7.0 (4)2.8 (3)1.4 (1)0.90 (1)1.1 (1)0.99 (1)0.99 (1)15/15
SLSQP-11 2019-gecco-surr 250 (435)∞∞∞∞∞∞1e50/15

Δ fopt
f17, 5-D		 1e1
5.0		 1e0
215		 1e-1
899		 1e-2
2861		 1e-3
3669		 1e-5
6351		 1e-7
7934		 #succ
15/15
CMA-ES 2019-gecco-surr 1.9 (2)0.72 (0.3)0.48 (0.1)↑00.63 (0.5)0.75 (0.4)0.79 (0.4)1.2 (0.4)15/15
lq-CMA-ES 2019-gecco-surr 3.1 (2)5.0 (7)2.6 (2)0.95 (0.7)1.1 (0.8)1.3 (0.6)1.2 (0.8)15/15
SLSQP+CMA 2019-gecco-surr 13 (19)2.6 (1)1.3 (2)0.55 (0.5)0.60 (0.4)0.95 (0.5)1.0 (0.4)15/15
SLSQP-11 2019-gecco-surr 31 (62)300 (273)∞∞∞∞∞1e50/15

Δ fopt
f18, 5-D		 1e1
103		 1e0
378		 1e-1
3968		 1e-2
8451		 1e-3
9280		 1e-5
10905		 1e-7
12469		 #succ
15/15
CMA-ES 2019-gecco-surr 0.82 (0.6)0.77 (0.2)↑10.36 (0.4)↑10.58 (0.6)0.66 (0.8)0.79 (0.6)0.93 (0.6)15/15
lq-CMA-ES 2019-gecco-surr 6.0 (0.7)2.3 (0.7)0.80 (0.9)0.69 (0.3)0.73 (0.3)0.79 (0.3)↑00.82 (0.1)↑115/15
SLSQP+CMA 2019-gecco-surr 2.1 (1)1.4 (0.3)0.78 (1.0)0.52 (0.5)0.69 (0.5)0.77 (0.3)↑10.85 (0.4)15/15
SLSQP-11 2019-gecco-surr 46 (39)∞∞∞∞∞∞1e50/15

Δ fopt
f19, 5-D		 1e1
1		 1e0
1		 1e-1
242		 1e-2
1.0e5		 1e-3
1.2e5		 1e-5
1.2e5		 1e-7
1.2e5		 #succ
15/15
CMA-ES 2019-gecco-surr 1 (0)1 (0)171 (176)1.0 (0.6)1.0 (0.6)1.0 (0.6)1.0 (0.6)15/15
lq-CMA-ES 2019-gecco-surr 1 (0)1 (0)72 (70)0.54 (0.5)0.55 (0.5)0.56 (0.5)0.57 (0.5)15/15
SLSQP+CMA 2019-gecco-surr 1 (0)1 (0)0.12 (0)↑01.0 (0.9)1.0 (0.6)1.0 (0.6)1.1 (0.6)15/15
SLSQP-11 2019-gecco-surr 1 (0)1 (0)0.12 (0)↑0∞∞∞∞1e50/15

Δ fopt
f20, 5-D		 1e1
16		 1e0
851		 1e-1
38111		 1e-2
51362		 1e-3
54470		 1e-5
54861		 1e-7
55313		 #succ
14/15
CMA-ES 2019-gecco-surr 1.7 (1)4.3 (5)1.9 (2)1.5 (1)1.5 (1)1.5 (1)1.5 (1)15/15
lq-CMA-ES 2019-gecco-surr 1.4 (1)6.1 (3)1.4 (1)1.1 (0.9)1.0 (0.9)1.0 (0.9)1.0 (0.9)15/15
SLSQP+CMA 2019-gecco-surr 0.81 (0)↑07.9 (6)2.2 (2)1.9 (1)1.9 (1)1.9 (1)2.0 (1)15/15
SLSQP-11 2019-gecco-surr 0.81 (0)↑02.3 (2)∞∞∞∞∞1e50/15

Δ fopt
f21, 5-D		 1e1
41		 1e0
1157		 1e-1
1674		 1e-2
1692		 1e-3
1705		 1e-5
1729		 1e-7
1757		 #succ
14/15
CMA-ES 2019-gecco-surr 1.5 (1)8.6 (9)15 (31)21 (32)21 (34)23 (35)23 (36)15/15
lq-CMA-ES 2019-gecco-surr 1.5 (1)2.0 (2)10 (8)11 (9)11 (9)11 (9)11 (9)14/15
SLSQP+CMA 2019-gecco-surr 1.3 (2)8.9 (9)38 (101)38 (98)38 (98)38 (97)38 (95)13/15
SLSQP-11 2019-gecco-surr 1.3 (2)0.81 (0.9)1.3 (0.7)1.2 (0.7)1.2 (0.7)1.3 (0.7)1.4 (1)15/15

Δ fopt
f22, 5-D		 1e1
71		 1e0
386		 1e-1
938		 1e-2
980		 1e-3
1008		 1e-5
1040		 1e-7
1068		 #succ
14/15
CMA-ES 2019-gecco-surr 3.4 (10)10 (15)155 (290)332 (370)323 (687)315 (528)307 (397)8/15
lq-CMA-ES 2019-gecco-surr 3.6 (5)7.0 (8)76 (162)170 (274)165 (268)161 (208)157 (179)8/15
SLSQP+CMA 2019-gecco-surr 1.4 (1)7.5 (13)333 (390)319 (526)311 (603)302 (492)294 (258)8/15
SLSQP-11 2019-gecco-surr 1.8 (2)1.1 (0.7)1.8 (2)1.7 (2)1.7 (2)1.7 (2)2.0 (2)14/15

Δ fopt
f23, 5-D		 1e1
3.0		 1e0
518		 1e-1
14249		 1e-2
27890		 1e-3
31654		 1e-5
33030		 1e-7
34256		 #succ
15/15
CMA-ES 2019-gecco-surr 2.9 (4)6.4 (6)1.9 (2)1.1 (1)1.0 (1)0.99 (1)0.98 (1)15/15
lq-CMA-ES 2019-gecco-surr 1.8 (3)3.3 (3)15 (15)7.9 (8)6.9 (7)10 (23)9.4 (22)13/15
SLSQP+CMA 2019-gecco-surr 96 (132)12 (15)2.3 (2)1.2 (1)1.1 (1)1.1 (1.0)1.1 (1.0)15/15
SLSQP-11 2019-gecco-surr 27 (8)13 (19)17 (17)25 (27)∞∞∞1e50/15

Δ fopt
f24, 5-D		 1e1
1622		 1e0
2.2e5		 1e-1
6.4e6		 1e-2
9.6e6		 1e-3
9.6e6		 1e-5
1.3e7		 1e-7
1.3e7		 #succ
3/15
CMA-ES 2019-gecco-surr 1.2 (1)9.1 (10)2.8 (2)∞∞∞∞1e60/15
lq-CMA-ES 2019-gecco-surr 1.2 (0.9)109 (123)3.7 (3)2.5 (2)2.5 (3)1.8 (2)1.8 (2)1/15
SLSQP+CMA 2019-gecco-surr 1.5 (1)13 (14)1.4 (2)1.9 (4)1.9 (2)1.4 (1)1.4 (1)1/15
SLSQP-11 2019-gecco-surr 6.4 (7)∞∞∞∞∞∞1e50/15

Previous dimension | Dimension = 10 | Next dimension

Δ fopt
f1, 10-D		 1e1
22		 1e0
23		 1e-1
23		 1e-2
23		 1e-3
23		 1e-5
23		 1e-7
23		 #succ
15/15
CMA-ES 2019-gecco-surr 3.6 (2)10 (2)17 (2)23 (4)30 (4)43 (3)56 (5)15/15
lq-CMA-ES 2019-gecco-surr 1.5 (0.1)1.7 (0.3)2.1 (0.5)2.5 (0.5)3.0 (0.3)3.7 (0.2)4.5 (0.3)15/15
SLSQP+CMA 2019-gecco-surr 0.68 (0)↑00.65 (0)↑00.65 (0)↑00.65 (0)↑00.65 (0)↑00.65 (0)↑00.65 (0)↑015/15
SLSQP-11 2019-gecco-surr 0.68 (0)↑00.65 (0)↑00.65 (0)↑00.65 (0)↑00.65 (0)↑00.65 (0)↑00.65 (0)↑015/15

Δ fopt
f2, 10-D		 1e1
187		 1e0
190		 1e-1
191		 1e-2
191		 1e-3
193		 1e-5
194		 1e-7
195		 #succ
15/15
CMA-ES 2019-gecco-surr 12 (4)15 (2)16 (2)17 (2)17 (2)19 (2)20 (2)15/15
lq-CMA-ES 2019-gecco-surr 1.7 (0.4)1.9 (0.3)2.1 (0.3)2.3 (0.3)2.5 (0.4)2.8 (0.4)3.2 (0.5)15/15
SLSQP+CMA 2019-gecco-surr 1.5 (0.4)1.6 (0.4)1.7 (0.4)1.9 (0.4)1.9 (0.4)2.1 (0.5)2.4 (0.6)15/15
SLSQP-11 2019-gecco-surr 1.6 (0.5)1.6 (0.5)1.7 (0.5)1.9 (0.5)2.0 (0.5)2.3 (0.6)2.6 (0.6)15/15

Δ fopt
f3, 10-D		 1e1
1739		 1e0
3600		 1e-1
3609		 1e-2
3636		 1e-3
3642		 1e-5
3646		 1e-7
3651		 #succ
15/15
CMA-ES 2019-gecco-surr 3.7 (4)1082 (870)1567 (1291)1555 (1637)1553 (1553)1552 (1723)1551 (1742)3/15
lq-CMA-ES 2019-gecco-surr 1.3 (2)4500 (5461)∞∞∞∞∞1e60/15
SLSQP+CMA 2019-gecco-surr 3.8 (3)2623 (3267)∞∞∞∞∞1e60/15
SLSQP-11 2019-gecco-surr ∞∞∞∞∞∞∞2e50/15

Δ fopt
f4, 10-D		 1e1
2234		 1e0
3626		 1e-1
3660		 1e-2
3695		 1e-3
3707		 1e-5
3744		 1e-7
28767		 #succ
12/15
CMA-ES 2019-gecco-surr 6.8 (9)∞∞∞∞∞∞1e60/15
lq-CMA-ES 2019-gecco-surr 4.6 (3)∞∞∞∞∞∞1e60/15
SLSQP+CMA 2019-gecco-surr 4.7 (3)∞∞∞∞∞∞1e60/15
SLSQP-11 2019-gecco-surr ∞∞∞∞∞∞∞2e50/15

Δ fopt
f5, 10-D		 1e1
20		 1e0
20		 1e-1
20		 1e-2
20		 1e-3
20		 1e-5
20		 1e-7
20		 #succ
15/15
CMA-ES 2019-gecco-surr 5.0 (2)6.3 (1)6.5 (2)6.7 (2)6.7 (2)6.7 (2)6.7 (2)15/15
lq-CMA-ES 2019-gecco-surr 1.6 (0.5)2.1 (0.6)2.2 (0.6)2.2 (0.6)2.2 (0.6)2.2 (0.6)2.2 (0.6)15/15
SLSQP+CMA 2019-gecco-surr 1.3 (0)2.5 (0)2.5 (0)2.5 (0)2.5 (0)2.5 (0)2.5 (0)15/15
SLSQP-11 2019-gecco-surr 1.3 (0)2.5 (0)2.5 (0)2.5 (0)2.5 (0)2.5 (0)2.5 (0)15/15

Δ fopt
f6, 10-D		 1e1
412		 1e0
623		 1e-1
826		 1e-2
1039		 1e-3
1292		 1e-5
1841		 1e-7
2370		 #succ
15/15
CMA-ES 2019-gecco-surr 1.7 (0.5)1.6 (0.3)1.7 (0.2)1.7 (0.2)1.7 (0.2)1.6 (0.1)1.6 (0.2)15/15
lq-CMA-ES 2019-gecco-surr 1.4 (0.4)1.5 (0.3)1.6 (0.2)1.7 (0.2)1.6 (0.2)1.6 (0.1)1.5 (0.2)15/15
SLSQP+CMA 2019-gecco-surr 2.4 (1)2.2 (0.8)2.3 (0.9)2.4 (0.8)2.7 (2)2.9 (0.9)3.2 (0.1)15/15
SLSQP-11 2019-gecco-surr 2.3 (0.9)2.0 (0.5)2.1 (0.4)2.3 (0.3)2.3 (0.5)2.4 (0.5)8.4 (11)4/15

Δ fopt
f7, 10-D		 1e1
172		 1e0
1611		 1e-1
4195		 1e-2
5099		 1e-3
5141		 1e-5
5141		 1e-7
5389		 #succ
15/15
CMA-ES 2019-gecco-surr 1.6 (0.6)0.62 (0.5)0.59 (0.3)↑10.64 (0.4)0.70 (0.3)0.70 (0.3)0.72 (0.3)15/15
lq-CMA-ES 2019-gecco-surr 1.5 (2)★0.72 (0.4)0.45 (0.4)↑00.54 (0.2)↑00.54 (0.2)↑00.54 (0.2)↑00.54 (0.2)↑015/15
SLSQP+CMA 2019-gecco-surr 2.0 (0.6)1.1 (0.7)0.92 (0.4)0.86 (0.3)0.86 (0.3)0.86 (0.3)0.87 (0.3)15/15
SLSQP-11 2019-gecco-surr ∞∞∞∞∞∞∞2e50/15

Δ fopt
f8, 10-D		 1e1
326		 1e0
921		 1e-1
1114		 1e-2
1217		 1e-3
1267		 1e-5
1315		 1e-7
1343		 #succ
15/15
CMA-ES 2019-gecco-surr 1.9 (0.6)3.4 (0.4)3.6 (0.4)3.6 (0.3)3.6 (0.3)3.7 (0.3)3.9 (0.3)15/15
lq-CMA-ES 2019-gecco-surr 1.1 (0.4)1.8 (3)1.6 (2)1.6 (2)1.6 (2)1.5 (2)1.5 (2)15/15
SLSQP+CMA 2019-gecco-surr 1.4 (0.9)1.4 (2)1.3 (2)1.3 (2)1.3 (2)1.3 (2)1.3 (2)15/15
SLSQP-11 2019-gecco-surr 1.4 (0.9)1.1 (0.7)0.97 (0.5)0.94 (0.5)0.93 (0.5)0.92 (0.5)0.91 (0.4)15/15

Δ fopt
f9, 10-D		 1e1
200		 1e0
648		 1e-1
857		 1e-2
993		 1e-3
1065		 1e-5
1138		 1e-7
1185		 #succ
15/15
CMA-ES 2019-gecco-surr 2.8 (0.7)5.2 (1)4.9 (0.9)4.6 (0.8)4.6 (0.7)4.6 (0.6)4.6 (0.6)15/15
lq-CMA-ES 2019-gecco-surr 1.6 (0.3)2.0 (2)1.7 (1)1.6 (1)1.5 (1)1.5 (0.9)1.4 (0.9)15/15
SLSQP+CMA 2019-gecco-surr 0.82 (0)↑00.39 (0)↑00.39 (0)↑00.39 (0)↑00.39 (0)↑00.39 (0)↑00.39 (0)↑015/15
SLSQP-11 2019-gecco-surr 0.82 (0)↑00.39 (0)↑00.39 (0)↑00.39 (0)↑00.39 (0)↑00.39 (0)↑00.39 (0)↑015/15

Δ fopt
f10, 10-D		 1e1
1835		 1e0
2172		 1e-1
2455		 1e-2
2728		 1e-3
2802		 1e-5
4543		 1e-7
4739		 #succ
15/15
CMA-ES 2019-gecco-surr 1.2 (0.2)1.2 (0.2)1.2 (0.1)1.2 (0.1)1.2 (0.1)0.80 (0.1)↑00.83 (0.1)↑015/15
lq-CMA-ES 2019-gecco-surr 0.21 (0.0)↑00.21 (0.0)↑00.21 (0.0)↑00.20 (0.0)↑00.21 (0.0)↑00.14 (0.0)↑00.18 (0.0)★↑015/15
SLSQP+CMA 2019-gecco-surr 0.16 (0.1)↑00.15 (0.1)↑00.14 (0.1)↑00.14 (0.1)↑00.14 (0.0)↑00.11 (0.0)↑00.97 (0.5)15/15
SLSQP-11 2019-gecco-surr 0.16 (0.1)↑00.15 (0.1)↑00.14 (0.0)↑00.14 (0.1)↑00.15 (0.0)↑00.10 (0.0)↑05.3 (8)9/15

Δ fopt
f11, 10-D		 1e1
266		 1e0
1041		 1e-1
2602		 1e-2
2954		 1e-3
3338		 1e-5
4092		 1e-7
4843		 #succ
15/15
CMA-ES 2019-gecco-surr 5.7 (1.0)1.7 (0.2)0.75 (0.1)↑00.73 (0.1)↑00.70 (0.1)↑00.64 (0.1)↑00.61 (0.0)↑015/15
lq-CMA-ES 2019-gecco-surr 1.6 (0.2)0.46 (0.1)↑00.20 (0.0)↑00.19 (0.0)↑00.18 (0.0)↑00.16 (0.0)↑00.17 (0.0)★2↑015/15
SLSQP+CMA 2019-gecco-surr 0.25 (0.1)↑00.09 (0.0)↑00.04 (0.0)↑00.05 (0.0)↑00.05 (0.0)↑00.40 (0.4)↑11.0 (0.6)15/15
SLSQP-11 2019-gecco-surr 0.25 (0.1)↑00.09 (0.0)↑00.04 (0.0)↑00.04 (0.0)↑00.05 (0.0)↑00.62 (0.3)↑185 (106)0/15

Δ fopt
f12, 10-D		 1e1
515		 1e0
896		 1e-1
1240		 1e-2
1390		 1e-3
1569		 1e-5
3660		 1e-7
5154		 #succ
15/15
CMA-ES 2019-gecco-surr 2.9 (2)3.7 (3)4.1 (3)4.3 (2)4.3 (2)2.4 (1.0)1.9 (0.8)15/15
lq-CMA-ES 2019-gecco-surr 0.79 (0.7)1.1 (0.9)1.1 (0.8)1.1 (0.7)1.2 (0.7)0.82 (0.5)0.82 (0.5)15/15
SLSQP+CMA 2019-gecco-surr 1.9 (1)1.5 (1)1.4 (0.8)1.3 (0.7)1.3 (0.7)1.3 (2)1.8 (2)15/15
SLSQP-11 2019-gecco-surr 1.9 (2)1.6 (1)1.4 (0.9)1.3 (0.8)1.3 (0.7)1.2 (1)4.2 (4)10/15

Δ fopt
f13, 10-D		 1e1
387		 1e0
596		 1e-1
797		 1e-2
1014		 1e-3
4587		 1e-5
6208		 1e-7
7779		 #succ
15/15
CMA-ES 2019-gecco-surr 2.2 (1.0)2.7 (1)3.5 (1)3.4 (0.8)0.83 (0.2)0.94 (0.2)0.95 (0.1)15/15
lq-CMA-ES 2019-gecco-surr 0.82 (0.2)0.89 (0.2)1.0 (0.3)0.93 (0.2)0.27 (0.1)↑00.42 (0.2)★2↑00.53 (0.1)★3↑015/15
SLSQP+CMA 2019-gecco-surr 0.65 (0.0)↑00.64 (0.0)↑00.62 (0.0)↑00.61 (0.0)↑01.3 (1)1.7 (0.4)1.6 (0.4)15/15
SLSQP-11 2019-gecco-surr 0.65 (0.0)↑00.64 (0.0)↑00.62 (0.0)↑00.61 (0.0)↑052 (57)266 (297)∞3e50/15

Δ fopt
f14, 10-D		 1e1
37		 1e0
98		 1e-1
133		 1e-2
205		 1e-3
392		 1e-5
687		 1e-7
4305		 #succ
15/15
CMA-ES 2019-gecco-surr 1.1 (0.8)2.7 (1.0)3.6 (0.8)3.7 (0.6)2.9 (0.5)3.1 (0.3)0.75 (0.0)↑015/15
lq-CMA-ES 2019-gecco-surr 2.1 (1)1.7 (0.5)1.9 (0.4)1.9 (0.2)1.3 (0.2)1.1 (0.1)0.36 (0.0)★4↑015/15
SLSQP+CMA 2019-gecco-surr 0.66 (0.2)↑10.58 (0.1)↑00.70 (0.1)↑00.74 (0.1)↑00.56 (0.1)↑00.54 (0.1)↑01.4 (0.6)15/15
SLSQP-11 2019-gecco-surr 0.66 (0.2)↑10.58 (0.1)↑00.70 (0.1)↑00.74 (0.1)↑00.56 (0.1)↑00.53 (0.1)↑0727 (753)0/15

Δ fopt
f15, 10-D		 1e1
4774		 1e0
39246		 1e-1
73643		 1e-2
74669		 1e-3
75790		 1e-5
77814		 1e-7
79834		 #succ
12/15
CMA-ES 2019-gecco-surr 1.3 (1)0.99 (0.7)0.81 (0.4)0.82 (0.4)0.82 (0.4)0.83 (0.4)0.84 (0.4)15/15
lq-CMA-ES 2019-gecco-surr 0.78 (0.7)0.79 (0.4)0.57 (0.2)↑10.57 (0.2)↑10.56 (0.1)↑10.55 (0.1)↑10.54 (0.1)↑115/15
SLSQP+CMA 2019-gecco-surr 1.2 (0.2)1.00 (0.8)0.95 (0.3)0.95 (0.3)0.96 (0.3)0.97 (0.3)0.98 (0.3)15/15
SLSQP-11 2019-gecco-surr ∞∞∞∞∞∞∞2e50/15

Δ fopt
f16, 10-D		 1e1
425		 1e0
7029		 1e-1
15779		 1e-2
45669		 1e-3
51151		 1e-5
65798		 1e-7
71570		 #succ
15/15
CMA-ES 2019-gecco-surr 3.0 (0.8)0.78 (0.5)0.99 (1)0.72 (0.3)↑10.78 (0.6)0.79 (0.7)0.75 (0.7)15/15
lq-CMA-ES 2019-gecco-surr 1.6 (0.7)1.0 (1.0)0.99 (0.8)0.47 (0.3)↑00.77 (0.5)0.62 (0.4)0.59 (0.4)↑115/15
SLSQP+CMA 2019-gecco-surr 4.5 (1)0.92 (0.5)1.0 (0.4)0.45 (0.3)↑10.41 (0.3)↑10.34 (0.2)↑10.33 (0.2)↑015/15
SLSQP-11 2019-gecco-surr 6940 (9285)∞∞∞∞∞∞2e50/15

Δ fopt
f17, 10-D		 1e1
26		 1e0
429		 1e-1
2203		 1e-2
6329		 1e-3
9851		 1e-5
20190		 1e-7
26503		 #succ
15/15
CMA-ES 2019-gecco-surr 0.90 (1)1.0 (0.5)0.82 (1)0.65 (0.5)1.1 (0.8)1.1 (0.1)1.2 (0.4)15/15
lq-CMA-ES 2019-gecco-surr 1.4 (2)3.0 (6)2.5 (2)1.3 (1)1.2 (0.8)1.1 (0.5)1.1 (0.4)15/15
SLSQP+CMA 2019-gecco-surr 7.9 (11)2.2 (0.9)1.3 (1)0.93 (0.5)0.85 (0.4)0.89 (0.3)1.1 (0.4)15/15
SLSQP-11 2019-gecco-surr 32 (64)∞∞∞∞∞∞2e50/15

Δ fopt
f18, 10-D		 1e1
238		 1e0
836		 1e-1
7012		 1e-2
15928		 1e-3
27536		 1e-5
37234		 1e-7
42708		 #succ
15/15
CMA-ES 2019-gecco-surr 0.85 (0.3)0.95 (0.2)0.47 (0.5)0.73 (0.5)0.56 (0.3)↑00.72 (0.3)0.77 (0.3)15/15
lq-CMA-ES 2019-gecco-surr 0.86 (0.4)3.0 (4)1.3 (0.7)1.0 (0.8)0.93 (0.4)0.83 (0.3)0.91 (0.3)15/15
SLSQP+CMA 2019-gecco-surr 3.1 (0.7)1.6 (0.5)0.98 (0.7)0.86 (0.6)0.64 (0.3)↑10.72 (0.3)0.78 (0.3)15/15
SLSQP-11 2019-gecco-surr 2547 (2150)∞∞∞∞∞∞2e50/15

Δ fopt
f19, 10-D		 1e1
1		 1e0
1		 1e-1
10609		 1e-2
9.8e5		 1e-3
1.4e6		 1e-5
1.4e6		 1e-7
1.4e6		 #succ
15/15
CMA-ES 2019-gecco-surr 1 (0)1 (0)11 (8)0.45 (0.3)↑00.44 (0.2)↑00.44 (0.2)↑00.44 (0.2)↑015/15
lq-CMA-ES 2019-gecco-surr 1 (0)1 (0)9.0 (6)0.53 (0.2)↑00.51 (0.2)↑00.51 (0.2)↑00.52 (0.2)↑015/15
SLSQP+CMA 2019-gecco-surr 1 (0)1 (0)6.8e-3 (5e-5)↑00.52 (0.2)↑00.42 (0.1)↑00.42 (0.1)↑00.43 (0.1)↑015/15
SLSQP-11 2019-gecco-surr 1 (0)1 (0)6.8e-3 (5e-5)↑0∞∞∞∞3e50/15

Δ fopt
f20, 10-D		 1e1
32		 1e0
15426		 1e-1
5.5e5		 1e-2
5.7e5		 1e-3
5.7e5		 1e-5
5.8e5		 1e-7
5.9e5		 #succ
15/15
CMA-ES 2019-gecco-surr 3.5 (2)2.5 (2)0.75 (0.4)0.76 (0.4)0.77 (0.4)0.77 (0.4)0.78 (0.4)15/15
lq-CMA-ES 2019-gecco-surr 1.7 (0.6)1.2 (0.8)0.62 (0.3)0.64 (0.3)0.63 (0.3)0.63 (0.3)0.62 (0.3)15/15
SLSQP+CMA 2019-gecco-surr 1 (0)2.0 (1)0.79 (0.4)0.80 (0.4)0.80 (0.4)0.81 (0.4)0.82 (0.4)15/15
SLSQP-11 2019-gecco-surr 1 (0)4.0 (5)∞∞∞∞∞2e50/15

Δ fopt
f21, 10-D		 1e1
130		 1e0
2236		 1e-1
4392		 1e-2
4487		 1e-3
4618		 1e-5
5074		 1e-7
11329		 #succ
15/15
CMA-ES 2019-gecco-surr 4.1 (9)1.2 (1)6.0 (14)6.3 (14)6.2 (13)5.9 (13)2.7 (6)15/15
lq-CMA-ES 2019-gecco-surr 7.8 (11)11 (7)7.4 (10)7.3 (9)7.1 (9)6.5 (8)2.9 (4)14/15
SLSQP+CMA 2019-gecco-surr 4.4 (11)30 (46)17 (24)17 (24)17 (24)15 (22)7.0 (10)14/15
SLSQP-11 2019-gecco-surr 1.8 (3)1.4 (2)1.2 (2)1.1 (2)1.1 (1)1.0 (1)0.70 (1)15/15

Δ fopt
f22, 10-D		 1e1
98		 1e0
2839		 1e-1
6353		 1e-2
6620		 1e-3
6798		 1e-5
8296		 1e-7
10351		 #succ
15/15
CMA-ES 2019-gecco-surr 30 (62)186 (427)258 (323)247 (424)241 (176)198 (161)158 (186)4/15
lq-CMA-ES 2019-gecco-surr 6.4 (6)46 (50)93 (115)89 (88)87 (107)71 (78)57 (48)4/15
SLSQP+CMA 2019-gecco-surr 38 (36)147 (204)∞∞∞∞∞6e50/15
SLSQP-11 2019-gecco-surr 1.2 (1)0.60 (0.3)↑10.82 (1)★0.79 (1)★0.78 (1)★0.67 (0.8)★2.9 (5)9/15

Δ fopt
f23, 10-D		 1e1
3.0		 1e0
915		 1e-1
16425		 1e-2
1.8e5		 1e-3
2.0e5		 1e-5
2.1e5		 1e-7
2.1e5		 #succ
15/15
CMA-ES 2019-gecco-surr 2.2 (2)33 (19)21 (19)32 (39)29 (22)28 (31)27 (34)5/15
lq-CMA-ES 2019-gecco-surr 3.3 (3)20 (56)16 (19)25 (24)23 (34)22 (27)22 (27)6/15
SLSQP+CMA 2019-gecco-surr 10 (7)15 (22)11 (9)19 (24)17 (25)16 (20)16 (19)7/15
SLSQP-11 2019-gecco-surr 9.3 (7)15 (18)87 (68)∞∞∞∞2e50/15

Δ fopt
f24, 10-D		 1e1
98761		 1e0
1.0e6		 1e-1
7.5e7		 1e-2
7.5e7		 1e-3
7.5e7		 1e-5
7.5e7		 1e-7
7.5e7		 #succ
1/15
CMA-ES 2019-gecco-surr 56 (101)10 (13)∞∞∞∞∞4e60/15
lq-CMA-ES 2019-gecco-surr 41 (53)22 (24)∞∞∞∞∞3e60/15
SLSQP+CMA 2019-gecco-surr 57 (91)10 (10)0.69 (0.7)0.69 (0.6)0.69 (0.6)0.69 (0.9)0.69 (0.5)1/15
SLSQP-11 2019-gecco-surr ∞∞∞∞∞∞∞2e50/15

Previous dimension | Dimension = 20 | Next dimension

Δ fopt
f1, 20-D		 1e1
43		 1e0
43		 1e-1
43		 1e-2
43		 1e-3
43		 1e-5
43		 1e-7
43		 #succ
15/15
CMA-ES 2019-gecco-surr 6.2 (0.9)13 (1)19 (2)25 (2)32 (3)44 (2)58 (3)15/15
lq-CMA-ES 2019-gecco-surr 1.5 (0.0)1.7 (0.3)2.2 (0.1)2.4 (0.1)2.7 (0.1)3.2 (0.1)3.7 (0.1)15/15
SLSQP+CMA 2019-gecco-surr 0.58 (0)↑00.58 (0)↑00.58 (0)↑00.58 (0)↑00.58 (0)↑00.58 (0)↑00.58 (0)↑015/15
SLSQP-11 2019-gecco-surr 0.58 (0)↑00.58 (0)↑00.58 (0)↑00.58 (0)↑00.58 (0)↑00.58 (0)↑00.58 (0)↑015/15

Δ fopt
f2, 20-D		 1e1
385		 1e0
386		 1e-1
387		 1e-2
388		 1e-3
390		 1e-5
391		 1e-7
393		 #succ
15/15
CMA-ES 2019-gecco-surr 22 (3)26 (4)28 (4)30 (3)31 (2)33 (2)34 (2)15/15
lq-CMA-ES 2019-gecco-surr 2.9 (0.7)3.6 (1)4.0 (1)4.3 (1)4.6 (1)5.1 (2)6.0 (2)15/15
SLSQP+CMA 2019-gecco-surr 2.4 (0.5)2.6 (0.5)2.8 (0.6)2.9 (0.7)3.1 (0.6)3.4 (0.6)3.7 (0.8)15/15
SLSQP-11 2019-gecco-surr 2.5 (0.5)2.6 (0.5)2.8 (0.6)3.0 (0.6)3.2 (0.8)3.5 (0.9)3.8 (1)15/15

Δ fopt
f3, 20-D		 1e1
5066		 1e0
7626		 1e-1
7635		 1e-2
7637		 1e-3
7643		 1e-5
7646		 1e-7
7651		 #succ
15/15
CMA-ES 2019-gecco-surr 15 (8)∞∞∞∞∞∞3e60/15
lq-CMA-ES 2019-gecco-surr 11 (12)∞∞∞∞∞∞2e60/15
SLSQP+CMA 2019-gecco-surr 11 (8)∞∞∞∞∞∞3e60/15
SLSQP-11 2019-gecco-surr ∞∞∞∞∞∞∞4e50/15

Δ fopt
f4, 20-D		 1e1
4722		 1e0
7628		 1e-1
7666		 1e-2
7686		 1e-3
7700		 1e-5
7758		 1e-7
1.4e5		 #succ
9/15
CMA-ES 2019-gecco-surr ∞∞∞∞∞∞∞2e60/15
lq-CMA-ES 2019-gecco-surr ∞∞∞∞∞∞∞2e60/15
SLSQP+CMA 2019-gecco-surr ∞∞∞∞∞∞∞2e60/15
SLSQP-11 2019-gecco-surr ∞∞∞∞∞∞∞4e50/15

Δ fopt
f5, 20-D		 1e1
41		 1e0
41		 1e-1
41		 1e-2
41		 1e-3
41		 1e-5
41		 1e-7
41		 #succ
15/15
CMA-ES 2019-gecco-surr 5.1 (0.8)6.1 (1)6.4 (1)6.4 (1)6.4 (1)6.4 (1)6.4 (1)15/15
lq-CMA-ES 2019-gecco-surr 2.0 (0.8)2.4 (0.7)2.4 (0.7)2.4 (0.7)2.4 (0.7)2.4 (0.7)2.4 (0.7)15/15
SLSQP+CMA 2019-gecco-surr 1.7 (0)2.2 (0)2.2 (0)2.2 (0)2.2 (0)2.2 (0)2.2 (0)15/15
SLSQP-11 2019-gecco-surr 1.7 (0)2.2 (0)2.2 (0)2.2 (0)2.2 (0)2.2 (0)2.2 (0)15/15

Δ fopt
f6, 20-D		 1e1
1296		 1e0
2343		 1e-1
3413		 1e-2
4255		 1e-3
5220		 1e-5
6728		 1e-7
8409		 #succ
15/15
CMA-ES 2019-gecco-surr 1.5 (0.3)1.2 (0.2)1.1 (0.1)1.1 (0.1)1.0 (0.1)1.1 (0.1)1.1 (0.1)15/15
lq-CMA-ES 2019-gecco-surr 1.4 (0.3)1.2 (0.2)1.1 (0.2)1.1 (0.1)1.1 (0.2)1.2 (0.2)1.2 (0.1)15/15
SLSQP+CMA 2019-gecco-surr 3.2 (3)2.6 (1)2.7 (0.8)2.7 (0.1)2.4 (0.1)2.1 (0.1)1.9 (0.1)15/15
SLSQP-11 2019-gecco-surr 2.5 (0.6)2.0 (0.5)2.1 (0.7)2.3 (0.9)2.4 (1)2.6 (0.9)8.7 (8)0/15

Δ fopt
f7, 20-D		 1e1
1351		 1e0
4274		 1e-1
9503		 1e-2
16523		 1e-3
16524		 1e-5
16524		 1e-7
16969		 #succ
15/15
CMA-ES 2019-gecco-surr 1.4 (1)2.3 (0.8)1.5 (0.6)0.92 (0.3)0.92 (0.3)0.92 (0.3)0.90 (0.3)15/15
lq-CMA-ES 2019-gecco-surr 1.0 (0.9)1.6 (0.6)0.99 (0.6)0.70 (0.3)0.70 (0.3)0.70 (0.3)0.69 (0.3)15/15
SLSQP+CMA 2019-gecco-surr 0.92 (0.2)2.4 (0.8)1.5 (0.7)1.0 (0.4)1.0 (0.4)1.0 (0.4)1.00 (0.3)15/15
SLSQP-11 2019-gecco-surr ∞∞∞∞∞∞∞4e50/15

Δ fopt
f8, 20-D		 1e1
2039		 1e0
3871		 1e-1
4040		 1e-2
4148		 1e-3
4219		 1e-5
4371		 1e-7
4484		 #succ
15/15
CMA-ES 2019-gecco-surr 3.3 (0.9)3.9 (3)4.2 (3)4.2 (3)4.3 (3)4.3 (3)4.3 (2)15/15
lq-CMA-ES 2019-gecco-surr 1.4 (0.4)1.4 (1)1.5 (1)1.5 (1.0)1.5 (1.0)1.5 (0.9)1.5 (0.9)15/15
SLSQP+CMA 2019-gecco-surr 0.79 (0.2)↑10.93 (1)1.0 (1)1.0 (2)1.0 (2)1.0 (2)1.0 (2)15/15
SLSQP-11 2019-gecco-surr 0.78 (0.2)↑10.76 (0.6)0.78 (0.5)0.78 (0.5)0.78 (0.5)0.77 (0.5)0.76 (0.5)15/15

Δ fopt
f9, 20-D		 1e1
1716		 1e0
3102		 1e-1
3277		 1e-2
3379		 1e-3
3455		 1e-5
3594		 1e-7
3727		 #succ
15/15
CMA-ES 2019-gecco-surr 3.6 (0.5)3.9 (0.3)4.2 (0.3)4.3 (0.3)4.3 (0.3)4.4 (0.3)4.4 (0.3)15/15
lq-CMA-ES 2019-gecco-surr 1.5 (0.2)1.5 (0.2)1.6 (0.2)1.6 (0.1)1.6 (0.1)1.6 (0.1)1.6 (0.1)15/15
SLSQP+CMA 2019-gecco-surr 0.35 (0)↑00.28 (0)↑00.32 (0)↑00.35 (0)↑00.36 (3e-3)↑00.36 (3e-3)↑00.36 (3e-3)↑015/15
SLSQP-11 2019-gecco-surr 0.35 (0)↑00.28 (0)↑00.32 (0)↑00.35 (0)↑00.36 (3e-3)↑00.36 (3e-3)↑00.36 (3e-3)↑015/15

Δ fopt
f10, 20-D		 1e1
7413		 1e0
8661		 1e-1
10735		 1e-2
13641		 1e-3
14920		 1e-5
17073		 1e-7
17476		 #succ
15/15
CMA-ES 2019-gecco-surr 1.1 (0.2)1.1 (0.1)1.0 (0.1)0.84 (0.1)↑00.80 (0.0)↑00.74 (0.0)↑00.75 (0.0)↑015/15
lq-CMA-ES 2019-gecco-surr 0.21 (0.0)↑00.21 (0.0)↑00.19 (0.0)↑00.16 (0.0)↑00.15 (0.0)↑00.15 (0.0)↑00.17 (0.0)★2↑015/15
SLSQP+CMA 2019-gecco-surr 0.13 (0.0)↑00.12 (0.0)↑00.10 (0.0)↑00.09 (0.0)↑00.08 (0.0)↑00.13 (0.0)↑00.84 (0.0)↑115/15
SLSQP-11 2019-gecco-surr 0.13 (0.0)↑00.12 (0.0)↑00.10 (0.0)↑00.09 (0.0)↑00.08 (0.0)↑00.09 (0.0)↑08.0 (11)1/15

Δ fopt
f11, 20-D		 1e1
1002		 1e0
2228		 1e-1
6278		 1e-2
8586		 1e-3
9762		 1e-5
12285		 1e-7
14831		 #succ
15/15
CMA-ES 2019-gecco-surr 4.3 (0.3)2.2 (0.1)0.85 (0.0)↑00.67 (0.0)↑00.62 (0.0)↑00.55 (0.0)↑00.49 (0.0)↑015/15
lq-CMA-ES 2019-gecco-surr 1.4 (0.1)0.70 (0.1)↑00.27 (0.0)↑00.21 (0.0)↑00.20 (0.0)↑00.18 (0.0)↑00.17 (0.0)★2↑015/15
SLSQP+CMA 2019-gecco-surr 0.15 (0.1)↑00.09 (0.0)↑00.04 (0.0)↑00.03 (6e-3)↑00.03 (7e-3)↑00.20 (0.3)↑10.60 (0.0)↑115/15
SLSQP-11 2019-gecco-surr 0.15 (0.1)↑00.09 (0.0)↑00.04 (0.0)↑00.03 (9e-3)↑00.03 (9e-3)↑00.64 (0.2)↑136 (41)0/15

Δ fopt
f12, 20-D		 1e1
1042		 1e0
1938		 1e-1
2740		 1e-2
3156		 1e-3
4140		 1e-5
12407		 1e-7
13827		 #succ
15/15
CMA-ES 2019-gecco-surr 2.9 (2)3.8 (2)3.9 (2)4.0 (2)3.5 (1)1.5 (0.5)1.5 (0.5)15/15
lq-CMA-ES 2019-gecco-surr 1.1 (0.4)★21.4 (0.9)1.5 (1.0)1.6 (0.9)1.4 (0.7)0.74 (0.2)↑10.85 (0.2)★215/15
SLSQP+CMA 2019-gecco-surr 2.8 (2)2.1 (1)1.9 (1.0)2.4 (3)2.0 (2)1.3 (0.9)2.0 (0.4)15/15
SLSQP-11 2019-gecco-surr 2.8 (2)2.1 (1)1.7 (1)1.6 (0.9)1.3 (0.7)1.8 (1)3.3 (2)9/15

Δ fopt
f13, 20-D		 1e1
652		 1e0
2021		 1e-1
2751		 1e-2
3507		 1e-3
18749		 1e-5
24455		 1e-7
30201		 #succ
15/15
CMA-ES 2019-gecco-surr 2.9 (2)2.8 (2)4.4 (3)4.6 (2)1.0 (0.3)1.2 (0.6)1.1 (0.5)15/15
lq-CMA-ES 2019-gecco-surr 2.0 (1)2.3 (1)2.7 (1)2.4 (0.9)0.48 (0.2)↑00.69 (0.4)★0.83 (0.4)15/15
SLSQP+CMA 2019-gecco-surr 1.2 (0.1)0.55 (0.0)↑00.51 (0.0)↑00.50 (0.0)↑01.0 (0.8)1.4 (0.7)1.6 (0.6)15/15
SLSQP-11 2019-gecco-surr 1.2 (0.1)0.55 (0.0)↑00.51 (0.0)↑00.50 (0.0)↑031 (64)131 (88)∞5e50/15

Δ fopt
f14, 20-D		 1e1
75		 1e0
239		 1e-1
304		 1e-2
451		 1e-3
932		 1e-5
1648		 1e-7
15661		 #succ
15/15
CMA-ES 2019-gecco-surr 2.9 (1)2.5 (0.5)3.4 (0.6)3.9 (0.5)3.2 (0.4)3.9 (0.3)0.66 (0.0)↑015/15
lq-CMA-ES 2019-gecco-surr 3.2 (1)2.2 (0.5)2.7 (0.4)2.8 (0.3)1.9 (0.1)1.6 (0.1)0.36 (0.0)★4↑015/15
SLSQP+CMA 2019-gecco-surr 0.75 (0.2)↑00.52 (0.1)↑00.69 (0.1)↑00.76 (0.1)↑00.53 (0.0)↑00.51 (0.1)↑00.97 (0.3)15/15
SLSQP-11 2019-gecco-surr 0.75 (0.2)↑00.52 (0.1)↑00.69 (0.1)↑00.76 (0.1)↑00.53 (0.0)↑00.51 (0.0)↑0∞4e50/15

Δ fopt
f15, 20-D		 1e1
30378		 1e0
1.5e5		 1e-1
3.1e5		 1e-2
3.2e5		 1e-3
3.2e5		 1e-5
4.5e5		 1e-7
4.6e5		 #succ
15/15
CMA-ES 2019-gecco-surr 1.0 (0.6)1.1 (0.3)0.65 (0.3)0.66 (0.3)0.66 (0.3)0.49 (0.2)↑00.50 (0.2)↑015/15
lq-CMA-ES 2019-gecco-surr 0.67 (0.5)0.81 (0.3)0.46 (0.2)↑00.46 (0.2)↑00.45 (0.2)↑00.32 (0.1)↑00.32 (0.1)↑015/15
SLSQP+CMA 2019-gecco-surr 0.98 (0.6)1.0 (0.3)0.68 (0.2)0.69 (0.2)0.69 (0.2)0.51 (0.2)↑00.52 (0.2)↑015/15
SLSQP-11 2019-gecco-surr ∞∞∞∞∞∞∞4e50/15

Δ fopt
f16, 20-D		 1e1
1384		 1e0
27265		 1e-1
77015		 1e-2
1.4e5		 1e-3
1.9e5		 1e-5
2.0e5		 1e-7
2.2e5		 #succ
15/15
CMA-ES 2019-gecco-surr 2.1 (1)0.94 (0.4)0.95 (0.6)0.69 (0.3)0.72 (0.5)0.84 (0.6)0.78 (0.5)15/15
lq-CMA-ES 2019-gecco-surr 1.0 (0.2)★30.93 (1)0.95 (0.7)0.67 (0.4)0.93 (0.6)0.94 (0.6)0.87 (0.6)15/15
SLSQP+CMA 2019-gecco-surr 3.3 (0.4)0.85 (0.6)0.79 (0.3)0.85 (0.6)1.4 (1)1.8 (1)1.6 (0.9)15/15
SLSQP-11 2019-gecco-surr ∞∞∞∞∞∞∞4e50/15

Δ fopt
f17, 20-D		 1e1
63		 1e0
1030		 1e-1
4005		 1e-2
12242		 1e-3
30677		 1e-5
56288		 1e-7
80472		 #succ
15/15
CMA-ES 2019-gecco-surr 1.1 (0.9)0.81 (0.2)0.94 (2)1.2 (1)0.77 (0.3)0.79 (0.4)0.87 (0.3)15/15
lq-CMA-ES 2019-gecco-surr 2.4 (1)1.7 (0.5)1.7 (2)1.9 (1)1.1 (0.8)1.2 (0.4)1.0 (0.0)15/15
SLSQP+CMA 2019-gecco-surr 12 (22)3.1 (0.8)2.0 (2)1.1 (0.7)0.93 (0.4)0.98 (0.4)0.99 (0.3)15/15
SLSQP-11 2019-gecco-surr 111 (71)∞∞∞∞∞∞4e50/15

Δ fopt
f18, 20-D		 1e1
621		 1e0
3972		 1e-1
19561		 1e-2
28555		 1e-3
67569		 1e-5
1.3e5		 1e-7
1.5e5		 #succ
15/15
CMA-ES 2019-gecco-surr 0.94 (0.3)0.75 (0.2)↑11.1 (0.6)1.2 (1)0.93 (0.9)1.0 (0.4)1.00 (0.4)15/15
lq-CMA-ES 2019-gecco-surr 0.90 (0.5)2.9 (6)1.6 (1)1.7 (0.6)1.1 (0.5)0.79 (0.4)0.78 (0.3)15/15
SLSQP+CMA 2019-gecco-surr 4.3 (1)2.1 (2)0.96 (0.8)1.1 (0.8)0.88 (0.2)0.77 (0.3)0.82 (0.3)15/15
SLSQP-11 2019-gecco-surr ∞∞∞∞∞∞∞4e50/15

Δ fopt
f19, 20-D		 1e1
1		 1e0
1		 1e-1
3.4e5		 1e-2
4.7e6		 1e-3
6.2e6		 1e-5
6.7e6		 1e-7
6.7e6		 #succ
15/15
CMA-ES 2019-gecco-surr 1 (0)1 (0)1.3 (0.5)0.49 (0.1)↑00.56 (0.2)↑10.61 (0.3)0.62 (0.5)13/15
lq-CMA-ES 2019-gecco-surr 1 (0)1 (0)0.56 (0.4)↑10.47 (0.3)↑00.45 (0.2)↑00.46 (0.2)↑00.47 (0.2)↑015/15
SLSQP+CMA 2019-gecco-surr 1 (0)1 (0)5.6e-4 (0)↑00.54 (0.2)↑00.46 (0.1)↑00.49 (0.2)↑00.49 (0.2)↑015/15
SLSQP-11 2019-gecco-surr 1 (0)1 (0)5.6e-4 (0)↑0∞∞∞∞5e50/15

Δ fopt
f20, 20-D		 1e1
82		 1e0
46150		 1e-1
3.1e6		 1e-2
5.5e6		 1e-3
5.5e6		 1e-5
5.6e6		 1e-7
5.6e6		 #succ
14/15
CMA-ES 2019-gecco-surr 3.7 (0.8)3.9 (1)0.69 (0.4)0.62 (0.2)↑00.62 (0.2)↑00.63 (0.2)↑00.63 (0.2)↑015/15
lq-CMA-ES 2019-gecco-surr 1.6 (0.6)3.0 (1.0)0.70 (0.3)0.56 (0.2)↑00.56 (0.2)↑00.55 (0.2)↑00.55 (0.2)↑015/15
SLSQP+CMA 2019-gecco-surr 0.65 (0)↑03.5 (1)0.74 (0.4)0.68 (0.2)↑10.68 (0.2)↑10.68 (0.2)↑10.68 (0.2)↑114/15
SLSQP-11 2019-gecco-surr 0.65 (0)↑014 (13)∞∞∞∞∞4e50/15

Δ fopt
f21, 20-D		 1e1
561		 1e0
6541		 1e-1
14103		 1e-2
14318		 1e-3
14643		 1e-5
15567		 1e-7
17589		 #succ
15/15
CMA-ES 2019-gecco-surr 4.7 (11)121 (165)62 (76)62 (77)61 (80)57 (70)51 (60)9/15
lq-CMA-ES 2019-gecco-surr 6.1 (2)47 (74)35 (46)34 (41)34 (36)32 (46)28 (45)7/15
SLSQP+CMA 2019-gecco-surr 4.5 (5)116 (179)55 (119)54 (98)53 (96)50 (90)44 (81)9/15
SLSQP-11 2019-gecco-surr 0.66 (0.8)0.56 (0.6)2.8 (6)2.8 (6)2.7 (6)2.6 (5)3.3 (5)7/15

Δ fopt
f22, 20-D		 1e1
467		 1e0
5580		 1e-1
23491		 1e-2
24163		 1e-3
24948		 1e-5
26847		 1e-7
1.3e5		 #succ
12/15
CMA-ES 2019-gecco-surr 6.6 (13)156 (209)∞∞∞∞∞1e60/15
lq-CMA-ES 2019-gecco-surr 4.5 (6)103 (135)∞∞∞∞∞4e50/15
SLSQP+CMA 2019-gecco-surr 2.1 (6)144 (100)∞∞∞∞∞1e60/15
SLSQP-11 2019-gecco-surr 1.0 (1)3.2 (2)4.1 (7)★34.0 (6)★33.8 (6)★33.6 (6)★34.3 (7)★31/15

Δ fopt
f23, 20-D		 1e1
3.0		 1e0
1614		 1e-1
67457		 1e-2
3.7e5		 1e-3
4.9e5		 1e-5
8.1e5		 1e-7
8.4e5		 #succ
15/15
CMA-ES 2019-gecco-surr 1.9 (2)1064 (2041)154 (182)∞∞∞∞7e60/15
lq-CMA-ES 2019-gecco-surr 6.8 (13)2.4 (0.8)152 (174)∞∞∞∞7e60/15
SLSQP+CMA 2019-gecco-surr 13 (11)1346 (2200)200 (340)74 (59)192 (233)116 (145)112 (104)1/15
SLSQP-11 2019-gecco-surr 13 (10)18 (27)∞∞∞∞∞4e50/15

Δ fopt
f24, 20-D		 1e1
1.3e6		 1e0
7.5e6		 1e-1
5.2e7		 1e-2
5.2e7		 1e-3
5.2e7		 1e-5
5.2e7		 1e-7
5.2e7		 #succ
3/15
CMA-ES 2019-gecco-surr 16 (16)10 (16)∞∞∞∞∞5e60/15
lq-CMA-ES 2019-gecco-surr 14 (26)8.5 (10)∞∞∞∞∞5e60/15
SLSQP+CMA 2019-gecco-surr ∞∞∞∞∞∞∞5e60/15
SLSQP-11 2019-gecco-surr ∞∞∞∞∞∞∞4e50/15

Previous dimension | Dimension = 40 | First dimension

Δ fopt
f1, 40-D		 1e1
83		 1e0
83		 1e-1
83		 1e-2
83		 1e-3
83		 1e-5
83		 1e-7
83		 #succ
15/15
CMA-ES 2019-gecco-surr 8.4 (0.8)14 (0.9)20 (1)26 (2)32 (2)45 (2)57 (2)15/15
lq-CMA-ES 2019-gecco-surr 1.5 (0.0)1.7 (0.2)1.9 (0.3)2.2 (0.0)2.4 (0.0)2.7 (0.1)3.1 (0.0)15/15
SLSQP+CMA 2019-gecco-surr 0.54 (0)↑00.54 (0)↑00.54 (0)↑00.54 (0)↑00.54 (0)↑00.54 (0)↑00.54 (0)↑015/15
SLSQP-11 2019-gecco-surr 0.54 (0)↑00.54 (0)↑00.54 (0)↑00.54 (0)↑00.54 (0)↑00.54 (0)↑00.54 (0)↑015/15

Δ fopt
f2, 40-D		 1e1
796		 1e0
797		 1e-1
799		 1e-2
799		 1e-3
800		 1e-5
802		 1e-7
804		 #succ
15/15
CMA-ES 2019-gecco-surr 38 (4)46 (3)52 (4)55 (3)58 (4)62 (2)63 (1)15/15
lq-CMA-ES 2019-gecco-surr 5.1 (2)6.1 (2)7.1 (2)7.8 (2)8.4 (2)9.5 (2)11 (3)15/15
SLSQP+CMA 2019-gecco-surr 3.7 (0.6)4.0 (0.7)4.2 (0.7)4.5 (0.7)4.7 (0.6)5.1 (0.6)5.5 (0.7)15/15
SLSQP-11 2019-gecco-surr 3.7 (0.6)4.0 (0.7)4.2 (0.7)4.5 (0.7)4.7 (0.6)5.1 (0.6)5.5 (0.7)15/15

Δ fopt
f3, 40-D		 1e1
15526		 1e0
15602		 1e-1
15612		 1e-2
15641		 1e-3
15646		 1e-5
15651		 1e-7
15656		 #succ
15/15
CMA-ES 2019-gecco-surr 4981 (4714)∞∞∞∞∞∞5e60/15
lq-CMA-ES 2019-gecco-surr ∞∞∞∞∞∞∞9e40/15
SLSQP+CMA 2019-gecco-surr 5009 (4342)∞∞∞∞∞∞6e60/15
SLSQP-11 2019-gecco-surr ∞∞∞∞∞∞∞8e50/15

Δ fopt
f4, 40-D		 1e1
15536		 1e0
15601		 1e-1
15659		 1e-2
15678		 1e-3
15703		 1e-5
15733		 1e-7
2.8e5		 #succ
6/15
CMA-ES 2019-gecco-surr ∞∞∞∞∞∞∞5e60/15
lq-CMA-ES 2019-gecco-surr ∞∞∞∞∞∞∞1e50/15
SLSQP+CMA 2019-gecco-surr ∞∞∞∞∞∞∞5e60/15
SLSQP-11 2019-gecco-surr ∞∞∞∞∞∞∞8e50/15

Δ fopt
f5, 40-D		 1e1
98		 1e0
116		 1e-1
120		 1e-2
121		 1e-3
121		 1e-5
121		 1e-7
121		 #succ
15/15
CMA-ES 2019-gecco-surr 4.7 (0.5)4.7 (0.7)4.6 (0.5)4.6 (0.5)4.6 (0.5)4.6 (0.5)4.6 (0.5)15/15
lq-CMA-ES 2019-gecco-surr 1.6 (0.4)1.6 (0.4)1.6 (0.3)1.6 (0.3)1.6 (0.3)1.6 (0.3)1.6 (0.3)15/15
SLSQP+CMA 2019-gecco-surr 1.3 (0)1.5 (0)1.4 (0)1.4 (0)1.4 (0)1.4 (0)1.4 (0)15/15
SLSQP-11 2019-gecco-surr 1.3 (0)1.5 (0)1.4 (0)1.4 (0)1.4 (0)1.4 (0)1.4 (0)15/15

Δ fopt
f6, 40-D		 1e1
3507		 1e0
5523		 1e-1
7168		 1e-2
9470		 1e-3
11538		 1e-5
15007		 1e-7
19222		 #succ
15/15
CMA-ES 2019-gecco-surr 1.5 (0.2)1.4 (0.2)1.4 (0.1)1.3 (0.1)1.3 (0.1)1.3 (0.1)1.2 (0.1)15/15
lq-CMA-ES 2019-gecco-surr 1.4 (0.2)1.4 (0.2)1.4 (0.2)1.3 (0.2)1.3 (0.2)1.4 (0.1)1.4 (0.2)15/15
SLSQP+CMA 2019-gecco-surr 3.8 (1)3.4 (1.0)3.2 (0.1)2.7 (0.1)2.4 (0.1)2.2 (0.1)1.9 (0.1)15/15
SLSQP-11 2019-gecco-surr 3.5 (2)3.5 (2)3.7 (2)3.6 (2)3.7 (1)4.1 (1.0)16 (14)0/15

Δ fopt
f7, 40-D		 1e1
10698		 1e0
17839		 1e-1
41037		 1e-2
66294		 1e-3
66294		 1e-5
66294		 1e-7
68145		 #succ
15/15
CMA-ES 2019-gecco-surr 1.1 (0.9)3.8 (1)1.9 (0.5)1.2 (0.3)1.2 (0.3)1.2 (0.3)1.2 (0.3)15/15
lq-CMA-ES 2019-gecco-surr 1.3 (0.5)2.1 (1)★21.2 (0.4)0.77 (0.2)0.77 (0.2)0.77 (0.2)0.76 (0.2)15/15
SLSQP+CMA 2019-gecco-surr 1.5 (0.7)3.6 (1)1.8 (0.6)1.1 (0.4)1.1 (0.4)1.1 (0.4)1.1 (0.4)15/15
SLSQP-11 2019-gecco-surr ∞∞∞∞∞∞∞8e50/15

Δ fopt
f8, 40-D		 1e1
7080		 1e0
10655		 1e-1
11012		 1e-2
11265		 1e-3
11430		 1e-5
11701		 1e-7
11969		 #succ
15/15
CMA-ES 2019-gecco-surr 5.6 (0.7)5.7 (4)5.8 (4)5.9 (4)5.9 (4)5.9 (4)5.8 (4)15/15
lq-CMA-ES 2019-gecco-surr 2.2 (0.3)2.0 (0.3)2.1 (0.3)2.1 (0.2)2.1 (0.2)2.1 (0.2)2.1 (0.2)15/15
SLSQP+CMA 2019-gecco-surr 0.94 (0.5)1.4 (3)1.4 (3)1.4 (3)1.4 (3)1.4 (3)1.4 (3)15/15
SLSQP-11 2019-gecco-surr 0.94 (0.5)0.94 (0.4)0.94 (0.3)0.94 (0.3)0.94 (0.3)0.93 (0.3)0.91 (0.3)15/15

Δ fopt
f9, 40-D		 1e1
6122		 1e0
12982		 1e-1
13300		 1e-2
13496		 1e-3
13651		 1e-5
13909		 1e-7
14142		 #succ
15/15
CMA-ES 2019-gecco-surr 6.5 (0.4)4.1 (0.2)4.3 (0.2)4.3 (0.1)4.3 (0.2)4.4 (0.1)4.4 (0.1)15/15
lq-CMA-ES 2019-gecco-surr 2.7 (0.3)1.7 (0.2)1.8 (0.2)1.8 (0.2)1.8 (0.2)1.9 (0.2)1.9 (0.2)15/15
SLSQP+CMA 2019-gecco-surr 0.34 (0)↑00.20 (0)↑00.22 (0)↑00.23 (0)↑00.24 (0)↑00.24 (0)↑00.24 (1e-3)↑015/15
SLSQP-11 2019-gecco-surr 0.34 (0)↑00.20 (0)↑00.22 (0)↑00.23 (0)↑00.24 (0)↑00.24 (0)↑00.24 (1e-3)↑015/15

Δ fopt
f10, 40-D		 1e1
25536		 1e0
29762		 1e-1
35676		 1e-2
51579		 1e-3
56007		 1e-5
65128		 1e-7
70824		 #succ
15/15
CMA-ES 2019-gecco-surr 1.3 (0.1)1.3 (0.1)1.2 (0.1)0.87 (0.1)↑00.84 (0.1)↑00.76 (0.0)↑00.72 (0.0)↑015/15
lq-CMA-ES 2019-gecco-surr 0.25 (0.0)↑00.25 (0.0)↑00.23 (0.0)↑00.18 (0.0)↑00.17 (0.0)↑00.16 (0.0)↑00.17 (0.0)★4↑015/15
SLSQP+CMA 2019-gecco-surr 0.12 (0.0)↑00.11 (0.0)↑00.10 (0.0)↑00.07 (0.0)↑00.07 (1e-2)↑00.12 (0.0)↑00.80 (0.0)↑015/15
SLSQP-11 2019-gecco-surr 0.12 (0.0)↑00.11 (0.0)↑00.10 (0.0)↑00.07 (0.0)↑00.07 (1e-2)↑00.07 (0.0)↑053 (46)0/15

Δ fopt
f11, 40-D		 1e1
2368		 1e0
4855		 1e-1
11681		 1e-2
25315		 1e-3
29749		 1e-5
38949		 1e-7
48211		 #succ
15/15
CMA-ES 2019-gecco-surr 5.8 (0.3)3.1 (0.1)1.4 (0.0)0.66 (0.0)↑00.59 (0.0)↑00.48 (0.0)↑00.42 (6e-3)↑015/15
lq-CMA-ES 2019-gecco-surr 2.2 (0.2)1.2 (0.1)0.55 (0.0)↑00.27 (0.0)↑00.24 (0.0)↑00.21 (0.0)↑00.19 (8e-3)★2↑015/15
SLSQP+CMA 2019-gecco-surr 0.09 (0.0)↑00.06 (0.0)↑00.03 (7e-3)↑00.02 (4e-3)↑00.02 (4e-3)↑00.13 (0.3)↑00.46 (0.0)↑015/15
SLSQP-11 2019-gecco-surr 0.09 (0.0)↑00.06 (0.0)↑00.03 (7e-3)↑00.02 (4e-3)↑00.02 (4e-3)↑00.04 (0.0)↑09.0 (10)0/15

Δ fopt
f12, 40-D		 1e1
4169		 1e0
7452		 1e-1
9174		 1e-2
10751		 1e-3
13146		 1e-5
22758		 1e-7
25192		 #succ
15/15
CMA-ES 2019-gecco-surr 2.0 (2)2.1 (2)2.3 (1)2.3 (1)2.2 (0.9)1.5 (0.6)1.6 (0.5)15/15
lq-CMA-ES 2019-gecco-surr 1.0 (0.6)1.2 (0.7)1.4 (0.6)1.4 (0.4)1.4 (0.4)1.1 (0.2)1.2 (0.2)15/15
SLSQP+CMA 2019-gecco-surr 2.8 (3)2.4 (2)2.3 (2)2.4 (2)2.3 (2)1.7 (1)2.0 (0.7)15/15
SLSQP-11 2019-gecco-surr 2.4 (2)1.6 (1)1.4 (0.9)1.3 (0.8)1.1 (0.7)4.0 (4)8.1 (18)11/15

Δ fopt
f13, 40-D		 1e1
2029		 1e0
6916		 1e-1
8734		 1e-2
11861		 1e-3
71936		 1e-5
98467		 1e-7
1.2e5		 #succ
15/15
CMA-ES 2019-gecco-surr 2.4 (3)3.7 (4)6.3 (6)6.9 (4)1.3 (0.6)2.0 (0.8)2.2 (0.9)15/15
lq-CMA-ES 2019-gecco-surr 1.9 (0.2)3.4 (3)5.2 (3)5.8 (1)1.0 (0.2)1.2 (0.6)1.4 (0.8)9/15
SLSQP+CMA 2019-gecco-surr 1.2 (0.0)0.48 (0.0)↑00.52 (0.0)↑00.45 (0.0)↑01.2 (1)1.7 (0.8)2.2 (1)15/15
SLSQP-11 2019-gecco-surr 1.2 (0.0)0.48 (0.0)↑00.52 (0.0)↑00.45 (0.0)↑025 (32)∞∞9e50/15

Δ fopt
f14, 40-D		 1e1
304		 1e0
616		 1e-1
777		 1e-2
1105		 1e-3
2207		 1e-5
4825		 1e-7
57711		 #succ
15/15
CMA-ES 2019-gecco-surr 2.1 (0.4)2.2 (0.3)2.8 (0.3)3.5 (0.3)3.6 (0.1)4.3 (0.2)0.65 (0.0)↑015/15
lq-CMA-ES 2019-gecco-surr 2.1 (0.6)2.1 (0.4)2.6 (0.4)3.3 (0.3)2.4 (0.2)1.9 (0.2)0.36 (0.0)★4↑015/15
SLSQP+CMA 2019-gecco-surr 0.50 (0.1)↑00.49 (0.1)↑00.60 (0.1)↑00.75 (0.1)↑00.56 (0.1)↑00.47 (0.0)↑00.76 (0.1)↑015/15
SLSQP-11 2019-gecco-surr 0.50 (0.1)↑00.49 (0.1)↑00.60 (0.1)↑00.75 (0.1)↑00.56 (0.1)↑00.47 (0.0)↑0∞8e50/15

Δ fopt
f15, 40-D		 1e1
1.9e5		 1e0
7.9e5		 1e-1
1.0e6		 1e-2
1.1e6		 1e-3
1.1e6		 1e-5
1.1e6		 1e-7
1.1e6		 #succ
15/15
CMA-ES 2019-gecco-surr 0.80 (0.3)0.64 (0.2)↑10.73 (0.2)↑10.74 (0.2)↑10.75 (0.2)↑10.76 (0.2)↑10.77 (0.2)↑115/15
lq-CMA-ES 2019-gecco-surr 0.76 (0.5)2.0 (2)1.5 (2)1.5 (1)1.5 (2)1.4 (1)1.4 (2)1/15
SLSQP+CMA 2019-gecco-surr 0.76 (0.3)0.60 (0.3)0.55 (0.2)↑00.55 (0.2)↑00.56 (0.2)↑00.57 (0.2)↑00.58 (0.2)↑015/15
SLSQP-11 2019-gecco-surr ∞∞∞∞∞∞∞8e50/15

Δ fopt
f16, 40-D		 1e1
5244		 1e0
72122		 1e-1
3.2e5		 1e-2
7.1e5		 1e-3
1.4e6		 1e-5
2.0e6		 1e-7
2.0e6		 #succ
15/15
CMA-ES 2019-gecco-surr 0.93 (0.3)0.92 (0.4)0.70 (0.3)0.47 (0.3)↑10.36 (0.3)↑00.43 (0.2)↑00.44 (0.2)↑015/15
lq-CMA-ES 2019-gecco-surr 0.58 (0.1)★3↑00.83 (0.3)1.4 (1)∞∞∞∞1e50/15
SLSQP+CMA 2019-gecco-surr 2.8 (0.4)1.1 (0.4)0.60 (0.3)↑10.55 (0.4)↑10.57 (0.4)↑10.52 (1.0)0.53 (1.0)15/15
SLSQP-11 2019-gecco-surr ∞∞∞∞∞∞∞8e50/15

Δ fopt
f17, 40-D		 1e1
399		 1e0
4220		 1e-1
14158		 1e-2
34948		 1e-3
51958		 1e-5
1.3e5		 1e-7
2.7e5		 #succ
14/15
CMA-ES 2019-gecco-surr 0.49 (0.3)↑00.45 (0.1)↑00.73 (1)1.0 (0.1)1.2 (0.9)0.92 (0.2)0.70 (0.3)15/15
lq-CMA-ES 2019-gecco-surr 0.87 (0.3)0.47 (0.2)↑00.73 (0.8)1.1 (0.5)1.4 (1)5.0 (6)∞9e40/15
SLSQP+CMA 2019-gecco-surr 3.4 (9)2.3 (0.8)1.9 (1)1.3 (0.5)1.2 (0.3)1.1 (0.3)0.68 (0.3)15/15
SLSQP-11 2019-gecco-surr 12 (38)∞∞∞∞∞∞8e50/15

Δ fopt
f18, 40-D		 1e1
1442		 1e0
16998		 1e-1
47068		 1e-2
1.3e5		 1e-3
1.9e5		 1e-5
6.7e5		 1e-7
9.5e5		 #succ
15/15
CMA-ES 2019-gecco-surr 0.82 (0.2)0.59 (0.9)1.1 (0.5)0.66 (0.4)0.80 (0.5)0.59 (0.3)↑10.56 (0.2)↑115/15
lq-CMA-ES 2019-gecco-surr 0.74 (0.1)↑10.63 (0.7)1.5 (1)1.4 (2)∞∞∞1e50/15
SLSQP+CMA 2019-gecco-surr 6.0 (2)1.4 (0.9)1.2 (0.5)0.73 (0.4)0.80 (0.2)↑10.56 (0.2)↑00.49 (0.2)↑015/15
SLSQP-11 2019-gecco-surr ∞∞∞∞∞∞∞8e50/15

Δ fopt
f19, 40-D		 1e1
1		 1e0
1		 1e-1
1.4e6		 1e-2
1.7e7		 1e-3
2.6e7		 1e-5
4.5e7		 1e-7
4.5e7		 #succ
8/15
CMA-ES 2019-gecco-surr 1 (0)1 (0)1.2 (0.8)0.58 (0.3)0.69 (0.5)0.64 (0.5)0.64 (0.7)5/15
lq-CMA-ES 2019-gecco-surr 1 (0)1 (0)∞∞∞∞∞2e50/15
SLSQP+CMA 2019-gecco-surr 1 (0)1 (0)4.7e-4 (0)↑00.99 (0.7)1.5 (1)1.7 (2)1.7 (3)2/15
SLSQP-11 2019-gecco-surr 1 (0)1 (0)4.7e-4 (0)↑0∞∞∞∞9e50/15

Δ fopt
f20, 40-D		 1e1
222		 1e0
1.3e5		 1e-1
1.6e8		 1e-2
∞		 1e-3
∞		 1e-5
∞		 1e-7
∞		 #succ
0/15
CMA-ES 2019-gecco-surr 3.1 (0.4)12 (4)0.99 (1)    0/15
lq-CMA-ES 2019-gecco-surr 1.6 (0.3)∞∞    0/15
SLSQP+CMA 2019-gecco-surr 0.42 (0)↑010 (4)0.99 (1)    0/15
SLSQP-11 2019-gecco-surr 0.42 (0)↑0∞∞    0/15

Δ fopt
f21, 40-D		 1e1
1044		 1e0
21144		 1e-1
1.0e5		 1e-2
1.0e5		 1e-3
1.0e5		 1e-5
1.0e5		 1e-7
1.0e5		 #succ
26/30
CMA-ES 2019-gecco-surr 1.8 (4)134 (205)98 (85)98 (86)98 (128)97 (164)97 (115)3/15
lq-CMA-ES 2019-gecco-surr 1.9 (3)9.0 (13)2.5 (4)2.5 (3)2.5 (2)2.5 (4)2.5 (2)5/15
SLSQP+CMA 2019-gecco-surr 4.6 (5)59 (148)28 (37)28 (31)28 (43)28 (37)28 (36)7/15
SLSQP-11 2019-gecco-surr 0.55 (0.7)0.84 (2)0.29 (0.3)↑10.29 (0.3)↑10.29 (0.3)↑10.30 (0.3)↑11.6 (4)5/15

Δ fopt
f22, 40-D		 1e1
3090		 1e0
35442		 1e-1
6.5e5		 1e-2
6.5e5		 1e-3
6.5e5		 1e-5
6.5e5		 1e-7
6.5e5		 #succ
8/30
CMA-ES 2019-gecco-surr 64 (15)105 (192)∞∞∞∞∞2e60/15
lq-CMA-ES 2019-gecco-surr 8.1 (20)3.7 (3)∞∞∞∞∞1e50/15
SLSQP+CMA 2019-gecco-surr 135 (410)69 (70)∞∞∞∞∞2e60/15
SLSQP-11 2019-gecco-surr 0.83 (0.9)5.2 (11)2.4 (2)2.4 (4)2.4 (3)2.4 (2)8.6 (15)0/15

Δ fopt
f23, 40-D		 1e1
7.0		 1e0
11925		 1e-1
75453		 1e-2
6.6e5		 1e-3
1.3e6		 1e-5
3.2e6		 1e-7
3.4e6		 #succ
15/15
CMA-ES 2019-gecco-surr 1.3 (1)800 (1205)440 (500)176 (161)∞∞∞8e60/15
lq-CMA-ES 2019-gecco-surr 8.5 (13)0.66 (0.1)↑00.69 (0.8)★3∞∞★2∞★2∞1e5★20/15
SLSQP+CMA 2019-gecco-surr 12 (11)813 (1240)451 (393)180 (158)∞∞∞9e60/15
SLSQP-11 2019-gecco-surr 12 (11)6.8 (8)∞∞∞∞∞8e50/15

Δ fopt
f24, 40-D		 1e1
5.8e6		 1e0
9.8e7		 1e-1
3.0e8		 1e-2
3.0e8		 1e-3
3.0e8		 1e-5
3.0e8		 1e-7
3.0e8		 #succ
1/15
CMA-ES 2019-gecco-surr 22 (23)1.3 (2)0.44 (0.7)0.44 (0.4)0.44 (0.6)0.44 (0.7)0.44 (0.4)1/15
lq-CMA-ES 2019-gecco-surr 0.30 (0.3)∞∞∞∞∞∞1e50/15
SLSQP+CMA 2019-gecco-surr 4.2 (4)∞∞∞∞∞∞9e60/15
SLSQP-11 2019-gecco-surr ∞∞∞∞∞∞∞8e50/15

Expected runtime (ERT in number of f-evaluations) divided by the respective best ERT measured during BBOB-2009 in different dimensions. This ERT ratio and, in braces as dispersion measure, the half difference between 10 and 90%-tile of bootstrapped run lengths appear for each algorithm and target, the corresponding reference ERT  in the first row. The different target ∆f-values are shown in the top row. #succ is the number of trials that reached the (final) target fopt+ 10−8. The median number of conducted function evaluations is additionally given in italics, if the target in the last column was never reached. Entries, succeeded by a star, are statistically significantly better (according to the rank-sum test) when compared to all other algorithms of the table, with p = 0.05 or p = 10−k when the number k following the star is larger than 1, with Bonferroni correction by the number of functions (24). A ↑ signifies the number of trials that were worse than the ERT of the best algorithm from BBOB 2009 shown only when less than 10 percent were worse and the ERT was better. Best results are printed in bold.