cma.fitness_models.LQModel

Up to a full quadratic model using the pseudo inverse to compute the model coefficients.

The full model has 1 + 2n + n(n-1)/2 = n(n+3) + 1 parameters. Model building "works" with any number of data.

Model size 1.0 doesn't work well on bbob-f10, 1.1 however works fine.

TODO: change self.types: List[str] to self.type: str with only one entry

>>> import numpy as np
>>> import cma
>>> import cma.fitness_models as fm
>>> # fm.Logger, Logger = fm.LoggerDummy, fm.Logger
>>> m = fm.LQModel()
>>> for i in range(30):
...     x = np.random.randn(3)
...     y = cma.ff.elli(x - 1.2)
...     _ = m.add_data_row(x, y)
>>> assert np.allclose(m.coefficients, [
...   1.44144144e+06,
...  -2.40000000e+00,  -2.40000000e+03,  -2.40000000e+06,
...   1.00000000e+00,   1.00000000e+03,   1.00000000e+06,
...  -4.65661287e-10,  -6.98491931e-10,   1.97906047e-09,
...   ], atol=1e-5)
>>> assert np.allclose(m.xopt, [ 1.2,  1.2,  1.2])
>>> assert np.allclose(m.xopt, [ 1.2,  1.2,  1.2])

Check the same before the full model is build:

>>> m = fm.LQModel()
>>> m.settings.min_relative_size = 3 * m.settings.truncation_ratio
>>> for i in range(30):
...     x = np.random.randn(4)
...     y = cma.ff.elli(x - 1.2)
...     _ = m.add_data_row(x, y)
>>> print(m.types)
['quadratic']
>>> assert np.allclose(m.coefficients, [
...   1.45454544e+06,
...  -2.40000000e+00,  -2.40000000e+02,  -2.40000000e+04, -2.40000000e+06,
...   1.00000000e+00,   1.00000000e+02,   1.00000000e+04,   1.00000000e+06,
...   ])
>>> assert np.allclose(m.xopt, [ 1.2,  1.2,  1.2,  1.2])
>>> assert np.allclose(m.xopt, [ 1.2,  1.2,  1.2,  1.2])

Check the Hessian in the rotated case:

>>> fitness = cma.fitness_transformations.Rotated(cma.ff.elli)
>>> m = fm.LQModel(2, 2)
>>> for i in range(30):
...     x = np.random.randn(4) - 5
...     y = fitness(x - 2.2)
...     _ = m.add_data_row(x, y)
>>> R = fitness[1].dicMatrices[4]
>>> H = np.dot(np.dot(R.T, np.diag([1, 1e2, 1e4, 1e6])), R)
>>> assert np.all(np.isclose(H, m.hessian))
>>> assert np.allclose(m.xopt, 4 * [2.2])
>>> m.set_xoffset([2.335, 1.2, 2, 4])
>>> assert np.all(np.isclose(H, m.hessian))
>>> assert np.allclose(m.xopt, 4 * [2.2])

Check a simple linear case, the optimum is not necessarily at the expected position (the Hessian matrix is chosen somewhat arbitrarily)

>>> m = fm.LQModel()
>>> m.settings.min_relative_size = 4
>>> _ = m.add_data_row([1, 1, 1], 220 + 10)
>>> _ = m.add_data_row([2, 1, 1], 220)
>>> print(m.types)
[]
>>> assert np.allclose(m.coefficients, [80, -10, 80, 80])
>>> assert np.allclose(m.xopt, [22, -159, -159])  # [ 50,  -400, -400])  # depends on Hessian
>>> # fm.Logger = Logger

For results see:

Hansen (2019). A Global Surrogate Model for CMA-ES. In Genetic and Evolutionary Computation Conference (GECCO 2019), Proceedings, ACM.

lq-CMA-ES at http://lq-cma.gforge.inria.fr/ppdata-archives/pap-gecco2019/figure5/