comocma.como.Sofomore : API documentation

Sofomore framework for multiobjective optimization, with the 
ask-and-tell interface.
See: [Toure, Cheikh, et al. "Uncrowded Hypervolume Improvement: 
    COMO-CMA-ES and the Sofomore framework." 
    GECCO'19-Genetic and Evolutionary Computation Conference. 2019.].
    
Calling Sequences
=================

- ``moes = Sofomore(list_of_solvers_instances, reference_point, opts)``

- ``moes = Sofomore(list_of_solvers_instances, reference_point)``

- ``moes = Sofomore(list_of_solvers_instances, 
                         reference_point).optimize(objective_fcts)``

Arguments   
=========
`list_of_solvers_instances`
    list of instances of single-objective solvers.
    It's generally created via a factory function.
    Let's take the example of a factory function that returns cma-es 
    instances, called `get_cmas`. Then:
    ``list_of_solvers_instances = get_cmas(11 * [x0], sigma0)`` 
    creates a list of 11 cma-es instances of initial mean `x0` and initial 
    step-size `sigma0`.
    A single-objective solver instance must have the following
    attributes and methods:
        - `incumbent`: an attribute that gives an estimate of 
        the single-objective solver.
        - `objective_values`: an attribute that stores the objective 
        values of the incumbent.
        - `stop`: a method returning a dictionary representing the 
        termination status.
        - `ask`: generates new candidate solutions.
        - `tell`: passes the objective values and updates the states of the
        single-objective solvers.
            
        
`opts`
    opts, a dictionary with optional settings related to the 
    Sofomore framework. It mainly contains the following keys:
        - 'archive': default value is `True`. 
        If its value is `True`, tracks the non-dominated
        points that dominate the reference point, among the points evaluated
        so far during the optimization.
        The archive will not interfere with the optimization 
        process.
        - 'indicator_front': default value is `None`. Used via
        `self.indicator_front = IndicatorFront(self.opts['indicator_front'])` 
        within the __init__ method of Sofomore. See the class IndicatorFront
        for more details.
        - 'restart': used to define in __init__ the attribute `restart` via
        `self.restart = self.opts['restart']`.
        - 'update_order': default value is a function that takes a natural 
        integer as input and returns a random number between 0 and 1.
        It is used as a `key value` in: `sorted(..., key = ...)`, and guides the
        order in which the kernels will be updated during the optimization.


`reference_point`  
    reference point of the multiobjective optimization.
    Its default value is `None` but should be set by the user 
    beforehand to guarantee an optimal p-distribution convergence of 
    the Hypervolume indicator of `p` points towards the Pareto set/front.
    It can be changed dynamically by the user if needed.
    

Main interface / usage
======================
The interface is inherited from the generic `OOOptimizer`
class, which is the same interface used by the `pycma` module. An object 
instance is generated as following::
    
    >>> import cma, comocma
    >>> import numpy as np
    >>> reference_point = [11, 11]
    >>> num_kernels = 11 # the number of points we seek to have on the Pareto front
    >>> dimension = 10 # the dimension of the search space
    >>> x0 = dimension * [0] # an initial mean for cma
    >>> sigma0 = 0.2 # initial step-size for cma
    >>> list_of_solvers_instances = comocma.get_cmas(num_kernels * [x0], sigma0, {'verbose':-9})
    >>> # `comocma.get_cmas` is a factory function that returns `num_kernels` cma-es instances
    >>> moes = comocma.Sofomore(list_of_solvers_instances,
    ...                      reference_point) #instantiation of our MO optimizer

The least verbose interface is via the optimize method::
    >>> fitness = comocma.FitFun(cma.ff.sphere, lambda x: cma.ff.sphere(x-1)) # a callable bi-objective function
    >>> moes.optimize(fitness) # doctest:+ELLIPSIS
    Iterat #Fevals   Hypervolume   axis ratios   sigmas   min&max stds***
    
More verbosely, the optimization of the callable multiobjective function 
`fitness` is done via the `ask-and-tell` interface::
 
    >>> moes = comocma.Sofomore(list_of_solvers_instances, reference_point)
    >>> while not moes.stop() and moes.countiter < 30:
    ...     solutions = moes.ask() # `ask` delivers new candidate solutions
    ...     objective_values = [fitness(x) for x in solutions]
    ...     moes.tell(solutions, objective_values)
    ...  # `tell` updates the MO instance by passing the respective function values.
    ...     moes.disp() # display data on the evolution of the optimization 

One iteration of the `optimize` interface is equivalent to one step in the 
loop of the `ask-and-tell` interface. But for the latter, the prototyper has
more controls to analyse and guide the optimization, due to the access of 
the instance between the `ask` and the `tell` calls.

Attributes and Properties
=========================

- `archive`: list of non-dominated points among all points evaluated 
so far, which dominate the reference point.
- `countevals`: the number of function evaluations.
- `countiter`: the number of iterations.
- `dimension`: is the dimension of the search space.
- `indicator_front`: the indicator used as a changing fitness inside an
iteration of Sofomore. By default it is the UHVI of all the objective values
of the kernels' incumbents, except the kernel being optimized.
See the class `IndicatorFront` for more details.
- `isarchive`: a boolean accessible via `self.opts['archive']`. If `True`,
we keep track of the archive, otherwise we don't.
- `kernels`: initialized with `list_of_solvers_instances`, 
and is the list of single-objective solvers.
- `key_sort_indices`: default value is `self.opts['update_order']`.
It is a function used as a key to sort some kernels' indices, in order to
select the first indices during the call of the `ask` method.
- `logger`: an attribute that accounts for the way we log the Sofomore data
during an optimization. `self.logger` is an instance of the SofomoreDataLogger
class.
- `NDA`: it is the non-dominated archiving method used within Sofomore.
By default the class `BiobjectiveNondominatedSortedList` is used
in two objectives and the class (non tested yet) `NonDominatedList` is used
for three or more objectives.
- `opts`: passed options.
- `offspring`: list of tuples of the index of a kernel with its 
corresponding candidate solutions, that we generally get with the cma's 
`ask` method.
- `pareto_front_cut`: list of non-dominated points among the incumbents of 
`self.kernels`, which dominate the reference point.
- `pareto_set_cut`: preimage of `pareto_front_cut`.
- `reference_point`: the current reference point.
- `restart`: accessible via `self.opts['restart']`. If not `None`, a callback
function that adds kernels after a kernel of the running Sofomore instance
stops. Its default value is `None`, meaning that we do not do any restart.
- `_active_indices`: the indices of the kernels that have not stopped yet.
- `_last_stopped_kernel_id`: the index of the last stopped kernel.
- `_told_indices`: the kernels' indices for which we will evaluate the 
objective values of their incumbents, before the next call of the `tell` 
method. And before the first call of `tell`, they are the indices of all the
initial kernels (i.e. `range(len(self))`). And before another call of 
`tell`, they are the indices of the kernels from which we have sampled new 
candidate solutions during the penultimate `ask` method. 
Note that we should call the `ask` method before any call of the `tell`
method.

Method	__init__	No summary
Method	__iter__	make `self` iterable.
Method	__getitem__	make `self` subscriptable.
Method	__len__	return length of the `Sofomore` instance by calling `len(.)`.
Method	sorted	return a reversed sorted list of kernels.
Method	ask	No summary
Method	tell	No summary
Method	pareto_front_cut	return the non-dominated solutions dominating the reference point, among the kernels' objective values. It's the image of `self.pareto_set_cut`.
Method	pareto_set_cut	return the non-dominated solutions whose images dominate the reference point, among the kernels' incumbents. It's the pre-image of `self.pareto_front_cut`.
Method	pareto_front_uncut	provisorial, return _all_ non-dominated objective values irrespectively
Method	stop	No summary
Method	termination_status	return a dictionary of the current termination states of the kernels.
Method	add	add `kernels` of type `list` to `self.kernels`. Generally, `kernels` are created from a factory function. If `kernels` is of length 1, the brackets can be omitted.
Method	remove	remove elements of the `kernels` (type `list`) that belong to `self.kernels`, and update the `_active_indices` attribute. If `kernels` is of length 1, the brackets can be omitted.
Method	median_stds	Undocumented
Method	max_max_stds	Undocumented
Method	inactivate	No summary
Method	activate	activate `kernel` when it was inactivated beforehand. Otherwise it remains quiet.
Method	disp_annotation	copy-pasted from `cma.evolution_strategy`. print annotation line for `disp` ()
Method	disp	copy-pasted from `cma.evolution_strategy`. print current state variables in a single-line. copy-pasted from `cma.evolution_strategy` module
Method	_UHVI_indicator_archive	return archive for uncrowded hypervolume improvement indicator for `kernel`.
Method	_UHVI_indicator	return indicator function(!) for uncrowded hypervolume improvement for `kernel`.
Method	_indices_to_ask	Undocumented

def __init__(self, list_of_solvers_instances, reference_point=None, opts=None):

Initialization:
- `list_of_solvers_instances` is a list of single-objective 
solvers' instances
- The reference_point is set by the user during the 
initialization.
- `opts` is a dictionary updating the values of 'indicator_front',
'archive', 'restart', 'update_order'; that respond respectfully to
the changing fitness we will choose within an iteration of Sofomore,
whether or not keeping an archive, how to do the restart in case of
any restart, and the order of update of the kernels. It also has
the keys 'verb_filename', 'verb_log' and 'verb_disp'; that 
respectfully indicate the name of the filename containing the Sofomore
data, the logging of the Sofomore data every 'verb_log' iterations
and the display of the data via `self.disp()` every 'verb_disp'
iterations.

def __iter__(self):

make self iterable.

def __getitem__(self, i):

make self subscriptable.

def __len__(self):

return length of the Sofomore instance by calling len(.).

The length is the number of (active and inactive) kernels and hence consistent with subscription like [moes[i] for i in range(len(moes)) if i in moes._active_indices].

def _UHVI_indicator_archive(self, kernel):

return archive for uncrowded hypervolume improvement indicator for kernel.

kernel can also be the respective index in self.

def _UHVI_indicator(self, kernel):

return indicator function(!) for uncrowded hypervolume improvement for kernel.

>>> import comocma, cma
>>> list_of_solvers_instances = comocma.get_cmas(13 * [5 * [1]], 0.7, {'verbose':-9})
>>> fitness = comocma.FitFun(cma.ff.sphere, lambda x: cma.ff.sphere(x-1))
>>> moes = comocma.Sofomore(list_of_solvers_instances, [11, 11])
>>> moes.optimize(fitness, iterations=37) # doctest:+ELLIPSIS
Iterat #Fevals   Hypervolume   axis ratios   sigmas   min&max stds***
>>> moes._UHVI_indicator(moes[1])(moes[2].objective_values) # doctest:+ELLIPSIS
***
>>> moes._UHVI_indicator(1)(moes[2].objective_values) # doctest:+ELLIPSIS
***

both return the UHVI indicator function for kernel 1 and evaluate kernel 2 on it:

>>> [[moes._UHVI_indicator(k)(k.objective_values)] for k in moes] # doctest:+ELLIPSIS
***

is the list of UHVI values for all kernels where kernels occupying the very same objective value have indicator value zero.

def sorted(self, key=None, reverse=True, **kwargs):

return a reversed sorted list of kernels.

By default kernels are reversed sorted by HV contribution or UHVI (which we aim to maximize) in the set of kernels. Exact copies have zero or negative UHVI value.

>>> import comocma, cma
>>> list_of_solvers_instances = comocma.get_cmas(13 * [5 * [1]], 0.7, {'verbose':-9})
>>> fitness = comocma.FitFun(cma.ff.sphere, lambda x: cma.ff.sphere(x-1))
>>> moes = comocma.Sofomore(list_of_solvers_instances, [11, 11])
>>> moes.optimize(fitness, iterations=31) # doctest:+ELLIPSIS
Iterat #Fevals   Hypervolume   axis ratios   sigmas   min&max stds***
>>> moes.sorted(key = lambda k: moes.archive.contributing_hypervolume(
...                          k.objective_values)) # doctest:+ELLIPSIS
[<comocma.como.CmaKernel object at***

sorts w.r.t. archive contribution (clones may get positive contribution).

def ask(self, number_to_ask=1):

get the kernels' incumbents to be evaluated and sample new candidate solutions from 
`number_to_ask` kernels.
The sampling is done by calling the `ask` method of the
`cma.CMAEvolutionStrategy` class.
The indices of the considered kernels' incumbents are given by the 
`_told_indices` attribute.

To get the `number_to_ask` kernels, we use the function `self.key_sort_indices` as
a key to sort `self._remaining_indices_to_ask` (which is the list of
kernels' indices wherein we choose the first `number_to_ask` elements.
And if `number_to_ask` is larger than `len(self._remaining_indices_to_ask)`,
we select the list `self._remaining_indices_to_ask` extended with the  
first `number_to_ask - len(self._remaining_indices_to_ask)` elements
of `range(len(self))`, sorted with `self.key_sort_indices` as key.

Arguments
---------
- `number_to_ask`: the number of kernels for which we sample solutions
 from, it is of TYPE int or str and is smaller or equal to `len(self)`.
 The unique case where it is a str instance is when it is equal to 
 "all", meaning that all the kernels are asked. That case is mainly used
 on parallel mode by distributing the evaluations of the kernels at the
 same time before calling the `tell` method. The opposite is when
 `number_to_ask` is equal to 1, which is the exact COMO-CMA-ES algorithm,
 where the evaluations are done sequentially by updating the kernels one
 by one.

Return
------
The list of the kernels' incumbents to be evaluated, extended with a
list of N-dimensional (N is the dimension of the search space) 
candidate solutions generated from `number_to_ask` kernels 
to be evaluated.

:See: the `ask` method from the class `cma.CMAEvolutionStrategy`,
    in `evolution_strategy.py` from the `cma` module.

def tell(self, solutions, objective_values):

pass objective function values to update the state variables of some 
kernels, `self._told_indices` and eventually `self.archive`.
Arguments
---------
`solutions`
    list or array of points (of type `numpy.ndarray`), most presumably 
    before delivered by the `ask()` method.
`objective_values`
    list of multiobjective function values (of type `list`)
    corresponding to the respective points in `solutions`.
`constraints_values`
    list of list of constraint values: each element is a list containing
    the values of one constraint function, that are obtained by evaluation
    on `solutions`.
    

Details
-------
To update a kernel, `tell()` applies the kernel's `tell` method
to the kernel's corresponding candidate solutions (offspring) along
with the "changing" fitness `- self.indicator_front.hypervolume_improvement`.

:See: 
    - the `tell` method from the class `cma.CMAEvolutionStrategy`,
    in `evolution_strategy.py` from the `cma` module.
    - the `hypervolume_improvement` method from the class
    `BiobjectiveNondominatedSortedList`, in the module `moarchiving.py`
    - the `hypervolume_improvement` method from the class
    `NonDominatedList`, in the module `nondominatedarchive.py`

@property
def pareto_front_cut(self):

return the non-dominated solutions dominating the reference point, among the kernels' objective values. It's the image of self.pareto_set_cut.

@property
def pareto_set_cut(self):

return the non-dominated solutions whose images dominate the reference point, among the kernels' incumbents. It's the pre-image of self.pareto_front_cut.

@property
def pareto_front_uncut(self):

provisorial, return _all_ non-dominated objective values irrespectively

of the current reference point. Only points whose contributing HV does not depend on the current reference point still have the same cHV in the resulting list. HV improvements in general may change.

def stop(self):

return a nonempty dictionary when all kernels stop, containing all the
termination status. Therefore it's solely ... on the kernels' `stop`
method, which also return dictionaries.
Return
------
For example with 5 kernels, stop should return either None, or a `dict`
of the form:
    {0: dict0,
     1: dict1,
     2: dict2,
     3: dict2,
     4: dict4},
where each index `i` is a key which value is the `dict` instance
`self.kernels[i].stop()`

@property
def termination_status(self):

return a dictionary of the current termination states of the kernels.

def add(self, kernels):

add kernels of type list to self.kernels. Generally, kernels are created from a factory function. If kernels is of length 1, the brackets can be omitted.

def remove(self, kernels):

remove elements of the kernels (type list) that belong to self.kernels, and update the _active_indices attribute. If kernels is of length 1, the brackets can be omitted.

@property
def median_stds(self):

Undocumented

@property
def max_max_stds(self):

Undocumented

def _indices_to_ask(self, number_to_ask):

Undocumented

def inactivate(self, kernel):

inactivate kernel, assuming that it's an element of self.kernels, or an index in range(len(self)). When inactivated, kernel is no longer updated, it is ignored. However we do not remove it from self.kernels, meaning that kernel might still play a role, due to its eventual trace in self.pareto_front_cut.

def activate(self, kernel):

activate kernel when it was inactivated beforehand. Otherwise it remains quiet.

We expect the kernel's stop method in interest to look like: kernel.stop() = {'callback': ['kernel turned off']}

def disp_annotation(self):

copy-pasted from cma.evolution_strategy. print annotation line for disp ()

def disp(self, modulo=None):

copy-pasted from cma.evolution_strategy. print current state variables in a single-line. copy-pasted from cma.evolution_strategy module

Prints only if iteration_counter % modulo == 0.

comocma.como.Sofomore(interfaces.OOOptimizer) class documentation

`comocma.como.Sofomore(interfaces.OOOptimizer)` class documentation