petab_select

Model selection extension for PEtab.

class petab_select.BackwardCandidateSpace(*args, predecessor_model=None, max_steps=None, **kwargs)

The backward method class.

__init__(*args, predecessor_model=None, max_steps=None, **kwargs)

accept(model, distance)

Add a candidate model to the candidate space.

Parameters:

• model (Model) – The model that will be added.

• distance (Union[None, float, int]) – The distance of the model from the predecessor model.

Return type:

None

FIXME(dilpath) #keep_others: # Whether to keep other models that were previously added to the # candidate space.

consider(model)

Add a candidate model, if it should be added.

Parameters:

model (Optional[Model]) – The candidate model. This value may be None if the ModelSubspace decided to exclude the model that would have been sent.

Return type:

bool

Returns:

Whether it is OK to send additional models to the candidate space. For example, if the limit of the number of accepted models has been reached, then no further models should be sent.

FIXME(dilpath) TODO change to return whether the model was accepted, and instead add self.continue to determine whether additional models should be sent.

distance(model)

Compute the distance between two models that are neighbors.

Parameters:

• model (Model) – The candidate model.

• predecessor_model – The initial model.

Return type:

int

Returns:

The distance from predecessor_model to model, or None if the distance should not be computed.

distances_in_estimated_parameters(model, predecessor_model=None)

Distance between two models in model space, using different metrics.

All metrics are in terms of estimated parameters.

Metrics:

l1:

The L_1 distance between two models.

size:

The difference in the number of estimated parameters between two models.

Parameters:

model (Model) – The candidate model.

Return type:

Dict[str, Union[int, float]]

Returns:

The distances between the models, as a dictionary, where a key is the name of the metric, and the value is the corresponding distance.

exclude(model)

Exclude model(s) from future consideration.

Parameters:

model (Union[Model, List[Model]]) – The model(s) that will be excluded.

Return type:

None

exclude_hashes(hashes)

Exclude models from future consideration, by hash.

Parameters:

hashes (Sequence[str]) – The model hashes that will be excluded

Return type:

None

excluded(model)

Whether a model is excluded.

Return type:

bool

is_plausible(model)

Determine whether a candidate model is plausible.

A plausible model is one that could possibly be chosen by model selection method, in the absence of information about other models in the model space.

For example, given a forward selection method that starts with an initial model self.predecessor_model that has no estimated parameters, then only models with one or more estimated parameters are plausible.

Parameters:

model (Model) – The candidate model.

Return type:

bool

Returns:

True if model is plausible, else False.

n_accepted()

Get the current number of accepted models.

Return type:

Union[float, int]

reset(predecessor_model=None, exclusions=None, limit=None)

Reset the candidate models, optionally reinitialize with a model.

Parameters:

• predecessor_model (Union[Model, str, None]) – The initial model.

• exclusions (Optional[List[str]]) – Whether to reset model exclusions.

• limit (Union[float, int]) – The new upper limit of the number of models in this candidate space.

Return type:

None

reset_accepted()

Reset the accepted models.

Return type:

None

update_after_calibration(*args, **kwargs)

Do work in the candidate space after calibration.

For example, this is used by the FamosCandidateSpace to switch methods.

Different candidate spaces require different arguments. All arguments are here, to ensure candidate spaces can be switched easily and still receive sufficient arguments.

wrap_search_subspaces(search_subspaces)

Decorate the subspace searches of a model space.

Used by candidate spaces to perform changes that alter the search. See BidirectionalCandidateSpace for an example, where it’s used to switch directions.

Parameters:

search_subspaces (Callable[[], None]) – The method that searches the subspaces of a model space.

class petab_select.BruteForceCandidateSpace(*args, **kwargs)

The brute-force method class.

__init__(*args, **kwargs)

accept(model, distance)

Add a candidate model to the candidate space.

Parameters:

• model (Model) – The model that will be added.

• distance (Union[None, float, int]) – The distance of the model from the predecessor model.

Return type:

None

FIXME(dilpath) #keep_others: # Whether to keep other models that were previously added to the # candidate space.

consider(model)

Add a candidate model, if it should be added.

Parameters:

model (Optional[Model]) – The candidate model. This value may be None if the ModelSubspace decided to exclude the model that would have been sent.

Return type:

bool

Returns:

Whether it is OK to send additional models to the candidate space. For example, if the limit of the number of accepted models has been reached, then no further models should be sent.

FIXME(dilpath) TODO change to return whether the model was accepted, and instead add self.continue to determine whether additional models should be sent.

distance(model)

Compute the distance between two models that are neighbors.

Parameters:

• model (Model) – The candidate model.

• predecessor_model – The initial model.

Return type:

Union[None, float, int]

Returns:

The distance from predecessor_model to model, or None if the distance should not be computed.

distances_in_estimated_parameters(model, predecessor_model=None)

Distance between two models in model space, using different metrics.

All metrics are in terms of estimated parameters.

Metrics:

l1:

The L_1 distance between two models.

size:

The difference in the number of estimated parameters between two models.

Parameters:

model (Model) – The candidate model.

Return type:

Dict[str, Union[int, float]]

Returns:

The distances between the models, as a dictionary, where a key is the name of the metric, and the value is the corresponding distance.

exclude(model)

Exclude model(s) from future consideration.

Parameters:

model (Union[Model, List[Model]]) – The model(s) that will be excluded.

Return type:

None

exclude_hashes(hashes)

Exclude models from future consideration, by hash.

Parameters:

hashes (Sequence[str]) – The model hashes that will be excluded

Return type:

None

excluded(model)

Whether a model is excluded.

Return type:

bool

is_plausible(model)

Determine whether a candidate model is plausible.

A plausible model is one that could possibly be chosen by model selection method, in the absence of information about other models in the model space.

For example, given a forward selection method that starts with an initial model self.predecessor_model that has no estimated parameters, then only models with one or more estimated parameters are plausible.

Parameters:

model (Model) – The candidate model.

Return type:

bool

Returns:

True if model is plausible, else False.

n_accepted()

Get the current number of accepted models.

Return type:

Union[float, int]

reset(predecessor_model=None, exclusions=None, limit=None)

Reset the candidate models, optionally reinitialize with a model.

Parameters:

• predecessor_model (Union[Model, str, None]) – The initial model.

• exclusions (Optional[List[str]]) – Whether to reset model exclusions.

• limit (Union[float, int]) – The new upper limit of the number of models in this candidate space.

Return type:

None

reset_accepted()

Reset the accepted models.

Return type:

None

update_after_calibration(*args, **kwargs)

Do work in the candidate space after calibration.

For example, this is used by the FamosCandidateSpace to switch methods.

Different candidate spaces require different arguments. All arguments are here, to ensure candidate spaces can be switched easily and still receive sufficient arguments.

wrap_search_subspaces(search_subspaces)

Decorate the subspace searches of a model space.

Used by candidate spaces to perform changes that alter the search. See BidirectionalCandidateSpace for an example, where it’s used to switch directions.

Parameters:

search_subspaces (Callable[[], None]) – The method that searches the subspaces of a model space.

class petab_select.CandidateSpace(method, predecessor_model=None, exclusions=None, limit=inf, summary_tsv=None, previous_predecessor_model=None)

A base class for collecting candidate models.

The intended use of subclasses is to identify suitable models in a model space, that will be provided to a model selection method for selection.

distances

The distances of all candidate models from the initial model.

FIXME(dilpath) change list to int? Is storage of more than one value useful?

predecessor_model

The model used for comparison, e.g. for stepwise methods.

previous_predecessor_model

The previous predecessor model.

models

The current set of candidate models.

exclusions

A list of model hashes. Models that match a hash in exclusions will not be accepted into the candidate space. The hashes of models that are accepted are added to exclusions.

limit

A handler to limit the number of accepted models.

method

The model selection method of the candidate space.

governing_method

Used to store the search method that governs the choice of method during a search. In some cases, this is always the same as the method attribute. An example of a difference is in the bidirectional method, where governing_method stores the bidirectional method, whereas method may also store the forward or backward methods.

summary_tsv

A string or pathlib.Path. A summary of the model selection progress will be written to this file.

FIXME(dilpath) #limited: # A descriptor that handles the limit on the number of accepted models. #limit: # Models will fail self.consider if len(self.models) >= limit.

__init__(method, predecessor_model=None, exclusions=None, limit=inf, summary_tsv=None, previous_predecessor_model=None)

accept(model, distance)

Add a candidate model to the candidate space.

Parameters:

• model (Model) – The model that will be added.

• distance (Union[None, float, int]) – The distance of the model from the predecessor model.

Return type:

None

FIXME(dilpath) #keep_others: # Whether to keep other models that were previously added to the # candidate space.

consider(model)

Add a candidate model, if it should be added.

Parameters:

model (Optional[Model]) – The candidate model. This value may be None if the ModelSubspace decided to exclude the model that would have been sent.

Return type:

bool

Returns:

Whether it is OK to send additional models to the candidate space. For example, if the limit of the number of accepted models has been reached, then no further models should be sent.

FIXME(dilpath) TODO change to return whether the model was accepted, and instead add self.continue to determine whether additional models should be sent.

distance(model)

Compute the distance between two models that are neighbors.

Parameters:

• model (Model) – The candidate model.

• predecessor_model – The initial model.

Return type:

Union[None, float, int]

Returns:

The distance from predecessor_model to model, or None if the distance should not be computed.

distances_in_estimated_parameters(model, predecessor_model=None)

Distance between two models in model space, using different metrics.

All metrics are in terms of estimated parameters.

Metrics:

l1:

The L_1 distance between two models.

size:

The difference in the number of estimated parameters between two models.

Parameters:

model (Model) – The candidate model.

Return type:

Dict[str, Union[int, float]]

Returns:

The distances between the models, as a dictionary, where a key is the name of the metric, and the value is the corresponding distance.

exclude(model)

Exclude model(s) from future consideration.

Parameters:

model (Union[Model, List[Model]]) – The model(s) that will be excluded.

Return type:

None

exclude_hashes(hashes)

Exclude models from future consideration, by hash.

Parameters:

hashes (Sequence[str]) – The model hashes that will be excluded

Return type:

None

excluded(model)

Whether a model is excluded.

Return type:

bool

is_plausible(model)

Determine whether a candidate model is plausible.

A plausible model is one that could possibly be chosen by model selection method, in the absence of information about other models in the model space.

For example, given a forward selection method that starts with an initial model self.predecessor_model that has no estimated parameters, then only models with one or more estimated parameters are plausible.

Parameters:

model (Model) – The candidate model.

Return type:

bool

Returns:

True if model is plausible, else False.

n_accepted()

Get the current number of accepted models.

Return type:

Union[float, int]

reset(predecessor_model=None, exclusions=None, limit=None)

Reset the candidate models, optionally reinitialize with a model.

Parameters:

• predecessor_model (Union[Model, str, None]) – The initial model.

• exclusions (Optional[List[str]]) – Whether to reset model exclusions.

• limit (Union[float, int]) – The new upper limit of the number of models in this candidate space.

Return type:

None

reset_accepted()

Reset the accepted models.

Return type:

None

update_after_calibration(*args, **kwargs)

Do work in the candidate space after calibration.

For example, this is used by the FamosCandidateSpace to switch methods.

Different candidate spaces require different arguments. All arguments are here, to ensure candidate spaces can be switched easily and still receive sufficient arguments.

wrap_search_subspaces(search_subspaces)

Decorate the subspace searches of a model space.

Used by candidate spaces to perform changes that alter the search. See BidirectionalCandidateSpace for an example, where it’s used to switch directions.

Parameters:

search_subspaces (Callable[[], None]) – The method that searches the subspaces of a model space.

class petab_select.Criterion(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

String literals for model selection criteria.

AIC = 'AIC'

The Akaike information criterion.

AICC = 'AICc'

The corrected Akaike information criterion.

BIC = 'BIC'

The Bayesian information criterion.

LH = 'LH'

The likelihood.

LLH = 'LLH'

The log-likelihood.

NLLH = 'NLLH'

The negative log-likelihood.

__init__(*args, **kwds)

class petab_select.CriterionComputer(model)

Compute various criteria.

__init__(model)

get_aic()

Get the Akaike information criterion.

Return type:

float

get_aicc()

Get the corrected Akaike information criterion.

Return type:

float

get_bic()

Get the Bayesian information criterion.

Return type:

float

get_lh()

Get the likelihood.

Return type:

float

get_llh()

Get the log-likelihood.

Return type:

float

get_n_estimated()

Get the number of estimated parameters.

Return type:

int

get_n_measurements()

Get the number of measurements.

Return type:

int

get_n_priors()

Get the number of priors.

Return type:

int

get_nllh()

Get the negative log-likelihood.

Return type:

float

property petab_problem: Problem

The PEtab problem that corresponds to the model.

Implemented as a property such that the petab.Problem object is only constructed if explicitly requested.

Improves speed of operations on models by a lot. For example, analysis of models that already have criteria computed can skip loading their PEtab problem again.

class petab_select.FamosCandidateSpace(*args, predecessor_model=None, critical_parameter_sets=[], swap_parameter_sets=[], method_scheme=None, n_reattempts=0, consecutive_laterals=False, **kwargs)

The FAMoS method class.

This candidate space implements and extends the original FAMoS algorithm (doi: 10.1371/journal.pcbi.1007230).

critical_parameter_sets

A list of lists, where each inner list contains parameter IDs. All models must estimate at least 1 parameter from each critical parameter set.

swap_parameter_sets

A list of lists, where each inner list contains parameter IDs. The lateral moves in FAMoS are constrained to be between parameters that exist in the same swap parameter set.

method_scheme

A dictionary that specifies how to switch between methods when the current method doesn’t produce a better model. Keys are n-tuples that described a pattern of length n methods. Values are methods. If the previous methods match the pattern in the key, then the method in the value will be used next. The order of the dictionary is important: only the first matched pattern will be used. Defaults to the method scheme described in the original FAMoS publication.

n_reattempts

Integer. The total number of times that a jump-to-most-distance action will be performed, triggered whenever the model selection would normally terminate. Defaults to no reattempts (0).

consecutive_laterals

Boolean. If True, the method will continue performing lateral moves while they produce better models. Otherwise, the method scheme will be applied after one lateral move.

__init__(*args, predecessor_model=None, critical_parameter_sets=[], swap_parameter_sets=[], method_scheme=None, n_reattempts=0, consecutive_laterals=False, **kwargs)

accept(model, distance)

Add a candidate model to the candidate space.

Parameters:

• model (Model) – The model that will be added.

• distance (Union[None, float, int]) – The distance of the model from the predecessor model.

Return type:

None

FIXME(dilpath) #keep_others: # Whether to keep other models that were previously added to the # candidate space.

check_critical(model)

Check if the model contains all necessary critical parameters

Return type:

bool

check_swap(model)

Check if parameters that are swapped are contained in the same swap parameter set.

Return type:

bool

consider(model)

Re-define consider of FAMoS to be the consider method of the inner_candidate_space. Update all the attributes changed in the consider method.

Return type:

bool

distance(model)

Compute the distance between two models that are neighbors.

Parameters:

• model (Model) – The candidate model.

• predecessor_model – The initial model.

Return type:

Union[None, float, int]

Returns:

The distance from predecessor_model to model, or None if the distance should not be computed.

distances_in_estimated_parameters(model, predecessor_model=None)

Distance between two models in model space, using different metrics.

All metrics are in terms of estimated parameters.

Metrics:

l1:

The L_1 distance between two models.

size:

The difference in the number of estimated parameters between two models.

Parameters:

model (Model) – The candidate model.

Return type:

Dict[str, Union[int, float]]

Returns:

The distances between the models, as a dictionary, where a key is the name of the metric, and the value is the corresponding distance.

exclude(model)

Exclude model(s) from future consideration.

Parameters:

model (Union[Model, List[Model]]) – The model(s) that will be excluded.

Return type:

None

exclude_hashes(hashes)

Exclude models from future consideration, by hash.

Parameters:

hashes (Sequence[str]) – The model hashes that will be excluded

Return type:

None

excluded(model)

Whether a model is excluded.

Return type:

bool

get_most_distant(calibrated_models)

Get most distant model to all the checked models. We take models from the sorted list of best models (self.best_models) and construct complements of these models. For all these complements we compute the distance in number of different estimated parameters to all models from history. For each complement we take the minimum of these distances as it’s distance to history. Then we choose the complement model with the maximal distance to history.

TODO: Next we check if this model is contained in any subspace. If so we choose it. If not we choose the model in a subspace that has least distance to this complement model.

Return type:

Model

insert_model_into_best_models(model_to_insert, criterion)

Inserts a model into the list of best_models which are sorted w.r.t. the criterion specified.

Return type:

None

is_plausible(model)

Determine whether a candidate model is plausible.

A plausible model is one that could possibly be chosen by model selection method, in the absence of information about other models in the model space.

For example, given a forward selection method that starts with an initial model self.predecessor_model that has no estimated parameters, then only models with one or more estimated parameters are plausible.

Parameters:

model (Model) – The candidate model.

Return type:

bool

Returns:

True if model is plausible, else False.

jump_to_most_distant(calibrated_models)

Jump to most distant model with respect to the history of all calibrated models.

n_accepted()

Get the current number of accepted models.

Return type:

Union[float, int]

reset(predecessor_model=None, exclusions=None, limit=None)

Reset the candidate models, optionally reinitialize with a model.

Parameters:

• predecessor_model (Union[Model, str, None]) – The initial model.

• exclusions (Optional[List[str]]) – Whether to reset model exclusions.

• limit (Union[float, int]) – The new upper limit of the number of models in this candidate space.

Return type:

None

reset_accepted()

Changing the reset_accepted to reset the inner_candidate_space as well.

Return type:

None

set_exclusions(exclusions)

Setting the exclusions for the inner_candidate_space as well.

set_limit(limit=None)

Setting the limit for the inner_candidate_space as well.

set_predecessor_model(predecessor_model)

Setting the predecessor model for the inner_candidate_space as well.

switch_inner_candidate_space(exclusions)

Switch the inner candidate space to match the current method.

Parameters:

exclusions (List[str]) – Hashes of excluded models.

switch_method(calibrated_models)

Switch to the next method with respect to the history of methods used and the switching scheme in self.method_scheme.

Return type:

None

update_after_calibration(*args, calibrated_models, newly_calibrated_models, criterion, **kwargs)

See CandidateSpace.update_after_calibration.

Return type:

None

update_from_newly_calibrated_models(newly_calibrated_models, criterion)

Update self.best_models with the latest newly_calibrated_models and determine if there was a new best model. If so, return False. True otherwise.

Return type:

bool

update_method(method)

Update self.method to method.

wrap_search_subspaces(search_subspaces)

Decorate the subspace searches of a model space.

Used by candidate spaces to perform changes that alter the search. See BidirectionalCandidateSpace for an example, where it’s used to switch directions.

Parameters:

search_subspaces – The method that searches the subspaces of a model space.

class petab_select.ForwardCandidateSpace(*args, predecessor_model=None, max_steps=None, **kwargs)

The forward method class.

direction

1 for the forward method, -1 for the backward method.

max_steps

Maximum number of steps forward in a single iteration of forward selection. Defaults to no maximum (None).

__init__(*args, predecessor_model=None, max_steps=None, **kwargs)

accept(model, distance)

Add a candidate model to the candidate space.

Parameters:

• model (Model) – The model that will be added.

• distance (Union[None, float, int]) – The distance of the model from the predecessor model.

Return type:

None

FIXME(dilpath) #keep_others: # Whether to keep other models that were previously added to the # candidate space.

consider(model)

Add a candidate model, if it should be added.

Parameters:

model (Optional[Model]) – The candidate model. This value may be None if the ModelSubspace decided to exclude the model that would have been sent.

Return type:

bool

Returns:

Whether it is OK to send additional models to the candidate space. For example, if the limit of the number of accepted models has been reached, then no further models should be sent.

FIXME(dilpath) TODO change to return whether the model was accepted, and instead add self.continue to determine whether additional models should be sent.

distance(model)

Compute the distance between two models that are neighbors.

Parameters:

• model (Model) – The candidate model.

• predecessor_model – The initial model.

Return type:

int

Returns:

The distance from predecessor_model to model, or None if the distance should not be computed.

distances_in_estimated_parameters(model, predecessor_model=None)

Distance between two models in model space, using different metrics.

All metrics are in terms of estimated parameters.

Metrics:

l1:

The L_1 distance between two models.

size:

The difference in the number of estimated parameters between two models.

Parameters:

model (Model) – The candidate model.

Return type:

Dict[str, Union[int, float]]

Returns:

The distances between the models, as a dictionary, where a key is the name of the metric, and the value is the corresponding distance.

exclude(model)

Exclude model(s) from future consideration.

Parameters:

model (Union[Model, List[Model]]) – The model(s) that will be excluded.

Return type:

None

exclude_hashes(hashes)

Exclude models from future consideration, by hash.

Parameters:

hashes (Sequence[str]) – The model hashes that will be excluded

Return type:

None

excluded(model)

Whether a model is excluded.

Return type:

bool

is_plausible(model)

Determine whether a candidate model is plausible.

A plausible model is one that could possibly be chosen by model selection method, in the absence of information about other models in the model space.

For example, given a forward selection method that starts with an initial model self.predecessor_model that has no estimated parameters, then only models with one or more estimated parameters are plausible.

Parameters:

model (Model) – The candidate model.

Return type:

bool

Returns:

True if model is plausible, else False.

n_accepted()

Get the current number of accepted models.

Return type:

Union[float, int]

reset(predecessor_model=None, exclusions=None, limit=None)

Reset the candidate models, optionally reinitialize with a model.

Parameters:

• predecessor_model (Union[Model, str, None]) – The initial model.

• exclusions (Optional[List[str]]) – Whether to reset model exclusions.

• limit (Union[float, int]) – The new upper limit of the number of models in this candidate space.

Return type:

None

reset_accepted()

Reset the accepted models.

Return type:

None

update_after_calibration(*args, **kwargs)

Do work in the candidate space after calibration.

For example, this is used by the FamosCandidateSpace to switch methods.

Different candidate spaces require different arguments. All arguments are here, to ensure candidate spaces can be switched easily and still receive sufficient arguments.

wrap_search_subspaces(search_subspaces)

Decorate the subspace searches of a model space.

Used by candidate spaces to perform changes that alter the search. See BidirectionalCandidateSpace for an example, where it’s used to switch directions.

Parameters:

search_subspaces (Callable[[], None]) – The method that searches the subspaces of a model space.

class petab_select.LateralCandidateSpace(*args, predecessor_model, max_steps=None, **kwargs)

Find models with the same number of estimated parameters.

__init__(*args, predecessor_model, max_steps=None, **kwargs)

Additional args:

max_number_of_steps:

Maximal allowed number of swap moves. If 0 then there is no maximum.

accept(model, distance)

Add a candidate model to the candidate space.

Parameters:

• model (Model) – The model that will be added.

• distance (Union[None, float, int]) – The distance of the model from the predecessor model.

Return type:

None

FIXME(dilpath) #keep_others: # Whether to keep other models that were previously added to the # candidate space.

consider(model)

Add a candidate model, if it should be added.

Parameters:

model (Optional[Model]) – The candidate model. This value may be None if the ModelSubspace decided to exclude the model that would have been sent.

Return type:

bool

Returns:

Whether it is OK to send additional models to the candidate space. For example, if the limit of the number of accepted models has been reached, then no further models should be sent.

FIXME(dilpath) TODO change to return whether the model was accepted, and instead add self.continue to determine whether additional models should be sent.

distance(model)

Compute the distance between two models that are neighbors.

Parameters:

• model (Model) – The candidate model.

• predecessor_model – The initial model.

Return type:

Union[None, float, int]

Returns:

The distance from predecessor_model to model, or None if the distance should not be computed.

distances_in_estimated_parameters(model, predecessor_model=None)

Distance between two models in model space, using different metrics.

All metrics are in terms of estimated parameters.

Metrics:

l1:

The L_1 distance between two models.

size:

The difference in the number of estimated parameters between two models.

Parameters:

model (Model) – The candidate model.

Return type:

Dict[str, Union[int, float]]

Returns:

The distances between the models, as a dictionary, where a key is the name of the metric, and the value is the corresponding distance.

exclude(model)

Exclude model(s) from future consideration.

Parameters:

model (Union[Model, List[Model]]) – The model(s) that will be excluded.

Return type:

None

exclude_hashes(hashes)

Exclude models from future consideration, by hash.

Parameters:

hashes (Sequence[str]) – The model hashes that will be excluded

Return type:

None

excluded(model)

Whether a model is excluded.

Return type:

bool

is_plausible(model)

Determine whether a candidate model is plausible.

A plausible model is one that could possibly be chosen by model selection method, in the absence of information about other models in the model space.

For example, given a forward selection method that starts with an initial model self.predecessor_model that has no estimated parameters, then only models with one or more estimated parameters are plausible.

Parameters:

model (Model) – The candidate model.

Return type:

bool

Returns:

True if model is plausible, else False.

n_accepted()

Get the current number of accepted models.

Return type:

Union[float, int]

reset(predecessor_model=None, exclusions=None, limit=None)

Reset the candidate models, optionally reinitialize with a model.

Parameters:

• predecessor_model (Union[Model, str, None]) – The initial model.

• exclusions (Optional[List[str]]) – Whether to reset model exclusions.

• limit (Union[float, int]) – The new upper limit of the number of models in this candidate space.

Return type:

None

reset_accepted()

Reset the accepted models.

Return type:

None

update_after_calibration(*args, **kwargs)

Do work in the candidate space after calibration.

For example, this is used by the FamosCandidateSpace to switch methods.

Different candidate spaces require different arguments. All arguments are here, to ensure candidate spaces can be switched easily and still receive sufficient arguments.

wrap_search_subspaces(search_subspaces)

Decorate the subspace searches of a model space.

Used by candidate spaces to perform changes that alter the search. See BidirectionalCandidateSpace for an example, where it’s used to switch directions.

Parameters:

search_subspaces (Callable[[], None]) – The method that searches the subspaces of a model space.

class petab_select.Method(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

String literals for model selection methods.

BACKWARD = 'backward'

The backward stepwise method.

BRUTE_FORCE = 'brute_force'

The brute-force method.

FAMOS = 'famos'

The FAMoS method.

FORWARD = 'forward'

The forward stepwise method.

LATERAL = 'lateral'

The lateral, or swap, method.

MOST_DISTANT = 'most_distant'

The jump-to-most-distant-model method.

__init__(*args, **kwds)

class petab_select.Model(petab_yaml, model_subspace_id=None, model_id=None, model_subspace_indices=None, predecessor_model_hash=None, parameters=None, estimated_parameters=None, criteria=None, petab_problem=None, model_hash=None)

A (possibly uncalibrated) model.

NB: some of these attribute names correspond to constants defined in the constants.py file, to facilitate loading models from/saving models to disk (see the saved_attributes attribute).

converters_load

Functions to convert attributes from YAML to Model.

converters_save

Functions to convert attributes from Model to YAML.

criteria

The criteria values of the calibrated model (e.g. AIC).

hash_attributes

This attribute is currently not used. Attributes that will be used to calculate the hash of the Model instance. NB: this hash is used during pairwise comparison to determine whether any two Model instances are unique. The model instances are compared by their parameter estimation problems, as opposed to parameter estimation results, which may differ due to e.g. floating-point arithmetic.

model_id

The model ID.

petab_yaml

The path to the PEtab problem YAML file.

parameters

Parameter values that will overwrite the PEtab problem definition, or change parameters to be estimated.

estimated_parameters

Parameter estimates from a model calibration tool, for parameters that are specified as estimated in the PEtab problem or PEtab Select model YAML. These are untransformed values (i.e., not on log scale).

saved_attributes

Attributes that will be saved to disk by the Model.to_yaml() method.

__init__(petab_yaml, model_subspace_id=None, model_id=None, model_subspace_indices=None, predecessor_model_hash=None, parameters=None, estimated_parameters=None, criteria=None, petab_problem=None, model_hash=None)

compute_criterion(criterion, raise_on_failure=True)

Compute a criterion value for the model.

The value will also be stored, which will overwrite any previously stored value for the criterion.

Parameters:

• criterion (Criterion) – The ID of the criterion (e.g. petab_select.constants.Criterion.AIC).

• raise_on_failure (bool) – Whether to raise a ValueError if the criterion could not be computed. If False, None is returned.

Return type:

float

Returns:

The criterion value.

static from_dict(model_dict, base_path=None, petab_problem=None)

Generate a model from a dictionary of attributes.

Parameters:

• model_dict (Dict[str, Any]) – A dictionary of attributes. The keys are attribute names, the values are the corresponding attribute values for the model. Required attributes are the required arguments of the Model.__init__() method.

• base_path (Union[str, Path]) – The path that any relative paths in the model are relative to (e.g. the path to the PEtab problem YAML file Model.petab_yaml() may be relative).

• petab_problem (Problem) – Optionally provide the PEtab problem, to avoid loading it multiple times. NB: This may cause issues if multiple models write to the same PEtab problem in memory.

Return type:

Model

Returns:

A model instance, initialized with the provided attributes.

static from_yaml(model_yaml)

Generate a model from a PEtab Select model YAML file.

Parameters:

model_yaml (Union[str, Path]) – The path to the PEtab Select model YAML file.

Return type:

Model

Returns:

A model instance, initialized with the provided attributes.

get_criterion(criterion, compute=True, raise_on_failure=True)

Get a criterion value for the model.

Parameters:

• criterion (Criterion) – The ID of the criterion (e.g. petab_select.constants.Criterion.AIC).

• compute (bool) – Whether to try to compute the criterion value based on other model attributes. For example, if the 'AIC' criterion is requested, this can be computed from a predetermined model likelihood and its number of estimated parameters.

• raise_on_failure (bool) – Whether to raise a ValueError if the criterion could not be computed. If False, None is returned.

Return type:

Optional[float]

Returns:

The criterion value, or None if it is not available. TODO check for previous use of this method before .get was used

get_hash()

Get the model hash.

Currently designed to only use pre-calibration information, such that if a model is calibrated twice and the two calibrated models differ in their parameter estimates, then they will still have the same hash.

This is not implemented as __hash__ because Python automatically truncates values in a system-dependent manner, which reduces interoperability ( https://docs.python.org/3/reference/datamodel.html#object.__hash__ ).

Return type:

int

Returns:

The hash.

get_mle()

Get the maximum likelihood estimate of the model.

Return type:

Dict[str, float]

get_parameter_values(parameter_ids=None)

Get parameter values.

Includes ESTIMATE for parameters that should be estimated.

The ordering is by parameter_ids if supplied, else self.petab_parameters.

Parameters:

parameter_ids (Optional[List[str]]) – The IDs of parameters that values will be returned for. Order is maintained.

Return type:

List[Union[float, int, Literal['estimate']]]

Returns:

The values of parameters.

has_criterion(criterion)

Check whether the model provides a value for a criterion.

Parameters:

criterion (Criterion) – The criterion (e.g. petab_select.constants.Criterion.AIC).

Return type:

bool

property petab_parameter_ids_estimated: List[str]

Get the IDs of all estimated parameters.

Returns:

The parameter IDs.

property petab_parameter_ids_fixed: List[str]

Get the IDs of all fixed parameters.

Returns:

The parameter IDs.

property petab_parameters_singular: Dict[str, float | int | Literal['estimate']]

TODO deprecate and remove?

set_criterion(criterion, value)

Set a criterion value for the model.

Parameters:

• criterion (Criterion) – The criterion (e.g. petab_select.constants.Criterion.AIC).

• value (float) – The criterion value for the (presumably calibrated) model.

Return type:

None

set_estimated_parameters(estimated_parameters, scaled=False)

Set the estimated parameters.

Parameters:

• estimated_parameters (Dict[str, float]) – The estimated parameters.

• scaled (bool) – Whether the estimated_parameters values are on the scale defined in the PEtab problem (True), or untransformed (False).

Return type:

None

to_dict(resolve_paths=True, paths_relative_to=None)

Generate a dictionary from the attributes of a Model instance.

Parameters:

• resolve_paths (bool) – Whether to resolve relative paths into absolute paths.

• paths_relative_to (Union[str, Path]) – If not None, paths will be converted to be relative to this path. Takes priority over resolve_paths.

Return type:

Dict[str, Any]

Returns:

A dictionary of attributes. The keys are attribute names, the values are the corresponding attribute values for the model. Required attributes are the required arguments of the Model.__init__() method.

to_petab(output_path=None, set_estimated_parameters=None)

Generate a PEtab problem.

Parameters:

• output_path (Union[str, Path]) – The directory where PEtab files will be written to disk. If not specified, the PEtab files will not be written to disk.

• set_estimated_parameters (Optional[bool]) – Whether to set the nominal value of estimated parameters to their estimates. If parameter estimates are available, this will default to True.

Return type:

Dict[str, Union[Problem, str, Path]]

Returns:

A 2-tuple. The first value is a PEtab problem that can be used with a PEtab-compatible tool for calibration of this model. If output_path is not None, the second value is the path to a PEtab YAML file that can be used to load the PEtab problem (the first value) into any PEtab-compatible tool.

to_yaml(petab_yaml, *args, **kwargs)

Generate a PEtab Select model YAML file from a Model instance.

Parameters:

• petab_yaml (Union[str, Path]) – The location where the PEtab Select model YAML file will be saved.

• args – Additional arguments are passed to self.to_dict.

• kwargs – Additional arguments are passed to self.to_dict.

Return type:

None

class petab_select.ModelSpace(model_subspaces)

A model space, as a collection of model subspaces.

model_subspaces

List of model subspaces.

exclusions

Hashes of models that are excluded from the model space.

__init__(model_subspaces)

static from_files(filenames)

Create a model space from model space files.

Parameters:

filenames (List[Union[str, Path]]) – The locations of the model space files.

Returns:

The corresponding model space.

reset_exclusions(exclusions=None)

Reset the exclusions in the model subspaces.

Return type:

None

search(candidate_space, limit=inf, exclude=True)

…TODO

Parameters:

• candidate_space (CandidateSpace) – The candidate space.

• limit (int) – The maximum number of models to send to the candidate space (i.e. this limit is on the number of models considered, not necessarily approved as candidates). Note that using a limit may produce unexpected results. For example, it may bias candidate models to be chosen only from a subset of model subspaces.

• exclude (bool) – Whether to exclude the new candidates from the model subspaces.

class petab_select.ModelSubspace(model_subspace_id, petab_yaml, parameters, exclusions=None)

Efficient representation of exponentially large model subspaces.

model_subspace_id

The ID of the model subspace.

petab_yaml

The location of the PEtab problem YAML file.

parameters

The key is the ID of the parameter. The value is a list of values that the parameter can take (including ESTIMATE).

exclusions

Hashes of models that have been previously submitted to a candidate space for consideration (CandidateSpace.consider()).

__init__(model_subspace_id, petab_yaml, parameters, exclusions=None)

property can_estimate: List[str]

Parameters that can be estimated, according to the subspace.

Parameters that are estimated as part of the PEtab problem are not considered.

property can_estimate_all: List[str]

All parameters than can be estimated in this subspace.

property can_fix: List[str]

Parameters that can be fixed, according to the subspace.

Parameters that are fixed as part of the PEtab problem are not considered.

check_compatibility_stepwise_method(candidate_space)

Check whether a candidate space is compatible with this subspace.

Directional methods (e.g. forward, backward) are not supported among different PEtab problems.

Parameters:

candidate_space (CandidateSpace) – The candidate space with a stepwise method.

Return type:

bool

Returns:

Whether the candidate space is compatible.

exclude_model(model)

Exclude a model from the model subspace.

Models are excluded in ModelSubspace.indices_to_model, which contains the only call to Model.__init__ in the ModelSubspace class.

Parameters:

model (Model) – The model that will be excluded.

Return type:

None

exclude_model_hash(model_hash)

Exclude a model hash from the model subspace.

Parameters:

model_hash (str) – The model hash.

Return type:

None

exclude_model_hashes(model_hashes)

Exclude model hashes from the model subspace.

Parameters:

model_hashes (Iterable[str]) – The model hashes.

Return type:

None

exclude_models(models)

Exclude models from the model subspace.

Models are excluded in ModelSubspace.indices_to_model(), which contains the only call to Model.__init__() in the ModelSubspace class.

Parameters:

models (Iterable[Model]) – The models that will be excluded.

Return type:

None

excluded(model)

Whether a model is excluded.

Return type:

bool

static from_definition(model_subspace_id, definition, parent_path=None)

Create a ModelSubspace from a definition.

Parameters:

• model_subspace_id (str) – The model subspace ID.

• definition (Union[Dict[str, str], Series]) – A description of the model subspace. Keys are properties of the model subspace, including parameters that can take different values.

• parent_path (Union[str, Path]) – Any paths in the definition will be set relative to this path.

Return type:

ModelSubspace

Returns:

The model subspace.

get_estimated(additional_parameters=None)

Get the IDs of parameters that are estimated.

Parameters:

additional_parameters (Optional[Dict[str, Union[float, int, Literal['estimate']]]]) – A specific parameterization that will take priority when determining estimated parameters.

Return type:

List[str]

Returns:

The parameter IDs.

get_models(estimated_parameters)

Get models in the subspace by estimated parameters.

All models that have the provided estimated_parameters are returned.

Parameters:

estimated_parameters (List[str]) –

The IDs of parameters that are estimated in the model. All other parameters will be fixed. Note that these parameters are in the subset of PEtab parameters that exist in the model subspace definition. Parameters in the PEtab problem but not the model subspace definition should not be included here.

FIXME(dilpath) TODO support the full set of PEtab parameters? Then would need to turn off estimation for parameters that are not provided in estimated_parameters – maybe unexpected for users.

Return type:

Iterator[Model]

Returns:

A list of models.

indices_to_model(indices)

Get a model from the subspace, by indices of possible parameter values.

Model exclusions are handled here.

Parameters:

indices (List[int]) – The indices of the lists in the values of the ModelSubspace.parameters dictionary, ordered by the keys of this dictionary.

Return type:

Optional[Model]

Returns:

A model with the PEtab problem of this subspace and the parameterization that corresponds to the indices. None, if the model is excluded from the subspace.

indices_to_parameters(indices)

Convert parameter indices to values.

Parameters:

indices (List[int]) – See ModelSubspace.indices_to_model().

Return type:

Dict[str, Union[float, int, Literal['estimate']]]

Returns:

The parameterization that corresponds to the indices.

property must_estimate: List[str]

Subspace parameters that must be estimated.

Does not include parameters that are estimated in the PEtab parameters table.

property must_estimate_all: List[str]

All parameters that must be estimated in this subspace.

property must_fix: List[str]

Subspace parameters that must be fixed.

Parameters that are fixed as part of the PEtab problem are not considered.

property must_fix_all: List[str]

All parameters that must be fixed in this subspace.

property parameters_all: Dict[str, float | int | Literal['estimate']]

Get all parameters, including those only in the PEtab problem.

Parameter values in the PEtab problem are overwritten by the model subspace values.

parameters_to_indices(parameters)

Convert parameter values to indices.

Parameters:

parameters (Dict[str, Union[float, int, Literal['estimate']]]) – Keys are parameter IDs, values are parameter values.

Returns:

The indices of the subspace that correspond to the parameterization.

parameters_to_model(parameters)

Convert parameter values to a model.

Parameters:

parameters (Dict[str, Union[float, int, Literal['estimate']]]) – Keys are parameter IDs, values are parameter values.

Return type:

Optional[Model]

Returns:

A model with the PEtab problem of this subspace and the parameterization. None, if the model is excluded from the subspace.

property petab_parameter_ids_estimated: List[str]

Get the IDs of all estimated parameters.

Returns:

The parameter IDs.

property petab_parameter_ids_fixed: List[str]

Get the IDs of all fixed parameters.

Returns:

The parameter IDs.

property petab_parameters_singular: Dict[str, float | int | Literal['estimate']]

TODO deprecate and remove?

search(candidate_space, limit=inf)

Search for candidate models in this model subspace.

Nothing is returned, as the result is managed by the candidate_space.

Parameters:

• candidate_space (CandidateSpace) – The candidate space.

• limit (int) – Limit the number of models.

send_model_to_candidate_space(model, candidate_space, exclude=False)

Send a model to a candidate space for consideration.

Parameters:

• model (Model) – The model.

• candidate_space (CandidateSpace) – The candidate space.

• exclude (Optional[bool]) – Whether to add the model to the exclusions.

Return type:

bool

Returns:

Whether it is OK to send additional models to the candidate space. For example, if len(candidate_space.models) == candidate_space.limit, then no further models should be sent.

class petab_select.Problem(model_space, candidate_space_arguments=None, compare=None, criterion=None, method=None, version=None, yaml_path=None)

Handle everything related to the model selection problem.

model_space

The model space.

calibrated_models

Calibrated models. Will be used to augment the model selection problem (e.g. by excluding them from the model space). FIXME(dilpath) refactor out

candidate_space_arguments

Custom options that are used to construct the candidate space.

compare

A method that compares models by selection criterion. See petab_select.model.default_compare() for an example.

criterion

The criterion used to compare models.

method

The method used to search the model space.

version

The version of the PEtab Select format.

yaml_path

The location of the selection problem YAML file. Used for relative paths that exist in e.g. the model space files.

TODO should the relative paths be relative to the YAML or the file that contains them?

__init__(model_space, candidate_space_arguments=None, compare=None, criterion=None, method=None, version=None, yaml_path=None)

exclude_model_hashes(model_hashes)

Exclude models from the model space, by model hashes.

Parameters:

model_hashes (Iterable[str]) – The model hashes.

Return type:

None

exclude_models(models)

Exclude models from the model space.

Parameters:

models (Iterable[Model]) – The models.

Return type:

None

static from_yaml(yaml_path)

Generate a problem from a PEtab Select problem YAML file.

Parameters:

yaml_path (Union[str, Path]) – The location of the PEtab Select problem YAML file.

Return type:

Problem

Returns:

A Problem instance.

get_best(models, criterion=None, compute_criterion=False)

Get the best model from a collection of models.

The best model is selected based on the selection problem’s criterion.

Parameters:

• models (Optional[Iterable[Model]]) – The best model will be taken from these models.

• criterion (Optional[str]) – The criterion by which models will be compared. Defaults to self.criterion (e.g. as defined in the PEtab Select problem YAML file).

• compute_criterion (bool) – Whether to try computing criterion values, if sufficient information is available (e.g., likelihood and number of parameters, to compute AIC).

Return type:

Model

Returns:

The best model.

get_path(relative_path)

Get the path to a resource, from a relative path.

Parameters:

relative_path (Union[str, Path]) – The path to the resource, that is relative to the PEtab Select problem YAML file location.

Return type:

Path

Returns:

The path to the resource.

new_candidate_space(*args, method=None, **kwargs)

Construct a new candidate space.

Parameters:

• args – Arguments are passed to the candidate space constructor.

• kwargs – Arguments are passed to the candidate space constructor.

• method (Method) – The model selection method.

Return type:

CandidateSpace

petab_select.TYPE_CRITERION

alias of float

petab_select.best(problem, models, criterion=None)

Get the best model from a list of models.

Parameters:

• problem (Problem) – The PEtab Select problem.

• models (List[Model]) – The list of models.

• criterion (Optional[str]) – The criterion by which models will be compared. Defaults to problem.criterion.

Return type:

Model

Returns:

The best model.

petab_select.calculate_aic(nllh, n_estimated)

Calculate the Akaike information criterion (AIC) for a model.

Parameters:

• nllh (float) – The negative log likelihood.

• n_estimated (int) – The number of estimated parameters in the model.

Return type:

float

Returns:

The AIC value.

petab_select.calculate_aicc(nllh, n_estimated, n_measurements, n_priors)

Calculate the corrected Akaike information criterion (AICc) for a model.

Parameters:

• nllh (float) – The negative log likelihood.

• n_estimated (int) – The number of estimated parameters in the model.

• n_measurements (int) – The number of measurements used in the likelihood.

• n_priors (int) – The number of priors used in the objective function.

Return type:

float

Returns:

The AICc value.

petab_select.calculate_bic(nllh, n_estimated, n_measurements, n_priors)

Calculate the Bayesian information criterion (BIC) for a model.

Args

nllh:

The negative log likelihood.

n_estimated:

The number of estimated parameters in the model.

n_measurements:

The number of measurements used in the likelihood.

n_priors:

The number of priors used in the objective function.

Returns:

The BIC value.

petab_select.candidates(problem, candidate_space=None, limit=inf, limit_sent=inf, calibrated_models=None, newly_calibrated_models=None, excluded_models=None, excluded_model_hashes=None, criterion=None)

Search the model space for candidate models.

A predecessor model is chosen from newly_calibrated_models if available, otherwise from calibrated_models, and is used for applicable methods.

Parameters:

• problem (Problem) – A PEtab Select problem.

• candidate_space (Optional[CandidateSpace]) – The candidate space. Defaults to a new candidate space based on the method defined in the problem.

• limit (Union[float, int]) – The maximum number of models to add to the candidate space.

• limit_sent (Union[float, int]) – The maximum number of models sent to the candidate space (which are possibly rejected and excluded).

• calibrated_models (Optional[Dict[str, Model]]) – All calibrated models in the model selection.

• newly_calibrated_models (Optional[Dict[str, Model]]) – All calibrated models in the most recent iteration of model selection.

• excluded_models (Optional[List[Model]]) – Models that will be excluded from model subspaces during the search for candidates.

• excluded_model_hashes (Optional[List[str]]) – Hashes of models that will be excluded from model subspaces during the search for candidates.

• criterion (Optional[Criterion]) – The criterion by which models will be compared. Defaults to the criterion defined in the PEtab Select problem.

Return type:

CandidateSpace

Returns:

The candidate space, which contains the candidate models.

petab_select.default_compare(model0, model1, criterion, criterion_threshold=0)

Compare two calibrated models by their criterion values.

It is assumed that the model model0 provides a value for the criterion criterion, or is the VIRTUAL_INITIAL_MODEL.

Parameters:

• model0 (Model) – The original model.

• model1 (Model) – The new model.

• criterion (Criterion) – The criterion by which models will be compared.

• criterion_threshold (float) – The value by which the new model must improve on the original model. Should be non-negative.

Return type:

bool

Returns:

True` if ``model1 has a better criterion value than model0, else False.

petab_select.model_to_petab(model, output_path=None)

Generate the PEtab problem for a model.

Parameters:

• model (Model) – The model.

• output_path (Union[Path, str, None]) – If specified, the PEtab problem will be output to files in this directory.

Return type:

Dict[str, Union[Problem, str, Path]]

Returns:

The PEtab problem, and the path to the PEtab problem YAML file, if an output path is provided.

petab_select.models_from_yaml_list(model_list_yaml, petab_problem=None, allow_single_model=True)

Generate a model from a PEtab Select list of model YAML file.

Parameters:

• model_list_yaml (Union[str, Path]) – The path to the PEtab Select list of model YAML file.

• petab_problem (Problem) – See Model.from_dict().

• allow_single_model (bool) – Given a YAML file that contains a single model directly (not in a 1-element list), if True then the single model will be read in, else a ValueError will be raised.

Return type:

List[Model]

Returns:

A list of model instances, initialized with the provided attributes.

petab_select.models_to_petab(models, output_path_prefix=None)

Generate the PEtab problems for a list of models.

Parameters:

• models (List[Model]) – The list of model.

• output_path_prefix (Optional[List[Union[str, Path]]]) – If specified, the PEtab problem will be output to files in subdirectories of this path, where each subdirectory corresponds to a model.

Return type:

List[Dict[str, Union[Problem, str, Path]]]

Returns:

The PEtab problems, and the paths to the PEtab problem YAML files, if an output path prefix is provided.

petab_select.parameter_string_to_value(parameter_string, passthrough_estimate=False)

Cast a parameter value from string to numeric.

Parameters:

• parameter_string (str) – The parameter value, as a string.

• passthrough_estimate (bool) – Whether to return ESTIMATE as ESTIMATE. If False, raises an exception if parameter_string == ESTIMATE.

Return type:

Union[float, int, str]

Returns:

The parameter value, as a numeric type.

petab_select.read_model_space_file(filename)

Read a model space file.

The model space specification is currently expanded and written to a temporary file.

Parameters:

filename (str) – The name of the file to be unpacked.

Return type:

TextIO

Returns:

A temporary file object, which is the unpacked file.