{ "cells": [ { "cell_type": "markdown", "id": "e28947de", "metadata": {}, "source": [ "# FAMoS in PEtab Select\n", "\n", "In this notebook, the FAMoS algorithm [1] is demonstrated. This is designed as an example for tool developers to be able to ensure that they utilize PEtab Select correctly, such that complex methods like FAMoS work.\n", "\n", "[1] Gabel M, Hohl T, Imle A, Fackler OT, Graw F (2019) FAMoS: A Flexible and dynamic Algorithm for Model Selection to analyse complex systems dynamics. PLOS Computational Biology 15(8): e1007230. https://doi.org/10.1371/journal.pcbi.1007230" ] }, { "cell_type": "markdown", "id": "484b7f3c", "metadata": {}, "source": [ "The model space contains 65536 models. In normal PEtab Select workflows, a calibration tool would take the place of the `example_cli_famos_calibration_tool.py` script. This script emulates a calibration tool: it takes PEtab Select models and assigns criterion values to them, based on previous calibration results for the same models." ] }, { "cell_type": "code", "execution_count": null, "id": "1f04dce0", "metadata": {}, "outputs": [], "source": [ "# Cleanup the state and candidate models output by a previous run of this notebook\n", "import shutil\n", "from pathlib import Path\n", "\n", "from example_cli_famos_helpers import (\n", " parse_summary_to_progress_list,\n", " petab_select_problem_yaml, # noqa: F401\n", ")\n", "\n", "output_path = Path().resolve() / \"output_famos\"\n", "output_path_str = str(output_path)\n", "if output_path.exists():\n", " shutil.rmtree(output_path_str)\n", "output_path.mkdir(exist_ok=False, parents=True)" ] }, { "cell_type": "code", "execution_count": null, "id": "a81560e6", "metadata": {}, "outputs": [], "source": [ "from petab_select import Method\n", "\n", "state = str(output_path / \"state.dill\")\n", "\n", "# Each iteration of model selection is described as a 2-tuple here.\n", "# First value is the model selection method.\n", "# Second value are relative change in parameter indices that correspond\n", "# to the best model from this iteration.\n", "# e.g. `(Method.FORWARD, {3})` states that the best model from a forward move\n", "# is the model that now estimates the parameter at index 3.\n", "expected_progress_list = [\n", " (Method.LATERAL, set()),\n", " (Method.LATERAL, {4, 15}),\n", " (Method.LATERAL, {9, 13}),\n", " (Method.FORWARD, set()),\n", " (Method.FORWARD, {3}),\n", " (Method.FORWARD, {11}),\n", " (Method.BACKWARD, set()),\n", " (Method.BACKWARD, {6}),\n", " (Method.BACKWARD, {10}),\n", " (Method.BACKWARD, {8}),\n", " (Method.BACKWARD, {14}),\n", " (Method.BACKWARD, {1}),\n", " (Method.BACKWARD, {16}),\n", " (Method.BACKWARD, {4}),\n", " (Method.FORWARD, set()),\n", " (Method.LATERAL, set()),\n", " (Method.MOST_DISTANT, {2, 3, 4, 5, 6, 7, 9, 11, 12, 13, 15}),\n", " (Method.LATERAL, {16, 7}),\n", " (Method.LATERAL, {5, 12}),\n", " (Method.LATERAL, {13, 15}),\n", " (Method.LATERAL, {1, 6}),\n", " (Method.FORWARD, set()),\n", " (Method.FORWARD, {3}),\n", " (Method.FORWARD, {7}),\n", " (Method.FORWARD, {2}),\n", " (Method.FORWARD, {11}),\n", " (Method.BACKWARD, set()),\n", " (Method.BACKWARD, {7}),\n", " (Method.BACKWARD, {16}),\n", " (Method.BACKWARD, {4}),\n", " (Method.FORWARD, set()),\n", " (Method.LATERAL, set()),\n", " (Method.LATERAL, {9, 15}),\n", " (Method.FORWARD, set()),\n", " (Method.BACKWARD, set()),\n", " (Method.LATERAL, set()),\n", "]" ] }, { "cell_type": "markdown", "id": "7202f6c6", "metadata": {}, "source": [ "The predecessor model is some model from the model space, and is defined in the PEtab Select problem YAML file." ] }, { "cell_type": "code", "execution_count": null, "id": "bb1a5144", "metadata": {}, "outputs": [], "source": [ "%%bash -s \"$petab_select_problem_yaml\" \"$output_path_str\"\n", "petab_select_problem_yaml=$1\n", "output_path_str=$2\n", "\n", "problem=$petab_select_problem_yaml\n", "state=$output_path_str/state.dill\n", "\n", "file_uncalibrated_models=$output_path_str/uncalibrated_models.yaml\n", "file_calibrated_models=$output_path_str/calibrated_models.yaml\n", "file_models=$output_path_str/iteration_models.yaml\n", "file_metadata=$output_path_str/metadata.yaml\n", "\n", "for i in {1..40}\n", "do\n", " echo \"Executing iteration $i\"\n", "\n", " petab_select start_iteration \\\n", " --problem=\"$problem\" \\\n", " --state=\"$state\" \\\n", " --output-uncalibrated-models=\"$file_uncalibrated_models\" \\\n", " --relative-paths\n", " \n", " # Replace this line with a tool that calibrates the models in `$output`\n", " # and stores the calibrated models in `$calibrated_output`.\n", " # This script also changes model IDs for easier analysis in this example.\n", " python example_cli_famos_calibration_tool.py $file_uncalibrated_models $file_calibrated_models\n", " \n", " petab_select end_iteration \\\n", " --state=\"$state\" \\\n", " --calibrated-models=\"$file_calibrated_models\" \\\n", " --output-models=\"$file_models\" \\\n", " --output-metadata=\"$file_metadata\" \\\n", " --relative-paths\n", " \n", " terminate=$(cat $file_metadata | grep terminate | awk '{print $NF}')\n", " if [ \"$terminate\" = \"true\" ]; then\n", " echo \"Model selection has terminated.\"\n", " break\n", " fi\n", "done" ] }, { "cell_type": "code", "execution_count": null, "id": "93caf071", "metadata": {}, "outputs": [], "source": [ "progress_list = parse_summary_to_progress_list(output_path / \"summary.tsv\")" ] }, { "cell_type": "code", "execution_count": null, "id": "cb61d0f7", "metadata": {}, "outputs": [], "source": [ "assert progress_list == expected_progress_list" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 5 }