Parameter Estimation for Dynamical Systems
EURANDOM, Eindhoven, The Netherlands
Programme & Abstracts| Speakers & Participants | Registration | Practical Information | Useful links
Anindya Bhadra (University of Michigan)
Malaria transmission dynamics: A formal comparison of rival hypotheses
Nicolas Brunel (L’université
d’Évry & ENSIIE)
Joint work with F. d'Alché-Buc
Estimation of parametric ODEs: two step estimators and orthogonality conditions
Estimation of parametric Ordinary Differential Equations from noisy time series is usually a difficult task. Direct approaches (such as least squares) gives rise to difficulties partly because of the intrinsic definition of the mathematical model. Alternative methods, such as two-step estimators, have been developped for more efficient (at least computationally) estimation procedure by using a nonparametric proxy. We will present then theoretical results about the asymptotics of these estimators, and this will motivate us to propose more adapted nonparametric estimators in order to ameliorate the behavior of two-step estimators. Finally, we will suggest the use of a new family of two step estimators consisting in a reinterpretation of the differential model as a set of orthogonality conditions that should be satisfied. This shed a new light onto two-step estimators and give the possibility to consider new ameliorations of two-step procedures.
Dave Campbell (Simon Fraser University)
Smooth Functional Tempering for Nonlinear Differential Equation models
Natalie Filmann (Goethe-University Frankfurt Main)
Joint work with Tje Lin Chung, Ingmar Mederacke, Heiner Wedemeyer, Eva Herrmann
Modeling of viral kinetics in patients chronically infected with hepatitis B and D
Viral kinetic models have become an important tool for understanding the main biological processes behind the dynamics of chronic viral diseases and for optimizing effectiveness of anti-viral therapy. We developed a model to analyze the viral kinetics of Hepatitis-B-Hepatitis-D-confection (HBV/HDV). Note that Hepatitis D virus is a defective virus that relies on HBV-particles (HBsAG) for replication and, therefore HDV-infection can only occur as confection with HBV-infection. Problem statement: Our scope was to develop a new mechanistic viral kinetic model for HBV/HDV/host-interaction. This model is fitted by analysing the dynamics of HBV/HDV-viremia after liver transplantation in recent patient data. Methods: Viral load (HBV and HDV), HBsAG and anti-HBs (administered to prevent reinfection with HBV after transplantation) of 25 coinfected patients undergoing liver transplantation were measured serially before and after liver transplantation. We propose an ODE-system which allows modeling the HBV-HDV-host-interplay after liver transplantation and using a maximum likelihood approach for parameter estimation during non-linear fitting of differential equations. Results and conclusions: Our findings show a strong correlation between HDV- and HBsAG- decline and anti-HBs increase. The results also suggest that this modeling approach may help to optimize anti-HB dosing schemes and further anti-viral treatment in patients undergoing HBV/HDV-indicated liver transplantation.
Piet Hemker (Synapse / CWI / UvA)
Some Good Old But Sometimes Neglected Aspects in Numerical Parameter Estimation
Grietje Holtrop (Biomathematics & Statistics Scotland)
Joint work with Daniel Lawson Biomathematics & Scotland, Aberdeen, UK
Bayesian analysis of non-linear differential equation models applied to a gut microbial system
Following a meal, undigested food particles, such as plant fibres, reach the large intestine where they can be fermented by bacteria. The fermentation products form an energy source and can be absorbed by the human host. Some of these products are believed to have an impact on health, e.g. butyrate is thought to play a protective role against colon cancer while lactate is associated with gut disorders such as Crohn’s disease. Studies in vitro, mimicking certain aspects of the colon, provide data on some of the relationships between substrate, bacteria and their metabolites.
Differential equation systems can be formulated that describe aspects of the biological processes taking place. These systems are non-linear, non-steady state, and contain many unknown parameters. Parameter estimation from a data set from a given condition often proves difficult due to poorly defined and interdependent relationships between parameters. Although inference techniques such as maximum likelihood are relatively easy to implement, they suffer from drawbacks such as not fully exploring the entire parameter space. Bayesian approaches eliminate some of these drawbacks and, in addition, allow for integrated analysis of data from experiments covering a range of environmental conditions. Although the Bayesian methodology is attractive in principle, implementation is challenging, due to a combination of non-linear non-steady state differential equations containing many parameters in conjunction with a limited amount of data.
We will present results from a Bayesian approach to allow for integrated analysis of data from gut bacteria experiments in vitro under a range of environmental conditions, using an MCMC algorithm that was run on a high performance computing cluster.
Dave Lunn (MRC Biostatistics Unit Cambridge)
BUGS/WBDiff software: Bayesian inference for dynamical systems
Kim McAuley (Queen's University)
Joint work with M. Saeed Varziri and P. James McLellan
Parameter Estimation in Continuous-Time Dynamic Models with Uncertainty
Chemical engineers who develop fundamental dynamic models estimate model parameters using noisy data. Often, modelers know that their models are imperfect due to simplifying assumptions. We have developed a parameter-estimation technique that explicitly considers model imperfections and measurement errors. Our proposed method is based on Maximum-Likelihood (ML) estimation where the likelihood criterion is approximated by means of smoothing splines, which are used to discretize the differential equations. The resulting objective function for parameter estimation contains three parts that account for measurement error, model uncertainty and uncertain initial conditions, respectively. The method extends the benefits of collocation techniques to stochastic dynamic models with uncertain inputs and nonstationary disturbances. Theoretical confidence intervals agree with empirical confidence intervals from Monte Carlo simulations.
Thomas Maiwald (Harvard Medical School)
Joint work with Andreas Raue, Peter K Sorger, Jens Timmer
Dynamical Modeling and Multi-Experiment Fitting with PottersWheel
(1) Harvard Medical School, Department of Systems Biology, Boston, MA, USA (2) University of Freiburg, Institute of Physics, Freiburg, Germany The program PottersWheel has been developed to provide an intuitive and yet powerful framework for data-based modeling of dynamical systems which can be expressed as sets of ordinary differential equations . We will exemplify its use and methodological approach on models and experimental data of biochemical cellular systems from Systems Biology.
The key functionality of PottersWheel is multi-experiment fitting, where several experimental data sets from different laboratory conditions are fitted simultaneously in order to improve the estimation of unknown model parameters, to check the validity of a given model, and to discriminate competing model hypotheses. Stochastic, deterministic, and hybrid optimization techniques can be applied in single fits or to generate fit-sequences which allow for parameter identifiability analysis. Chi-square and Likelihood ratio tests rule out insufficient models and exemplify what can be concluded from experimental data given a certain noise level. Interactive design of external driving input functions helps to optimize the expected information of further experiments.
Models are either created using text files comprising a chemical reaction network or differential equations or by drag and drop via a graphical model designer. Dynamically generated C MEX files of the differential equations and the use of FORTRAN integrators provide fast simulation and fitting procedures. PottersWheel is designed as a Matlab toolbox, comprises 200.000 lines of Matlab and C code, and includes numerous graphical user interfaces. The program is freely available for academic usage from www.potterswheel.de. It is intensively used by experimentalists and modelers since 2005. A comprehensive application programming interface is available for customization and use within own MATLAB scripts.
 Thomas Maiwald and Jens Timmer, Dynamical modeling and multi-experiment fitting with PottersWheel, Bioinformatics 2008, 24:2037-2043
Jana Nĕmcovà (CWI)
Joint work with Jan H. Schuppen
Structural and global indentifiability for the classes of parametrized polynomial and parametrized rational systems
James Ramsay (McGill University)
Paramete Estimation for Differential Equations: A Generalized Smoothing Approach
Andreas Raue (University of Freiburg)
Joint work with Thomas Maiwald (2), Clemens Kreutz (1), Jens Timmer (1,3)
Structural and practical identifiability analysis of partially observed dynamical systems
(1) Physics Institute, University of Freiburg, 79104 Freiburg, Germany (2) Department of Systems Biology, Harvard Medical School, 02115 Boston, MA, USA (3) Freiburg Institute for Advanced Studies, University of Freiburg, 79104 Freiburg, Germany
Parameter estimation in reaction networks described by differential equations faces the challenge of structural and practical non-identifiability. If only parts of the network can be observed directly, structural non-identifiability may arise. It manifests in functionally related model parameters which cannot be estimated uniquely. Practical non-identifiability occurs if the amount and quality of experimental data is limited, leading to infinite confidence intervals for the parameter estimates. Knowledge about ambiguities of parameter estimates is essential for investigation of model predictions.
We present two data-based approaches complementing one another in coping with non-identifiability. The first approach aims to detect both structural and practical non-identifiability by exploiting the profile likelihood . It simultaneously calculates confidence intervals for the parameter estimates and allows for experimental planning to reduce parameter uncertainties. The second approach applies the mean optimal transformation to group structurally non-identifiable parameters according to their functional relations . Thereby it allows to infer reasons for the occurrence of the underlying parameter estimation problem. Both approaches can be applied to large systems containing arbitrary non-linearities.
 Andreas Raue, Clemens Kreutz, Thomas Maiwald, Julie Bachmann, Marcel Schilling, Ursula Klingmüller, Jens Timmer. Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood, Bioinformatics, submitted
 Stefan Hengl, Clemens Kreutz, Jens Timmer, Thomas Maiwald Data-based identifiability analysis of nonlinear dynamical models, Bioinformatics 2007, 23, 2612-2618
Domingo Salazar (Syngenta)
Circadian Clocks and Flowering Induction Network
Plants and animals are able to schedule their daily activities thanks to "circadian clocks": internal biochemical networks which can be entrained by environmental cues and take the form of multiple interlocked genetic loops with positive and negative feedback. I will present in this talk an approach to quantify the flexibility of this kind of regulatory networks and an analytical characterization of their evolutionary goals. We addressed the question of why they have such complex structures and argue that it is to provide the flexibility necessary to simultaneously attain multiple desirable properties; such as robust entrainment, temperature compensation, and environmental adaptability. As part of our analysis, we introduced new concepts such as infinitesimals response curves and the flexibility dimension. Our results suggests that regulatory networks might be much less flexible and lower dimension than their apparent complexity would suggests.
In plants, the circadian clock in involved in the timing of flowering as part of a complex genetic network that integrates information about daylength, the passing of winter (vernalisation) and plant age. In the last part of this talk I will explain briefly how we modelled this network from published messenger RNA waveforms of the relevant genes. I will also discuss how an exhaustive parameter search for these models allowed us to conclude what features of the mRNA waveforms could and could not be simulated. Based on the defects of the simulated waveforms, we were able to make predictions about missing interactions in the network.
References: 1. Design principles underlying circadian clocks. D. A. Rand, B. V. Shulgin, D. Salazar and A. J. Millar. Journal of The Royal Society Interface, 2004. DOI: 10.1098/rsif.2004.0014. 2. Uncovering the design principles of circadian clocks: Mathematical analysis of flexibility and evolutionary goals. D.A. Rand, B.V. Shulgin, J.D. Salazar and A.J. Millar. Journal of Theoretical Biology, volume 238, Issue 3, 7 February 2006, Pages 616-635. 3. Mathematical Model of the Epigenetic Control of Vernalisation in Arabidopsis thaliana. J.D. Salazar, J. Foreman, I.A. Carré, D.A. Rand and A.J. Millar. Proc. VIIIth IS on Modelling in Fruit Research. Ed.: J. Samietz. Acta Hort. 803, ISHS 2008. 4. Prediction of Photoperiodic Regulators from Quantitative Gene Circuit Models. J. Domingo Salazar, Treenut Saithong, Paul E Brown, Julia Foreman, James Locke, Karen J Halliday, Isabelle A Carre, David A Rand and Andrew J Millar. In preparation.
Eberhard O. Voit (Georgia Institute of Technology)
Joint work with Wallace H. Coulter, Georgia Institute of Technology
Parameter Estimation and Structure Identification in Metabolic Pathway Systems
Modern methods of biology permit the simultaneous generation of large quantities of data characterizing metabolic systems. The data either consist of biochemical and kinetic information on metabolites, enzymes and modulators, or of metabolic time series, and require genuinely different methods for extracting information that can be used for the design of pathway models. The presentation will describe recent methods of parameter estimation and structure identification from time series, along with challenges and open issues. Some new findings suggest that the traditional criteria of speed and quality of fit need to be augmented with measures of uniqueness, extrapolability, and the dependence of each method on implicit assumptions regarding the chosen modeling format.
Victor Zavala (Argonne National Laboratory)
Computational Strategies for Large-Scale Estimation in Dynamic Systems
We present a nonlinear programming (NLP) framework for the solution of estimation problems constrained by partial differential equations (PDEs) and/or differential and algebraic equations (DAEs). The framework is based on recent developments in interior point NLP solvers and on automatic differentiation and sparse linear algebra techniques. We demonstrate that these components enable the solution of highly complex problems including many experimental data sets and degrees of freedom. We then discuss extensions to accelerate solutions in on-line environments and to perform inference analysis. An industrial case study is presented to illustrate the developments.
Tje Lin Chung (Goethe University Frankfurt Main)
Joint work with Natalie Filmann, Stefan Potente and Eva Herrmann
The use of post-mortem continuous temperature measurements for estimating the time of death
Temperature based techniques are commonly used in forensic medicine to describe the early postmortem interval and to estimate the time of death. Henssge  introduced a practical and robust algorithm based on single body and ambient temperature measurements. Two well known shortcomings are the need of choosing corrective factors to account for “non-standard” cooling conditions and the theoretic assumption of a constant ambient temperature. We propose a non-linear regression model to describe the characteristics of the body cooling curve. Regression parameters were fitted using continuous temperature measurements of 11 forensic cases. Model validation was performed by comparing the reported and predicted time of death. The main advantage of the presented method is the discard of corrective factors which purely depend on the examiner’s subjective choice. On average our method yielded a higher accuracy of death time estimation for the inspected cases. However, it only showed a moderate degree of improvement. In a next step we want to investigate the impact of variations in ambient temperature on the body cooling curve.
 Hennsge C. (1988), Death time estimation in case work. I. The rectal temperature time of death nomogram, Forensic Science International, 38, pp 209-236
Rosário Laureano (IBS — ISCTE
Business School Lisboa)
Joint work with Diana Mendes
Efficient synchronization with continuous chaotic dynamical systems
The possibility of two (or more) chaotic systems oscillate in a coherent and synchronized way is not an obvious phenomenon, since it is not possible to reproduce exactly the initial conditions and infinitesimal perturbations of the initial conditions lead to the divergence of nearby starting orbits. However, when ensembles of chaotic oscillators are coupled, the attractive effect of a suitable coupling can counterbalance the trend of the trajectories to diverge. In many cases there are parameters that control the strength of coupling between the systems. So, the results of stability of synchronous chaotic state depend on them (coupling parameters).
In order to obtain identical and generalized synchronization, we apply various unidirectional and bidirectional coupling schemes between Lorenz or Rössler systems, with control parameters that lead to chaotic behavior. We combine some of these with total or partial substitution on the nonlinear terms of the second system, a coupling version that was less explored. In some cases we only conclude about local stability of the synchronous state, but we present coupling schemes where the global stability is guaranteed. The conditions of global stability are obtained from a different approach of the Lyapynov direct method for the transversal system. In fact, the effectiveness of a coupling between systems with equal dimension follows of the analysis of the synchronization error. In optimal situation, the coupling leads to their asymptotic synchronization between the systems, but it is also possible to consider practical synchronization where is only expected to stabilize the synchronization error.
Coupled dynamical systems are constructed from simple, low-dimensional dynamical systems and form new and more complex organizations. The chaotic dynamics introduces new degrees of freedom in ensembles of coupled systems. However, when two or more chaotic oscillators are coupled and synchronization is achieved, in general the number of dynamic degrees of freedom for the coupled system effectively decreases.
Our motivation for researching chaos synchronization methods is to explore their practical application in various scientific areas, such as physics, biology or economics.
The ability of nonlinear oscillators to synchronize with each other is a basis for the explanation of many processes of nature.Therefore, synchronization of chaos is thus a robust property expected to hold in mademan devices and plays a significant role in science.
Using the singular value decomposition, in this paper it is presented a method for unidirectional coupling that suppresses exponential divergence of the dynamics of the synchronization error (due to the sensitive dependence on initial conditions), and exploits the existing contraction properties of the given systems. In this way, systems can be synchronized using a minimum of transmitted information and is guaranted linear stability of the synchronized state in all points of the state space.
Onyango Nelson Owuor (Technical University of Munich)
Characterising Optimal Vaccination Strategies in Periodic Settings
Using a simple SIR model, we characterize optimal vaccination strategies in periodic settings for childhood diseases such as measles, chicken pox, and others. Such diseases are characterized by outbreaks that last only a short time, due to short infective periods. Elegant mathematical techniques such as Floquet theory or singular perturbation are used to define the stability criterion of the model. Using the stability criterion, an optimal control problem is defined, which is then used to characterize an optimal vaccination strategy. But the parameters are widely not known. Data sets that define this problem are rare, or generally unavailable, making it difficult to verify model assumptions.
Key words: Floquet Theory, Singular Pertubation theory, Disease Free Periodic Orbit, Optimal Vaccination Strategies.
Alexei Sharpanskykh (Vrije Universiteit
Joint work with Fiemke Both, Mark Hoogendoorn, S. Waqar Jaffry, Rianne van Lambalgen, Rogier Oorburg, Jan Treur, and Michael de Vos
An Approach to Validate of an Agent Model for Human Work Pressure
In demanding working circumstances the quality of the tasks performed by a human might degrade. To improve task performance, personal assistant agents may be used. A personal assistant agent is able to reason about the current state of the human, and to give the most appropriate and effective support, when required. To enable this, the agent may use diverse human cognitive models. One of such models is a human work pressure model has been developed previously (Bosse et al., 2008). To ensure that human states are recognized correctly and a proper support is provided to the human by the agent, the model used by the agent should be valid. In this work an approach is proposed to validate the existing work pressure model. The approach comprises a number of steps. First, human experiments have been designed and conducted, whereby measurements related to the model have been performed. Next, this data has been used to obtain appropriate parameter settings for the work pressure model. To this end, parameter estimation for the work pressure model has been performed using an approach based on the maximum likelihood principle and an approach based on probabilistic search. The estimation quality obtained is discussed in this work using a number of standard measures. Finally, the work pressure model, with the tailored parameter settings, has been used to predict human behavior to investigate predictive capabilities of the model. The prediction results are considered in this work.
References Bosse, T., Both, F., Lambalgen, R. van,, & Treur, J. (2008). An Agent Model for a Human's Functional State and Performance. In: Jain, L. et al. (eds.), Proceedings of International Conference IAT'08. (pp. 302-307). IEEE Computer Society Press.
José Carlos Simon de Miranda (University of São Paulo )
Functional parameter estimation in partial diferential equations
S. Waqar Jaffry (Vrije Universiteit Amsterdam)
Joint work with Mark Hoogendoorn and Jan Treur
Parameter Estimation for Human Trust in Information Sources
In this paper, an approach has been presented to learn parameters of a given trust model based upon observed experiences of a human. This approach has been introduced to enable a personal assistant agent to take such human trust into account when giving advice. Hereby, an existing trust model has been taken as a basis. Several methods have been used to enable learning of these parameters, including exhaustive search, Simulated Annealing, bisection, and an extended form of bisection. The process is adaptive in the sense that new experiences can come in, and are taking into consideration by finding the most appropriate parameter setting. The algorithms have been rigorously tested for various cases, and the results thereof have been analyzed using formal verification techniques. The results show that the computation time of the exhaustive search scales up worst, whereas the Simulated Annealing approach scales up best. When looking at the accuracy however, the inverse is true: exhaustive search finds the most accurate point, whereas Simulated Annealing sometimes only comes up with poor solutions. The bisection, and the more advanced extended bisection approach are right in the middle: They do have a higher accuracy and are computationally less expensive. The choice of which method to choose ultimately depends on the domain. For particular domains a higher computation time might be acceptable as long as the results are good, whereas in other more time critical domain speed could be a necessity. In this respect, the bisection approaches are a good combination of both worlds.
Muhammad Umair (Vrije Universiteit Amsterdam)
Joint work with Jan Treur
Parameter Estimation for Adaptive Model-Based Reasoning about Environmental Dynamics
The environments in which agents are used often may be described by dynamical models, for example in the form of a set of differential equations. In this paper an agent model is proposed that can perform model-based reasoning about the environment, based on a numerical (dynamical system) model of the environment. Moreover, it does so in an adaptive manner by adjusting the parameter values in the environment model that represent believed environmental characteristics, thus adapting these beliefs to the real characteristics of the environment.