When: 1:00-1:45 PM September 3, 2014

Chairs: Daniel A Beard (University of Michigan, beardda@umich.edu) and C. Anthony Hunt (UCSF, a.hunt@ucsf.edu)

Modular design is established in software engineering, but less defined in the computational physiology/bioengineering domain. Specifying, ingratiating, and composing component models (representing sub-systems or “modules”) for applications in biology is typically accomplished in an ad-hoc manner, often involving hand coding of composite models using various standards (CellML, SBML) and authoring/simulation tools (MATLAB, JSim, and OpenCOR, as well as general-purpose “low-level” languages, such as C and Fortran).

It is recognized that we will be able to achieve faster and more reliable progress by developing and using methods that enable and facilitate modular multi-scale model construction.  The process necessitates stating new requirements that, themselves, may differ depending on model and module usage.

The purposes of this session will be to describe the challenges in realizing this goal and to identify current approaches to meeting these challenges.  

Summary of points raised:

MSBbreakoutsummary.pptx

Modularity

• biological modularity is different from software engineering
• "white box modularity”, semi-automated merging of modules e.g. bioreceptor with cardiac dynamics
• joining models by establishing equivalence relations between variables or parameters in different modules
• what is the minimal specification that should be required to enable shared modules to be integrated? (A. Popel)
• isolated cell might not operate like the same cell in the body - limit to modularity (Meyer Katzper)

Model sharing

• need standards for how to annotate what is in the model, use of common ontology, not two ontologies for the same thing, standards for annotation (DB)
• each module has its own assumptions, parameters, variables, etc., how to join models together in such a circumstance? (Ashley Kia)
• journals do in principle agree on the need for standards, but don't want to create barriers to submission (DB)
• cellML, SBML - strict in certain domains, but in multiscale models of physiology, the standards are yet to be set, but we still need annotation of what exists to allow interoperability (DB)
• challenge: getting into and out of a standard format, e.g. when code written in Fortran or C for high performance, how to get it into standard format (Moshin)
• password controlled access to code in order to count the number of users to then report to funding agencies. journals could collect download statistics (A. Popel)
• Need for a repository of models that can't adhere to standards that exist (M. King)
• Movement in publishing on experimental side to share protocols but no comparative protocol journal for computational modeling (MK)

Reproducibility (models vs. experiments)

• released code must be more robust - in house use to released code takes a lot of effort - need to guard against spurious conclusions obtained from improper use. (Anon)
• Fear of sharing: sharing code to everybody might make a researcher redundant, no problem to share with collaborators (Anon)
• sharing of models distinct from sharing of code, code does not contain the model, it runs the model (A. McCulloch)
• separation of simulation code (available as compiled program) from model specification code (completely open) (AM)
• workflow is an important part to be shared (e.g. for combining pde simulation with machine learning, sensitivity analysis etc) (AM)
• standards often lag behind aspects being represented in models being currently developed (Gary An)
• Are models reagents or tools? (comparison with experimental sharing) J. Linderman, F. MacCabhann, I. Melas
• different compilers can give different results (Meyer Katzper)
• how to collect remote usage statistics (R. Fleming)

Complete transcript (Scribe: Ronan Fleming) - please edit or amend as appropriate

Dan Beard

• model modularity - biological modularity is different from software engineering
• "white box modularity"
• semi-automated merging of modules e.g. bioreceptor with cardiac dynamics
• joining models by establishing equivalence relations between variables or parameters in different modules
• most suited to ode modeling
• in general the publication is insufficient to reproduce the modeling
• some models cannot be specified mathematically, so how can it be reproduced without the original code?
• funding source may obligate the sharing of code
• credit for sharing code e.g. King-Altmann
• need standards for how to annotate what is in the model
• use of common ontology, not two ontologies for the same thing,
• standards for annotation

Tony Hunt

• multi-scale modeling
• start from the use case
• modularity built into the requirements
• select modeling methods to meet requirements
• we must strive to make our modeling reproducible - reproducibility is one of the cornerstones of science
• what is the scientific value of a modeling publication that takes years to reproduce (due to lack of specification)
• modularity can help with sharing, that is sharing individual modules

Ashley Kia

• each module has its own assumptions, parameters, variables, etc.
• how to join models together in such a circumstance?

Dan Beard

• tools to annotate models in reproducible way
• need to capture experimental protocol
• tools exist for formalized representation of modeling experiments
• existing repositories for data
• need standards to make things interoperable
• http://www.opencor.ws/
• SBML has resources for modeling
• solver versus model
• journals do in principle agree on the need for standards, but don't want to create barriers to submission
• IMAG wiki offer a way to per
• fieldML - standards for everything - is it realistic?
• cellML, SBML - strict in certain domains, but in multiscale models of physiology, the standards are yet to be set, but we still need annotation of what exists to allow interoperability
• input, output, interface, how to define?

Moshin

• almost impossible to reproduce a paper with description given in the majority of papers
• challenge: getting into and out of a standard format, e.g. when code written in Fortran or C for high performance, how to get it into standard format
• parsing of equation blocks

Aleksander Popel

• provision of sufficient material to reproduce the modeling results
• multi-scale models exist beyond biology, e.g. boeing - how do they do it? are there general principles we could apply?
• what is the minimal specification that should be required to enable shared modules to be integrated?
• not sure that sharing should be obligatory?
• password controlled access to code in order to count the number of users to then report to funding agencies
• journals could collect download statistics

How many people share models routinely

• 30% routinely share simulation source code simulation engine
• % routinely share compiled simulation engine
• % routinely share model

Michael King

• Need for a repo of models that can't adhere to standards that exist
• Movement in publishing on experimental side to share protocols but no comparative protocol journal for computational modeling
• Incentives for sharing is better than making it a requirement
• Open access publications get more downloads and more citations
• released code must be more robust - in house use to released code takes a lot of effort - need to guard against spurious conclusions obtained from improper use
• download statistics are not equivalent to usage statistics

Anon

• fear of sharing: sharing code to everybody might make a researcher redundant, no problem to share with collaborators

Andrew McCulloch

• incentive for sharing: grant mechanisms (mp3)
• sharing of models distinct from sharing of code
• developing code can take years to develop
• code does not contain the model, it runs the model
• separation of simulation code versus code that describes the model
• separation of simulation code (available as compiled program) from model specification code (completely open)
• workflow is an important part to be shared (e.g. for combining de simulation with machine learning, sensitivity analysis etc)

Gary An

• what constitutes sufficient specification to reproduce modeling result without the actual source code?
• published wet-lab protocols often not sufficient to experiments
• standards often lag behind aspects being represented in models being currently developed

Feilim MacCabhann, John Hopkins

• Microfluidic devices must be shared when requested
• Are models reagents or tools?
• modeling code can be difficult to make work, it may be faster to reproduce it from scratch
• model must be created in a way that it is shareable

Conceptual questions

• is it possible to integrate different modeling software
• is it feasible to build a standard

Denise Kirschner

• standardization can impede sharing

Ioannis Melas, FDA

• in electronics, one may be able to create the whole device being modeled, but not so in systems biology
• data used to train model can be noisy

Meyer Katzper

• isolated cell might not operate like the same cell in the body - limit to modularity
• pulmonary function - 1st, 2nd, 3rd order interactions
• integration time and effort necessary, rather than just building models
• pseudocode could be used to specify the properties of a model
• more valuable if one knows the mechanisms of a simulation, rather than using it as a black box (e.g. blind use of statistical software)
• different compilers can give different results

Ronan Fleming

• start with the basic parts of a model that can be standardized
• e.g agree on common identifier for metabolites, genes, etc
• sharing code helps to improve the code
• how to collect remote usage statistics