Working Group 10: Model Standards and Sharing

Announcements

JimBassingthwaighte 7feb12: Donna R. Lochner (Associate Director, Office of Science and Engineering Laboratories, Center for Devices and Radiological Health U.S. Food and Drug Administration 301-796-6309) announces:

   FDA/ PhUSE computational science conference

This conference, co-sponsored by FDA and the Pharmaceutical Users Software Exchange, may be of interest to some in IMAG. It seems to be geared towards manipulation of large clinical data sets for development of new drugs. Perhaps folks in the Data Sharing Working Group and maybe others would find the conference working groups of interest:

  http://www.phuse.eu/css
  http://www.phuse.eu/Working-Groups.aspx

Bassingthwaighte/Erdemir 25oct10 WG10 PPT for 27oct10 MSM mtg in DC: Media:IMAG.WG10rep.ppt

JimBassingthwaighte 20oct10. The latest draft version of the White paper on Modular Modeling is provided here. Media:Mod20oct.docx Media:Mod20oct.doc Media:Mod23oct.d.doc

ErdemirA 14:42, 1 July 2010 (EDT) A few of us in WG 6 decided to work on reporting recommendations for models based on finite element analysis Reporting in FEA. Feedback and/or contributions from WG10 is highly appreciated as we are attempting to apply the conceptual framework of the standards proposal to that area. We anticipate to contribute to the modular modeling white paper initiative.

ErdemirA 09:06, 7 June 2010 (EDT) Myself (Ahmet) and a few WG6 members will be at the ASME Summer Bioengineering Conference. I will be happy to meet any of you there to discuss WG10 related issues and future. Please let me know or find me at the conference. If you need information on how I look like, see end of IEEE EMBS Article for a mugshot.

ErdemirA 09:20, 1 March 2010 (EST) Page reorganized, please edit, provide comments.

JimBassingthwaighte 8AM 1jul10 PDT IMAG WG10 Suggestions for on-line discussion (This will be removed after discussion in July and replaced but a report on the discussion.)

1jul10

  • Standards for Modules. (These need to be developed and given preliminary approval for further editing.)
    • Identification of external connections.
    • Definitions for computational innards driven or modified by the external conditions
    • Units consistent with multiple spatial domains in which modules live.
    • Rules for combining modules and computing changes in external variables, e.g. for two or more K+ channels.
    • SemSim (Max Neal) methodology
  • Standards for FE models: (Ahmet Erdomir has led off: See Erdomir 1jul2010 "Reporting in FEA" above)
    • Store model along with computational platform?
    • SimBios standards, well enough defined?
    • Data analysis methods: how to define these?
    • Validation Measures (fitting data and testing validation by prediction against other data sets)
    • Units, verification methods, examples,
  • Standards for biophysical/biochemical models: (The Standards of Sept2009 look overly demanding?)
    • Refining and reducing these to make them practical as a check list)
    • Should a Checklist accompany each model?
    • Require an assumption list for each model?
  • Ontologies and using them for modeling: Practicalities for biochemical and electrophysiological (single level cell models with subsidiary modular components as "simple " examples ( ?Dan Cook and John Gennari and Max Neal)
    • FMA, Protege, SemSim (Semantic Simulation) and modular construction of multiscale models.
  • Databasing physiol data and use of such data for modeling systems
    • PhysioNet reqs: Moody/Goldberger suggestions are needed
    • Publication description or more detail including detailed protocols
    • Archive the raw data AND data processed to subsequent stages. (assumptions built into analyses?)
    • Ontologies for data?
    • Both graphs and numbers for time course data?
    • Evaluation of data accuracy
    • Sage: Stephen H. Friend is developing a new company to provide Open Source data:
  • OpenScience | Open Culture www.openculture.com/2009/08/making_the_web_work_for_science.html (Discussion forum participants; Tim O'Reilly (Media), Jimmy Wales (Wikipedia), Stephen Friend (Rosetta, Sage), John Willbanks (Creative Commons)

Working Group 10: Model Standards and Sharing

Working Group Leads

Working Group 10 Participants

See Working_Group_10_Participants

Goals and Objectives

Mathematically defined computer models of biological systems represent integrated summaries of the knowledge captured in the model's equations, definitions and assumptions. They are fundamental packages for the spread of concepts and information. The goal of the Model Sharing Working Group is to maximize the efficiency of distribution of scientific ideas by designing the mechanisms of providing reliable and useful models to the research community in understandable and convenient forms.

Categories

Models

The goal is to make a list of expectations for modeling standards to update draft of standards to be reviewed and improved by others, see #Standards Proposal.

Background

Modular Modeling: Given the existence of a set of well-curated models of the components of a system, "modules", the composing of integrated models from these components can be greatly facilitated by automation of the process, with the advantage that curated modules can be drawn from a database. The automation process requires adherence to additional features (ontologies, input/outputs, internal definitions of nodes and edges) than is required of stand-alone models. A little essay on this topic follows: Media:ModuleRules.pdf. See also Media:Standards.12sep09.xls.

Can we examine the virtues and vices, the difficulties, and reluctances in giving the results of one's hard labor to others so they may: build upon it, criticize it, disprove it, etc.? See #Standards Proposal.

Expectations

  • cleanly written
  • computes accurately and is thoroughly verified to demonstrate that accuracy
  • runs under specified solvers and simulations systems
  • discipline/domain specific standards: for biophysical models and for detailed 3D models with mechanical and anatomical representations
  • Is it possible to expect a unified solver platform for archiving, e.g. OpenFoam? Useful but compliance will be difficult. Maybe the way is to encourage model developers to utilize FREE and accessible simulation software. This way, those who download models can also find the software to simulate.
  • validation: comparisons against experimentation, sensitivity analysis. This component will likely establish the perceived value of the model, a scientific tool (research value) or a learning tool (education value)
  • no feature creep

Documentation

The goal is to make a list of expectations for documentation standards for models to create a draft of standards to be reviewed and improved by others.

Expectations

  • description of model: inputs, outputs, parameters, assumptions
  • definition of its math
  • verification procedures and results
  • validation procedures and results
  • citation of publications
  • instructions for successful simulation with test inputs
  • instructions for assessment of simulation results
  • identification of experimental resources for model development
  • contact information for developers and/or maintainers

Dissemination

The goal is to make a list of expectations for dissemination standards for models to create a draft of standards to be reviewed and improved by others. Note that incentives may be necessary for scientists to spend time to document and disseminate rather than write grants and publications. Support from institutes and funding agencies is necessary, particularly for young investigators (graduate students, post-docs, junior faculty) to contribute.

Expectations

  • timing dof 0issemination: should it follow completing model and documentation, at least for testing? Or early dissemination for collaborative development, documentation and testing? Partial dissemination will help others, too. E.g. discretized geometry, meshes of 3D anatomy.
  • online sharing
  • online simulations
  • access to simulation software
  • quality assurance: online reviewing and rating system
  • licensing allowing use, further development and dissemination
  • version control

Standards Proposed for Physiological/Biophysical Modeling

Working document: Media:Standards.12sep09.xls

Working document in PDF format: Media:Standards.12sep09.pdf

Please edit the page WG-10 - Review of Standards Proposal to provide your comments. Also see Modeling Standards.

Replicating Simulation Experiments

Standards such as SBML (sbml.org) or CellML (cellml.org) allow us to unambiguous describe models of subcellular processes such as metabolism, signaling and gene regulatory events. What is missing is a means to describe actual simulations. It is now possible to download many published models of cellular processes, however it is very difficult to then take that model and reproduce a given graph or table of results from the paper. As a result, some of us in the computation community have begun to develop the first stage in a language that can be used to describe simulation experiments, particularly experiments from the subcellular modeling as this has the most developed community.

Collaborations

  • Functional Collaborations, exchanging code, models, methods, tools:
    • Beard-Bassingthwaighte on intermdiary metabolism an energetics.
    • Chizeck-Bassingthwaighte on model reduction, and on signal detection.
    • Chizeck-Beard-Bassingthwaighte. on comparing JSim and Matlab
  • Intellectual Collaborations, exchanging ideas, principles and approaches:
    • Beard/Cabrera/Bassingthwaighte on the use of thermodynamic constraints in biochemical network modeling.
    • Working Group 10 - Physionet on the ways for dissemination and standardization of it.

Presentations

Working Group 10

23 August 2007

BREEZE presentation of Modeling with JSim, followed by Discussion on model sharing and archiving

Title: JSim Simulation System

Presented by: James Bassingthwaighte, Erik Butterworth, Gary Raymond, Brian Carlson, Kay Sterner, Eric Lawson, University of Washington

Slides: Media:IMAG23aug.ppt

13 March 2009

BREEZE presentation on Reproducibility and Markup Languages, SBML and CellML

Title: REPRODUCIBILITY IS THE ISSUE

Date & Time: Friday March 13, 2009 4-5pm EST

Presented by: Jim Bassingthwaighte, Herbert Sauro, Frank Bergmann, University of Washington

Abstract: Working Group 10 was established in order to define and test and implement a practical set of standards for models. The target area was biophysical and biochemical models rather than finite element or finite volume 3-dimensional models which were considered to be a different class and more difficult to capture for dissemination. The standards provided on the WG10 website have gotten little discussion and probably little usage, so it is clear that a different approach is needed. Various standards and efforts have emerged from the SBML and CellML communities, things such as MIASE (how to describe a simulation), MIRIAM (minimal annotation standards for models), SBO (ontology to describe models), SBGN (visualizing networks), TEDDY (Ontology to describe model behaviors), KiSAO (ontology for kinetics) and SBRML (a sort of NetCDF for simulation). The emergence of accepted standards has also spawned the development of model repositories including BioModels Database, the CellML repository, JWS Online, and at www.physiome.org using JSim-based Wikis for on-line simulation. The goal of the discussion is to define a set of goals for WG10 and the means of achieving them.

Slides (in PPT): Media: UWgroup13mar09.ppt

Slides (in PDF): Media:UWgroup13mar09.pdf

Manuscript: Media:units_rev.pdf, in press on the automation of unit balance check and unit conversion in models

Other Related

Coming Events

MODELING COURSE 2011

The NSR Physiome Project will be holding a five-day simulation and modeling courses on "Cardiovascular and Respiratory Systems Modeling: From Cell to Organ" on August 22-26, 2011 at the University of Washington, Seattle. This courses is for graduate students, investigators, and clinicians wanting to gain experience in mathematical modeling for the analysis of physiological, biochemical and pharmacological data. Please see [1] for more information.

The course fee is $400/person. There are a small number of scholarships and travel stipends available. Please register using our online form or writing Mr Gary Raymond at garyr@washington.edu. Course is supported by NIH/NHLBI 1 T15 HL008516.

Past Events

Developing Multi-Scale,Multi-Cell Developmental and Biomedical Simulations with CompuCell3D and SBW: August 2nd - August 13th, 2010.

We are pleased to announce Joint User-Training Workshop "Developing Multi-Scale,Multi-Cell Developmental and Biomedical Simulations with CompuCell3D and SBW ". It will focus on teaching the basics of multi-cell, multi-scale modeling using the open-source packages CompuCell3D and SBW. The workshop will be taught by many of the CompuCell3D and SBW developers. In addition to participating in lectures and hands-on exercises, each participant should prepare a 30 min presentation covering her/his area of research. Based on our previous experience such presentations lead to many future collaborations as well as make the workshop more scientifically stimulating event.

Event Dates: August 2nd - August 13th, 2010. Deadline: Applications must be submitter before June 15,2010 All submissions must be by email to Dr. Maciej Swat - mswat at indiana dot edu Non-US participants: Before making travel arrangements, please consult us about visa type you need to enter the US. Using certain types of US visas may result in your financial support to be taxed according to US tax code. (Please note: Your reimbursement may be reduced due to U.S. tax withholding.) Consulting US visa regulations is recommended for all foreign participants in order to avoid last minute hassles.

For further details see [2] --SauroH 14:43, 21 June 2010 (EDT)

Systems Biology Workbench at the International Systems Biology Conference, Edinburgh (Oct 2010). Tutorial on "Systems Biology Workbench", Oct 10th, at 8.30am.

April 10, 2007: Model Sharing Strategies - One Day Mini-Workshop

SauroH 14:37, 9 September 2010 (EDT)

Articles

  • Klipp, E; Liebermeister, W; Helbig, A; Kowald, A; Schaber, J Systems Biology Standards--The Community Speaks Nature Biotechnology 25, 390 - 391 (2007) Full Text Link]
  • Lloyd, Catherine; Lawson, James; Hunter, Peter; Nielsen, Poul ""The CellML Model Repository"" Bioinformatics 2008 24(18):2122-2123, Auckland Bioengineering Institue, The University of Auckland, Auckland 1010, New Zealand [Full Text link]
  • Anderson JC and Bassingthwaighte JB. Tracers in physiological systems modeling. In: Mathematical Modeling in Nutrition and Agriculture. Proc 9th Internat Conf on Mathematical Modeling in Nutrition, Roanoke, VA, August 14-17, 2006, edited by Mark D. Hanigan JN and Casey L Marsteller. Virginia Polytechnic Institute and State University Blacksburg, VA, 2007, pp 125-159. Media: 737nutritCh8.pdf
  • Bassingthwaighte JB. Linking cellular energetics to local flow regulation in the heart. In: Control and Regulation of Transport Phenomena in the Cardiac System. Ann. New York Acad Sci 1123:, edited by Sideman S, Beyar R, and Landesberg A. 2008, pp 126-133.
  • Carlson BE, Anderson JC, Raymond GR, Dash RK, and Bassingthwaighte JB. Modeling oxygen and carbon dioxide transport and exchange using a closed loop circulatory system. In: Oxygen Transport to Tissue XXIX, edited by Kang KA, Harrison DK, and Bruley DF. New York, NY: Springer, 2008, pp 353-360.
  • Bassingthwaighte JB. Microcirculation and the Physiome Projects. Microcirculation 15: 835-839, 2008.
  • Bassingthwaighte JB and Chizeck HJ. The Physiome Projects and multiscale modeling. IEEE Signal Processing , 2008 (In press)
  • Alessio A, Butterworth E, Caldwell J, and Bassingthwaighte JB. PET, SPECT and the quantification of coronary blood flow. In Image Guided Therapy Systems. Artech House 2009. (still in press).
  • Bassingthwaighte JB, Noble D, and Hunter PJ. The Cardiac Physiome: perspectives for the future. Experimental Physiology 94.5: 597-605, 2009. Media:623noble.pdf
  • Chizeck HJ, Butterworth E, and Bassingthwaighte JB. Automated Unit Balancing in Modeling Interface Systems. IEEE Eng Med Biol 28(3): 50-58, 2009. Media:units_rev.pdf

Model Repositories

  • The University of Washington Physiome Project: Models for physiological transport (convection, diffusion, filtration , osmosis, enzymatic reaction, flow), particularly for the cardiovascular and respiratory systems can be found at the University of Washington Physiome site. ~300 models and related tutorials on physiological topics (mechanical, transport and biochemical) See this site for LINKS to many modeling sites. Run models over the Web.
  • Biological Components Databank: This databank is being built to provide an online clearinghouse of mechanistic models of functional biological components to use as building blocks to construct computational models of biological systems.
  • The CellML Project: CellML is a markup language for cellular physiology and biophysics. The site houses ~400 models.
  • SimTk: Site of Simbios, the National NIH Center for Biomedical Computing. Repository of models developed for physics-based simulation of biological structures.

Resources

  • JSim: a multi-platform environment (Linux, Windows, MacOS) targeting modeling analysis of data (ODES, PDEs and DAEs), includes automated unit balance checking.
  • IUPS Physiome Project: The Physiome Project is a worldwide public domain effort to provide a computational framework for understanding human and other eukaryotic physiology. The project is the home for CellML.
  • PhysioNet: The research resource for complex physiological signals.
  • Software Carpentry: Good practices for scientists and engineers for software development.

Discussions

There are some corrections and comments to the above 20100424 discussion document which I wish to make here:

1. SBML did not start "out to accommodate differential algebraic equations and ordinary differential equations (ODE)." SBML was devised to describe biochemical reaction based models. SBML stores models in the form of biochemical reactions which could in principle be converted into any modeling approach, including ODE, stochastic, Boolean etc. SBML does have the provision to include explicit ODE and DAE equations, but only when it is not possible to describe a particular biological aspect in terms of biochemical reactions (eg some aspects of electrophysiology - we currently do not have a means to describe electrophysiology models other than by specifying the mathematical model itself).

2. SBML has annual hackathons and annual meetings. These are separate types of meeting, one to serve interoperability and software issues (hackathon) and the other to discuss new developments.

3. "Leadership for standards is also at Cambridge". I am not sure if I would entirely agree with this statement. It is true that much of the comput-infrastructure is at Cambridge (EBI has more resources) but standards development and leadership is distributed across the community. For example SED-ML is currently led from UW Seattle and Rostock, Germany. UW, Seattle also employs two of six worldwide SBML editors. The remaining four editors are at Caltech, Heidelberg, NewCastle (UK) and Connecticut. Cambridge has led the development of the systems biology ontology and the graphical notation, SBGN (also a joint effort by the community).

4. Reproducibility of model simulations is a key issue in modeling. SED-ML is part of a community wide effort (Currently centered at UW) to try and develop a model agnostic approach that will allow researchers to reproduce published simulations directly from journal papers.

--SauroH 15:10, 21 June 2010 (EDT) (Herbert Sauro is one of the founders of SBML (with Kitano, Bolouri and Doyle. Note added by JBB))

Updates

Aug 2008

MODULAR MODELING concepts are broadening. Jim Bassingthwaighte, at the SIAM Life Science meeting on August 5th, reported the success of Gary Raymond's system for the automated combining of previously constructed running models. Three lines (comments) had to be added to each model program to allow the "combine" to recognize the divisions of the model. With this capability, complex models can be assembled, or reassembled in variant forms, from a listing of the component operational modules without coding.

Shiva Ayyadurai and Forbes Dewey reported there a remarkable success in modular computing in which a model solution was computed from three components each running on a computer in a different country. They present Cytosolve, a new method that dynamically integrates a distributed ensemble of biological pathway models without the need to merge the source codes of the individual biological pathway models. Cytosolve allows each biological pathway model to reside at its own location on any computer worldwide, where the authors of each model can independently maintain and update the model’s source code. See their abstract for the SIAM meeting in Session 1.

The curators for SBML and CellML are developing plans for similar approaches. This is also the intent of the developers of "little b", a LISP based approach to the modeling of molecular interactions in cellular biochemistry. See:

http://arstechnica.com/news.ars/post/20080729-little-b-project-creates-biology-specific-programming-system.html and http://blog.wired.com/wiredscience/2008/07/open-source-bio.html

It is also the basis for the developments described by Dan Cook under Current Discussion (below) linking ontologies, anatomic structures, and physiological modeling.

July 2008

THOUGHTS ON RULES FOR CONSTRUCTION OF MODULES for Multiscale Modeling. J.Bassingthwaighte 22jul08

ModulesRules Media:ModuleRules.pdf. This is a draft suggestion for constructing modules for multi-modular multiscale models. Please critique. If you want the original file to work from, please send email to me at my "new" address: jbb2@washington.edu.

12dec07

(Bassingthwaighte). We have just submitted for possible publication in the IEEE special issue on Multiscale Modeling our article entitled "Automated Unit balancing in the JSim Modeling System". It will be reviewed by referees prior to acceptance so any comments and criticisms will be gratefully received, either here or through direct email to jbb2@bioeng.washington.edu. Media:Bassing.Units_11dec07.pdf. It covers Units Definitions, Automated Unit Balance Checking and automated Unit Conversion.

3dec07

(Bassingthwaighte) There is now considerable international effort to embark on the automation of combining multiple modules into integrated, composite models. It will be most important to do this in an automated fashion, as that will allow each new composite model, or a recomposed integrated model, to be generated from the latest and presumable most correct and well-documented version of the set of modules. An update to the Checklist for Standards Spreadsheet is available for critique. Media: Standards.list.dec07.xls

27oct07

(Bassingthwaighte): Through the discussions, the Modeling Standards description has been improved. An abbreviated list Media:Standards.list.xls is provided here as a checklist against which individual model programs may be assessed.

June 2007

Document on Modeling standards under revision by members of WG10 and interested others. Plan to finish the working draft in July. See previous draft Modeling Standards.

Table sorting checkbox
Off