Agent-based Models to address the Crisis of Reproducibility and Precision Medicine

Investigators
Gary An and Daniel Faissol
Contact info (email)
gan@med.uvm.edu
1. Define context(s)
aid in clinical decision making
aid in clinical trial design
reveal new biological insights
other
Current Conformance Level / Target Conformance Level
Partial
Primary goal of the model/tool/database

Development of machine learning, evolutionary computing and artificial intelligence approaches to increase clinical applicability of agent-based models. Context is to foster a new, ideally more clinically relevant, perspective of how to validate agent-based models and compare them to real-world data and situations. The key component of this is being able to more fully capture and utilize the heterogeneity of both agent-based models and aid in their use for the discovery of clinical interventions.

Biological domain of the model
NA
Structure(s) of interest in the model
NA
Spatial scales included in the model
NA
Time scales included in the model
NA
Additional comments about the model’s context (optional)

The IIRABM is being used primarily as a testing and development platform for the methods that form the core of this grant. The suitability of the IIRABM is based on prior validation against clinical behavior, and the overall complexity of the model.

2. Data for building and validating the model
Data for building the model Published? Private? How is credibility checked? Current Conformance Level / Target Conformance Level
in vitro (primary cells cell, lines, etc.) X Lit Review Adequate
ex vivo (excised tissues)
in vivo pre-clinical (lower-level organism or small animal) X Lit Review Adequate
in vivo pre-clinical (large animal) X Lit Review Adequate
Human subjects/clinical X X Adequate
Other: ________________________
Data for validating the model Published? Private? How is credibility checked? Current Conformance Level / Target Conformance Level
in vitro (primary cells cell, lines, etc.)
ex vivo (excised tissues)
in vivo pre-clinical (lower-level organism or small animal)
in vivo pre-clinical (large animal)
Human subjects/clinical X X Goal of the Project Partial
Other: ________________________
3. Validate within context(s)
Who does it? When does it happen? How is it done? Current Conformance Level / Target Conformance Level
Verification Inside group During code development Standard debugging tools and procedures Adequate
Validation Inside group Method Purpose Developed Method Partial
Uncertainty quantification
Sensitivity analysis
Other:__________
Additional Comments The goal of this project is to develop new perspectives and methods for assessing validity. Code development follows standard verification procedures. We are attempting to redefine the means of assessing appropriate context for validation for agent-based models.
4. Limitations
Disclaimer statement (explain key limitations) Who needs to know about this disclaimer? How is this disclaimer shared with that audience? Current Conformance Level / Target Conformance Level
Currently highly theoretical Users Stated in publications Partial
5. Version control
Current Conformance Level / Target Conformance Level
Adequate
Naming Conventions? Repository? Code Review?
individual modeler
within the lab X
collaborators X X
6. Documentation
Current Conformance Level / Target Conformance Level
Code commented? Partial
Scope and intended use described? Yes
User’s guide? No
Developer’s guide? No
7. Dissemination
Current Conformance Level / Target Conformance Level
Partial
Target Audience(s): “Inner circle” Scientific community Public
Simulations X X
Models X X
Software X
Results X X
Implications of results X X
8. Independent reviews
Current Conformance Level / Target Conformance Level
NA
Reviewer(s) name & affiliation:
When was review performed?
How was review performed and outcomes of the review?
9. Test competing implementations
Current Conformance Level / Target Conformance Level
NA
Yes or No (briefly summarize)
Were competing implementations tested? Yes. The point of the project is to demonstrate the benefit of capturing system behavioral heterogeneity (primarily in terms of trajectory hyperplanes/manifolds) compared to standard calibration/fitting processes.
Did this lead to model refinement or improvement? Yes. In terms of the project workflow, this is the subject of the grant and is continuing to be refined. In terms of the IIRABM, findings from this work has led to development of the next iteration of the immune model (also in response to COVID19).
10. Conform to standards
Current Conformance Level / Target Conformance Level
Yes*
Yes or No (briefly summarize)
Are there operating procedures, guidelines, or standards for this type of multiscale modeling? Yes
How do your modeling efforts conform? Follows standard procedures for code and rule documentation/testing. However, primary goal of project is to expand these operating procedures in terms of using agent-based models
11. (optional) Additional information to support items 1-10

As noted above, the primary goal of this project is to develop new methods and workflows to improve the ability to appropriated contextualize, validate and use agent-based models for translational/clinical research. More specifically, this involves developing methods to better capture the heterogeneity of the real world, and in so doing enhance the robustness and generalizability of research involving agent-based models. This involves developing novel methods for characterizing the behavioral hyperplane/topology of an agent-based model without having to resort to brute force computation, developing means of training neural networks that can serve as ABM surrogates, and applying reinforcement learning and genetic algorithms to facilitate control discovery using ABMs.