Introduction

JSim Data Sets are portions of a JSim project that represent numeric data captured from experimental measurements or previous model runs. This document describes how to load data sets into a JSim project, some typical uses for them once they are loaded, and how to export data to external files.

Prerequisites:

• Introduction to the JSim GUI (required)
• JSim Projects (recommended)

Contents:

• Data set Structure
• Loading data sets from external files
• Loading data sets from model runs
• Create 1D CSV data curves
• Modify an existing, imported 1D data curve
• Copying data sets between projects
• Typical uses for Data sets
• Exporting data sets to external files
• TAC Data Format
• Columnar Text Data (CDATA) Format
• Comma Separated Value (CSV) Data Format
• JSML Data Format
• Pretty Data (PDATA) Format
• Line Data (LDATA) Format
• Matlab Data Format
• I4 Bullseye Data Format
• Plug-in Data Formats
• Comments or Questions?
  Give feedback

JSim Data Set Structure

JSim data sets are collections of data curves. Each curve has a name that is unique within the data set by which the curve is known to the rest of JSim. A curve name must also start with a letter and contains only letters, numbers and the underscore character. A curve may optionally possess a free-form text description and physical unit information. Curves may be of any dimension, however 1-dimensional data is most common and is best supported in JSim at present. Each data set in a JSim project will appear as a leaf in the JSim project tree and as a tab on the left-hand side of the JSim project window.

Loading data sets from external files

Most data sets are loaded from external data files. Currently, the following import formats are supported:

• TAC format is a standard for time-activity curve representation defined by NSR that offers easy interchange with other NSR simulation products such as Simcon, XSIM and earlier versions on JSim (1.4 and 1.5).
• Columnar text (CDATA) format is an informal format, described below, offering easy interchange with spreadsheet programs.
• Comma separated value (CSV) format is similar to columnar text format, but uses commas a separator characters is becoming more popular for scientific data exchange.
• JSML format (for JSim Markup Language) is an XML dialect used for internal data storage within JSim project files. It supports multi-dimensional data and offers easy interchange with XML based data-bases and applications.
• I4 Bullseye format is a functional image format for 2D and 3D cardiac functional image data used the the I4 imaging system. This format is useful only if you use the I4 system.

JSim also offers a data format plug-in feature that allows users to define support for other file formats within JSim. Serious users of JSim should become familiar with the relative advantages of each supported format. The list of supported import formats will grow as JSim matures.

To load an external data file of a supported format:

1) obtain a copy of the file on your computer system and make sure if conforms to the applicable specification (see above);

2) select "Import data file ..." from the Add menu in a project's "Project" tab;

3) navigate the file system and select the desired file.

If the file conforms to specification, a new data set leaf will appear in the project tree. If not, a diagnostic error message will appear in the project's Message tab.

Loading data sets from model runs

It is sometimes useful to save output data from model runs into data sets. Some possible uses include:

• using one model's output to feed another's input;
• generating reference data for comparison of different models (e.g. optimizing one model to fit another's output).

To do this:

1) run a model and plot the desired data in a plot page;

2) select "Store project dataset ..." from the plot page's File menu;

3) name the new data set at the prompt.

Once completed a new data set leaf will appear in the project tree.

Note that a data set may be created from the model optimization graph in an analogous manner.

Creating new 1D CSV curves from within JSim

(Available in JSim version 2.19 or greater) JSim allows you to enter 1D data curves directly into the project file through the JSim gui:

• From within the JSim GUI, select the 'Project' tab.
• Select 'Add' --> 'New data set' from the drop-down menu. Either type in a name for the new data set or use the default.
• Select the 'DataSets' tab and select the name of the data set created above.
• Now select 'Edit' --> 'create' to create a new data curve. A pop-up window should appear with an example data curve. Currently only 1D CSV format is supported.
• Type in your curve following the template shown. You may copy/paste a curve into the window, if desired.
• Finally, select 'Curve' --> 'Save' to save the curve.

The new data set and curve should now be available for plotting and analysis within JSim (Do not forget to save the updated project file). You can also create and add a new data curve to an existing data set.

 

Modify existing, 1D curves from within Jsim

(Available in JSim version 2.19 or greater) You can modify an existing 1D data curve from within JSim:

• Pick the data set containing the curve you wish to modify through selecting it in the 'DataSets' tab in the top right panel of the JSim gui.
• Next, select the curve you wish to modify and then select 'Edit' --> 'modify'. A pop-up window should appear containg the data curve in CSV format.
• Now edit curve as needed. You can delete, add data points, and change values. All additions must follow the CSV formatting.
• Finally, select 'Curve' --> 'Save' to save the curve.

The modified curve will have a '_mod' appended to the original curve name. Note, the original curve is not deleted. To delete an unneeded curve just select the curve and select 'Edit' --> 'cut'

 

Copying data sets between projects

Data sets may be copied directly between project using the cut/copy/paste mechanism that applied to all items in a project tree. This is preferred to the old-style mechanism of writing data sets to external file and then reimporting the external files because:

• It is simpler, faster and less error prone;
• The only current supported file format supports all information in a data set is JSML. Some information may be lost if other formats are used.
• Local file access is when JSim is run as a untrusted WWW applet.

To do this:

1) make sure both project windows were created in the same JSim program execution (copying between different executions is not yet supported);

2) select the Project tab in the project you wish to copy from;

3) highlight the data set (or other project tree leaf) you wish to copy by clicking on it. Additional leaves may be highlighted by clicking with the Control key depressed;

4) select "copy" from the Edit menu;

5) select the Project tab in the project you wish to copy to;

6) select "paste" from the Edit menu.

Typical uses for Data Sets

Plotting. Data may be selected for plotting in plot pages. Once a data set is loaded, it becomes an option in the data source selection box in the plot page configurator. This data source may be either a model or a data set. When it is a data set, the data select box will list the curves in a data set by name.

Function Generators. Model function generators are used to generate useful functions that can be assigned to model input variables. The "DataCurve" option allows the function generator to assume values taken from a particular curve in a data set. For further information see Using Function Generators in JSim

Optimization. JSim's model optimizers allow for automated analysis of what parameter values generate output that best match given experimental data. For further information see Using JSim's Parameter Optimizer

Exporting data sets to external files

Data files may currently be exported from two places within JSim:

• Project data sets may be exported using the "Export data file..." menu item in a data set's File menu;
• Plot data may be exported using the "Export data file..." menu item in a plot page's File menu;

Exported data files may be written in any of the supported input formats or in three "output only" formats below:

• Pretty format presents 0-D, 1-D and 2-D in somewhat attractive text tables. Higher dimensional data is rather less attractive. This format is used for text display of data within JSim (e.g. plot page text). File is saved with a .pdata extension.
• Line format presents data for each variable on a single line. This presents some advantages in model verification and debugging for researchers familiar with Unix text file tools.
• Matlab format writes data in Matlab's .m format. This format supports data of arbitrarily large dimension.

When exporting data, the user must supply a file suffix, and that file suffix must match the recommended suffix for the desired data format. Formats and corresponding suffixes are displayed in the export dialog. They are also detailed in this document under the appropriate format.

TAC Data Format

TAC (for Time Activity Curve) format is an ASCII text specification designed by NSR. It is strongly recommended that TAC files have the .tac extension. An example of a TAC file is 

TAC format strengths are:

• supports data interchange between various NSR simulation interfaces (Simcon, XSIM, and JSim);
• promotes good documentation;
• format well suited to multiple indicator dilution experiment data.

TAC format weaknesses are:

• files are complex to create from scratch. Some utilities are available from NSR;
• only 1-dimensional data is supported.

JSim assigns names to curves within TAC files using the same convention as XSIM. For those unfamiliar with XSIM, here is a brief synopsis. The highest level of TAC file structure is the "run". Each run may contain any number of physiological, input function and/or sample curves. JSim's assigned curve names are of the form R#x# where:

• R is the letter "R";
• the first # is the run number, starting count at 1;
• x is the letter "p" for physiological, "i" for input function or "s" for sample data curves;
• the second # is curve number of the specified type within the run, starting count at 1.

Columnar Text Data (CDATA) Format

Columnar text data files are ASCII files containing a single 2-dimensional table representing one or more 1-dimensional data curves. It is strongly recommended such file have the .cdata extension. An example, generated from a JSim plot page, is 

The format is described below.

Columnar text format strengths:

• quick and easy to generate from spreadsheets or custom utility programs;
• exporting tabular text from multi-column numeric table in a spreadsheet program will normally generate a file compatible with this format.

Columnar text format weaknesses:

• not a formal standard, so conventions not necessarily supported by other products and may change in the future;
• no provision for in-file documentation;
• only 1-dimensional data is supported.

Comma Separated Value (CSV) Data Format

CSV data files are ASCII files containing a single 2-dimensional table representing one or more 1-dimensional data curves. They are very similar to Columnar text format except that they use the comma as a separator character. It is strongly recommended such file have the .csv extension. An example of a CSV file is 

CSV format strengths:

• this format is becoming more popular for scientific data exchange;
• quick and easy to generate from spreadsheets or custom utility programs;
• exporting tabular text from multi-column numeric table in a spreadsheet program will normally generate a file compatible with this format.

CSV format weaknesses:

• CSV data file are sometimes used to represent data which does not conform to JSim's interpretation of a set of 1D curves;
• units are not supported;
• only 1-dimensional data is supported.

Formal format specification:

File must be ASCII text. Blank lines are ignored. Each non-blank line must have the same number of columns. The first non-blank line may be either a header line or a data line. The second non-blank line may be either a units line or a data line. All subsequent non-blank lines must be data lines. Data lines must contain only space or tab delimited numbers. Numbers follow standard ASCII floating point representation. The keyword NaN, in any capitalization, translates to an IEEE 754 NaN. The first non-blank line is considered a header line if the content of first column is not a number. If present, a header line must contain a unique name for each column that starts with a letter. If present, a units line must contain a unit name for each column. Each name or unit may or may not be enclosed in matching single or double quotes. The first column represents the independent variable. Subsequent columns represent dependent variable values corresponding to the 1st column. Values of the independent variable must be strictly increasing and may be either regularly or irregularly spaced.

The imported data set will contain one curve for each dependent variable, that is, one less than the number of columns. If a header line is present, curve names are taken from the header with any enclosing quotes removed and any unacceptable characters replaced by underscore. If a header line is not present, curves are automatically named y1, y2, etc.

Examples:

Two examples should make the formal specification easier to grasp. The first file includes a header line. JSim will name the two resulting curves u and v. The columns line up in this example for clarity, but alignment is not required by the specification:

        time      u         v
        0         1         2
        1         .5        4
        2         .25       8
        5         .125      16
        10        6.25e-2   32

In this second, there is no header line. JSim will name the two resulting curves y1 and y2:

        0         1         2
        1         .5        4
        2         .25       8
        5         .125      16
        10        6.25e-2   32

In this third example, there is both a header an a units line:

        time      u         v
        sec       watt/M^2  m/sec
        0         1         2
        1         .5        4
        2         .25       8
        5         .125      16
        10        6.25e-2   32

JSML Data Format

This format (JSim Markup Language) is an XML dialect used for internal data storage within JSim project files. It supports multi-dimensional data and offers easy interchange with XML based data-bases and applications. XML tags for this format are not yet finalized. The recommended file suffix for this format is .jsml . An example, generated from a JSim plot page, is 

There are two encoding options for JSML: legacy and ascii. Ascii encoding is now recommended in all cases because:

• disk storage is roughly 1/3 that required by legacy encoding, with corresponding decrease in loading time;
• run-time memory allocation for loaded data is roughly one half that required by legacy encoding;
• better support for multi-dimensional data curves and physical unit import.

Ascii is the default encoding for JSML data files. The jsbatch -oencoding switch allows you to specify legacy encoding if desired.

Pretty Data (PDATA) Format

This format presents 0-D, 1-D and 2-D in somewhat attractive text tables. Higher dimensional data is rather less attractive. This format is used for text display of data within JSim (e.g. plot page text). It is an output format only, and may not be imported into JSim. The recommended file suffix for this format is .pdata. An example, generated from a JSim plot page, is 

Line Data (LDATA) Format

This format presents data for each variable on a single line. This presents some advantages in model verification and debugging for researchers familiar with Unix text file tools. It is an output format only, and may not be imported into JSim. The recommended file suffix for this format is .ldata . An example, generated from a JSim plot page, is 

Matlab Data Format

This format allows export of JSim numeric results to Matlab via .m files. This is an output only format, and may not be imported into JSim. An example, generated from a JSim plot page, is 

 This format supports data of any dimensionality. A 1D MML variable will be exported as a Matlab vector, a 2D MML variable will be exported as a 2D Matlab matrix, a 3D MML variable will be exported as a 3D Matlab matrix, and so on. The domain (e.g. time) values corresponding the matrix dimensions are also written to Matlab vectors in a systematic way. For example, a 2D MML variable u(t,x) would generate 3 Matlab variables:

• u: a 2D matrix containing the values of u for each t and x;
• u__1: a 1D vector containing the values of t corresponding to u's 1st dimension;
• u__2: a 1D vector containing the values of x corresponding to u's 2nd dimension.

Limitations: Note that JSim import support for .m files would require a complete reimplementation of Matlab, so don't even think about it. Also note that only numeric data is transferred. Translating JSim MML source code into corresponding Matlab executable code would require a very significant additional programming effort, and is not anticipated at this time. It is possible, however, that Matlab binary .mat files may be supported for both import and export as some future time. If this is of pressing interest to you, please contact the JSim development team.

I4 Bullseye Data Format

This format is used by the I4 Imaging System for storage of static (2D) and dynamic (3D) cardiac bullseye data. JSim supports this format (both input and output) as part of its functional imaging facility which is currently under development. The recommended file suffix for this format is .bul .

Plug-in Data Formats

JSim also offers a data format plug-in feature that allows users to define support for other file formats within JSim.

Comments or Questions?

Give feedback

 

Model development and archiving support at https://www.imagwiki.nibib.nih.gov/physiome provided by the following grants: NIH U01HL122199 Analyzing the Cardiac Power Grid, 09/15/2015 - 05/31/2020, NIH/NIBIB BE08407 Software Integration, JSim and SBW 6/1/09-5/31/13; NIH/NHLBI T15 HL88516-01 Modeling for Heart, Lung and Blood: From Cell to Organ, 4/1/07-3/31/11; NSF BES-0506477 Adaptive Multi-Scale Model Simulation, 8/15/05-7/31/08; NIH/NHLBI R01 HL073598 Core 3: 3D Imaging and Computer Modeling of the Respiratory Tract, 9/1/04-8/31/09; as well as prior support from NIH/NCRR P41 RR01243 Simulation Resource in Circulatory Mass Transport and Exchange, 12/1/1980-11/30/01 and NIH/NIBIB R01 EB001973 JSim: A Simulation Analysis Platform, 3/1/02-2/28/07.