BioUML - open source Java framework for systems biology
 
 Site Content 
 About 
    Home 
    Features 
    Worldwide 
    Team 
    Acknowledgements 
    Contact us 
 
 
 Download 
    BioUML workbench 
    Help & manuals 
 
 
 Development 
    Roadmap 
    History 
    Tests 
 
 
 Community 
    News 
    Forum 
    Conferences 
 
     
  Comparing SBML capable Simulators

Original idea: http://www.sys-bio.org/sbwWiki/compare

The models each simulator had to go through were models from biomodels.net. We used the ninth release (25th September 2007). Each simulator had to simulate the model for 10 seconds, delivering 1000 simulation points. In this comparison we only compared the concentration data returned for floating species.

Results summary
last update - 2008-03-28 - ByoDyn results are added

Simulator passed* no difference small difference significant difference
BioUML v.0.7.8 beta 150 136 129 4 8 10 13
ByoDyn 149 134 129 2 7 13 13
SBWOdeSolver 143 134 126 7 13 2 4
RoadRunner 137 115 107 4 9 18 21
Copasi 133 120 106 5 10 8 17
MathSBML 134 115 104 8 16 11 14
SBToolbox2 106 54 50 38 36 14 20
Oscill8 Core 106 69 57 2 4 35 45
JSim 88 75 73 0 0 13 15
Jarnac 72 66 65 0 1 6 6
VCell 62 54 52 2 2 6 8
CVODE 9 8 7 1 2 0 0

Implementation note
* To pass a test a simulator should provide at least values for one variable excluding time. So if the CSV file contains only time values the test is not passed.

Details

Simulator data (CSV files):

  • CSV files for Copasi, CVODE, RoadRunner, Jarnac, JSim, Oscill8 Core, SBWOdeSolver were obtained by e-mail from Frank Bergmann at 13.11.2007
  • CSV files for MathSBML and SBToolbox2 were downloaded from web site: http://www.sys-bio.org/sbwWiki/compare

Comparison criteria

Passed CSV file was generated by simulator and the file contains values at least for one variabel excluding time.

interval criteria

  •  no difference - 0.999 * min < x < 1.001 * max or 
    x < ZERO and max < ZERO 
  • small difference 0.5 * min < x < 1.5 * max 
  • significant difference - otherwise

median criteria

  •  no difference - Abs( (x-median)/median ) < 0.01  or 
    x < ZERO and median < ZERO 
  • small difference Abs( (x- median)/median ) < 0.5
  • significant difference otherwise

x variable value provided by compared simulator 
min, max, median calculated from values provided by other simulators with which the specified simulator is being compared.

ZERO values (it also can be considered as relative error)  were specified explicitly for following models:

 Model ZERO
BIOMD0000000002 1e-12
BIOMD0000000005 1e-10
BIOMD0000000007 1e-10
BIOMD0000000009 1e-10
BIOMD0000000009 1e-10
BIOMD0000000011 1e-10
BIOMD0000000013 1e-10
BIOMD0000000014 1e-12
BIOMD0000000016 1e-10
BIOMD0000000017 1e-10
BIOMD0000000018 1e-20
BIOMD0000000025 1e-25
BIOMD0000000028 1e-10
BIOMD0000000032 1e-10
BIOMD0000000033 1e-12
BIOMD0000000038 1e-10
BIOMD0000000040 1e-10
BIOMD0000000041 1e-12
BIOMD0000000046 1e-15
BIOMD0000000048 1e-14
BIOMD0000000049 1e-13
BIOMD0000000051 1e-10
BIOMD0000000061 1e-10
BIOMD0000000064 1e-15
BIOMD0000000069 1e-12
BIOMD0000000070 1e-10
BIOMD0000000071 1e-10
BIOMD0000000072 1e-12
BIOMD0000000075 1e-10
BIOMD0000000080 1e-10
BIOMD0000000081 1e-10
BIOMD0000000082 1e-13
BIOMD0000000085 1e-14
BIOMD0000000086 1e-15
BIOMD0000000087 1e-11
BIOMD0000000088 1e-15
BIOMD0000000091 1e-10
BIOMD0000000093 1e-11
BIOMD0000000094 1e-12
BIOMD0000000103 1e-14
BIOMD0000000104 1e-10
BIOMD0000000105 1e-11
BIOMD0000000106 1e-15
BIOMD0000000108 1e-10
BIOMD0000000109 1e-9
BIOMD0000000111 1e-10
BIOMD0000000112 1e-10
BIOMD0000000122 1e-10
BIOMD0000000123 1e-10
BIOMD0000000125 1e-10
BIOMD0000000126 1e-12
BIOMD0000000137 1e-10
BIOMD0000000139 1e-12
BIOMD0000000140 1e-13
BIOMD0000000143 1e-10
BIOMD0000000146 1e-09
BIOMD0000000148 1e-10
BIOMD0000000150 1e-10
BIOMD0000000151 1e-12

 

   
BioUML forum
We have launched new forum (phpBB) to provide on-line users support and answer on questions of developers. Please use your BioStore account to login into the forum.
 
Version 0.9.4
BioUML workbench version 0.9.4 (beta) was released on March 20, 2013. More...
 
SBML tests passed 100%
BioUML version 0.9.4 has passed all tests from SBML test suite version 2.1.0. Test details.
 


Copyright © 2001-2013 Institute of Systems Biology. All rights reserved.