SBML Semantic Test Suite

Currently BioUML workbench provides two alternative simulation engines:

• Java simulation engine
  workbench automatically generates and compiles Java code. For simulation we have adopted odeToJava library and explicit Runge Kutta scheme known as the Dormand-Prince routine is used. For solving algebraic equations Newton solver is used.
  
• MATLAB simulation engine
  workbench automatically generates code for MATLAB and invokes MATLAB engine to simulate model behavior. While code generator supports events, delays, rules and algebraic rules, then generated code is not trivial.

SBML semantic test suite provides a set of valid SBML models each with a simulated time course data. This test suite is designed by Andrew Finney to be used by developers to check that their simulators produce results that are consistent with the SBML standard and thus with each other. Overall aim of the semantic test suite is to facilitate the consistent interpretation of SBML models.

SBML semantic test suite was used to validate BioUML workbench simulation engines. Test results are summarized below

 Legend:

• Tests - total number of tests in the category.
• Failed - number of tests where some exception has occurred during the test.
• Successful - number of successful tests (see the test methodology for details).
• Errors - number of tests where simulation results significantly differ from the known ones.
• Needs tuning - relative error is not small enough.
• CSV error - original results (in CSV format) is missing or can not be parsed.
• Result differs - some variable or time point is missing in simulation engine output.
• Time - pure simulation  time for simulating all models behaviour in the tests category.

Testing methodology

.. to be described ..