Description of Testcases for OMK
================================

Each test resides in a separate directory. The directories can be
nested. There is no difference between the tests in different levels of
directory structure.

Structure of a Testcase
-----------------------

A directory contains the testcase if it contains at least one file
that matches runtest* pattern.

* Testcase Definition:

  This is the description of special files in the testcase
  directory. Besides these files, testcase directory contains all the
  other files needed to run the testcase (e.g. sources, config.omk
  etc.)

  - runtest*: is the executable that runs the test(s). It is possible
    to have more files in a testcase directory which begins with
    runtest. In that case all these files will be executed in
    alphabetical order as a separate testcases. The goal is to be able
    to write several tests on the same structure of files and directories.

  - rules: Specifies the rules this testcase applies to. The syntax of
    this files is describes in section Rules description.

* Testcase Execution:

  Runtest is executed with OMK_RULES environment variable set to the
  name of the actual rules tested and OMK_TESTSROOT to the directory
  containing the runtests.py script.

* Testcase Output:

  Each execution of testcase should produce the following outputs:

  - Exit status of make or runtest: 
        0 - the testcase was successfully passed
        1 - testcase failed
        2 - testcase cannot be executed (e.g. because the needed
            compiler is not available)


  - Error message:
    In the case of error, the file '_error' should contain more
    detailed description of the error.

  - stdout and stderr: TODO

Rules Description
-----------------

If the 'rules' file is contained in the testcase, this file defines
a set of Makefile.rules this testcase can be applied to. If there is
no rules file, this testcase is applied to all the rules.

The rules file contains one line of one of the following forms:

* <rules name> - This selects only one rules file

* all: - this selects all the rules files

* snip:<snippet name> - this selects all the rules, which contain the
  given snippet.

* python:<python expression> - The python expression is evaluated. The
  global namespace contains the following variables:

    - rules: the name of tested rules

    - snippets: is of type list and contains the snippet names from
      which the tested rules are composed.

  If the expression evaluates as True, the testcase is executed with
  this rules, otherwise it is not.

  Example: python: "config_h" in snippets and rules != "rtems"