1 Description of Testcases for OMK
2 ================================
4 Each test resides in a separate directory. The directories can be
5 nested. There is no difference between the tests in different levels of
8 Structure of a Testcase
9 -----------------------
11 A directory contains the testcase if it contains at least one file
12 that matches runtest* pattern.
14 * Testcase Definition:
16 This is the description of special files in the testcase
17 directory. Besides these files, testcase directory contains all the
18 other files needed to run the testcase (e.g. sources, config.omk
21 - runtest*: is the executable that runs the test(s). It is possible
22 to have more files in a testcase directory which begins with
23 runtest. In that case all these files will be executed in
24 alphabetical order as a separate testcases. The goal is to be able
25 to write several tests on the same structure of files and directories.
27 - rules: Specifies the rules this testcase applies to. The syntax of
28 this files is describes in section Rules description.
32 Runtest is executed with OMK_RULES environment variable set to the
33 name of the actual rules tested and OMK_TESTSROOT to the directory
34 containing the runtests.py script.
38 Each execution of testcase should produce the following outputs:
40 - Exit status of make or runtest:
41 0 - the testcase was successfully passed
43 2 - testcase cannot be executed (e.g. because the needed
44 compiler is not available)
48 In the case of error, the file '_error' should contain more
49 detailed description of the error.
51 - stdout and stderr: TODO
56 If the 'rules' file is contained in the testcase, this file defines
57 a set of Makefile.rules this testcase can be applied to. If there is
58 no rules file, this testcase is applied to all the rules.
60 The rules file contains one line of one of the following forms:
62 * <rules name> - This selects only one rules file
64 * all: - this selects all the rules files
66 * snip:<snippet name> - this selects all the rules, which contain the
69 * python:<python expression> - The python expression is evaluated. The
70 global namespace contains the following variables:
72 - rules: the name of tested rules
74 - snippets: is of type list and contains the snippet names from
75 which the tested rules are composed.
77 If the expression evaluates as True, the testcase is executed with
78 this rules, otherwise it is not.
80 Example: python: "config_h" in snippets and rules != "rtems"