+* FRSH/FORB FRAMEWORK
-WvTest: the dumbest cross-platform test framework that could possibly work
-==========================================================================
-
-I have a problem with your unit testing framework. Yes, you.
-The person reading this. No, don't look away guiltily. You
-know your unit testing framework sucks. You know it has a
-million features you don't understand. You know you hate it,
-and it hates you. Don't you?
-
-Okay, fine. Let's be honest. Actually, I don't know who you
-are or how you feel about your unit testing framework, but I've
-tried a lot of them, and I don't like any of them. WvTest is
-the first one I don't hate, at least sort of. That might be
-because I'm crazy and I only like things I design, or it might
-be because I'm crazy and therefore I'm the only one capable of
-designing a likable unit testing framework. Who am I to say?
-
-Here are the fundamental design goals of WvTest:
-
- - Be the stupidest thing that can possibly work. People are
- way, way too serious about their testing frameworks. Some
- people build testing frameworks as their *full time job*.
- This is ridiculous. A test framework, at its core, only does
- one thing: it runs a program that returns true or false. If
- it's false, you lose. If it's true, you win. Everything
- after that is gravy. And WvTest has only a minimal amount of
- gravy.
-
- - Be a protocol, not an API. If you don't like my API, you can
- write your own, and it can still be WvTest and it can still
- integrate with other WvTest tools. If you're stuck with
- JUnit or NUnit, you can just make your JUnit/NUnit test
- produce WvTest-compatible output if you want (although I've
- never done this, so you'll have to do it yourself). I'll
- describe the protocol below.
-
- - Work with multiple languages on multiple operating systems.
- I'm a programmer who programs on Linux, MacOS, and Windows,
- to name just three, and I write in lots of programming
- languages, including C, C++, C#, Python, Perl, and others.
- And worse, some of my projects use *multiple* languages and I
- want to have unit tests for *all* of them. I don't know of
- any unit testing framework - except maybe some horrendously
- overdesigned ones - that work with multiple languages at
- once. WvTest does.
-
- - NO UNNECESSARY OBJECT ORIENTATION. The big unit testing
- craze seems to have been started by JUnit in Java, which is
- object-oriented. Now, that's not a misdesign in JUnit; it's
- a misdesign in Java. You see, you can't *not* encapsulate
- absolutely everything in Java in a class, so it's perfectly
- normal for JUnit to require you to encapsulate everything in
- a class. That's not true of almost any other language
- (except C#), and yet *every* clone of JUnit in *every*
- language seems to have copied its classes and objects. Well,
- that's stupid. WvTest is designed around the simple idea of
- test *functions*. WvTest runs your function, it checks a
- bunch of stuff and it returns or else it dies horribly. If
- your function wants to instantiate some objects while it does
- that, then that's great; WvTest doesn't care. And yes, you
- can assert whether two variables are equal even if your
- function *isn't* in a particular class, just as God intended.
-
- - Don't make me name or describe my individual tests. How many
- times have you seen this?
-
- assertTrue(thing.works(), "thing didn't work!");
-
- The reasoning there is that if the test fails, we want to be
- able to print a user-friendly error message that describes
- why. Right? NO!! That is *awful*. That just *doubled* the
- amount of work you have to do in order to write a test.
- Instead, WvTest auto-generates output including the line
- number of the test and the code on that line. So you get a
- message like this:
-
- ! mytest.t.cc:431 thing.works() FAILED
-
- and all you have to write is this:
-
- WVPASS(thing.works());
-
- (WVPASS is all-caps because it's a macro in C++, but also
- because you want your tests to stand out. That's what
- you'll be looking for when it fails, after all. And don't
- even get me started about the 'True' in assertTrue. Come
- on, *obviously* you're going to assert that the condition is
- true!)
-
- - No setup() and teardown() functions or fixtures. "Ouch!" you
- say. "I'm going to have so much duplicated code!" No, only
- if you're an idiot. You know what setup() and teardown() are
- code names for? Constructor and destructor. Create some
- objects and give them constructors and destructors, and I
- think you'll find that, like magic, you've just invented
- "test fixtures." Nothing any test framework can possibly do
- will make that any easier. In fact, everything test
- frameworks *try* to do with test fixtures just makes it
- harder to write, read, and understand. Forget it.
-
- - Big long scary test functions. Some test frameworks are
- insistent about the rule that "every function should test
- only one thing." Nobody ever really explains why. I can't
- understand this; it just causes uncontrolled
- hormone-imbalance hypergrowth in your test files, and you
- have to type more stuff... and run test fixtures over and
- over.
-
- My personal theory for why people hate big long test
- functions: it's because their assertTrue() implementation
- doesn't say which test failed, so they'd like the *name of
- the function* to be the name of the failed test. Well,
- that's a cute workaround to a problem you shouldn't have had
- in the first place. With WvTest, WVPASS() actually tells you
- exactly what passed and what failed, so it's perfectly okay -
- and totally comprehensible - to have a sequence of five
- things in a row where only thing number five failed.
-
-
-The WvTest Protocol
--------------------
-
-WvTest is a protocol, not really an API. As it happens, the
-WvTest project includes several (currently four)
-implementations of APIs that produce data in the WvTest format,
-but it's super easy to add your own.
-
-The format is really simple too. It looks like this:
-
- Testing "my test function" in mytest.t.cc:
- ! mytest.t.cc:432 thing.works() ok
- This is just some crap that I printed while counting to 3.
- ! mytest.t.cc.433 3 < 4 FAILED
-
-There are only four kinds of lines in WvTest, and each of the
-lines above corresponds to one of them:
-
- - Test function header. A line that starts with the word
- Testing (no leading whitespace) and then has a test function
- name in double quotes, then "in", then the filename, and then
- colon, marks the beginning of a test function.
-
- - A passing assertion. Any line that starts with ! and ends with
- " ok" (whitespace, the word "ok", and a newline) indicates
- one assertion that passed. The first "word" on that line is
- the "name" of that assertion (which can be anything, as long
- as it doesn't contain any whitespace). Everything between the
- name and the ok is just some additional user-readable detail
- about the test that passed.
-
- - Random filler. If it doesn't start with an ! and it doesn't
- look like a header, then it's completely ignored by anything
- using WvTest. Your program can print all the debug output it
- wants, and WvTest won't care, except that you can retrieve it
- later in case you're wondering why a test failed. Naturally,
- random filler *before* an assertion is considered to be
- associated with that assertion; the assertion itself is the
- last part of a test.
-
- - A failing assertion. This is just like an 'ok' line, except
- the last word is something other than 'ok'. Generally we use
- FAILED here, but sometimes it's EXCEPTION, and it could be
- something else instead, if you invent a new and improved way
- to fail.
-
-
-Reading the WvTest Protocol: wvtestrun
---------------------------------------
-
-WvTest provides a simple perl script called wvtestrun, which
-runs a test program and parses its output. It works like this:
-
- cd python
- ../wvtestrun ./wvtest.py t/twvtest.py
-
-(Why can't we just pipe the output to wvtestrun, instead of
- having wvtestrun run the test program? Three reasons: first, a
- fancier version of wvtestrun could re-run the tests several
- times or give a GUI that lets you re-run the test when you push
- a button. Second, it handles stdout and stderr separately.
- And third, it can kill the test program if it gets stuck
- without producing test output for too long.)
-
-If we put the sample output from the previous section through
-wvtestrun (and changed the FAILED to ok), it would produce this:
-
- $ ./wvtestrun cat sample-ok
-
- Testing "all" in cat sample-ok:
- ! mytest.t.cc my ok test function: .. 0.010s ok
-
- WvTest: 2 tests, 0 failures, total time 0.010s.
-
- WvTest result code: 0
-
-What happened here? Well, wvtestrun took each test header (in
-this case, there's just one, which said we're testing "my test
-function" in mytest.t.cc) and turns it into a single test line.
-Then it prints a dot for each assertion in that test function,
-tells you the total time to run that function, and prints 'ok'
-if the entire test function failed.
+ FRSH/FORB is a contract-based resource reservation framework for
+ distributed real-time applications. In general, it provides timing
+ isolation between applications, i.e. multiple applications can use
+ the same resources such as CPU, networks, etc., without influencing
+ timing of the other applications. The main principle is that
+ application developers use FRSH API to specify their resource
+ requirements needed to achieve desired timeliness and the framework
+ uses schedulability analysis to check these requirements. If the
+ check is successful an application is granted a "virtual resource"
+ which allow the application to use the requested resource while
+ enforcing the application not to use more than requested.
-Note that the output of wvtestrun is *also* valid WvTest output.
-That means you can use wvtestrun in your 'make test' target in a
-subdirectory, and still use wvtestrun as the 'make test' runner
-in the parent directory as well. As long as your top-level
-'make test' runs in wvtestrun, all the WvTest output will be
-conveniently summarized into a *single* test output.
+ The development of the framework begun in FRESCOR project
+ (http://frescor.org) and now it is developed as a stand-alone
+ project on SourceForge (http://frsh-forb.sf.net).
-Now, what if the test had failed? Then it would look like this:
+* BUILDING FRSH/FORB FRAMEWORK
- $ ./wvtestrun cat sample-error
+1) Install prerequisites. On Debian/Ubuntu:
- Testing "all" in cat sample-error:
- ! mytest.t.cc my error test function: .
- ! mytest.t.cc:432 thing.works() ok
- This is just some crap that I printed while counting to 3.
- ! mytest.t.cc.433 3 < 4 FAILED
- 0.000s ok
+ apt-get install libidl-dev libcpufreq-dev libacpi-dev \
+ libcgroup-dev libncurses5-dev
- WvTest: 2 tests, 1 failure, total time 0.000s.
+ To build camera demo, you need to:
- WvTest result code: 0
+ apt-get install freeglut3-dev
-What happened there? Well, because there were failed tests,
-wvtestrun decided you'd probably want to see the detailed output
-for that test function, so it expanded it out for you. The line
-with the dots is still there, but since it doesn't have an 'ok',
-it's considered a failure too, just in case.
+2) Go to build directory and configure the build:
+
+ cd build/aquosa
+ make default-config
-Watch what happens if we run a test with both the passing, and
-then the failing, test functions:
+ If you are not satisfied with configuration found in
+ config.omk-default or config.target, you can override it in
+ config.omk.
- $ ./wvtestrun cat sample-ok sample-error
+ To be able to use CPU reservations AQuoSA
+ (http://aquosa.sourceforge.net) has to be installed. If you cannot
+ use AQuoSA, the framework can also be compiled without CPU support
+ (echo CONFIG_AQUOSA=n >> config.omk) or can use cgroups (echo
+ CONFIG_CPUCG=y >> config.omk). Note, that cgroups support is not
+ well tested, but we plan to work on it.
- Testing "all" in cat sample-ok sample-error:
- ! mytest.t.cc my ok test function: .. 0.000s ok
- ! mytest.t.cc my error test function: .
- ! mytest.t.cc:432 thing.works() ok
- This is just some crap that I printed while counting to 3.
- ! mytest.t.cc.433 3 < 4 FAILED
- 0.000s ok
+3) Compile it:
- WvTest: 4 tests, 1 failure, total time 0.000s.
+ make
- WvTest result code: 0
+4) Test it:
-Notice how the messages from sample-ok are condensed; only the
-details from sample-error are expanded out, because only that
-output is interesting.
+ make test
+* DIRECTORY STRUCTURE
-How do I actually write WvTest tests?
--------------------------------------
+ * build/* - configuration for different build targets
-Sample code is provided for these languages:
+ * build/aquosa - default build for linux
- C++: try typing "cd cpp; make test"
- C# (mono): try typing "cd dotnet; make test"
- Python: try typing "cd python; make test"
- Shell: try typing "cd sh; make test"
+ * build/marte - build for MarteOS. Not completely supported now.
-There's no point explaining the syntax here, because it's really
-simple. Just look inside the cpp, dotnet, python, and sh
-directories to learn how the tests are written.
+ * src - all sources
+ * src/forb - CORBA-like middle for interprocess and inter-node
+ communication.
-How should I embed WvTest into my own program?
-----------------------------------------------
+ * src/frsh - The core of resource reservation framework.
-The easiest way is to just copy the WvTest source files for your
-favourite language into your project. The WvTest protocol is
-unlikely to ever change - at least not in a
-backwards-incompatible way - so it's no big deal if you end up
-using an "old" version of WvTest in your program. It should
-still work with updated versions of wvtestrun (or wvtestrun-like
-programs).
+ * src/frsh-include - FRSH API headers from FRESCOR project. Our FRSH
+ framework implements this API.
-Another way is to put the WvTest project in a subdirectory of
-your project, for example, using 'svn:externals',
-'git submodule', or 'git subtree'.
+ * src/fosa - Operating system adaptation layer
+ * src/ulut - library providing generic data types and algorithms
+ (AVL trees etc.).
-How do I run just certain tests?
---------------------------------
+ * src/fna - Network adaptation layer = unified API for plugging in
+ different network protocols.
-Unfortunately, the command-line syntax for running just *some*
-of your tests varies depending which WvTest language you're
-using. For C++ or C#, you link an executable with wvtestmain.cc
-or wvtestmain.cs, respectively, and then you can provide strings
-on the command line. Test functions will run only if they have
-names that start with one of the provided strings:
+ * src/fwp - Communication protocol and resource management for WiFi
+ (also works with Ethernet).
- cd cpp/t
- ../../wvtestrun ./wvtest myfunc otherfunc
+* MISCELLANEOUS
-With python, since there's no linker, you have to just tell it
-which files to run:
+** Advantages of OMK make system
- cd python
- ../wvtestrun ./wvtest.py ...filenames...
+ * It is not easy to test FRSH/FORB simultaneously on multiple
+ platforms. With OMK, you can have the same sources compiled for
+ multiple platforms/targets at the same time because it uses out
+ of source directory compilation.
+ * Automatic handling of dependencies. When a file is changed,
+ only the files dependent on it are rebuilt. Developers don't
+ have to care about this.
+ * With OMK it is easy to combine multiple components/libraries (from
+ different developers) together and compile them with the same
+ configuration (e.g. PLATFORM variable). The structure of leaf
+ makefiles (Makefile for every component) is very simple and *well
+ specified*. Because of this it is easy to combine components from
+ multiple developers.
+ * OMK already supports compilation for user-space
+ programs/libraries, Linux and RTLinux modules, RTEMS and
+ several other platforms. Now we have also added support for
+ Marte. OMK works under MinGW and Cygwin, so it can be used to
+ compile for OSE. In Pisa, they already use OMK for Aquosa.
+** Further remarks
-What else can parse WvTest output?
-----------------------------------
+- If you are not interested in compilation of some component (e.g .
+ because it is currently in uncompilable state), you can simply
+ delete the link to it.
-It's easy to parse WvTest output however you like; for example,
-you could write a GUI program that does it. We had a tcl/tk
-program that did it once, but we threw it away since the
-command-line wvtestrun is better anyway.
+- It might be possible that you will need to change some configuration
+ value from config.target. You can override any variable declared
+ there in config.omk - just create it and put variable definitions to
+ it. For example, if you have MARTE installed in a different
+ directory that the one specified in config.target, you can put the
+ following in config.omk:
-One other program that can parse WvTest output is gitbuilder
-(http://github.com/apenwarr/gitbuilder/), an autobuilder tool
-for git. It reports a build failure automatically if there are
-any WvTest-style failed tests in the build output.
+ MARTE_PATH=/path/to/marte/
+- FRSH/FORB support for Marte OS is not maintained. However, if you
+ want to try it, it is not necessary to set PATH variable to GNAT
+ compiler since OMK uses full paths to call the compiler.
-Other Assorted Questions
-------------------------
+- Since we don't want to modify the Makefiles developed for Marte OS,
+ in order to compile only a part of the tree, you cannot simply run
+ make in the desired directory, but you have to specify -f flag with
+ the path to Makefile.rules. For this reason, we recommend using omk
+ script instead of make. The script can be obtained from
+ http://rtime.felk.cvut.cz/gitweb/omk.git/blob_plain/HEAD:/omk.
+** FAQ
-What does the "Wv" stand for?
+- How do I debug my Makefile.omk set-up ?
- Either "Worldvisions" or "Weaver", both of which were part of the
- name of the Nitix operating system before it was called Nitix, and
- *long* before it was later purchased by IBM and renamed to Lotus
- Foundations.
+ Just activate verbose compilation (V=1 or V=2):
- It does *not* stand for World Vision (sigh) or West Virginia.
+ make V=1
-Who owns the copyright?
+ or
- While I (Avery) wrote most of the WvTest framework in C++, C#, and
- Python, and I also wrote wvtestrunner, the actual code I wrote is
- owned by whichever company I wrote it for at the time. For the most
- part, this means:
+ make -f /path/to/Makefile.rules binary-pass V=1
- C++: Net Integration Technologies, Inc. (now part of IBM)
- C#: Versabanq Innovations Inc.
- Python: EQL Data Inc.
+- How do I get back syntax highlighting in Emacs while editing .omk files ?
-What can I do with it?
+ Just add these lines to your $(HOME)/.emacs:
- WvTest is distributed under the terms of the GNU LGPLv2. See the
- file LICENSE for more information.
-
- Basically this means you can use it for whatever you want, but if
- you change it, you probably need to give your changes back to the
- world. If you *use* it in your program (which is presumably a test
- program) you do *not* have to give out your program, only
- WvTest itself. But read the LICENSE in detail to be sure.
-
-Where did you get the awesome idea to use a protocol instead of an API?
-
- The perl source code (not to be confused with perlunit)
- did a similar trick for the perl interpreter's unit
- test, although in a less general way. Naturally, you
- shouldn't blame them for how I mangled their ideas, but
- I never would have thought of it if it weren't for them.
-
-Who should I complain to about WvTest?
-
- Email me at: Avery Pennarun <apenwarr@gmail.com>
-
- I will be happy to read your complaints, because I actually really
- like it when people use my programs, especially if they hate them.
- It fills the loneliness somehow and prevents me from writing bad
- poetry like this:
-
- Testing makes me gouge out my eyes
- But with WvTest, it takes fewer tries.
- WvTest is great, wvtest is fun!
- Don't forget to call wvtestrun.
+ (setq auto-mode-alist
+ (append '(("\.omk$" . makefile-mode))
+ auto-mode-alist))
projects / frescor / frsh-forb.git / blobdiff