Revision 185926 of How to add a build-time test

<p>
Writing a test is good, but it is even more helpful if that test gets executed. We have <a class="external" href="https://wiki.mozilla.org/MozillaQualityAssurance:Test_Suite_and_Tool_Inventory">various automation tools</a> that help make this possible. The first link, though, is the "<code>check</code>" target in a Makefile. If a test is added, that test or its directory needs to be mentioned in a Makefile.in file somewhere so that the build system knows about it.
</p>
<h4 name="standalone_executables">standalone executables</h4>
<p>To add a test that is written in C or C++ and which is called as a standalone executable, a few things must be done. For standalone executables, if one sets up the right variables, then the rules.mk file will do lots of magic and most of the heavy lifting. A simple example of adding a test is in https://lxr.mozilla.org/mozilla/source/netwerk/test/Makefile.in.
</p><p>Note that the following variables are supplied by the rules.mk file: <code>CPPSRCS</code>, <code>SIMPLE_PROGRAMS</code>, <code>RUN_TEST_PROGRAM</code>.
</p>
<h4 name="xpcshell_tests">xpcshell tests</h4>
<p>Here are the simple guidelines for adding an xpcshell test to the build system.
</p>
<ol><li> For now, enclose your test-related code (in the makefile) with <code>ifdef ENABLE_TESTS</code>
</li><li> Run the test program from the "check" target in the makefile. ({{wiki.template('Named-source', [ "netwerk/test/Makefile.in", "example" ])}} to run the TestCookie program)
</li><li> In the test program:
<ul><li> If the test fails, exit with a non-zero status and/or print the string "FAIL" to stdout
</li><li> If the test passes, exit with a zero status and don't print the string "FAIL" (bonus points for printing "PASS" :) )
</li></ul>
</li></ol>
<p>Write the test so that you expect it to pass on all platforms, since if the test fails, the tree will go orange (once we've set this up - see {{template.Bug(352240)}} for status).
</p><p>For example, to add an xpcshell test to a module, do the following:
</p>
<ol>
<li>Copy {{template.Source("tools/test-harness/xpcshell-simple/example")}} to <code><i>yourmoduledir</i>/<i>tests_type</i></code>, where <i>tests_type</i> is something that describes your tests.  You can of course use <code>unit</code> or <code>test</code> if you're not feeling creative.</li>
<li>In <tt><i>yourmoduledir</i>/<i>tests_type</i>/Makefile.in</tt> change <code><a href="en/DEPTH">DEPTH</a></code>, <code><a href="en/MODULE">MODULE</a></code>, and <code><a href="en/XPCSHELL_TESTS">XPCSHELL_TESTS</a></code> appropriately:</li>
<ul>
<li><code>DEPTH</code> should be a relative path pointing to <tt>mozilla/</tt>, e.g. if you're in <tt>netwerk/test</tt>, set <code>DEPTH = ../..</code></li>
<li><code>MODULE</code> should be defined to <code>test_<i>yourmodule</i></code> or just <code><i>yourmodule</i></code> -- either will work fine.</li>
<li><code>XPCSHELL_TESTS</code> be a list of subdirectories of the current directory which contain xpcshell tests.  You can of course just use the one <i>tests_type</i> directory here, but if you want to subdivide your tests by functionality, separate directories is the way to go.</li>
</ul>
<li>Reference the test dir in a parent makefile (<tt><i>yourmoduledir</i>/Makefile.in</tt>):

<pre>ifdef ENABLE_TESTS
DIRS  += tests_type
endif</pre>
</li>
<li>(Optional, but recommended) Add the new makefile to <a href="en/Allmakefiles.sh">allmakefiles.sh</a> (TODO: need more details about this)</li>
<li>Reconfigure (e.g. with <tt>make -f client.mk configure</tt>)</li>
<li><i>make</i> / <i>make check</i> in <tt><i>yourmodule</i></tt></li>
</ol>