There are two test suites, affectionately called "jstests" and "jit-tests". Sometimes jstests are called JS reftests because of how they are run in the browser. Both sets of tests can be used from a normal build of the JS shell.
Running jstests
Running jstests on a JS shell
The jstests shell harness is js/src/tests/jstests.py. Basic usage is:
jstests.py PATH_TO_JS_SHELLDevelopers will usually want to run something like this, where NPROC is the number of processors on your system:
jstests.py -d -j NPROC PATH_TO_JS_SHELLThe -d option skips tests that are marked as randomly failing; random failures are usually just noise when being run for day-to-day developer testing. The -j option runs the suite in parallel (the default is 2, so if you have a 2-core machine you don't need that option); on a fast 8-core machine, the entire suite runs in under a minute.
Another common use case is to run just one test, or all the tests in just one directory:
jstests.py PATH_TO_JS_SHELL TEST_PATH_SUBSTRING [ TEST_PATH_SUBSTRING_2 ... ]This runs all the tests whose paths contain TEST_PATH_SUBSTRING. Thus, you can run one test by giving its filename, or any substring that is a substring of that test filename only. You can run all tests in a directory by giving the directory path. You can run three specific tests by giving the three names.
Other options allow you to show the test command lines being run, command output and return codes, run tests named in a given file, exclude tests named in a given file, hide the progress bar, change the timeout, run skipped tests, print output in Tinderbox format, run a test in the debugger, or run tests in valgrind. Run with the -h option to see all the options.
Running JSTest in a browser
[TBA]
Running jstests on TinderBox
When viewing the TinderboxPushLog after a push to the Mozilla repositories, jstests are shown as R(J) meaning "Javascript Reftests".
Running jit-tests
The jit-test suite uses a Python harness that is similar to the jstests shell harness, but with some minor differences. Basic usage is the same:
jit_test.py PATH_TO_JS_SHELLDevelopers will usually want to run like this to skip the slow tests and cover the most important options:
jit_test.py --no-slow PATH_TO_JS_SHELL --jitflags=mjp,m,mj,mja You can select specific tests to run in the same way as the jstests shell harness. Most of the options in the jstests harness are also available for this one, but a few have different names, and -j is omitted. Use -h to see all the options.
The --jitflags option allows you to test the JS executable with different flags. The flags are used to control which features are run: mjp represents the same flags the browser uses when it's run. You can combine different sets of options with a comma, for example mjp,m,j,mj runs each test 4 times, once with each set of options.
One helpful option specific to jit-tests is -R filename (or --retest filename). The first time this is run, it runs the suite and writes a list of tests that failed to the given filename. The next time it is run, it will run only the tests in the given filename, and then will write the list of tests that failed when it is done. Thus, you can keep retesting as you fix bugs and only the tests that failed the last time will run, speeding things up a bit.
Running jit-tests on TinderBox
When viewing the TinderboxPushLog after a push to the Mozilla repositories, jit-tests are run by calling make check, and appear under B.
Creating new tests for jit-tests
Creating new tests for jit-tests is easy. Just add a new JS file to the tests/basic directory (or any appropriate directory under 'tests'). The test harness will automatically find the test and run it. The test is considered to pass if the exit code of the JS shell is zero (i.e., JS didn't crash and there were no JS errors). Use the assertEq function to verify values in your test.
There are some advanced options for tests. See the README (in js/src/jit-tests) for details.