Running Tests
Marionette-based tests can be run in-tree with a locally built Firefox through mach marionette-test, as well as with a downloaded Firefox binary against in- and out-of-tree tests with runtests.py. The mach command calls runtests.py on the binary built from your mozilla-central checkout. All the same options should be available in both cases. (Aside: Are you looking to run unit tests for the Marionette Test Runner? That's described over in Developer setup under "Marionette Harness Unit Tests")
Running in-tree tests, mach will automatically manage the Python virtual environment in which your tests are run. The Marionette client that is picked up is the one that is in-tree at testing/marionette/client.
If you want to run tests from a downloaded tests.zip file, you will first need to set up the Python Marionette client. Second, you will need to download the tests from the tests.zip package found attached to Treeherder jobs for your system. If you are using a tests.zip package, extract it, and find the tests under marionette/tests/testing/marionette/client/marionette/tests.
You can launch the tests using one of the methods below. If desired, you can replace the path to unit-tests.ini with the path to a particular test. See the Marionette Python TestRunner for more options.
For Desktop Firefox
With a local build
Assuming you've run ./mach build successfully, you can use mach (German for “do”) to run all the tests:
% ./mach marionette-test
Individual tests can be run by specifying the file name:
% ./mach marionette-test testing/marionette/harness/marionette/tests/unit/test_findelement.py
You can see what extra options are provided using the -h flag.
If you are developing the Marionette server component, it is often useful to redirect Gecko’s standard output to your shell’s, as such:
% ./mach marionette-test TEST --gecko-log -
Similarily the --jsdebugger flag is occasionally useful in order to attach a Developer Tools Browser Debugger to the Marionette session.
With a downloaded desktop Firefox build
In case you want to run the tests with eg. a Nightly build of Firefox, mach let you do this with the --binary option:
% ./mach marionette-test TEST --binary=/path/to/firefox-executable
With a downloaded desktop Firefox build and tests.zip
Activate the virtualenv that you installed Marionette in (if you used one), then:
cd /path/to/extracted/tests.zip/marionette/marionette python runtests.py --binary=/path/to/firefox-executable ../tests/testing/marionette/client/marionette/tests/unit-tests.ini
For Firefox for Android (Fennec)
With a local build
Assuming you've run ./mach build, ./mach package and ./mach install successfully (see Simple Firefox for Android build; you may need to add ENABLE_MARIONETTE=1 to your mozconfig), you can use mach (German for “do”) to run all the tests. As a result of setting up a Marionette-enabled Fennec build, you will have Android SDK set up, likely somewhere under ~/.mozbuild -- this will be useful shortly.
The Marionette Test Runner can take care of launching an Android emulator and a Marionette-enabled Fennec for you, but it needs to know where to find an emulator and an AVD, which AVD to use, etc. You can use mach commands to set up these pieces.
# This command will offer to download and set up an AVD, and use it to launch an emulator # The AVD is likely set up at ~/.mozbuild/android-device/avd ./mach android-emulator # Install Fennec on the emulator ./mach install # Run Fennec at least once to set up a profile ./mach run
To launch an emulator and launch Fennec in it and run tests:
./mach marionette-test --emulator --app='fennec' --avd-home /path/to/.mozbuild/android-device/avd --emulator-binary /path/to/.mozbuild/android-sdk-macosx/tools/emulator
If you set an AVD_HOME environment variable and if emulator command is in your PATH, you may omit the --avd-home and --emulator-binary arguments. See ./mach marionette-test -h for additional options that allow to you specify x86 vs arm and so on.
To connect to an already-running emulator and launch Fennec:
./mach marionette-test --emulator --app='fennec' --address=localhost:2828
To connect to an already-running Fennec instance either in an emulator or on a device, you will need to enable Marionette manually. You can do this by building with ENABLE_MARIONETTE=1 and adding a boolean preference named marionette.defaultPrefs.enabled with a value of true in your profile. Once this preference has been set you will need to restart Fennec for it to take effect.
adb forward tcp:2828 tcp:2828 ./mach marionette-test --address=localhost:2828