Please note, this is a STATIC archive of website developer.mozilla.org from 03 Nov 2016, cach3.com does not collect or store any user information, there is no "phishing" involved.

Firefox OS reftests

Les tests de référence (ou reftests), sont des tests simples qui comparent, deux rendus d'interface utilisateur, d'une application web séparément (Des éléments HTML, par exemple) pour vérifier si ils sont identiques ou non. Cet article vous fournit tout ce que vous devez savoir pour lancer la suite reftests de Mozilla sur B2G. Pour en savoir plus sur reftests, lisez Créer des tests unitaires basés sur reftest (Ressource en anglais).

Prérequis pour lancer reftests

Vous pouvez lancer reftests sur B2G. Il est principalement essayé sur l'émulateur, mais en théorie doit fonctionner tout aussi bien sur des appareils. Dans cet article $B2G_HOME fait référence au clone du dépôt B2G. Vous trouverez ci-dessous ce que vous aurez besoin de compiler ou paramétrer, avant de pouvoir essayer de lancer les reftests B2G.

Si vous obtenez ce stacktrace pendant que vous jouez les reftests:

ImportError: cannot import name expected

File "gecko-dev/objdir-b2g-ics-emulator/_tests/reftest/runreftestb2g.py", line 16, in <module>
    from b2g_desktop import run_desktop_reftest
File "Projects/gecko-dev/objdir-b2g-ics-emulator/_tests/reftest/b2g_desktop.py", line 14, in <module>
     from marionette import Marionette, expected
 
Ceci est un bogue connu sur https://bugzilla.mozilla.org/show_bug.cgi?id=1114474#c7

Build B2G for the target you're testing

Regular B2G build instructions apply — no need for any special build options.

The B2G config that you will be using (what you pass to ./config.sh) will probably be emulator as that is what reftests are typically run on. Before building B2G for the emulator config, pay special attention to the Emulator build issues section in the Firefox OS build prerequisites page.

As usual, make sure adb is in your path and that it successfully connects to your running emulator or device. Try adb devices to make sure it is working ok.

Running reftests

Having satisfied the above prerequisites, you can then run reftests using one of the following sets of terminal commands.

For the emulator

Use mach:

cd $B2G_HOME
​./mach reftest-remote

This will run all reftests. If you want to run a specific reftest, do the following:

cd $B2G_HOME
./mach reftest-remote relative/path/from/B2G/to/reftest.list

For a real device

Running reftests on a device is not officially supported yet. One problem is that most devices don't support the minimum required resolution (800x1000) for many of the tests. But if you want to try anyway, you should be able to do so using the following steps:

cd $B2G_HOME/objdir-gecko
make package-tests
source _virtualenv/bin/activate
cd dist/test-package-stage/reftest
python runreftestb2g.py --b2gpath $B2G_HOME --xre-path /path/to/dir/containing/desktop/xpcshell --ignore-window-size relative/path/from/B2G/objdir-gecko/to/reftest.list

Running crashtests

To run crashtests, do the following:

cd $B2G_HOME
​./mach crashtest-remote

This will run all crashtests. If you want to run a specific crashtest, do the following:

cd $B2G_HOME
./mach reftest-remote relative/path/from/B2G/to/crashtests.list

Running jsreftests

JSReftests are not officially supported yet. See bug 972870 for progress updates on adding jsreftests to mach.

Running reftest with non-default options

You can optionally add --total-chunks and --this-chunks arguments as you would with regular desktop reftests. Use:

mach help reftest-remote

for a full list of supported arguments.

Running reftest with a downloaded emulator

If you've built B2G for another config (like otoro) and want to run the tests on an emulator, you can do so without building an emulator yourself. Just download the latest trunk arm emulator and do the following:

pip install marionette-client
python runreftestb2g.py --emulator arm --b2gpath path/to/emulator --xre-path /path/to/dir/containing/desktop/xpcshell --ignore-window-size --emulator-res 800x1000 <path_to_reftest_manifest>

Running reftest with a downloaded emulator and downloaded tests.zip package

If you want to run reftests against a downloaded emulator and a downloaded tests.zip package, you should extract them both.

Then run the following commands:

cd $TESTS_ZIP_DIR/mozbase
python setup_development.py
cd $TESTS_ZIP_DIR/marionette
python setup.py develop
cd $TESTS_ZIP_DIR/reftest
python runreftestb2g.py --emulator arm --b2gpath $EMULATOR_DIR --xre-path /path/to/dir/containing/desktop/xpcshell --ignore-window-size --emulator-res 800x1000 <path_to_reftest_manifest>

Note: For best results, you should perform the above steps using a Python virtualenv.

Troubleshooting

The following sections provide some answers to common problems encountered when trying to use reftests.

I can't build the emulator!

If you have trouble building B2G in the emulator config, check out the Emulator build issues section in the Firefox OS build prerequisites page.

Check that your B2G build/emulator works and that ADB can connect to it

First check that your B2G build runs normally. If this is an emulator build, then ./run-emulator.sh should work: it should start up the emulator and boot B2G inside it. If it doesn't, get the log from adb logcat and ask for help. If ADB fails to connect to the running emulator, make sure that you have no other (Android or B2G) devices connected, and try passing in the -e option to tell adb to focus on the emulator:

adb logcat -e

Once you have verified that your B2G build/emulator works normally and that ADB can connect to it, the next step is to verify that your xpcshell desktop executable runs normally. If you built B2G for a real device and ADB can't connect to it, check out Connecting a Firefox OS device to the desktop.

Note: None of the errors below should occur if you are using mach. If you have problems with mach, please file a bug under Testing/Reftest and CC :ahal

Check that you can run your desktop xpcshell executable

The python runreftestb2g.py command line above will try to run xpcshell on your desktop with the path specified by the --xre-path argument. Check that running xpcshell from the current directory with this relative path actually works.

For example, suppose that your command line contains:

--xre-path /hack/mozilla-central/objdir/dist/bin

Try running this from the same directory as you want to run python runreftestb2g.py from:

$ /hack/mozilla-central/objdir/dist/bin/xpcshell

This should land you in a special-looking shell like this:

js>

If this fails with an error message complaining about being unable to find or load certain shared libraries, like this

/hack/mozilla-central/objdir/dist/bin/xpcshell: error while loading shared libraries: libxul.so: cannot open shared object file: No such file or directory

then you need to set the right environment variable so that it looks for these libraries in the same directory. On Linux, you would set LD_LIBRARY_PATH to the same directory as you want to pass as --xre-path. For example, this should work:

LD_LIBRARY_PATH=/hack/mozilla-central/objdir/dist/bin /hack/mozilla-central/objdir/dist/bin/xpcshell

On Mac OSX, the environment variable to set is DYLD_LIBRARY_PATH.

Check that reftest can find httpd.js

If reftest still fails to run — returning early with an error — the next most likely cause of trouble is it failing to find certain files that it needs to load. The first file that it could fail to find is httpd.js. Typically, the reason why it would not find it is that you accidentally built xpcshell with --disable-tests. So, make sure that the path you pass to --xre-path does contain a httpd.js file under the components subdirectory.

This is good:

$ find /hack/mozilla-central/objdir/dist/bin/ -name httpd.js
/hack/mozilla-central/objdir/dist/bin/components/httpd.js
/hack/mozilla-central/objdir/dist/bin/modules/commonjs/sdk/test/httpd.js

This is bad (was caused by --disable-tests):

$ find /hack/mozilla-central/objdir/dist/bin/ -name httpd.js
/hack/mozilla-central/objdir/dist/bin/modules/commonjs/sdk/test/httpd.js

Check that you didn't forget to make package-tests

Forgetting the make package-tests step described above would certainly explain why nothing works.

Check that you passed a correct relative path to the reftest.list

The final argumant in the command line running reftest is the relative/path/from/B2G/objdir-gecko/to/reftest.list. This must be a relative path from the B2G/objdir-gecko directory to a reftest.list file under it. So check that a reftest.list file is actually present there! If it isn't, compare your command line to the sample command line below, and/or check that you didn't forget to make package-tests as explained above.

Sample command line:

$ LD_LIBRARY_PATH=/hack/mozilla-central/objdir/dist/bin python runreftestb2g.py --b2gpath /hack/b2g/B2G/ --xre-path /hack/mozilla-central/objdir/dist/bin  --ignore-window-size --emulator arm --emulator-res 800x1000 tests/layout/reftests/css-gradients/reftest.list

Here, we are running only the css-gradients directory of reftests.

Étiquettes et contributeurs liés au document

 Contributeurs à cette page : jwhitlock, sousmangoosta
 Dernière mise à jour par : jwhitlock,