在 Firefox OS 中运行测试程序: 开发者指南

入门

如果您是一位 Gecko 开发者，则需要对您已经熟悉的有关 Firefox OS 特定的自动化测试文档进行回顾：  mochitest, reftest, 和 xpcshell.

如果您是一位 Gaia App 开发者，或者是一位 Gecko 开发者，但是对更深层次的终端用户样式测试感兴趣，则需要了解下 Gaia 测试套件。 一共有两个主测试套件：

• Gaia UI 测试:  这些测试以基于 Python 的， 可以运行在桌面和设备上。这些测试会兼顾到 Gaia UI 的各个方面，例如耐久性测试（在 Contacts 应用中添加许多联系人， 重复多次执行锁屏界面的解锁动作， 在 Alarm 应用中设置许多闹钟等），功能性测试（例如在 FM 收音机中收台， 使用 Camera app 来拍照）以及可访问性测试（例如，使能和禁止屏幕阅读器）。
• Gaia 集成测试: 这些测试是基于 JavaScript， 使用 marionette-js-runner，现在仅支持 B2G 桌面版本（对实际设备的支持也即将实现）。由于 Gaia 与其他 app 和服务进行交互的内容不同， 这些测试也会有区别。例如，Calednar app 是否可以成功的添加一个 CalDAV 服务器， 或者 Browser app 是否成功处理与搜索引擎的交互。

可以依据您喜爱的工具链，以及测试的内容来决定究竟要选择何种测试。

让我们现在来查看和运行这些测试。

运行 Gaia UI 测试

The Gaia UI Test suite can be run on real devices and B2G Desktop builds, but in this section we'll concentrate on running them on a real device, as real devices are always best where possible.

Note that this test is destructive and as such, you should back up anything you care about on the phone before running these tests. Depending on which tests you run, they can also make phone calls. So be aware that you want to be very careful about what you run and how you back up the phone, remove the SIM card, etc.  That said, if you've already created an engineering build they are really easy to get running. Here's how.

One Time Set up

You only need to perform the following steps once, assuming you do not change the location of your Gaia directory. Create a python virtualenv (install the virtualenv tool first if you haven't already), activate it, and install the gaia UI test tool into your virtualenv. By creating the virtual environment using the steps below, you ensure that you are running the gaia UI test harness code that lives in your Gaia repo (that's useful in case you need to debug anything).

$ virtualenv gaia_ui_venv # This will create a gaia_ui_venv directory where the virtual environment lives. It can be anywhere on your system.
$ source gaia_ui_venv/bin/activate # This activates our virtualenv
(gaia_ui_venv)$ cd <b2groot>/gaia/tests/python/gaia-ui-tests;python setup.py develop # This installs the gaia ui harness into your virtual environment. 

If you have already created a virtual environment for gaia ui tests, you can simply do the following:

$ source gaia_ui_venv/bin/activate

To Run the Tests

First you need to create the testvars file. To do this, copy the standard one over, and add in the attributes to turn off the warnings that this test will destroy all content on your phone. These are good tests, they leave no state around and as such, you will need to make sure your phone is backed up before running them. Instructions from here on will assume you've activated the virtual environment and are working in the gaia/tests/python/gaia-ui-tests directory.

(gaia_ui_venv)$ cp gaiatest/testvars_template.json testvars.json
# Now edit your copy of testvars.json and add in the following attributes into the json:
"acknowledged_risks": true,
"skip_warning": true,

Now you just need to connect our phone via USB, forward the marionette port so your test runner can access it and run our tests. The tests are in gaiatest/tests and you can pick whichever one you want to run. For example, if you wanted to run contacts tests you would do the following:

(gaia_ui_venv)$ adb forward tcp:2828 tcp:2828
(gaia_ui_venv)$ gaiatest --testvars=testvars.json --address=localhost:2828 gaiatest/tests/functional/contacts/

To get out of the python virtualenv, just use the special virtualenv command deactivate:

(gaia_ui_venv)$ deactivate
$

运行 Gaia 集成测试

To run the Gaia Integration tests tests you currently have to use a B2G Desktop build (note that these are also going to be available for devices soon as well). Let's look at how this is done.

These just require a Gaia tree and NodeJS to be installed on your computer; the following command will do the rest:

$ cd gaia $ make test-integration 

That's it — this instruction will download a B2G desktop build, and start running the tests in that build.

总结

As always, work is underway to make all our tests easier to run both locally for developers as well as in our automation systems. Feel free to drop into the #ateam channel any time you have questions about test automation for Firefox OS or any of the Mozilla automation tools.