Introduction to Robocop
- Robocop is a java based testing framework. Robocop is an extention of the Android Robotium test tool with features needed to perform automation on Firefox for Android builds.
- Before starting to wright tests you need to Setup the Auromation Enviroment.
- Here are some more resources where you can read about how to set up the enviroments, run and create automated tests using Robocop:
- If you have a Pandaboard that you need to setup, check out the Pandaboard setup Wiki page. Here you will find information on how to install the Automation build on the board, some usefull adb commands to help you work with it and some info on the SUTAgent tool.
Limitations of Robocop and where it can be used
- All Robocop tests will be run on Tegra and Panda boards. In order to make sure the test runs correctly make sure it works on phones or tablets running Android 2.3 - this would ensure it runs on the Tegraboards (Android 2.3 and Gongerbread phones UI) - and on 7" Tablets running Android 4.0 - this would ensure it runs on the Pandaboards (Android 4.0.4 and ICS 7" Tablet UI)
- No real urls should be used because the time it takes to load them is not constant, the pages are not static, they aren't always available. Only use local webpages created for the purpuse of the test.
- Being based on the Robotium test API it can be used to test the UI elements of Firefox for Android
- Being a UI test tool it can be used to test Basic Functional Tests and is not the best tool for performance or endurance testing
- The Firefox for Android Reader Mode feature is not completely supported yet
- Robocop receives limited information about the Gecko layer so any Gecko content (Web Content) is seen as a picture. You can tap on the content, get the color of a pixel in the content, you can drag the content but you don't get any information about the scroll position, if elements on the page are displayed or have been loaded, if everything on the page is where it should be, if the content is correctly displayed or what type of content it is (text, picture, empty space, video, flash)
- It works on some devices but not all. This is also caused by the Android fragmentation, the different Android OS versions and build
- There are a lot of issues with event timings. The tests will always have to wait for events to happen before doing other actions
Getting started with Robocop tests
- Please make sure to look over this page for some general information about Robocop tests.
- The existing test cases can be found in the sorce directory under the folder mobile/android/base/tests
- The class that will cover the text will have to be named the same as the test file
- The test file name should be test[Feature_to_be_tested].java.in
- The test must be included in the same package as all other Robocop tests
Start wrighting a new test
- Download sources. More info can be found here about how to do it.
- If you already have the sources downloaded update to the lates version by running the command in the sources directory:
hg pull -u
- Make the build by running in superuser (sudo su) and make sure the PATH variable is updated with the Android SDK location:
make -f client.mk
- Create a new file named "test" followed by the test name and with the extention "java.in" : for e.g. testBookmarks.java.in
- Copy the basic structure of the test presented in the next section to the test making sure to match the test name and the class and constructure names.
- Create a new mercurial queue patch on the repository by running in the source folder the command:
hg qnew <patch_name>
- Add the new test file to the queue by running the command:
hg add mobile/android/base/tests/<test_file_complete_name>
- Open robocop.ini from mobile/android/base/tests and add after the last Robocop test a new line with the test name between sqare brackets ( "[" and "]")
- Write the test, compile robocop (here is how) and run the test in order to test that it works
- After the test works follow the instruction to create a patch so the test can be uploaded to a bug and integrated in mozilla.
- Before updating the sources or after the patch is completed make sure to return the sources to the unmodified version by running the command in the source folder until there are no patches in the queue:
hg qpop
Basic structure of a test
#filter substitution package @[email protected]; import @ANDROID_PACKAGE_NAME@.*; /* Insert info about the test here - a basic description of what the test does */ public class test[Feature_to_be_tested] extends BaseTest { @Override // This method is needed in the BaseTest class protected int getTestType() { return TEST_MOCHITEST; // Test type should remain Mochitest } /* * This is the class constructor * This method will contatin the test that will be ran */ public void test[Feature_to_be_tested]() { } }
- This is a basic structure of a Robocop test
- Each test class must have three methods:
- protected void setUp() // Starts Fennec and sets up commonly used member variables. This is usually very similar for every test class
- public void test[YourFeature]() // Your test code goes here. Use the Robocop API to access Fennec elements, click, send keys, and assert conditions
- public void tearDown() // Clean up
- If your tests extends BaseTest then you will not need to add the methods setUp() and tearDown() as they are already defined in BaseTest
- If you need to do extra clean-up, like removing entries you added in a database for e.g., the tearDown() method can be overwrittern but make sure to call super.tearDown() as needed in the method. The same is applicable for setUp().
APIs
Robotium itself provides a rich API through the Solo class - javadocs for Solo are available at [1].
"Robocop" provides an additional API to make common tasks easier. The main interfaces are Actions, Elements, and Driver.
Actions provides commonly used non-element specific actions that can be taken on the application, such as dragging and sending key events.
Actions //This will cause this process to spin until the gecko fires a specific JSON event, such as DOMContentLoaded void waitForGeckoEvent(String geckoEvent); //Clicks the given Key (Actions.SpecialKey.[DOWN|UP|LEFT|RIGHT|ENTER]) void sendSpecialKey(SpecialKey button) //Sends a string of characters to the system. (most have been implemented but not all) void sendKeys(String keysToSend); //Sends a drag action across the screen void drag(int startingX, int endingX, int startingY, int endingY) // Run a sql query on the specified database public Cursor querySql(String dbPath, String sql);
Element represents each of the available UI objects in Fennec including the Awesomebar, the 'tabs' button, and different lists and menus.
Element //To click the element. void click() //Returns true if the element is currently displayed boolean isDisplayed(); //Returns the text currently displayed on the element, or direct sub-elements. String getText();
Driver finds elements and provides info about the UI.
Driver //This is used to find elements given their id's name. Element findElement(String name); //This is used for getting information on scrolls. NOTE: It must be used for the next three methods to return useful information void setupScrollHandling(); int getPageHeight(); //The total height of the page. int getScrollHeight(); //How far down the screen the client has scrolled. int getHeight(); //The height of the client's view. //The following are used to give information on the graphical location of the Gecko view on the screen. int getGeckoTop(); int getGeckoLeft(); int getGeckoHeight(); int getGeckoWidth();
Finally, an evolving set of test base classes - BaseTest, PixelTest, etc - can be leveraged for some types of tests.
Usable IDs
The following is a list of ids that can be used with Driver.findElement(). Most of the following ids have not been tested, so might have unexpeced results, or require increased APIs for them. To know how a given object is used, in mobile/android/base, grep R.id.[id-name] * (from Objdir/mobile/android/base/R.java#id)
abouthome_content add_tab addons address_bar agent_mode all_pages_list awesome_bar awesome_screen awesomebar_button awesomebar_tabs awesomebar_text background bookmark bookmark_icon bookmark_title bookmark_url bookmarks_list browser_toolbar close container doorhanger_choices doorhanger_container doorhanger_title favicon forward gecko_layout grid history_list info list main_layout notification_image notification_progressbar notification_text notification_title outline plugin_container preferences quit recommended_addon_list reload save_as_pdf screenshot select_list share site_security stop tabs tabs_count title url
Basic code sequences to do different actions
- Most of the code is already covered by existing methods but this code bits will help you understand how Robop works. Besides looking over these you should also check out the existing tests
Load Page
- Note that the tests need to use local pages in order to run on the mozilla setup. Outside pages are not accessible
- Using an URL
String url = "<insert_link_here>"; loadUrl(url);
- Using the robocop tests blank page
String url = getAbsoluteUrl("/robocop/robocop_blank_01.html"); loadUrl(url);
Check Page Title
- There has been a method implemented for this in BaseTest.java.in called verifyPageTitle(String title) but the core code to verify the title is:
Element awesomebar = mDriver.findElement(getActivity(), "awesome_bar_title"); // search the awesomebar title mAsserter.isnot(awesomebar, null, "Got the awesomebar"); // log "Got the awesomebar" in the logs assertMatches(awesomebar.getText(), "<insert_title_here", "page title match"); // do a match between expected and actual title
Open app menu
- There has been a method implemented for this in BaseTest.java.in called selectMenuItem(String item) but the core code for opening the menu is:
mActions.sendSpecialKey(Actions.SpecialKey.MENU); //Open the More menu for devices with old style menues if (mSolo.waitForText("^More$")) { mSolo.clickOnText("^More$"); }
Simulate a Back key press:
mActions.sendSpecialKey(Actions.SpecialKey.BACK);
Setup a listener to catch a new page load in a new tab
- This is included in the addTab(String url) method from BaseTest.java.in
Actions.EventExpecter tabEventExpecter = mActions.expectGeckoEvent("Tab:Added"); // Look for a new tab event Actions.EventExpecter contentEventExpecter = mActions.expectGeckoEvent("DOMContentLoaded"); // Look for page load complete
Wait for the tab and page load
tabEventExpecter.blockForEvent(); contentEventExpecter.blockForEvent();
Wait for Geko to complete loading
- Include this code at the start of each test to wait for Firefox to load completely before starting to do any other actions
blockForGeckoReady();
Open the Bookmarks Tab
private ListView openBookmarksList() { Activity awesomeBarActivity = clickOnAwesomeBar(); // Click the "Bookmarks" tab to switch to bookmarks list mSolo.clickOnText("Bookmarks"); TabHost tabHost = (TabHost)mSolo.getView(TabHost.class, 0); return (ListView)tabHost.getCurrentView(); }
Open the bookmark context menu
View child = list.getChildAt(<nr>); //bookmark nr- numbering starts at 0 mSolo.clickLongOnView(child); //long tap to open context menu
- Similarly you can open the context menu for History and Top Sites items
Open a new tab and check the tab count
expectedTabCount = <nr_of_expected_tabs>; tabCountText = tabCount.getText(); tabCountInt = Integer.parseInt(tabCountText); mAsserter.is(tabCountInt, expectedTabCount, "Number of tabs increased");
Loging info in the Android Logs
- The Assert class can be found in the source folder in the subfolder /obj-android/build/mobile/robocop
- Please see the java code for the Assert methods for more info about how they work
- Each Assert will actually determin if a test fails or passes
Test that an object is not the same as a second object
- Method signature:
public void isnot(Object a, Object b, String name)
- Example for checking that there is a title set for the current page:
Element awesomebar = mDriver.findElement(getActivity(), "awesome_bar_title"); mAsserter.isnot(awesomebar, null, "Got the awesomebar"); // Checks that the awesomebar title is not null and posts in the logs
Test that an object is the same as a second object
- Method signature:
public void is(Object a, Object b, String name)
- Example for checking the default bookmarks:
ListView list = getAllPagesList(url); mSolo.waitForText(url); mAsserter.is(list.getChildCount(), 5, "all pages list has 5 children (the default bookmarks)"); // Checks that there are 5 entries in the list and posts in the logs
Write text in the Logs
- Method signature:
public void dumpLog(String message) // writes text in the logs
Log info about test progression
- Method signature:
public void ok(boolean condition, String name, String diag)
- Example
ListView list = getAllPagesList(url); mSolo.waitForText(url); mAsserter.ok(list != null, "checking that all pages list exists", list.toString()); // tests if the list is null and if it isn't it prints the list after printing the message of what the test does
Updating your test directory
t is not always necessary to rebuild Fennec to use new tests. If you are updating the build with your own test, ensure that your test has been added to the manifest in /PATH/TO/FENNEC/SOURCE/mobile/android/base/tests/robocop.ini .
Otherwise, you can retrieve tests from the repository by executing:
hg pull hg update
Now you can build the robocop part of Fennec by executing:
cd objdir sudo make -C build/mobile/robocop/ sudo make package
Testing and submitting the test as a patch
- Make sure your code follows the basic coding style guidelines. In this article there are a few guidelines that should be followed.
- Make sure you run the test at least 5 times to make sure there are no intermitent fails caused by timings
- If you have access to the tryserver(Commit access level 1) run the patch on the tryserver and it passes the tests
- Once you are sure the test works you can create a patch and add it to a bug. If there isn't a bug already to cover the creation of the test create one
- If you need help creating a patch please read the documentation covering the this from this wiki page