We learnt in Part 2 that we can easily control Firefox OS using Marionette client commands, but typing them into a Python console is as slow and tedious. The key advantage of test automation is that it can run autonomously. We will learn how to do that in this part, as we put all of our code commands into a Python file that can then be run all in one go.
Test case recap
In Part 2 we went through the steps to run a typical test case — opening up the Contacts app and adding a new contact:
- Unlock Firefox OS (optional; in Part 2 we turned off the lock screen manually, therefore we won't include this in the code below.)
- Switch to Contacts app
- Tap add new contact icon
- Type in the contact’s name
- Tap done
- Wait and check that the contact is present
Putting our test into a Python file
If we put all of these steps into a Python file, we can re-use it and run it much more quickly. Create a new text file called test_add_contact.py in a convenient directory of your choosing.
Into this file, enter the commands we saw in Part 2, as listed below. We'll use a Python class structure, as it is good practice, and forms a good base to build on in future steps of the tutorial.
import time
from marionette import Marionette
class TestContacts:
def __init__(self):
self.test_add_contacts()
def test_add_contacts(self):
# Create the client for this session. Assuming you're using the default port on a Marionette instance running locally
self.marionette = Marionette()
self.marionette.start_session()
# Switch context to the homescreen iframe and tap on the contacts icon
time.sleep(2)
home_frame = self.marionette.find_element('css selector', 'div.homescreen iframe')
self.marionette.switch_to_frame(home_frame)
contacts_icon = self.marionette.find_element('xpath', "//div[@class='icon']//span[contains(text(),'Contacts')]")
contacts_icon.tap()
# Switch context back to the base frame
self.marionette.switch_to_frame()
time.sleep(2)
# Switch context to the contacts app
contacts_frame = self.marionette.find_element('css selector', "iframe[data-url*='contacts']")
self.marionette.switch_to_frame(contacts_frame)
# Tap [+] to add a new Contact
self.marionette.find_element('id', 'add-contact-button').tap()
time.sleep(2)
# Type name into the fields
self.marionette.find_element('id', 'givenName').send_keys('John')
self.marionette.find_element('id', 'familyName').send_keys('Doe')
# Tap done
self.marionette.find_element('id', 'save-button').tap()
time.sleep(2)
# Close the Marionette session now that the test is finished
self.marionette.delete_session()
if __name__ == '__main__':
TestContacts()
Note: one additional thing you'll notice in the code that we didn't cover in Part 2 is the Python time.sleep() function — this makes the script pause for a certain length of time (defined in seconds) before continuing onto the next line. We have added these lines into the automated test because we need to simulate the user manually tapping buttons, etc. and waiting for Firefox OS to complete the resulting actions. If we ran this script without any delays, Python would complete everything instantaneously, probably causing the test to fail, as Firefox OS wouldn't be able to keep up.
Now you can run the test by navigating to the directory the test is saved in in your terminal and running the following command:
python test_add_contact.py
Note: Be aware of Python’s indentation rules. After copying and pasting you may need to indent everything correctly for the code to run. If you get an error related to this, make sure that all indentation levels are separated by a tab.
Note: You'll also notice that the name inserted using the code above is "John Doe", different to the "Foo Bar" name in Part 2. We've done this so that the code will run successfully and add another contact. If you try to add a contact with the same name, Firefox OS will warn you about duplicate contacts. For the moment, the best way to repeat run the test is to go into the Firefox OS interface and manually delete the contact before each run.
Adding an assertion
One thing we're still missing from our test, which is important to automated tests, is an assertion — a report or measure of whether Firefox OS has reached the state we want it to reach; whether the test was successful. We’ll do this by adding some code to check whether the new contact is present in the app.
Just before the # Close the Marionette session... line, add in this code, making sure it is indented to the same level as the other lines in the class:
# Now let's find the contact item and get its text
contact_name = self.marionette.find_element('css selector', 'li.contact-item:not([data-group$="ice"]) p').text
assert contact_name == 'John Doe'
Delete the old contact and try running the test again, with the following:
python test_add_contact.py
If it all runs well then great, now we have a functional test!
Note: If the assertion fails, be sure that the previous 'Foo Bar' contact does not exist anymore. The CSS selector before the assert is actually picking up the first contact in the list (that can be seen by calling print "Contact name: %s" % contact_name before calling the assert).
Note: The assertion won't currently appear to do anything, but assertions are very important when we start to use test runners, as introduced in Part 5: Introducing a test runner. Test runners like unittest use assertions to check whether tests have completed successfully or not, and then return the results of these tests (OK or FAIL.)
A note on timing
One of the most difficult things to deal with when writing an automated test is the timing. If the test moves onto the next step before Firefox OS completes the last one, then we’re likely to get a failure.
As mentioned above, In the sample code we added time.sleep(x) commands to solve this problem. However, using time.sleep(x) is not a good practice. Using a hardcoded set time can cause your test to run too long or not long enough. The latter is the worst case; it will cause false negative test results — meaning a test that reports a failure when in fact the app is perfectly functional but behaved a bit slower thn the test was expecting.
In the next part, we’ll progress onto abstracting out certain parts of the test into separate Python functions, and replacing the sleep() functions with proper dynamic waits.