Mozmill commit policy

The following details our policy for committing patches to the mozmill-tests and litmus-data repositories. If you have any questions or need help email Anthony Hughes or ping ashughes on irc.mozilla.org/#automation.

What is a committer?

A committer is someone who physically checks code into the repository. This is a privilege which comes with a lot of responsibility and is typically only granted to those who have proven themselves over an extended period of contribution.

How do I become a committer?

When the team feels you are ready to become a committer, you will need to go through the application process. Mozilla has a standard Committer Agreement with everyone who wishes to check code into any repository. If you want to be able to land patches to the mozmill-tests and litmus-data repositories, get in touch with Anthony Hughes.

Privileges

Once granted commit access you will be granted the following privileges:

• Landing patches to skip/disable a failing test (patch must be approved by one peer)
• Landing patches to fix a failing test (patch must be approved by one peer and one super-reviewer)
• Landing patches to add a new test (patch must be approved by one peer and one super-reviewer)
• Landing patches to litmus-data (patch must be approved by one peer and one super-reviewer)
• Landing patches to endurance tests (patch must be approved by one peer, one super-reviewer, and the module owner)
• Landing patches to shared modules (patch must be approved by one peer, one super-reviewer, and the module owner)

Responsibilities

Once granted commit access you will be granted the following responsibilities:

• Landing your own patches after they have passed review (ideally within 24 hours of r+)
• Checking the live test results the following day to see if your patch introduced a regression
• Promptly backing out your patch when a regression is introduced
• Landing patches for those without commit access after they have passed review

Life-cycle of a patch

Patches for Endurance Tests

• Land the patch on the default branch
• Update the bug with the following changes
  • [landed] added to the patch summary
  • comment with the URL to the changeset
  • status set to RESOLVED FIXED
• The next day, verify the test has not caused regressions by checking the Endurance Dashboard for failures
  • if no failures are found, set the bug status to VERIFIED FIXED
  • if the test fails, the bug status is set to REOPENED and the patch is backed out

Patches for Functional Tests

• Land the patch on the default branch
• Update the bug with the following changes
  • [landed:default] added to the patch summary
  • comment with the URL to the changeset
  • status set to RESOLVED FIXED
• The next day, verify the test has not caused regressions by checking the Functional Dashboard for failures
• If no failures are found, land the patch on the mozilla-aurora, mozilla-beta, and mozilla-release branches, and update the bug with the following changes:
  • change [landed:default] to [landed] in the patch summary
  • comment with the URLs to the changesets for each of the branches
  • change the status to VERIFIED FIXED
• The next day, verify the test has not caused regressions by checking the Functional Dashboard for failures
  • if the test fails, the bug status is set to REOPENED and the patch is backed out

Patches for Test Failures

• Land the patch on the affected branch
• Update the bug with the following changes
  • [landed] added to the patch summary
  • comment with the URL to the changeset
  • status set to RESOLVED FIXED
• The next day, verify the test has not caused regressions by checking the Functional Dashboard for failures
• If the test is still failing, set the bug status to REOPENED and back out the patch
• Otherwise, change the status of the bug to VERIFIED FIXED

Litmus

After you have landed a new test, or disabled a failing one, you need to update Litmus.

• If the test has been disabled
  • find the Mozmill Tests subgroup for each branch in Litmus
  • remove the test from the subgroup
  • amend the Mozmill Test block to include the word DISABLED and strike-through the link to the test file
  • Remember to revert this change when the test is being re-enabled
• If the test is new
  • find the Litmus testcase corresponding to your Mozmill test
  • add a comment to the test stating it is covered by Mozmill and provide a link to the file

    • This test is covered by Mozmill: 
      testPreferences/testPreferredLanguage.js
      

  • add the Litmus test to the Mozmill Tests subgroup for that branch
  • This needs to be done for each and every branch the test has landed

Nomenclature for patch landing

When you land a patch you should do the following in Bugzilla:

• Edit Details for the attached patch
  • If the patch is landed on all branches, add [landed] to the attachment summary
  • If the patch is landed on a single branch, add [landed:branch_name] to the attachment summary
• Add a comment to the bug indicating where the patch was landed and provide a link to the changeset

Example:

Comment on attachment 588432 [details] [diff] [review]
test v7 [landed]

Landed:
https://hg.mozilla.org/qa/mozmill-tests/rev/829eb7f3fc87 (mozilla-aurora)
https://hg.mozilla.org/qa/mozmill-tests/rev/49cea842da76 (mozilla-beta)
https://hg.mozilla.org/qa/mozmill-tests/rev/819e3aa37289 (mozilla-release)

Non-responsibilities

The following are not your responsibility as a committer and should never be done unless given explicit permission. These will be take care of by the module owners.

• Landing patches for new tests on the mozilla-1.9.2 branch
• Landing patches on the mozilla-esrN branch
• Merging branches