This article presents an overview of why we do sign-off reviews of localizations, the details on the criteria used for the sign-off reviews, and the process for requesting a review and for following its progress.
The sign-off review process has been notoriously shrouded in mystery. We use this review process to perform quality assurance (QA) testing on your L10n efforts. This ensures that every user who downloads your localized Mozilla application receives all of your hard work in a functional product. Each QA review helps us in these ways:
- Ensures that your L10n work is visible and functional within the Mozilla application.
- Allows us to understand how each L10n team is performing their localization work.
- Allows us to discover potential bugs within L10n tools.
Overview
You've done some localization work and are ready for it to be released. The merge date for migrating from one product release channel to the next is rapidly approaching. You need to make sure that your work can effectively be compiled into a build for the next release channel. Here is a detailed overview of what a sign-off review is and who it's meant for:
A sign-off review:
- is performed as a QA measure for maintaining a localization from release-to-release.
- only evaluates new L10n work. This is done by generating a diff between the new release revision being considered and the previously approved release revision.
- focuses on technical accuracy and completion of L10n work, not translation quality.
- evaluates new release revisions, not entire L10n repos.
- does not need to be approved before revision merge between release channels, although it is recommended.
Sign-off review criteria
Since the scope of each QA review process varies, the criteria used to evaluate a localizer's work also varies. In principle, they share basic elements, but sign-off reviews are only for evaluating the changes between your latest approved revision and the new revision you've submitted for release approval. Hence, unless a localizer makes a change within the new revision that nullifies the approved work in the previous revision, the sign-off criteria will be significantly less comprehensive than a technical review.
Here is some of the criteria we use to perform sign-off reviews:
- Increased amount of untranslated strings
- We look for previously translated strings changing to untranslated strings in the new revision. This is a common problem with some tools.
- Encoding
- Some tools add BOM to UTF-8 encoded files, and some change the encoding altogether.
- Mistakenly localized code
- We look for changes in localized code that should not have been localized or wasn't localized in the previous revision.
- Search plugin changes
- This greatly impacts productization. We look for these changes and file bugs when they're found in a new revision.
- Further indications of tool-originated corruption
- This is common. Most bugs filed from sign-offs are caused by a tool corrupting your localization's files.
- Newly translated strings that shouldn't have been translated
- Self-explanatory enough, right?
- Access key changes
- We ensure that when access keys are changed that they are consistent with the keyboard layout for that particular locale and not blended between two or more distinct layouts.
How it all works
A large part of the review process takes place before you even submit your request for a sign-off review. To put it simply, when we receive your request, we expect that you have spent time testing your work and making sure all strings have been localized and that your changes are relatively functional. Your review request symbolically represents that you have worked on your localization, tested it successfully, and found it to represent the best of your work. So before you request your sign-off review, be sure that it is as ready as it can be.
There is a specific process to follow in Bugzilla for requesting these reviews:
- File a bug requesting the review.
- Attach your files to the bug. Once attached, open the attachment details.
- In the attachment details, select the ? from the dropdown menu next to the appropriate approval flags (see image). This will display a text box to assign the review to someone.
- Enter :Pike as the Requestee.
Once you have made the formal request in Bugzilla and you have pushed your work into your L10n repo, refer to the dashboards to follow your review's progress.
Sign-off dashboards
To the right, you'll see the current sign-offs page for a German localization. Sign-offs are approved using the buttons in the farthest column to the right.
You can work with the L10n-drivers to track the progress of your review and correlate your review/sign-off's approval progress with specific Bugzilla bugs.
As your work is approved, you'll see a green circle in the farthest column to the right.
If bugs are discovered, your technical review/sign-off is rejected and the bugs are filed in Bugzilla. Once filed, they appear on your dashboard for you to fix. Once they are fixed, push your fix to your repo, attach it to the corresponding bug using the same Mercurial revision number as before, and then request another review. The process is then repeated until the bug is fixed to satisfaction.
The diff generated for sign-off reviews
The image to the right illustrates the revisions diff between the current release revision and your newly proposed release revision. This is generated from the revision attachment in your Bugzilla bug and evaluated to verify the technical accuracy of your new changes.
Text in green font represent additions you've made in this new revision. Text in red font represents content you've removed since the last released revision. Unseen here, orange font represents content that was replaced in your new release revision. Using these colors helps us to identify potential bugs and approve your latest revision with speed while maintaining critical accuracy.
All approved!
Once your work is entirely approved, be prepared to celebrate! You've accomplished a massive undertaking and you deserve to be rewarded! Go grab a drink with friends and enjoy your latest success!