Reducing testcases

A reduced testcase is the simplest possible Web page that still demonstrates the bug. A bug report with a reduced testcase is easier to tackle than a bug report that merely refers to a web page that Firefox displays incorrectly.

Why make reduced testcases?

• Every hour Gecko engineers spend decomposing bug reports is an hour they can't spend fixing bugs.
• Overworked engineers tend to focus on bugs with testcases, so writing a testcase is the best and most productive way to vote for a bug.
• Writing a reduced testcase ensures that the bug report will still be useful if the page changes.
• Once the bug is fixed, a reduced testcase can be converted into an automated test, ensuring that the problem will not return.

How do I make a reduced testcase?

To turn a web page that shows a bug into a reduced test case:

1. Download a recent nightly build of Firefox to avoid wasting time on a bug that is already fixed.
2. Download the Web page that shows the bug to your local machine.
   • Use the "Web Page, Complete" option in the save dialog to download all linked files, including JavaScript (.js), CSS (.css), frameset (.html), and image files.
   • Or, use the "Web Page, HTML only" option and then add <base href="original url"> to the top of the HEAD element to maintain relative URLs.
   • Or, use "wget -E -H -k -K -p" (aka wget --html-extension --span-hosts --convert-links --backup-converted --page-requisites) in an empty directory.
3. Ensure you can reproduce the bug with the local copy.  If not:
   • Jesse's XMLHttpRequest interceptor can work around one issue.
   • Jeff's recording proxy will work on just about anything.
4. Using a text editor, remove irrelevant HTML markup, CSS rules, and lines of JavaScript from the page. As you remove parts of the page, reload it in Firefox to make sure it still reproduces the bug. If you remove too much, just Undo.
5. When you've cut away as much HTML, CSS, and JavaScript as you can, and cutting away any more causes the bug to disappear, you're done!

What do I do with a reduced testcase?

Once you have made a reduced testcase, attach the testcase to the bug report using the "Add an attachment" link. Then enter "testcase" into the Keywords¹ field so engineers know the bug is ready to be crushed.

Now that you know what HTML tags or CSS properties are involved in triggering the bug, search Bugzilla again to determine whether others have reported similar problems. You may learn that Gecko's behavior is intentional, in which case you can turn¹ the bug report into a (tech evangelism bug). Or you may learn that the bug is a duplicate of another open bug report.

Consider rewriting the bug's summary¹ to reflect the testcase, so it can be found by someone doing a similar search. For example, you might change "Can't open this page's dropdown" to "Can't open dropdown in table in position:relative span".

¹ A standard Bugzilla account does not have sufficient permission to add keywords or change a bug summary. After you have created testcases for two bugs, mail Gerv with links to the two bugs, and he will upgrade your permissions.

External files

If the testcase uses multiple files, try to modify the testcase so all of the information is in a single file.

• Replace <link rel="stylesheet" href="..."> with a <style> element containing the text of the stylesheet.
• Replace <script src="..."></script> with a <script> element containing the text of the script.
• Replace src URLs in IMG and IFRAME tags with equivalent data: URIs using the data: URI kitchen. Note that Internet Explorer 7 and below did not support data: URIs, so you shouldn't use this technique if the bug is about a difference between IE7 and Gecko.

As a last resort, you can upload ancillary files to Bugzilla, then modify the main file to point to the URLs of the attachments before uploading it. This is better than uploading a zip file because it allows the testcase to be viewed merely by clicking the link in Bugzilla.

Standards and quirks modes

Gecko uses the page's document type declaration to decide whether to display the page in "Quirks mode", "Almost Standards mode", or "Standards Compliance mode". Firefox's "Page Info" window can show you which mode a page uses, but it does not yet distinguish between Almost Standards mode and Standards Compliance mode (bug 154359).

If a page looks bad in Standards Compliance mode, try removing the doctype from the top of the file. If the Web page then display correctly, then the Web page author probably is relying on the (incorrect) rendering behaviors of older browsers but mistakenly asking browsers to render it according to modern web standards. Search for the name of the feature along with "quirk" or "standards mode" to find related bug reports.

Increasing testcase clarity

It is best for a reduced testcase to use a standards-mode doctype. This adds to the size of the file, but it effectively makes the testcase simpler by instructing Gecko not to apply compatibility quirks such as the rules in quirk.css. We prefer the HTML5 doctype

<!DOCTYPE html>

If the bug involves tables or margins, you can make it easier to visualize the bug by adding borders or outlines.

You can also increase the clarity of a testcase by using simpler elements:

• Replace block elements such as P with the generic block element DIV.
• Replace inline elements such as B, I, EM, STRONG, and even A with the generic inline element SPAN.
• Replace an image with an inline-block element of the same size, if you don't mind excluding browsers that don't support inline-block.

Useful tools

• A text editor with syntax highlighting can make it easier to scan code as you reduce it.
  • Windows: Notepad++ highlights HTML markup and embedded scripts.
  • Mac: TextWrangler highlights HTML markup, embedded stylesheets (if they are marked with type="text/css"), and embedded scripts (if they are marked with type="text/javascript").
• Firebug lets you view the DOM tree of a page and see what CSS rules apply to each element.
• DOM Inspector is great for modifying DOM trees on the fly.  You can delete nodes with the Delete key and undo their deletion with Cmd+Z.  This is often much faster than reducing HTML in a text editor.  When you're done, you can serialize the DOM tree to HTML using Firefox's "save as, complete" feature.
• JSFiddle gives you easy CSS, Javascript, and HTML with results all in one window.
• Web development bookmarklets such as "ancestors", "show blocks", "view scripts", and "view style sheets" can be useful for finding relevant code in the page.
• The Real-Time HTML Editor lets you build testcases from the ground up.
• Lithium lets you automate the process of reducing testcases for bugs that can be detected automatically, such as crashes and layout-instability bugs (tips).
• The W3C HTML Validator points out markup errors, which are sometimes the reason the page looks bad in Firefox. You can also use the validator to ensure that your reduced testcase file is valid HTML.

See also

• Mozilla Triage Guide: Minimal testcases
• WebKit testcase reduction guide
• Chromium testcase reduction guide