Link prefetching FAQ

What is link prefetching?

Link prefetching is a browser mechanism, which utilizes browser idle time to download or prefetch documents that the user might visit in the near future. A web page provides a set of prefetching hints to the browser, and after the browser is finished loading the page, it begins silently prefetching specified documents and stores them in its cache. When the user visits one of the prefetched documents, it can be served up quickly out of the browser's cache.

Does prefetching work with HTTPS?

Starting in Gecko 1.9.1 (Firefox 3.5), HTTPS content can be prefetched.

What are the prefetching hints?

The browser looks for either an HTML <link> or an HTTP Link: header with a relation type of either next or prefetch. An example using the link tag follows:

<link rel="prefetch" href="/images/big.jpeg">

The same prefetching hint using an HTTP Link: header:

Link: </images/big.jpeg>; rel=prefetch

The Link: header can also be specified within the HTML document itself by using a HTML meta tag:

<meta http-equiv="Link" content="</images/big.jpeg>; rel=prefetch">

The format for the Link: header is described in RFC 5988 section 5.

The browser observes all of these hints and queues up each unique request to be prefetched when the browser is idle. There can be multiple hints per page, as it might make sense to prefetch multiple documents. For example, the next document might contain several large images.

Some more examples follow:

<link rel="prefetch alternate stylesheet" title="Designed for Mozilla" href="mozspecific.css">
<link rel="next" href="2.html">

Are anchor (<a>) tags prefetched?

No, only <link> tags with a relation type of next or prefetch are prefetched. However, if there is sufficient interest, we may expand link prefetching support to include prefetching <a> tags, which include a relation type of next or prefetch in the future. Doing so would probably help content providers avoid the problem of stale prefetching links.

Is link prefetching standards compliant?

Yes, link prefetching as outlined in this document does not violate any existing web standards. In fact, the HTML 4.01 specification explicitly allows for the definition of new link relation types (see Section 6.12: Link types). However, the exact mechanism employed by Mozilla is not yet standardized. An Internet-Draft is in the works.

Standardization of this technique is part of the scope of HTML 5, see the current working draft, section §5.11.3.13. Link type "prefetch" .

How is browser idle time determined?

In the current implementation (Mozilla 1.2), idle time is determined using the nsIWebProgressListener API. We attach a listener to the toplevel nsIWebProgress object ("@mozilla.org/docloaderservice;1"). From this, we receive document start & stop notifications, and we approximate idle time as the period between the last document stop and the next document start. The last document stop notification occurs roughly when the onLoad handler would fire for the toplevel document. This is when we kick off prefetch requests. If a subframe contains prefetching hints, prefetching will not begin until the top-most frame and all its "child" frames finish loading.

What happens if I click on a link while something is being prefetched?

When the user clicks on a link, or initiates any kind of page load, link prefetching will stop and any prefetch hints will be discarded. If a prefetched document is partially downloaded, then the partial document will still be stored in the cache provided the server sent an "Accept-Ranges: bytes" response header. This header is typically generated by webservers when serving up static content. When the user visits a prefetched document for real, the remaining portion of the document will be fetched using a HTTP byte-range request.

What if I'm downloading something in the background? Will link prefetching compete for bandwidth?

Yes and no. If you are downloading something using Mozilla, link prefetching will be delayed until any background downloads complete. For example, if you load a bookmark group (which opens several tabs), any prefetch requests initiated by one of the bookmarked pages will not begin until all of the tabs finish loading. If you are using another application which uses the network, link prefetching in Mozilla may compete for bandwidth with the other application. This is a problem that we hope to address in the future by leveraging operating system services to monitor network idle time.

Are there any restrictions on what is prefetched?

Yes, only https:// (and, starting in Gecko 1.9.1 https://) URLs can be prefetched. Other protocols (such as FTP) do not provide rich enough support for client side caching.

Will Mozilla prefetch documents from a different host?

Yes. There is no same-origin restriction for link prefetching. Limiting prefetching to only URLs from the the same server would not offer any increased browser security.

Do prefetched requests contain a Referer: header?

Yes, prefetched requests include a HTTP Referer: header indicating the document from which the prefetching hint was extracted.

This may impact referrer tracking that is commonly used on many sites. For this reason, link prefetching may not be appropriate for all content. However, it is possible to instruct Mozilla to validate a prefetched document when the user follows a href to the prefetched document by specifying the Cache-control: must-revalidate HTTP response header. This header enables caching, but requires an If-Modified-Since or If-None-Match validation request before the serving the document out of the browser's cache.

As a server admin, can I distinguish prefetch requests from normal requests?

Yes, we send the following header along with each prefetch request:

X-moz: prefetch

Of course, this request header is not at all standardized, and it may change in future Mozilla releases.

Is there a preference to disable link prefetching?

Yes, there is a hidden preference that you can set to disable link prefetching. Add this line to your prefs.js file located in your profile directory (or make the appropriate change via about:config):

user_pref("network.prefetch-next", false);

However, the theory is that if link prefetching needs to be disabled then there must be something wrong with the implementation. We would rather improve the implementation if it does not work correctly, than expect users to locate and tweak some obscure preference.

What about folks who pay-per-byte for network bandwidth?

Basically, there are two ways of looking at this issue: websites can already cause things to be silently downloaded using JS/DOM hacks. prefetching is a browser feature; users should be able to disable it easily.

It is important that websites adopt <link> tag based prefetching instead of trying to roll-in silent downloading using various JS/DOM hacks. The <link> tag gives the browser the ability to know what sites are up to, and we can use this information to better prioritize document prefetching. The user preference to disable <link> tag prefetching may simply encourage websites to stick with JS/DOM hacks, and that would not be good for users. This is one reason why prefetching is enabled by default.

Which browsers support link prefetching?

Browsers based on Mozilla 1.2 (or later) as well as browsers based on Mozilla 1.0.2 (or later) support prefetching. This includes Firefox and Netscape 7.01+. Camino builds as of March 2003 are based on Mozilla 1.0.1, and therefore do not support prefetching. Test your browser to see if it supports Link Prefetching.

Privacy implications

Along with the referral and URL-following implications already mentioned above, prefetching will generally cause the cookies of the prefetched site to be accessed. (For example, if you google amazon, the google results page will prefetch www.amazon.com, causing amazon cookies to be sent back and forth. You can block 3rd party cookies in Firefox, see Disabling third party cookies.)

What about...?

If you have any questions or comments about link prefetching, please feel free to send them my way :-)

See also...

Prefetching Hints