Intercepting Page Loads

There are several ways to detect and intercept loading of web pages and their content, be it only to realize when it happens, modify their contents, or to block them and do something else instead. Some of the techniques presented here apply only to content loaded in the main browser area, while others detect content being loaded in other XUL windows, or even detect XUL content being loaded. Also, the different techniques tap into different steps of the load process. Which one you should use solely depends on your needs. We will start with the simplest one, which is also the most common to use.

The Easy Way: Load Events

This comes from the tabbrowser code snippets page. In a nutshell, from the chrome code in the overlay we add an event listener for the load event.

this._loadHandler = function() {that._onPageLoad(); };

gBrowser.addEventListener("load", this._loadHandler, true);

gBrowser is a global object that corresponds to the tabbrowser element in the main browser window. It is full of useful functions, so you should always keep it in mind when you want to handle tabs and web content windows. Attaching the load event handler to gBrowser allows us to listen to these events for all tabs, without having to worry how many tabs are open. gBrowser exists in every browser window. We store the handler function in a private variable because later we want to remove it when we do not need it anymore.

gBrowser.removeEventListener("load", this._loadHandler, true);

Finally, the actual code for the handler, which is very simple:

_onPageLoad : function(event) {
  // this is the content document of the loaded page.
  let doc = event.originalTarget;

  if (doc instanceof HTMLDocument) {
    // is this an inner frame?
    if (doc.defaultView.frameElement) {
      // Frame within a tab was loaded.
      // Find the root document:
      while (doc.defaultView.frameElement) {
        doc = doc.defaultView.frameElement.ownerDocument;
      }
    }
  }
}

The second if validation is necessary if you need to make a distinction for HTML documents being loaded in inner page frames. Few modern sites use framesets, but it is common for ads to appear inside iframe elements. In most cases you'll need to compare the page URL with some string or regular expression:

if (SOME_REGULAR_EXPRESSION.test(doc.defaultView.location.href))

You can access and modify the DOM of the loaded page, just like you normally would for XUL and HTML documents.

You can't, however, easily cancel the page load. You can close the tab, redirect the tab to about:blank or another page, or tell the browser to stop loading this page, but in general you don't want to do this because it will be visible to the user and it will look like a bug. There are better ways to intercept page loads before any content is downloaded, and before the user sees anything in the tab.

HTTP Observers

Another common way of detecting and intercepting loads is using HTTP observer topics. This is how the Tamper Data extension and others do it.

HTTP notifications are fired for all HTTP requests originating from Firefox. They are window-independent, so it is better to keep your observer code in non-chrome objects (your XPCOM service or jsm module). Otherwise you have to make sure to avoid duplicated work if you have 2 or more windows open.

There are 2 HTTP topics you can listen to, as specified in the Observer Notifications page:

The subject argument on the observe method is the nsIHttpChannel object that corresponds to the HTTP channel being opened or already opened, depending on the topic.

observe : function(aSubject, aTopic, aData) {
  if (TOPIC_MODIFY_REQUEST == aTopic) {
    let url;

    aSubject.QueryInterface(Components.interfaces.nsIHttpChannel);
    url = aSubject.URI.spec;

    if (RE_URL_TO_MODIFY.test(url)) { // RE_URL_TO_MODIFY is a regular expression.
      aSubject.setRequestHeader("Referer", "https://example.com", false);
    } else if (RE_URL_TO_CANCEL.test(url)) { // RE_URL_TO_CANCEL is a regular expression.
      aSubject.cancel(Components.results.NS_BINDING_ABORTED);
    }
  }
}

This example shows how you can obtain the URL for the request, analyze it using regular expressions, and perform actions on it such as modifying HTTP headers, or even canceling the request.

HTTP observers are great for load detection, and filtering by URL. You can also use these in conjunction with nsITraceableChannel to get and modify the response text before it gets to the original requester.

You won't be able to make DOM modifications like with the load events, since these notifications are triggered before the response arrives and is parsed into DOM tree, so this is not the best approach for the typical Greasemonkey-style extensions.

WebProgressListeners

When used in the chrome, this is a more sophisticated way of intercepting and modifying the various stages in page loads. But of course, there's always a price to pay: web progress listeners in the chrome are attached to specific instances of the browser element. What does this mean? It means you have to keep track of tabs being opened and closed, in order to add and remove your listener. Here's a code sample that keeps track of your progress listeners for all tabs:

init : function() {
  gBrowser.browsers.forEach(function (browser) {
    this._toggleProgressListener(browser.webProgress, true);
  }, this);

  gBrowser.tabContainer.addEventListener("TabOpen", this, false);
  gBrowser.tabContainer.addEventListener("TabClose", this, false);
},

uninit : function() {
  gBrowser.browsers.forEach(function (browser) {
    this ._toggleProgressListener(browser.webProgress, false);
  }, this);

  gBrowser.tabContainer.removeEventListener("TabOpen", this, false);
  gBrowser.tabContainer.removeEventListener("TabClose", this, false);
},

handleEvent : function(aEvent) {
  let tab = aEvent.target;
  let webProgress = gBrowser.getBrowserForTab(tab).webProgress;

  this._toggleProgressListener(webProgress, ("TabOpen" == aEvent.type));
},

_toggleProgressListener : function(aWebProgress, aIsAdd) {
  if (aIsAdd) {
    aWebProgress.addProgressListener(this, aWebProgress.NOTIFY_ALL);
  } else {
    aWebProgress.removeProgressListener(this);
  }
}

This shouldn't be too hard to follow. We register and unregister the progress listeners for the first tab manually, and then add tab open and close event listeners so we can keep track of the rest of the listeners for all the tabs. We're being careful about removing all listeners, as not doing it has the potential of causing memory leaks.

There are a couple of things left to do: implementing the progress listener methods, and figuring out what NOTIFY_ALL is about. For that, we recommend that you first read the documentation on nsIWebProgressListener and the nsIWebProgress.Constants (WebProgress NOTIFY constants). In a nutshell, there are a lot of state and status changes going on when a page loads, and the NOTIFY constants allow you to filter out the events you don't need to listen to. Picking the right filter not only simplifies your code, but also reduces the performance impact your extension will have in regular navigation.

Here are a couple of common use cases and the ways to implement them with web progress listeners:

• If you want simple detection and filtering like in the page load events, you can use onLocationChange. You can use aLocation.spec to get the URL and match it against a regular expression. The request object aRequest holds the request being processed, and you can run aRequest.cancel(NS_BINDING_ABORTED) to cancel it. aWebProgress.DOMWindow gives you access to the window where the content was going to be loaded.
• Sometimes you do not care about redirects, and only want to detect the final page being loaded, the one holding the actual content. In this case, your best bet is to use onStateChange, and filter to when the state flags indicate a document has begun being loaded:

  if ((aStateFlags & Components.interfaces.nsIWebProgressListener.STATE_START) &&
      (aStateFlags & Components.interfaces.nsIWebProgressListener.STATE_IS_DOCUMENT))
  

  Note the use of the binary mask & operator.

  To detect if this is a frame being loaded or not, you can do this:

  if (aWebProgress.DOMWindow != aWebProgress.DOMWindow.top) {
    // this is a frame.
  }
  

  In this case, the URL can be obtained from aRequest.name. Make sure you access it from inside the state validation if condition. There may be other cases where accessing this property will throw an exception. Canceling the request works just like with onLocationChange.

• See "Recognizing page load failure" at mozilla.dev.extensions for tips on detecting connection failures and HTTP response codes.

XPCOM Solutions

There are a couple more solutions you may be interested in trying if the previous ones are insufficient. These require creating XPCOM components that implement existing Firefox interfaces. They may be useful in case you have most of your application logic in XPCOM, or if you absolutely need a single point of inspection for loads. The previously explained solutions are enough for most cases, so these ones will be mentioned briefly.

Document Loader Service

nsIDocumentLoader is nothing but a global Web Progress Listener. You can create an XPCOM component that extends nsIWebProgressListener and use the addProgressListener method in the service to include it. Everything mentioned before applies here as well, except you'll be receiving all the events for all tabs and windows in a single object, and you don't have to worry about adding and removing listeners every time a tab is opened or closed.

This also has the advantage of detecting page loads anywhere in the application, not only in browser windows.

If you are building a web filtering extension, you should keep in mind that XUL windows such as the DOM Inspector window and the Add-ons Manager window allow (limited) web navigation. Other extensions might add XUL windows that allow navigation as well, so in those cases it's best to use a global solution like this one.

Content Policy

Finally, there is the option of implementing nsIContentPolicy. You can create an XPCOM component that extends nsIContentPolicy and register it to the "content-policy" category using the nsICategoryManager.

The nsIContentPolicy.shouldLoad() method is the only one in this interface that is really useful. It actually allows for cleaner-looking code than most of the previously seen solutions, because you get the content URI directly as an argument, and you indicate if the content should be loaded or not with the return value, which has well-defined possible values. The context parameter gives you access to the window loading the content.

As with all other solutions, you need to be efficient, and a good way to do this is to filter out unneeded cases right from the start. shouldLoad is called for every load operation Firefox tries to do, including images, scripts and XUL documents. A good filter would look like this:

shouldLoad : function(aContentType, aContentLocation, aRequestOrigin, aContext, aMimeTypeGuess, aExtra) {
  let result = Components.interfaces.nsIContentPolicy.ACCEPT;

  // we should check for TYPE_SUBDOCUMENT as well if we want frames.
  if ((Components.interfaces.nsIContentPolicy.TYPE_DOCUMENT == aContentType) &&
      SOME_REGULAR_EXPRESSION.test(aContentLocation.spec)) {
    // do stuff here, possibly changing result.
  }

  return result;
}

The content policy is applied very early in the process, even before the request is made, allowing a very clean cancel operation. This characteristic brings 2 limitations to this approach. The first one is that you can't easily read or modify the content to be loaded. The second one is that shouldLoad is not invoked for redirects. You only get one call, for the first URL requested. If you let it pass, it can redirect anywhere without you noticing it. This approach is used in popular filtering extensions, such as AdBlock Plus. We recommend you look into the other solutions first, though. Maybe a combination of some or all of these will be what your extension needs.

This tutorial was kindly donated to Mozilla by Appcoast.