Creating Sandboxed HTTP Connections

Introduction

Starting with Gecko 1.8.1 (Firefox 2), it is possible to create sandboxed HTTP connections which don't affect the user's cookies. This article will cover the basics of doing HTTP connections from XPCOM JavaScript, and should easily translate to C++ XPCOM.

Setting up an HTTP connection

The first step in setting up an HTTP connection from an URL (stored in a string) is to create an nsIURI out of it. nsIURI is an XPCOM representation of an URI, with useful methods to query and manipulate the URI. To create an nsIURI from an string, we use the newURI method of nsIIOService:

// the IO service
var ioService = Components.classes["@mozilla.org/network/io-service;1"]
                          .getService(Components.interfaces.nsIIOService);

// create an nsIURI
var uri = ioService.newURI(myURLString, null, null);

Once the nsIURI has been created, a nsIChannel can be generated from it using nsIIOService's newChannelFromURI method:

// get a channel for that nsIURI
var channel = ioService.newChannelFromURI(uri);

To initiate the connection, the asyncOpen method is called. It takes two arguments: a listener and a context that is passed to the listener's methods.

channel.asyncOpen(listener, null);

HTTP notifications

The above mentioned listener is a nsIStreamListener, which gets notified about events such as HTTP redirects and data availability.

onStartRequest - gets called when a new request is initiated.
onDataAvailable - new data is available. Since this is a stream, it could be called multiple times (depending on the size of the returned data, networking conditions, etc).
onStopRequest - the request has finished.
onChannelRedirect - when a redirect happens, a new nsIChannel is created, and both the old and new ones are passed in as arguments.

Since nsIStreamListener does not cover cookies, the current channel being used will need to be stored as a global, since another listener will be used for cookie notifications (covered in the next section). It is usually best to use a JavaScript wrapper that implements all the required methods and calls the specified callback function when the connection has completed. Below is an example:

// global channel
var gChannel;

// init the channel

// the IO service
var ioService = Components.classes["@mozilla.org/network/io-service;1"]
                          .getService(Components.interfaces.nsIIOService);

// create an nsIURI
var uri = ioService.newURI(myURLString, null, null);

// get a channel for that nsIURI
gChannel = ioService.newChannelFromURI(uri);

// get an listener
var listener = new StreamListener(callbackFunc);

gChannel.notificationCallbacks = listener;
gChannel.asyncOpen(listener, null);

function StreamListener(aCallbackFunc) {
  this.mCallbackFunc = aCallbackFunc;
}

StreamListener.prototype = {
  mData: "",

  // nsIStreamListener
  onStartRequest: function (aRequest, aContext) {
    this.mData = "";
  },

  onDataAvailable: function (aRequest, aContext, aStream, aSourceOffset, aLength) {
    var scriptableInputStream = 
      Components.classes["@mozilla.org/scriptableinputstream;1"]
        .createInstance(Components.interfaces.nsIScriptableInputStream);
    scriptableInputStream.init(aStream);

    this.mData += scriptableInputStream.read(aLength);
  },

  onStopRequest: function (aRequest, aContext, aStatus) {
    if (Components.isSuccessCode(aStatus)) {
      // request was successfull
      this.mCallbackFunc(this.mData);
    } else {
      // request failed
      this.mCallbackFunc(null);
    }

    gChannel = null;
  },

  // nsIChannelEventSink
  onChannelRedirect: function (aOldChannel, aNewChannel, aFlags) {
    // if redirecting, store the new channel
    gChannel = aNewChannel;
  },

  // nsIInterfaceRequestor
  getInterface: function (aIID) {
    try {
      return this.QueryInterface(aIID);
    } catch (e) {
      throw Components.results.NS_NOINTERFACE;
    }
  },

  // nsIProgressEventSink (not implementing will cause annoying exceptions)
  onProgress : function (aRequest, aContext, aProgress, aProgressMax) { },
  onStatus : function (aRequest, aContext, aStatus, aStatusArg) { },

  // nsIHttpEventSink (not implementing will cause annoying exceptions)
  onRedirect : function (aOldChannel, aNewChannel) { },

  // we are faking an XPCOM interface, so we need to implement QI
  QueryInterface : function(aIID) {
    if (aIID.equals(Components.interfaces.nsISupports) ||
        aIID.equals(Components.interfaces.nsIInterfaceRequestor) ||
        aIID.equals(Components.interfaces.nsIChannelEventSink) || 
        aIID.equals(Components.interfaces.nsIProgressEventSink) ||
        aIID.equals(Components.interfaces.nsIHttpEventSink) ||
        aIID.equals(Components.interfaces.nsIStreamListener))
      return this;

    throw Components.results.NS_NOINTERFACE;
  }
};

Quick note: storing the channel in a global (especially in an extension) isn't a good idea, but was done to make the code easier to read. It would be better to have the entire implementation inside a class and storing the channel as a member:

function myClass() {
  this.mChannel = null;
  // ...
  var listener = new this.StreamListener(callbackFunc);
  // ...
}
 
myClass.prototype.StreamListener = function (aCallbackFunc) {
  return ({
    mData: "",
    // ...
  })
}

Handling cookies

When sending a request, cookies that apply to the URL are sent with the HTTP request. The HTTP response can also contain cookies, which the browser processes. As of Mozilla 1.8.1 (Firefox 2), it is now possible to intercept those two cases.

For example, this means that if the user was logged into an webmail account, another account on the same domain could be checked without changing the user's cookies.

The observer service (nsIObserverService) is used to send general notifications, including the two cookie ones. The addObserver method is used to add an observer for a certain topic and takes in three agruments:

an object than implements nsIObserver
the topic to listen for. For cookies, the two topics are:
- http-on-modify-request - happens after the cookie data has been loaded into the request, but before the request is sent.
- http-on-examine-response - happens after the response is received, but before the cookies are processed
whether to hold a weak reference to the observer argument. Use false.

In order to avoid memory leaks, the observer needs to be removed at one point. The removeObserver method takes in the listener object and the topic and removes it from the notification list.

As with the above stream listener, an nsIObserver implementing object is needed, which only needs to implement one method, observe. The observe method gets passed in three arguments, which for the two cookie topics are:

aSubject: the channel (nsIChannel) that caused this notification to happen.
aTopic: the notification topic.
aData: null for the two topics.

Since the observers get notified for the registered topic for any connection, the listener needs to make sure that the notification is for the HTTP connection our code created. Since the channel that causes the notification is passed in as the first argument, comparing it to the globally stored channel (gChannel) in the previous section (which also gets updated each time a redirect happens).

// create an nsIObserver implementor
var listener = {
  observe : function(aSubject, aTopic, aData) {
    // Make sure it is our connection first.
    if (aSubject == gChannel) {
      var httpChannel = aSubject.QueryInterface(Components.interfaces.nsIHttpChannel);
      if (aTopic == "http-on-modify-request") {
         // ...
      } else if (aTopic == "http-on-examine-response") {
         // ...
      }
    }
  },

  QueryInterface : function(aIID) {
    if (aIID.equals(Components.interfaces.nsISupports) ||
        aIID.equals(Components.interfaces.nsIObserver))
      return this;
    throw Components.results.NS_NOINTERFACE;
  }
};

// get the observer service and register for the two coookie topics.
var observerService = Components.classes["@mozilla.org/observer-service;1"]
                                .getService(Components.interfaces.nsIObserverService);
observerService.addObserver(listener, "http-on-modify-request", false);
observerService.addObserver(listener, "http-on-examine-response", false);

The final piece is to manipulate the cookies. In order to manipulate cookies, the nsIChannel needs to be converted into a nsIHttpChannel by using QueryInterface (QI):

var httpChannel = aSubject.QueryInterface(Components.interfaces.nsIHttpChannel);

Cookies are actually part of the HTTP header, nsIHttpChannel provides four methods for working with headers: two for getting and setting request headers, and two for getting and setting response headers. The cookie header for requests is called "Cookie", while for responses it is "Set-Cookie".

getRequestHeader(aHeader) - returns the request header value for the requested header.
setRequestHeader(aHeader, aValue, aMerge) - sets the request header's value. If aMerge is true, the new value is appened, otherwise the old value is overwritten.
getResponseHeader(aHeader) - returns the response header value for the requested header.
setResponseHeader(aHeader, aValue, aMerge) - sets the response header's value. If aMerge is true, the new value is appened, otherwise the old value is overwritten.

These methods provide all the required functionality needed to modify cookies before they are processed/sent, allowing for sandboxed cookie connections that don't affect the user's cookies.

HTTP referrer

If the HTTP request needs to have a referrer set, two additional steps are needed after the nsIChannel is created, but before it is opened. First, a nsIURI needs to be generated for the referrer URL. Like before, the nsIIOService is used:

var referrerURI = ioService.newURI(referrerURL, null, null);

Next, the nsIChannel is QIed to nsIHttpChannel and the referrer property is set to the generated nsIURI:

var httpChannel = channel.QueryInterface(Components.interfaces.nsIHttpChannel);
httpChannel.referrer = referrerURI;

Creating HTTP POSTs

To create an HTTP POST, a few additional steps are required after the nsIChannel is created.

First, a nsIInputStream instance is created, after which the setData method is called. The first argument is the POST data as a string, while the second argument is the length of that data. In this case, the data is URL encoded, meaning that the string should look like this: foo=bar&baz=eek.

var inputStream = Components.classes["@mozilla.org/io/string-input-stream;1"]
                  .createInstance(Components.interfaces.nsIStringInputStream);
inputStream.setData(postData, postData.length);

Next, the nsIChannel is QIed to an nsIUploadChannel. Its setUploadStream method is called, passing in the nsIInputStream and the type (in this case, "application/x-www-form-urlencoded"):

var uploadChannel = gChannel.QueryInterface(Components.interfaces.nsIUploadChannel);
uploadChannel.setUploadStream(inputStream, "application/x-www-form-urlencoded", -1);

Due to a bug, calling setUploadStream will reset the nsIHttpChannel to be a PUT request, so now the request type is set to POST:

// order important - setUploadStream resets to PUT
httpChannel.requestMethod = "POST";