Add to iPhoto

This extension for Mac OS X serves as a demonstration of how to use js-ctypes to call Mac OS X Carbon, Core Foundation, and other system frameworks from an extension written entirely in JavaScript.

You can download an installable version of this extension on AMO.

Once installed, when you right-click on an image, you'll see among the options in the contextual menu an option to "Add Image to iPhoto". Choose it, and iPhoto will start up (if it's not already running) and import the image.

Declaring the APIs

The first thing we have to do is declare the Mac OS X APIs we'll be using. This extension uses a number of methods and data types, as well as constants, from three system frameworks.

Since a lot of this stuff is repetitive, we'll only look at selected parts of the code to get an idea how things work. You can download the extension and poke through the code inside it if you'd like to see all of it.

For the sake of organization, I chose to implement each system framework (and, mind you, I only declare the APIs I actually use, not all of them) as a JavaScript object containing all the types and methods that framework's API.

Some global types

There are a few global data types used by all of our frameworks. These are declared near the top of the code:

const OSStatus = ctypes.int32_t;
const CFIndex = ctypes.long;
const OptionBits = ctypes.uint32_t;

OSStatus

Used to represent the status code resulting from an operation.

CFIndex

A Core Foundation long integer type used to represent indexes into lists. I debated including this in the CoreFoundation object, and probably would have if I were being more formal, but opted against it for clarity's sake.

OptionBits

A 32-bit bit field data type.

Core Foundation

The majority of the system routines we'll be using come from Core Foundation. Among these are routines for managing CFString, CFURL, and CFArray objects, among others. These are core system data formats that are used by other frameworks, and we'll be making use of them.

The Core Foundation API is implemented by the CoreFoundation object, which consists of two methods to initialize and shut down the library, a reference to the library, and all the types and methods declared to support Core Foundation.

Initializing Core Foundation

The init() method, which sets everything up, looks like this:

init: function() {
  this.lib = ctypes.open("/System/Library/Frameworks/CoreFoundation.framework/CoreFoundation");

  // declaring all the APIs goes here
}

Shutting down Core Foundation

While the Core Foundation system framework itself doesn't need to be shut down, we do need to close the library we opened using the js-ctypes API; that's where the shutdown() method comes in:

shutdown: function() {
  this.lib.close();
}

Select API declarations

Let's take a look at a few of the key APIs we declare for Core Foundation, to see how it's done.

CFRange

CFRange is a structure that identifies a range; that is, it identifies an offset to an item in a list and a number of items. In C, the declaration looks like this:

typedef struct {
    CFIndex location;
    CFIndex length;
} CFRange;

To declare this for use with js-ctypes, we use the following code:

this.CFRange = new ctypes.StructType("CFRange",
                    [ {'location': ctypes.int32_t},
                      {'length': ctypes.int32_t}]);

This defines CoreFoundation.CFRange to represent this data type, comprised of two 32-bit integer fields called location and length.

Generic CFType routines

All Core Foundation data types are based upon a core CFType data type. Basic CFType routines handle memory management, dumping CFType objects to the console, comparing CFType values, and so forth. We'll be using a number of these methods, but for brevity's sake, since these are generally simple declarations, let's look at only the CFRelease() and CFRetain() declarations.

In C, these are declared thusly:

void CFRelease(CFTypeRef cf);
void CFRetain(CFTypeRef cf);

In JavaScript, this translates to:

    this.CFRelease = this.lib.declare("CFRelease",
                                ctypes.default_abi,
                                ctypes.void_t,
                                ctypes.voidptr_t);        // input: object to release
    
    this.CFRetain = this.lib.declare("CFRetain",
                                ctypes.default_abi,
                                ctypes.void_t,
                                ctypes.voidptr_t);        // input: object to retain

These methods are used to manage the reference counting for Core Foundation objects.

CFString

A CFString is an opaque data type that contains a string. The string can be stored in any of a number of encodings, so you use assorted functions that know how to cope with different encodings to set and get values of CFStrings, as well as to perform typical string operations.

The first declaration to be done here is to actually declare the CFStringRef data type; this is an opaque pointer to a CFString object.

this.CFStringRef = new ctypes.PointerType("CFStringRef");

Now that we've declared the core type, we can declare the methods we use that work with CFString objects. Let's take a look at one of these.

    this.CFStringCreateWithCharacters = this.lib.declare("CFStringCreateWithCharacters",
                                ctypes.default_abi,
                                this.CFStringRef,         // returns a new CFStringRef
                                ctypes.voidptr_t,         // allocator
                                ctypes.jschar.ptr,        // pointer to the Unicode string
                                ctypes.int32_t);          // length of the string

CFStringCreateWithCharacters() is used to create a new CFString object using a Unicode string as the source string, which is copied into the new CFString object. It returns a CFStringRef, which is a pointer to the new string, and accepts, as input, three parameters: an allocator, which is a pointer to a routine that will allocate the memory to contain the new object (we use the ctypes.voidptr_t type for this), a pointer to the Unicode string to copy into the new string object (ctypes.jschar.ptr), and the length of the Unicode string in characters.

CFURL

The CFURL type is used to describe a URL. It differs from a string in that it offers URL-specific methods for managing the content, and includes methods for converting between URLs and file system routine data formats such as FSRef and Unix pathnames. We use a few of these routines because the Launch Services routine we'll be using to launch iPhoto and pass it the image to import uses CFURL for the file references.

Let's take a look at two of the routines declared here:

    this.CFURLCreateFromFileSystemRepresentation = this.lib.declare("CFURLCreateFromFileSystemRepresentation",
                                ctypes.default_abi,
                                this.CFURLRef,            // returns
                                ctypes.voidptr_t,         // input: allocator
                                ctypes.unsigned_char.ptr, // input: pointer to string
                                CFIndex,                  // input: string length
                                ctypes.bool)              // input: isDirectory

This method is used to convert a Unix pathname into an URL. The interesting things to note about the declaration of CoreFoundation.CFURLCreateFromFileSystemRepresentation() are:

• It returns a CFURLRef, which is an opaque pointer similar to the CFStringRef we noted above.
• The pathname is specified as a value of type ctypes.unsigned_char.ptr, which is a pointer to an unsigned character. "File system representation" strings on Mac OS X are in UTF-8 format.
• We use our CFIndex type here to specify the length of the string.

    this.CFURLGetFSRef = this.lib.declare("CFURLGetFSRef",
                                ctypes.default_abi,
                                ctypes.bool,              // Returns a bool
                                this.CFURLRef,            // input: URL to convert
                                ctypes.voidptr_t);        // input: Pointer to FSRef to fill

The CoreFoundation.CFURLGetFSRef() method is used to fill out a Carbon FSRef structure to describe the location of a file represented by a CFURL object. The main reason I include this here is because of the last parameter, which should be a pointer to an FSRef, but that's not declared until we get around to declaring the Carbon API, and I think that's worth noting.

CFArray

The CFArray type is used to create arrays of objects; the objects in the array can be of any type, thanks to a set of callbacks you can provide to handle managing their memory and performing operations such as comparisons. The most interesting thing we'll look at here is how to reference the system-provided default callback record, which is exported by Core Foundation under the name kCFTypeArrayCallBacks.

In C, the callback structure, and the predefined callback record, look like this:

typedef const void *	(*CFArrayRetainCallBack)(CFAllocatorRef allocator, const void *value);
typedef void		(*CFArrayReleaseCallBack)(CFAllocatorRef allocator, const void *value);
typedef CFStringRef	(*CFArrayCopyDescriptionCallBack)(const void *value);
typedef Boolean		(*CFArrayEqualCallBack)(const void *value1, const void *value2);
typedef struct {
    CFIndex				version;
    CFArrayRetainCallBack		retain;
    CFArrayReleaseCallBack		release;
    CFArrayCopyDescriptionCallBack	copyDescription;
    CFArrayEqualCallBack		equal;
} CFArrayCallBacks;

CF_EXPORT const CFArrayCallBacks kCFTypeArrayCallBacks;

The kCFTypeArrayCallBacks constant refers to a predefined callback structure referencing callback routines for managing arrays whose values are all CFType-based objects, such as CFURL, which is what we'll be using.

We need to be able to reference that predefined structure from our code. To do that, we first need to declare the CFArrayCallBacks structure:

    this.CFArrayCallBacks = new ctypes.StructType("CFArrayCallBacks",
      [ {'version': CFIndex},
        {'retain': ctypes.voidptr_t},
        {'release': ctypes.voidptr_t},
        {'copyDescription': ctypes.voidptr_t},
        {'equal': ctypes.voidptr_t} ]);

Having done this, we can then import the kCFTypeArrayCallBacks structure. This is done using the js-ctypes library object's declare() method, just like importing a function:

    this.kCFTypeArrayCallBacks = this.lib.declare("kCFTypeArrayCallBacks",
                              this.CFArrayCallBacks);

CFMutableArray

One thing about Core Foundation types that is interesting is the use of regular and mutable versions of the same data types. For example, the CFArray type describes an array, but CFArray objects can't be changed once they've been created.

However, obviously there are cases in which you'll want to be able to manipulate the contents of an array by adding and removing items, sorting them, and so forth. That's where the CFMutableArray type comes into play. All CFArray functions accept CFMutableArray objects, so you can use CFMutableArray with any routine that accepts a CFArray as input, but CFMutableArray supports additional functions that let you change the contents of the array.

There's nothing particularly interesting about how we declare this API, but it will be noteworthy when we look at how we use CFMutableArray objects with methods that accept a CFArray as input, so I introduce this concept here.

Carbon

The Carbon API is the core operating system API derived from the classic Mac operating system. We actually aren't using any Carbon methods, but we are using one Carbon data type, the previously mentioned FSRef structure. FSRef is an opaque object describing a file.

In C, the FSRef is declared thusly:

struct FSRef {
  UInt8               hidden[80];             /* private to File Manager; •• need symbolic constant */
};
typedef struct FSRef                    FSRef;

We declare it using js-ctypes like this:

    this.struct_FSRef = new ctypes.StructType("FSRef",
                [ {"hidden": ctypes.char.array(80)}]);

The Carbon library init() and shutdown() routines are otherwise similar to how we do things for Core Foundation.

Application Services

The Application Services framework consists of a number of different APIs that provide special services to applications. The Application Services API we'll be using is the Launch Services API, which is used to launch applications and open files in default (or specific, in our case) applications.

The function we'll be using is LSOpenURLsWithRole(), whose declaration looks like this:

this.LSOpenURLsWithRole = this.lib.declare("LSOpenURLsWithRole",
                                ctypes.default_abi,                 // ABI type
                                OSStatus,                           // Returns OSStatus
                                CoreFoundation.CFArrayRef,          // Array of files to open in the app
                                OptionBits,                         // Roles mask
                                ctypes.voidptr_t,                   // inAEParam
                                this.struct_LSApplicationParameters.ptr, // description of the app to launch
                                ctypes.voidptr_t,                   // PSN array pointer
                                CFIndex);                           // max PSN count

This function returns an OSStatus indicating the result of the launch attempt, and accepts these parameters:

• A CFArrayRef providing a list of CFURL objects for the files to open in the application.
• An OptionBits value providing a bit field of special options.
• A pointer to an Apple Event parameter; we aren't using this, so I'm just using a voidptr_t here.
• A pointer to an LSApplicationParameters structure that describes what application to launch
• A pointer to the first element of an array to receive the serial numbers of the launched applications; we're not using this field, but if you do, you'll probably have to declare this differently.
• A CFIndex indicating the size of the array specified by the previous parameter.

The LSApplicationParameters structure is declared like this:

    this.struct_LSApplicationParameters = new ctypes.StructType('LSApplicationParameters',
                                              [ {'version': CFIndex},
                                                {'flags': OptionBits},
                                                {'application': ctypes.voidptr_t},  // FSRef of application to launch
                                                {'asyncLaunchRefCon': ctypes.voidptr_t},
                                                {'environment': ctypes.voidptr_t},  // CFDictionaryRef
                                                {'argv': ctypes.voidptr_t},         // CFArrayRef of args
                                                {'initialEvent': ctypes.voidptr_t}]); // AppleEvent *

Most of these fields, we won't be using. We'll get a look at how we use this shortly.

There are also a few constants used for the flags field in the LSApplicationParameters structure:

this.kLSRolesNone = 1;
this.kLSRolesViewer = 2;
this.kLSRolesEditor = 4;
this.kLSRolesAll = 0xffffffff;

Implementing the extension

Now that the Mac OS X APIs we'll be using have been declared, we can write the core of the extension itself. This is done in the iPhoto object in the extension's code.

Hooking up to the context menu

On startup, we find the content area's context menu and add an event listener to it that will be called when the context menu is displayed. We'll use our handler for this event to add the "Add Image to iPhoto" option if the user has right-clicked on an image.

if (document.getElementById("contentAreaContextMenu")) {
  document.getElementById("contentAreaContextMenu").addEventListener("popupshowing", iPhoto.onPopup, false);
}

Responding when the context menu is clicked

When the user right-clicks an image, our handler gets called:

onPopup: function() {
  var node = iPhoto.getCurrentNode();

  var item = document.getElementById("add-to-iphoto_menuitem");

  if (item) {
    item.hidden = (node == null); // Hide it if we're not on an image
  }
}

This code finds the image node the user right-clicked in by calling our getCurrentNode() method, then sets the hidden state of the "Add Image to iPhoto" menu item based on whether or not an image node was found.

The code to identify the node looks like this:

getCurrentNode: function() {
  var node = document.popupNode;
  
  // If no node, just return null now
  
  if (node == undefined || !node) {
    return null;
  }
  
  // Is it an image node?
  
  var elemName = node.localName.toUpperCase();
  
  if (elemName == "IMG") {
    return node;
  }
  
  // Nope, return null
  
  return null;
}

This starts by getting the node the popup was opened from. If this is null or undefined, we immediately return null, indicating there is no node associated with the context menu.

Otherwise, we fetch the name of the element and look to see if it's an <img> element. If so, we return that node; otherwise, we return null.

The important thing to take away from this is that this method returns either null or the image node the user right-clicked on. If they right-clicked anything other than an image, it returns null.

Responding when the "Add Image to iPhoto" option is chosen

When the user chooses to add the image to iPhoto, the add() method is executed.

add: function() {
  var node = iPhoto.getCurrentNode();
  
  if (node) {
    var src = node.getAttribute("src"); // Get the URL of the image
    if (src && src != "") {
      iPhoto.addImageByURL(src);
    }
  }
}

This fetches the node representing the image the user wants to add, and, if it's an image, fetches the image's URL from its src attribute, then passes it into our addImageByURL() method, which will do all the heavy lifting.

Adding the image to iPhoto

The addImageByURL() method handles actually retrieving the image and adding it to iPhoto. Let's take a look at its code, then explore how it works. This is where all our js-ctypes usage occurs.

addImageByURL: function(src) {
  CoreFoundation.init();
  Carbon.init();
  AppServices.init();
  
  // Download the image
  
  var filePath = this.downloadImage(src);
  
  var mutableArray = CoreFoundation.CFArrayCreateMutable(null, 1, CoreFoundation.kCFTypeArrayCallBacks.address());
  if (mutableArray) {
    var url = CoreFoundation.CFURLCreateFromFileSystemRepresentation(null, filePath, filePath.length, false);
    CoreFoundation.CFArrayAppendValue(mutableArray, url);
    CoreFoundation.CFRelease(url);
    
    // Call Launch Services to open iPhoto and deliver the image
    
    var ref = new Carbon.struct_FSRef;
    var appParams = AppServices.struct_LSApplicationParameters(0, 1, ref.address(), null, null, null, null);
  
    var appstr = "file:///Applications/iPhoto.app";
    var appstrCF = CoreFoundation.CFStringCreateWithCharacters(null, appstr, appstr.length);
    var appurl = CoreFoundation.CFURLCreateWithString(null, appstrCF, null);
    CoreFoundation.CFRelease(appstrCF);
    var b = CoreFoundation.CFURLGetFSRef(appurl, ref.address());
    
    if (!b) {
      var stringsBundle = document.getElementByID("string-bundle");
      
      alert(stringsBundle.getString('alert_download_error_string'));
    } else {
      var array = ctypes.cast(mutableArray, CoreFoundation.CFArrayRef);
      AppServices.LSOpenURLsWithRole(array, 0, null, appParams.address(), null, 0);
    }
    CoreFoundation.CFRelease(appurl);
          
    // Clean up
    
    CoreFoundation.CFRelease(array);
  }
  
  AppServices.shutdown();
  Carbon.shutdown();
  CoreFoundation.shutdown();
}

This code begins by initializing all the system frameworks we're using, by calling the init() methods on the CoreFoundation, Carbon, and AppServices objects.

Then the downloadImage() method is used to actually download the image to a temporary file. Once we have the file, we start making use of our native APIs.

Creating the array of files to import

The first step is to construct an array of URLs for all the files we want to open in iPhoto. In our case, we have only one file, but we still need an array. So we start by calling CoreFoundation.CFArrayCreateMutable() to create a mutable array with room for one item, specifying the address of the standard callback routines exported by Core Foundation using the syntax CoreFoundation.kCFTypeArrayCallBacks.address().

If creating the array succeeded, we continue by creating a new CFURL object from the pathname of the image file returned by the downloadImage() method. This is done by calling the Core Foundation routine CFURLCreateFromFileSystemRepresentation(). Conveniently, we can simply pass in the JavaScript string, filePath, as the string and filePath.length as its length.

The array is then built by using CFArrayAppendValue() to add the new CFURL to the array. Doing this causes the array to retain the URL object, so we can use CFRelease() to release it now.

Calling Launch Services to launch iPhoto

Next, we need to build the parameters for the LSOpenURLsWithRole() function, then call it to start up iPhoto.

The first step here is to create a new FSRef object to contain the reference to the iPhoto application itself, since LSOpenURLsWithRole() uses an FSRef to specify the application to launch.

Then we build the LSApplicationParameters structure describing the application to launch. Let's take a closer look at this syntax:

    var appParams = AppServices.struct_LSApplicationParameters(0, 1, ref.address(), null, null, null, null);

Here you're calling a constructor, created for you by js-ctypes, that creates and fills out the structure, specifying the values of all of the parameters. To specify a pointer to the FSRef indicating the application to launch, we pass ref.address(), which obtains the actual memory address of the C data structure.

Note that so far, we haven't actually obtained a value for the FSRef in question. We do that next by following these steps:

1. Create a CFString referring to file:///Applications/iPhoto.app, which is iPhoto's default path, using CFStringCreateWithCharacters().
2. Create the corresponding CFURL object by calling CFURLCreateWithString().
3. Release the string using CFRelease(), since we're done with it.
4. Call CFURLGetFSRef() to fill out the FSRef structure to reference the same file as the CFURL.

If that fails, we display an error; otherwise, we cast the CFMutableArrayRef into a CFArrayRef by calling ctypes.cast(), then call LSOpenURLsWithRole() to actually send the image to iPhoto.

After doing that, we clean up after ourselves by releasing the CFURL object and the array, then shutting down the three libraries we used.

Downloading the image

The downloadImage() method handles actually downloading the image to a temporary file; it then returns the local pathname of the downloaded file to the caller.

downloadImage: function(src) {
  // Get the file name to download from the URL
  
  var fileName = src.slice(src.lastIndexOf("/")+1);

  // Build the path to download to

  var dest = Components.classes["@mozilla.org/file/directory_service;1"]
                          .getService(Components.interfaces.nsIProperties)
                          .get("TmpD", Components.interfaces.nsIFile);
  dest.append(fileName);
  
  dest.createUnique(dest.NORMAL_FILE_TYPE, 0600);
  var wbp = Components.classes['@mozilla.org/embedding/browser/nsWebBrowserPersist;1']
            .createInstance(Components.interfaces.nsIWebBrowserPersist);
  var ios = Components.classes['@mozilla.org/network/io-service;1']
            .getService(Components.interfaces.nsIIOService);
  
  var uri = ios.newURI(src, document.characterSet, gBrowser.selectedBrowser.contentDocument.documentURIObject);
  wbp.persistFlags &= ~Components.interfaces.nsIWebBrowserPersist.PERSIST_FLAGS_NO_CONVERSION; // don't save gzipped
  wbp.saveURI(uri, null, null, null, null, dest);
  
  return dest.path;
}

This is pretty straightforward, typical Mozilla code. It gets the filename of the file being download by slicing it off the end of the specified image URL, then obtains the path to the temporary items folder and appends the image file's name to that path. Then we call createUnique() to create a unique file by that name (or a derivative thereof if the name is already in use), and download the contents of the image file to that local file.

Closing remarks

This is a fairly simple example of how to use js-ctypes, but it actually does something useful, and should be a helpful demonstration not just for how to use js-ctypes, but also more specifically for developers that want to interface with Mac OS X system frameworks.

See also

• ctypes.jsm
• js-ctypes reference
• Using js-ctypes