IPDL Tutorial

IPDL, short for "Inter-process-communication Protocol Definition Language", is a Mozilla-specific language allowing C++ code to pass messages between processes or threads in an organized and secure way. All messages for multi-process plugins and tabs in Firefox are declared in the IPDL language.

All IPDL messages are sent between parent and a child end points, called actors. An IPDL protocol declares how actors communicate: it declares the possible messages that may be sent between actors, as well as a state machine describing when messages are allowed to be sent.

The parent actor is typically the more permanent side of the conversation:

Each protocol is declared in a separate file. The IPDL compiler generates several C++ headers from each IPDL protocol. This generated code manages the details of the underlying communication layer (sockets and pipes), constructing and sending messages, ensuring that all actors adhere to their specifications, and handling many error conditions. The following IPDL code defines a very basic interaction of browser and plugin actors:

async protocol PPlugin
{
child:
  Init(nsCString pluginPath);
  Shutdown();

parent:
  Ready();
};

This code declares the PPlugin protocol. Two messages can be sent from the parent to the child, Init() and Shutdown(). One message can be sent from the child to the parent, Ready().

Generated C++ Code

When PPlugin.ipdl is compiled, the headers PPluginParent.h, and PPluginChild.h will be generated in the ipc/ipdl/_ipdlheaders/ directory of the build tree. The PPluginParent and PPluginChild classes are abstract classes that must be subclassed. Each outgoing message is a C++ method which can be called. Each incoming message is a pure-virtual C++ method which must be implemented:

class PPluginParent
{
public:
  bool SendInit(const nsCString& pluginPath) {
    // generated code to send an Init() message
  }

  bool SendShutdown() {
    // generated code to send a Shutdown() message
  }

protected:
  /**
   * A subclass of PPluginParent must implement this method to handle the Ready() message.
   */
  bool RecvReady() = 0;
};

class PPluginChild
{
protected:
  bool RecvInit(const nsCString& pluginPath) = 0;
  bool RecvShutdown() = 0;

public:
  bool SendReady() {
    // generated code to send a Ready() message
  }
};

These Parent and Child abstract classes take care of all the "protocol layer" concerns: serializing data, sending and receiving messages, and checking protocol safety. It is the responsibility of the implementor to create subclasses to perform the actual work involved in each message. Here is a dirt-simple example of how a browser implementor might use PPluginParent.

class PluginParent : public PPluginParent
{
public:
  PluginParent(const nsCString& pluginPath) {
    // launch child plugin process
    SendInit(pluginPath);
  }

  ~PluginParent() {
    SendShutdown();
  }

protected:
  bool RecvReady() {
    mObservers.Notify("ready for action");
  }
};

Here's how the PPluginChild might be used by a C++ implementor in the plugin process:

class PluginChild : public PPluginChild
{
protected:
  void RecvInit(const nsCString& pluginPath) {
    mPluginLibrary = PR_LoadLibrary(pluginPath.get());
    SendReady();
  }
  void RecvShutdown() {
    PR_UnloadLibrary(mPluginLibrary);
  }

private:
  PRLibrary* mPluginLibrary;
};

Launching the subprocess and hooking these protocol actors into our IPC "transport layer" is beyond the scope of this document. See IPDL Processes and Threads for more details.

Because protocol messages are represented as C++ methods, it's easy to forget that they are in fact asynchronous messages: by default the C++ method will return immediately, before the message has been delivered.

The parameters of the Recv* methods (const nsCString& pluginPath in the example) are references to temporary objects, so copy them if you need to keep their data around.

Direction

Each message type includes a "direction." The message direction specifies whether the message can be sent from-parent-to-child, from-child-to-parent, or both ways. Three keywords serve as direction specifiers; child was introduced above. The second is parent, which means that the messages declared under the parent label can only be sent from-child-to-parent. The third is both, which means that the declared messages can be sent in both directions. The following artificial example shows how these specifiers are used and how these specifiers change the generated abstract actor classes.

// PDirection.ipdl
async protocol PDirection
{
child:
  Foo();  // can be sent from-parent-to-child
parent:
  Bar();  // can be sent from-child-to-parent
both:
  Baz();  // can be sent both ways
};

// PDirectionParent.h
class PDirectionParent
{
protected:
  virtual void RecvBar() = 0;
  virtual void RecvBaz() = 0;

public:
  void SendFoo() { /* boilerplate */ }
  void SendBaz() { /* boilerplate */ }
};

// PDirectionChild.h
class PDirectionChild
{
protected:
  virtual void RecvFoo() = 0;
  virtual void RecvBaz() = 0;

public:
  void SendBar() { /* boilerplate */ }
  void SendBaz() { /* boilerplate */ }
};

You can use the child, parent, and both labels multiple times in a protocol specification. They behave like public, protected, and private labels in C++.

Parameters

Message declarations allow any number of parameters. Parameters specify data that are sent with the message. Their values are serialized by the sender and deserialized by the receiver. IPDL supports built-in and custom primitive types, as well as unions and arrays.

using mozilla::plugins::NPRemoteEvent;

sync protocol PPluginInstance
{
child:
  HandleEvent(NPRemoteEvent);
};

using struct mozilla::void_t from "ipc/IPCMessageUtils.h";

union Variant
{
  void_t;
  bool;
  int;
  double;
  nsCString;
  PPluginScriptableObject;
};

struct Variant
{
  enum Type {
    Tvoid_t, Tbool, Tint, Tdouble, TnsCString, TPPlugionScriptableObject
  };
  Type type();
  void_t& get_void_t();
  bool& get_bool();
  int& get_int();
  double& get_double();
  nsCString& get_nsCString();
  PPluginScriptableObject* get_PPluginScriptableObject();
};

aVariant = false;

struct NameValuePair
{
  nsCString name;
  nsCString value;
};

NameValuePair entry(aString, anotherString);
foo(entry.name(), entry.value()); // Named accessor functions return references to the members

InvokeMethod(nsCString[] args);

virtual bool RecvInvokeMethod(nsTArray<nsCString>& args);

IPDL's generated data structures can be used in several protocols if they are defined in a separate .ipdlh file. These files must be added to the ipdl.mk makefile like regular .ipdl files, and they use the same syntax (except they cannot declare protocols). To use the structures defined in Foo.ipdlh, include it as follows.

// in a .ipdl file
include Foo;

Synchronous and RPC Messaging

Up until now, all the messages have been asynchronous. The message is sent, and the C++ method returns immediately. But what if we wanted to wait until the message was handled, or get return values from a message?

In IPDL, there are three different semantics:

1. asynchronous semantics; the sender is not blocked.
2. Wait until the receiver acknowledges that it received the message. We call this synchronous semantics, as the sender blocks until the receiver receives the message and sends back a reply. The message may have return values.
3. rpc semantics are a variation on synchronous semantics, see below.

Note that the parent can send messages to the child, and vice versa, so 'sender' and 'receiver' in the above three cases can be either the parent or the child. The messaging semantics applies in the same way to both directions. So, for example, in synchronous semantics from the child to the parent, the child will block until the parent receives the message and the response arrives, and in asynchronous semantics from the parent to the child the parent will not block.

When creating a plugin instance, the browser should block until instance creation is finished, and needs some information returned from the plugin:

protocol PPluginInstance
{
child:
    sync Init() returns (bool windowless, bool ok);
};

We added two new keywords to the Plugin protocol, sync and returns. sync marks a message as being sent synchronously (async is the default if no semantic is specified). The returns keyword marks the beginning of the list of values that are returned in the reply to the message. It is an error to add a returns block to an asynchronous message.

To make the blocking nature more noticeable to programmers, the C++ method names for synchronous and RPC messages are different:

Message Semantics Strength

The above protocol will fail the IPDL type checker. IPDL protocols also have "semantics specifiers", just like messages. A protocol must be declared to have semantics at least as "strong" as its strongest message semantics. Synchronous semantics is called "stronger than" asynchronous. Like message declarations, the default protocol semantics is asynchronous; however, since the Plugin protocol declares a synchronous message, this type rule is violated. The fixed up Plugin protocol is shown below.

sync protocol PPluginInstance
{
child:
    sync Init() returns (bool windowless, bool ok);
};

The generated C++ code for this method uses outparam pointers for the returned values:

class PPluginInstanceParent
{
  ...
  bool SendInit(bool* windowless, bool* ok) { ... };
};

class PPluginInstanceChild
{
  ...
  virtual bool RecvInit(bool* windowless, bool* ok) = 0;
}

RPC semantics

"RPC" stands for "remote procedure call," and this third semantics models procedure call semantics. A quick summary of the difference between RPC and sync semantics is that RPC allows "re-entrant" message handlers: while an actor is blocked waiting for an "answer" to an RPC "call", it can be unblocked to handle a new, incoming RPC call.

In the example protocol below, the child actor offers a "CallMeCallYou()" RPC interface, and the parent offers a "CallYou()" RPC interface. The rpc qualifiers mean that if the parent calls "CallMeCallYou()" on the child actor, then the child actor, while servicing this call, is allowed to call back into the parent actor's "CallYou()" message.

rpc protocol Example {
child:
    rpc CallMeCallYou() returns (int rv);

parent:
    rpc CallYou() returns (int rv);
};

If this were instead a sync protocol, the child actor would not be allowed to call the parent actor's "CallYou()" method while servicing the "CallMeCallYou()" message. (The child actor would be terminated with extreme prejudice.)

Preferred semantics

Use async semantics whenever possible.

Blocking on replies to messages is discouraged. If you absolutely need to block on a reply, use sync semantics very carefully. It is possible to get into trouble with careless uses of synchronous messages; while IPDL can check and/or guarantee that your code does not deadlock, it is easy to cause nasty performance problems by blocking.

Please don't use RPC semantics. RPC semantics exists mainly to support remoting plugins (NPAPI), where we have no choice.

Message Delivery Order

Delivery is "in-order", that is, messages are delivered to the receiver in the order that they are sent, regardless of the messages' semantics.  If an actor A sends messages M1 then M2 to actor B, B will be awoken to process M1 then M2.

Subprotocols and Protocol Management

So far we've seen a single protocol, but no real-world situation would have a single protocol in isolation. Instead, protocols are arranged in a managed hierarchy of subprotocols. A sub-protocol is bound to a "manager" which tracks its lifetime and acts as a factory. A protocol hierarchy begins with a single top-level protocol from which all subprotocol actors are eventually created. In Mozilla there are two main top-level protocols: PPluginModule for remote plugins, and PContent for remote tabs.

The following example extends the toplevel plugin protocol to manage plugin instances.

// ----- file PPlugin.ipdl

include protocol PPluginInstance;

rpc protocol PPlugin
{
    manages PPluginInstance;
child:
    rpc Init(nsCString pluginPath) returns (bool ok);
    rpc PPluginInstance(nsCString type, nsCString[] args) returns (int rv);
};

// ----- file PPluginInstance.ipdl

include protocol PPlugin;

rpc protocol PPluginInstance
{
    manager PPlugin;
child:
    rpc __delete__();
    SetSize(int width, int height);
};

This example has several new elements: `include protocol` imports another protocol declaration into this file. Note that this is not a preprocessor directive, but a part of the IPDL language. The generated C++ code will have proper #include preprocessor directives for the imported protocols.

The `manages` statement declares that this protocol manages PPluginInstance. The PPlugin protocol must declare constructor and destructor messages for PPluginInstance actors. The `manages` statement also means that PPluginInstance actors are tied to the lifetime of the Plugin actor that creates them: if this PPlugin instance is destroyed, all the PPluginInstances associated with it become invalid or are destroyed as well.

The mandatory constructor and destructor messages (PPluginInstance and __delete__ respectively) exist, confusingly, in separate locations.  The constructor must be located in the managing protocol, while the destructor belongs to the managed subprotocol.  These messages have syntax similar to C++ constructors, but the behavior is different. Constructors and destructors have parameters, direction, semantics, and return values like other IPDL messages. A constructor and destructor message must be declared for each managed protocol.

Each subprotocol must include a `manager` statement.

At the C++ layer, the subclasses in both the child and the parent must implement methods for allocating and deallocating the subprotocol actor. The constructor and destructor are translated into standard C++ methods for messages.

Note: __delete__ is a built-in construct, and is the only IPDL message which does not require an overridden implementation (ie. Recv/Answer__delete__).  However, overridden implementations are encouraged when some action should happen on protocol destruction in lieu of using the DeallocPProtocol function.

class PPluginParent
{
  /* Allocate a PPluginInstanceParent when the first form of CallPluginInstanceConstructor is called */
  virtual PPluginInstanceParent* AllocPPluginInstance(const nsCString& type, const nsTArray<nsCString>& args, int* rv) = 0;

  /* Deallocate the PPluginInstanceParent after PPluginInstanceDestructor is done with it */
  virtual bool DeallocPPluginInstance(PPluginInstanceParent* actor) = 0;

  /* constructor message */
  virtual CallPPluginInstanceConstructor(const nsCString& type, const nsTArray<nsCString>& args, int* rv) { /* generated code */ }

  /* alternate form of constructor message: supply your own PPluginInstanceParent* to bypass AllocPPluginInstance */
  virtual bool CallPPluginInstanceConstructor(PPluginInstanceParent* actor, const nsCString& type, const nsTArray<nsCString>& args, int* rv)
  { /* generated code */ }

  /* destructor message */
  virtual bool Call__delete__(PPluginInstanceParent* actor) { /* generated code */ }

  /* Notification that actor deallocation is imminent, IPDL mechanisms are now unusable */
  virtual void ActorDestroy(ActorDestroyReason why);

  ...
};

class PPluginChild
{
  /* Allocate a PPluginInstanceChild when we receive the PPluginInstance constructor */
  virtual PPluginInstanceChild* AllocPPluginInstance(const nsCString& type, const nsTArray<nsCString>& args, int* rv) = 0;

  /* Deallocate a PPluginInstanceChild after we handle the PPluginInstance destructor */
  virtual bool DeallocPPluginInstance(PPluginInstanceChild* actor) = 0;

  /* Answer the constructor message. Implementing this method is optional: it may be possible to answer the message directly in AllocPPluginInstance. */
  virtual bool AnswerPPluginInstanceConstructor(PPluginInstanceChild* actor, const nsCString& type, const nsTArray<nsCString>& args, int* rv) { }

  /* Answer the destructor message. */
  virtual bool Answer__delete__(PPluginInstanceChild* actor) = 0;

  /* Notification that actor deallocation is imminent, IPDL mechanisms are now unusable */
  virtual void ActorDestroy(ActorDestroyReason why);

  ...
};

Subprotocol Actor Lifetime

AllocPProtocol and DeallocPProtocol are a matched pair of functions. The typical implementation of these functions uses `new` and `delete`:

class PluginChild : PPluginChild
{
 virtual PPluginInstanceChild* AllocPPluginInstance(const nsCString& type, const nsTArray<nsCString>& args, int* rv)
  {
    return new PluginInstanceChild(type, args, rv);
  }

  virtual bool DeallocPPluginInstanceChild(PPluginInstanceChild* actor)
  {
    delete actor; // actor destructors are always virtual, so it's safe to call delete on them!
    return true;
  }

  ...
};

In some cases, however, external code may hold references to actor implementations which require refcounting or other lifetime strategies. In this case, the alloc/dealloc pairs can perform different actions. Here is an example of refcounting:

class ExampleChild : public nsIObserver, public PExampleChild { ... };

virtual PExampleChild* TopLevelChild::AllocPExample()
{
  nsRefPtr<ExampleChild*> actor = new ExampleChild();
  return actor.forget();
}

virtual bool TopLevelChild::DeallocPExample(PExampleChild* actor)
{
  NS_RELEASE(static_cast<ExampleChild*>(actor));
  return true;
}

If an object that implements a protocol can't be constructed inside AllocPFoo, or has been previously constructed and doesn't require an IPDL connection throughout its lifetime, there is a second form of SendPFooConstructor which can be used:

class ExampleChild
{
public:
    void DoSomething() {
        aManagerChild->SendPExampleConstructor(this, ...);
    }
};

Internally, the first constructor form simply calls

PExample(Parent|Child)* actor = AllocPExample(...);
SendPExampleConstructor(actor, ...);
return actor;

with the same effect.

Subprotocol Deletion

It is worth understanding the protocol deletion process.  Given the simple protocols:

// --- PExample.ipdl
include protocol PSubExample;

async protocol PExample
{
    manages PSubExample;

parent:
    PChild();
};

// --- PSubExample.ipdl
include protocol PExample;

async protocol PSubExample
{
    manager PExample;

child:
    __delete__();
};

We assume that there is a PSubExampleParent/Child pair in existence, such that some element now wishes to trigger the protocol's deletion from the parent side.

aPSubExampleParent->Send__delete__();

will trigger the following ordered function calls:

PSubExampleParent::ActorDestroy(Deletion)
/* Deletion is an enumerated value indicating
   that the destruction was intentional */
PExampleParent::DeallocPSubExample()

PSubExampleChild::Recv__delete__()
PSubExampleChild::ActorDestroy(Deletion)
PExampleChild::DeallocPSubExample()

ActorDestroy is a generated function that allows code to run with the knowledge that actor deallocation is imminent.  This is useful for actors with lifetimes outside of IPDL - for instance, a flag could be set indicating that IPDL-related functions are no longer safe to use.

Accessing the protocol tree from C++

The IPDL compiler generates methods that allow actors to access their manager (if the actor isn't top-level) and their managees (if any) from C++.  For a protocol PFoo managed by PManager, that manages PManagee, the methods are

PManager* PFoo::Manager()
const InfallibleTArray<PManagee*> PFoo::ManagedPManagee();
void PFoo::ManagedPManagee(InfallibleTArray<PManagee*>&);

Shutdown and Error Handling

The C++ methods which implement IPDL messages return bool: true for success, and false for catastrophic failure. Message implementations should return false from a message implementation if the data is corrupted or otherwise malformed. Any time a message implementation returns false, IPDL will immediately begin catastrophic error handling: the communication channels for the child process (tab or plugin) will be disconnected, and the process will be terminated. Do not return false from message handlers for "normal" error conditions such as inability to load a network request! Normal errors should be signaled with a message or return value.

Note: the following paragraphs are not yet implemented. IPDL tracks all active protocols between two endpoints. If if the child side crashes or becomes hung:

• any synchronous or RPC messages currently active will return false
• no further messages will be accepted (C++ methods will return false)
• each IPDL actor will receive an OnError message
• DeallocPSubprotocol will be called on each manager protocol to deallocate any active subprotocols.

When a manager protocol is destroyed, any subprotocols will be notified:

• no further messages will be accepted
• DeallocPSubprotocol will be called on the manager protocol to deallocate any active subprotocols

When the toplevel protocol is destroyed, this is equivalent to shutting down the entire IPDL machinery for that connection, because no more messages can be sent and all subprotocols are destroyed.

Protocol state machines

The astute reader might question why IPDL includes the word "protocol" when all that has been introduced so far are unstructured grab-bags of messages. IPDL allows protocol authors to define the order and structure of how messages may be sent/received by defining protocol state machines (finite state machines).

[Note that the state machine portion of the IPDL compiler is not complete as of this writing, 22 October 2009. IPDL code for state machines is accepted by the compiler, but it does not affect the generated C++, yet.]

IPDL parent and child actors communicating via a protocol are paired. Each actor in the pair follows the same state machine. The pair attempts to keep their single collective state synchronized. Though, it is possible that the parent and child actors may be momentarily out of sync while messages are transmitted.

IPDL (arbitrarily) requires state machines to be defined from the perspective of the parent side of the protocol. For example, when you see the send Msg syntax, it means "when the parent actor sends Msg".

The following example shows one such state machine for the Plugin protocol.

include protocol PPluginInstance;

sync protocol PPlugin {
  manages PPluginInstance;

child:
  sync Init() returns (int rv);
  Deinit();

  sync PPluginInstance(String type, StringArray args) returns (int rv);

// NOTE: state machine follows
state START:
  send Init goto IDLE;

state IDLE:
  send PPluginInstance goto ACTIVE;

state ACTIVE:
  send PPluginInstance goto ACTIVE;
  send ~PPluginInstance goto ACTIVE;
  send Deinit goto DYING;

state DYING:
  send ~PPluginInstance goto DYING;
};

There are three new syntactic elements, above. First are "state declarations": the code state FOO: declares a state "FOO". (States are capitalized by convention, not because of syntactic rules.) The first state to be declared is the protocol's "start state"; when an actor is created, its initial state is the "start state."

The second new syntactic element is the trigger. The syntax send MsgDecl defines a trigger for a state transition; in this case, the trigger is sending the async or sync message "MsgDecl." The triggers are:

1. sending an async or sync message
2. recving an async or sync message
3. calling an RPC
4. answering an RPC

Aside: this is why actor ctors/dtors act like normal messages, with directions etc.: this allows them to be checked against the protocol state machine like any other message.

The third new syntactic element is a state transition. The syntax is: goto NEXT_STATE. When the trigger preceding this transition occurs, the protocol actor's internal state is changed to, in this case, "NEXT_STATE."

Another example state machine, for PluginInstance, follows.

sync protocol PluginInstance {
  manager Plugin;

child:
  SetWindow(PluginWindow window);
  Paint();

parent:
  sync GetBrowserValue(String key) returns (String value);

state START:
  send SetWindow goto SENT_WINDOW;
  recv GetBrowserValue goto START;

state SENT_WINDOW:
  send SetWindow goto SENT_WINDOW;
  send Paint goto SENT_WINDOW;
  recv GetBrowserValue goto SENT_WINDOW;
};

Note:

• The following points will apply when the IPDL compiler fully supports states. It is incomplete as of this writing, on 22 October 2009.
• Protocol state machines are optional, but strongly encouraged. Even simple state machines are useful.
• All actor states, trigger matching, and transitions are managed by IPDL-generated code. Your C++ code need not manage these things.
• All messages sent and received are checked against the protocol's state machine. If a message violates the state machine semantics defined in the IPDL code, the child process containing the child actor will be terminated with extreme prejudice, and all parent actors will be made invalid! It is up to the developer to make sure that this never happens.
• Lots of syntactic sugar is possible for state machine definitions. Ping the Electrolysis team if you have a good proposal