Customizing the keyboard in B2G OS

Status of the Gaia Built-in Keyboard app & system architecture

Two major keyboard features have been implemented in Firefox OS, as of October 2013.

• A build flag to turn keyboard/spellcheck dictionaries to include in the build in the default Gaia Keyboard app on and off (see bug 884752).
• An OS-level feature that allows users to install third-party keyboard apps from the Firefox Marketplace. This will be available starting from Firefox OS v2.0. See bug 816869 and bug 964670 for details.

To include a new keyboard layout on your phone, you could put a custom layout file in Gaia and build the keyboard app using a build config, or develop your own keyboard app and install it on the phone. Let's have a look at how to do each of these.

Developing your own keyboard app

You can use Gaia's built-in keyboard app as a template, or fork it. The basic ideas are:

• In the manifest file, declare your app as having a role of input in the role property.
• Make sure you have the input permission specified in the permissions property.

After that, you can install your keyboard, and it will show up on the bottom half of the screen whenever the user is focused to an input field. The space issues that arise from this can be dealt with by:

• Using the old, classic window.resizeTo to update the area your app should occupy.
• Using the new, shiny navigator.mozInputMethod API to mutate the content of the input given; see the WebIDL of the API for details. Many of these methods output the same characters but generate different events (KeyboardEvent, CompositionEvent, etc), so please ensure you are using the right method for your use cases.

Including a custom layout in the Gaia keyboard app

1. See the keyboard CONFIGURE details for more information. Depending on the language you might want to add a dictionary, or include an interactive IME (for Asian languages).
2. If you add layouts for a newly-supported language, you may want to add them as default layouts for that language in build/config/keyboard-layouts.json.

Asian IME

East Asian languages (Chinese, Japanese, and Korean) use a fairly complex script that involves thousands of characters. As there is no way to put thousands of keys on a hardware or software keyboard, users rely on an indexing program called input method editor, or IME for short, to convert a series of symbols into selections of characters.

How symbols should be converted and how "smart" the IME should be depends on the method chosen, the implementation itself, and often NLP science, which is outside the scope of this document. To conserve memory, it is advised that data should be compiled into a binary blob and read out as an array buffer; complex JSON structures will not work on actual devices.

IME interface in Gaia keyboard app

Assuming you have an IME library readily available for converting symbols into characters (''IM Engine'' in our terminology), you would need to use the defined interface within the Gaia Keyboard app to receive symbols from the layout and output the characters. The current API is documented on Github.

We invite implementations for further refining this API within Gaia.

Make your IM engine work with your layout

To get your IM Engine working with your keyboard layout:

1. Introduce a new layout for your IM Engine, as described below.
2. Reference the IM engine with the imEngine property. When your layout is enabled, the keyboard app will start loading the script located at keyboard/js/imes/<imEngine>/<imEngine>.js.
3. Set up the APIs to receive and send the keys/characters when the script is loaded and init'd.
4. Do whatever you want in response to the user input. When the user taps a key on the keyboard, it will typically get sent to the IM Engine.

Known implementations

• JSZhuyin is the first implementation of a Gaia Keyboard Asian IM Engine for Traditional Chinese, incorporating Zhuyin IME.
• JSKanji is a prototype IM Engine written for Japanese.
• JSPinyin is written for Simplified Chinese and works by tapping Pinyin symbols.
• As a demonstration, Chewing, an C/C++ open-source Zhuyin IME for Traditional Chinese, was cross-compiled with Emscripten and linked with the Gaia keyboard.
• JSHangul is a prototype IM Engine for Korean, and is currently in development.

New locales from the localizers perspective

Generally two things are required to add a new locale: a keyboard layout and a lexicon (for the predictive text feature).

Creating a keyboard layout

The keyboard layout is contained in a JavaScript file and defines which keys are on the keyboard and which of them result in which letters and symbols, including the secondary options available through long presses. The most straightforward way is to grab an existing keyboard from GitHub, rename it and then adjust it.

First you need to change the header. Let's says your keyboard is close to en-US and the locale you want is Scottish Gaelic (gd). You would change this:

Keyboards.en = {
 label: 'English',
 shortLabel: 'En',
 imEngine: 'latin',
 types: ['text', 'url', 'email', 'number', 'password'],
 autoCorrectLanguage: 'en_us',
 menuLabel: 'English',

to this:

Keyboards.gd = {
 label: 'Scottish Gaelic',
 shortLabel: 'gd',
 imEngine: 'latin',
 types: ['text', 'url', 'email', 'number', 'password'],
 autoCorrectLanguage: 'gd_gb',
 menuLabel: 'Gàidhlig',

the label field should be the name of your language in English, the shortLabel field should be three characters long at the most; it's an abbreviation shown on the onscreen keyboard so the user knows which language they're currently using. You can use the ISO code but you can use something else too if appropriate. The menuLabel field is the name of your language written in your language: this is what the user sees in the menu options.

Now if the layout is close, the only thing you are likely to tweak are the alternative characters (which you access in the UI via a longpress):

alt: {
  a: 'áàâäåãāæ',
  c: 'çćč',
  e: 'éèêëēę€ɛ',
  i: 'ïíìîīį',
  o: 'öõóòôōœøɵ',
  u: 'üúùûū',
  s: 'ßśš$',
  S: 'ŚŠ$',
  n: 'ñń',
  l: 'ł£',
  y: 'ÿ¥',
  z: 'žźż',
  '.': ',?!;:'
},

You can change both the order and number of characters for the long press. In this case, because Gaelic uses the grave more often than the acute, we changed the order to:

a: 'àáâäåãāæ',

We also wanted to give people the option of using the dotted consonants, so we added new lines like this one:

b: 'ḃ',

You probably also want to check the alternative layout (for numbers, punctuation, etc.) and you probably also want to check that the right currency symbol is displayed.

Creating a lexicon

It's better to have a lexicon too because otherwise users have to type out each word and don't get any word predictions. For this you need (ideally) some form of corpus to rank your words according to frequency. For example, the word there is a lot more common than thespian so when the user types the, there should be suggested before thespian.

This is done by assigning numbers from 1 (least frequent) to 255 (most frequent) to each word (aka line) in the lexicon (an XML file; you can have a look at the existing ones here). This is what you need to create:

<wordlist locale="gd" description="Gàidhlig" date="1401554807" version="1">
 <w f="255" flags="">a</w>
 <w f="254" flags="">an</w>
 <w f="247" flags="">agus</w>

Even a quick and dirty corpus (for example made by grabbing a large text and counting the frequencies) will help. If there is no such data at all for your locale and not much in the way of digital texts, use your knowledge of the language and give some manual rankings to the words you know from experience to be common.

Building the files

Once you have the js and the lexicon file, go in this gaia directory and open Makefile. Append the name of your dictionary to the list (e.g., if it's called gd_wordlist.xml), put:

ga.dict \
gd.dict \

Make sure the gd_worldlist.xml file is in the folder and run $ make gd.dict

If you have gotten this far but get stuck at the last step of building the .dict file, file a bug (Component: Gaia Keyboard) and attach your files and explain what the problem is. Someone should be able to help. Make sure to submit the patch as a pull request, and flag for review to someone, otherwise the bug will be likely unattended.

Testing your new locale

Once you have built and committed the files for the new locale, the quickest way of testing the new keyboard and lexicon is our web-based demo page that runs the keyboard app. Click on En to cycle through the locales. If you can't see yours, click the spanner symbol (top right) and tick/untick the keyboards you want to test.

Discussion and QA

Please come to the dev-gaia mailing list or the #gaia channel on irc.mozilla.org (see Mozilla IRC for more details). #mozilla-taiwan is the de-facto Chinese-language channel for this topic.