Mercurial Queues

Mercurial Queues, or MQ, is a Mercurial extension that lets you keep work in progress as mutable patches, instead of as immutable changesets.

Before you start

You can destroy work with MQ. MQ puts you in a position where you're doing fairly complicated stuff to your uncommitted work. Certain operations make it easy to lose work. Watch your step.

For instance, unless you're running the mqext, hg qrefresh is destructive. It replaces your previous version of the current patch with what's in your working directory. The previous version is lost. Other things to keep in mind:

• Don't use MQ in a repository anyone might pull from. MQ creates temporary changesets in your repo. If someone pulls one of them, you'll never get rid of it.
• Avoid the -f option. It is sharp and can mess up your repository if used incorrectly.
• Ensure you use the git diff format. Otherwise you will lose any changes to binary files.
• Ensure you use the latest stable release of Mercurial.
• Version your patch queue to save changes. The mqext extension can make this much easier.

Introduction

The output of a developer (on a good day, anyway) is patches. The MQ extension lets you treat a stack of patches as works-in-progress. You can apply them as Mercurial changesets, unapply them, edit them, and when they're done, turn them into permanent changesets and push them.

Each repository has its own queue of patches managed by MQ. They're just stored as files in the .hg/patches directory under the repository.

To enable MQ, put this in your Mecurial.ini file for Windows (see mozilla-build in Windows Build Prerequisites)  or $HOME/.hgrc file:

[extensions]
mq =

[diff]
git = 1
unified = 8
showfunc = 1

Don't forget the git line. This allows changing binary files in your patches. The unified line give 8 lines patch.

Basic Commands

The basic MQ commands are:

How to use MQ for Mozilla development

The following commands demonstrate how to create an MQ entry for a single bug fix, record the changes you make, and turn them into a diff file named "bug-123456-fix.patch" that is ready to be attached to the corresponding Bugzilla bug.

$ hg qnew bug-123456-fix

... change some files ...

$ hg qrefresh

... change some more files ...

$ hg qrefresh -m "Bug 123456: A brief summary of the changes you have made."
$ hg export qtip > ~/bug-123456-fix.patch

While the patch is being reviewed, you might want to work on another bug. The process is exactly the same:

$ hg qnew bug-341896-fix

... change some files ...

$ hg qrefresh -m "Bug 341896: The fix, summarized briefly."
$ hg export qtip > ~/bug-341896-fix.patch

If you look at the new patch file you have exported, you'll notice that it only contains the changes for your bug-341896-fix mq entry. MQ allows you to keep your unrelated changes isolated from each other. hg qseries demonstrates this:

$ hg qseries -v -s
0 A bug-123456-fix: Bug 123456: A brief summary of the changes you have made.
1 A bug-341896-fix: Bug 341896: The fix, summarized briefly.

This shows all of the patches you have in your queue. Let's say that the your fix for bug 123456 is reviewed, and you need to make a couple changes. MQ makes it easy to go back and fix up earlier work:

$ hg qpop
popping bug-341896-fix
now at: bug-123456-fix

$ hg qseries -v -s
0 A bug-123456-fix: Bug 123456: A brief summary of the changes you have made.
1 U bug-341896-fix: Bug 341896: The fix, summarized briefly.

Using qpush and qpop, you can apply and revert your patches until the one you want to modify is current.

... make the necessary changes ...
$ hg qrefresh
$ hg export qtip > ~/bug-123456-fix-v2.patch

If you compare bug-123456-fix.patch and bug-123456-v2.patch, you'll see your most recent changes reflected in the second. You can now qpush your way back to bug-341896-fix if you want to keep working on it, or qnew yourself an entirely new patch to work on!

The last important step is updating your mozilla-central repository. It is very dangerous to pull remote changes while you have MQ patches applied (unless you are using the rebase extension). Instead, you should pop them all before updating, like so:

$ hg qpop -a
popping bug-341896-fix
popping bug-123456-fix
patch queue now empty
$ hg pull -u
...
$ hg qpush bug-341896-fix
applying bug-123456-fix
patching file extensions/cookie/test/test_loadflags.html
Hunk #1 FAILED at 5
1 out of 1 hunks FAILED -- saving rejects to file extensions/cookie/test/test_loadflags.html.rej
errors during apply, please fix and refresh patch

It turns out that the recent changes to mozilla-central have modified the same files as your bug-123456-fix patch. If you haven't setup a merge tool, you'll need to open the files that had conflicts, fix up the bits of your patch that were in conflict (look at the .rej files that are listed), and hg qrefresh the patch to save your changes.

$ hg qrefresh
$ hg qpush
applying bug-314896-fix
now at: bug-314896-fix
$ hg qpop -a
popping bug-341896-fix
popping bug-123456-fix
patch queue now empty
$ hg qpush
applying bug-123456-fix
now at: bug-123456-fix

Your patches that conflicted should now apply cleanly until the next time you update and a conflict occurs. You can learn more about advanced MQ usage from the reference page, but you should now know enough to be able to use MQ effectively for Mozilla development work.

Bugzilla integration

Rob Arnold has made a Mercurial extension called qimportbz that makes it much easier to transfer patches between Bugzilla and your patch queue. See his blog post for more details. (Since that post was written, the command syntax has changed to: hg qimport bz:1234567)

Ted Mielczarek has made a tool that works the other way: bzexport can take patches from your patch queue and attach them to an existing bug in Bugzilla. See his blog post for details.

Reference

hg qnew patch-name

Make a new empty patch and give it a name. The new patch is applied when it's created. That means it's a changeset. If you do hg log -r . you'll see it. All the other hg commands, like hg annotate and hg grep, see it as a regular changeset.

Note: If you want to create a patch from a bugzilla attachment or other URL, you actually want qimport not qnew.

hg qrefresh

Update the current patch to include your latest uncommitted changes. You'll do this often.

• qrefresh -e will open an editor to let you set the commit message for a patch.
• qrefresh -m "My commit message" will set the commit message directly from the command line.

hg commit --mq -m "backup"

Save a snapshot of your .hg/patches directory as a Mercurial revision in the .hg/patches repository. Do this often, too. This does not save your uncommitted work. Use qrefresh to put edits into patches and commit --mq to save snapshots of those patches. The mqext extension should make managing this mostly automatic.

hg qpop

Unapply a patch. This removes the changeset. The patch is set aside in your .hg/patches directory. You can do other work, then re-apply your patch later.

• hg qpop -a will unapply all of the applied patches in a queue.

hg qpush

Apply the next unapplied patch. If the patch is out of date—that is, if you've done hg update since the last time you refreshed it—then you might get rejects! The only thing to do in that case is to apply the rejects manually and hg qrefresh.

• hg qpush patch_name will apply patches until the desired patch is on top.
• hg qpush -a will apply all the patches in the queue (until a merge conflict is found).

hg qdelete patchname

Throw an unapplied patch away. This removes the patch from your .hg/patches directory (and from MQ's oversight generally).

• hg qdelete -r qbase will commit a patch as a permanent changeset, the same as hg qfinish.

hg qfinish

"Finish" the bottommost applied patch. It becomes a permanent changeset, suitable for pushing, using the commit message set with qrefresh. You can only finish patches from the bottom (i.e. you can't finish a patch that's applied over unfinished patches).

• hg qfinish -a  (or hg qdelete -r qbase:qtip) will finish all applied patches. See also #Reordering the queue below.

hg qimport filename_or_url

Import a patch into your queue, e.g. from Bugzilla. It is unapplied by default and the filename_or_url is the patchname. You can directly import a Bugzilla patch by using the Bugzilla attachment URL as the argument. In that case you may also want to use -n patch_name to specify the patch name.

• hg qimport -r tip with no mq patches applied, will reimport the last committed changeset into your mq. This is useful when, e.g., you forget to set a commit message for a patch before qfinishing it.
• hg qimport filename -n name will allow you to name the patch you are importing.

Just as important are the commands that show you what's going on:

hg status

Show you only uncommitted edits — work that's neither committed nor in an MQ patch. After a qrefresh, they'll both be empty.

hg qdiff

Show what the topmost applied MQ patch looks like, including any uncommitted edits you've made.

• hg qdiff -p -U 8 will produce patches for review.
• hg diff -r qparent will export all applied patches in your queue as a single patch, without disturbing your queue.

hg qseries -v

List all MQ patches. Applied patches are listed as A, and unapplied patches as U.

hg qapplied

List all the MQ patches that are applied, in queue order (that is, the order they were applied).

hg qunapplied

List all the MQ patches that aren't applied, in queue order (that is, the order they would be applied, if you did hg qpush -a).

hg log -r qbase:qtip

List all applied patches, just like hg qapplied, but also show the changesets. (qbase and qtip are tags provided by MQ. They always point to the bottom-most and topmost applied patch.)

To see more advanced commands, use hg help.

Advanced topics

Rebasing patches manually

Rebasing for dummies. When you write a patch, it's based on whatever version of the repo you had when you started working. Of course, changes are constantly being pushed to the central repository, so by the time you want to push, you'll be out of date. Your changes are based on an old revision.

       $ hg glog --template '{node|short} - {author} - {desc|firstline}\n' 
TIP -> o  cd9f8db8ee0f - Devin Naquin <[email protected]> - Bug 383223
       |
       o  103f04f54b14 - L. David Baron <[email protected]> - Tests for z-ordering of text-decorations.
       |
       o  e3a4c136455b - Robert O'Callahan <[email protected]> - Support system proxy settings on OS X, fixi
       |
       o  745e0f997344 - Robert O'Callahan <[email protected]> - Support system proxy settings on OS X, fixi
       |
       o  9d80a1461309 - Robert O'Callahan <[email protected]> - Support system proxy settings on OS X, fixi
       |
       o  4721deb1dd19 - Diane Trout <[email protected]>, James Bunton <[email protected]> - Support syst
       |
       o  3e166c19d130 - Michael Ventnor <[email protected]> - text-shadow tests (bug 10713)
       |
       o  c06307605f98 - Michael Ventnor <[email protected]> - Implement text-shadow rendering.
       |
YOU ---->@  d243d3af29ed - Jason Orendorff <[email protected]> - [mq] Implement trebled fromps.
       |/
       o  1df6e4240511 - Shawn Wilsher <[email protected]> - Bug 429987
       |
       .
       .
       . (the past)

You have two choices:

• Go ahead and finish your patch now, then hg merge with the tip; or
• Rebase your patch before you finish and push it.

Rebasing is considered the polite thing. Merging leaves a merge changeset in the history.

It's best not to hg pull while you have patches applied. The most foolproof way to pull and update is:

$ hg qpop -a             # Unapply all patches
$ hg pull
$ hg update
$ hg qpush patchname     # Reapply patches -- watch out for rejects!

Rebasing for smarties. If you're used to Mercurial and MQ and you dislike .rej files, you might want to consider MqMerge. This technique lets you rebase using your merge program, but it's a bit complex.

Reordering the queue

Sometimes the queue ends up not being in the order you want. For example, maybe you've been working on two patches, and the second one (the topmost one in your queue) is ready to be pushed before the first one is.

If you have Mercurial 1.6 or newer, the best way to reorder your queue is hg qpush --move. For example:

$ hg qpop -a                   # Unapply all patches
$ hg qpush --move patchname    # Apply only one patch, reordering as needed

With older Mercurial versions, you can do this:

$ hg qpop -a                   # Unapply all patches
$ $EDITOR .hg/patches/series   # Rearrange the lines of the series file
$ hg qpush patchname           # Reapply patches -- watch out for rejects!

Folding multiple patches into one

The hg qfold command allows you to merge a patch into another one:

$ hg qpush --move my-first-patch      # apply my first patch
$ hg qfold my-second-patch     # fold second patch into it

Splitting a patch, the easy case: per-file splitting

If you have a patch that modifies file1 and file2, and you want to split it into two patches each modifying only one file, do:

$ hg qgoto my-patch
$ hg qref -X path/to/first/file # take changes out of current patch and back into `hg diff`
$ hg qnew -f patch-modifying-first-file # and take that into a new MQ patch

Here, the qref -X command takes the changes to the first file out of the patch, so that they now show up in hg diff and therefore get picked up by the hg qnew.

Splitting a patch: the general case, including per-hunk and per-line splitting

If you need to perform finer patch splitting, for example per-hunk or even per-line, there's a great tool for that: hg qcrecord. It's provided by the Crecord extension. Follow the instructions on that page to install it. Update your Mercurial.ini/.hgrc as follows to preserve authorship information.

[defaults]
qcrecord = -Ue

The qcrecord command creates a new patch in your queue from the changes in your working directory (as shown by hg diff) - it does not edit existing mercurial queue patches directly. To split an existing mercurial queue patch you must first move the changes out from the patch and into your working directory, then delete the (now empty) patch from your queue (since qcrecord will create a new patch, not add to that old patch). Say you have a patch in your queue called 'my-patch' that you want to split it into 'my-patch' and 'my-other-patch', you would do something like this:

$ hg diff                  # Check there are no local changes in the working directory
$ hg qgoto my-patch        # Make sure my-patch is the most recently applied
$ hg qref -X .             # Move all changes out of my-patch and into the working directory
$ hg qpop -f               # Pop my-patch from the queue so that we can delete it, leaving the changes in the working directory
$ hg qdelete my-patch      # Delete my-patch (which is now empty) from the queue
$ hg qcrecord my-patch     # Select the pieces of the diff that you want in my-patch, and save
$ hg qnew my-other-patch   # Save the remaining pieces of the original patch to the queue as 'my-other-patch'

Or, with the latest version of the Crecord extension, that can be simplified to:

$ hg diff                  # Check there are no local changes in the working directory
$ hg qgoto my-patch        # Make sure my-patch is the most recently applied
$ hg qcrefresh             # Select the pieces of the diff that you want to keep in my-patch
$ hg qnew my-other-patch   # Save the remaining pieces of the original patch to the queue as 'my-other-patch'

When you invoke qcrecord, it will open a console-based dialog allowing you to select file-by-file, hunk-by-hunk, and even line-by-line, what changes you want to record into the patch that qcrecord will create. When you first launch hg qcrecord, it shows you a list of modified files:

SELECT CHUNKS: (j/k/up/dn/pgup/pgdn) move cursor; (space/A) toggle hunk/all
 (f)old/unfold; (c)ommit applied; (q)uit; (?) help | [X]=hunk applied **=folded
[X]**M hello.cpp

Pressing 'f' toggles between folding and unfold (collapsing and expanding) the diff of hello.cpp:

SELECT CHUNKS: (j/k/up/dn/pgup/pgdn) move cursor; (space/A) toggle hunk/all
 (f)old/unfold; (c)ommit applied; (q)uit; (?) help | [X]=hunk applied **=folded
[X]    diff --git a/hello.cpp b/hello.cpp
       2 hunks, 4 lines changed

   [X]     @@ -1,4 +1,5 @@
            #include <iostrea>
      [X]  +#include <cmath>
           #include <cstdlib>

           double square(double x)

   [X]     @@ -8,5 +9,6 @@

            int main()
            {
      [X]  -  std::cout << square(3.2) << std::endl;
      [X]  +  double x = 2.0;
      [X]  +  std::cout << std::sqrt(square(x)) << std::endl;
           }

This allows us to select the lines to record in the patch. When we're done, we press 'c'.

See also

• Managing change with Mercurial queues
• Sharing patch repositories useful as a backup and for collaboration

Wanted

• Using MQ to thaw/edit/refreeze history
• guards, maybe
• multiple queues, maybe
• rebase extension instructions (hg pull --rebase)