Completion annotations

,

One of the most useful features of Emacs is its interactive completion. Whenever you need to input the name of some item, Emacs allows you to type some parts of the name and then type TAB to see a list of items that match what you have typed so far. The screenshot shows the simplest form of match: here the user has typed M-x set-cursor-color RET Dark TAB and is shown a list of colour names that match the prefix ‘Dark’. But completion is more sophisticated than just prefix-matching. You can use wildcards (for example *green TAB will show colour names containing the substring ‘green’) and when the completion items have multiple words (separated by hyphens or spaces), completion works on each word individually, so that you can type M-x t-d-o-e RET to run the command toggle-debug-on-error. Naturally you can customize the set of completion mechanisms using the completion-styles setting. The completion system makes it straightforward to select:

The Emacs/Perforce integration that I maintain, p4.el, implements completion for each kind of Perforce item that you might need to name: branches, numbered pending changelists, clients, filespecs, groups, help topics, jobs, labels, and users. However, this works poorly in the case of changelists and jobs, because all you get when you type TAB is a list of opaque identifiers that give you no help in choosing the item you want. See the screenshot, which shows the result of typing C-u M-x p4-job RET TAB.

What is needed here is a description or annotation next to each completion in the *Completions* buffer. And this looks as if it ought to be easy to implement, because the function that displays the list of completions apparently supports exactly this feature:

display-completion-list completions

This function displays completions to the stream in standard-output, usually a buffer. The argument completions is normally a list of completions just returned by all-completions, but it does not have to be. Each element may be a symbol or a string, either of which is simply printed. It can also be a list of two strings, which is printed as if the strings were concatenated. The first of the two strings is the actual completion, the second string serves as annotation.

But how do you use this feature? The trouble is that user commands implemented in Emacs Lisp almost never call display-completion-list directly. Instead, they call completing-read, passing either:

  1. a static table of completions; or
  2. a dynamic completion function that is called when the user types TAB and which returns a table of completions matching what has been typed so far. The required interface here is quite complex, so usually you build this using the helper function completion-table-dynamic.

Either way, by time the table completions reach display-completion-list, the annotations (if you tried adding them) have been stripped. Try it yourself! Here’s code for case 1 (the static table of completions): 

(defvar my-completions '(("a" "description of a") ("b" "b's description")))

(completing-read "Prompt: " my-completions)

and for case 2 (the dynamic completion function): 

(completing-read "Prompt: " (completion-table-dynamic (lambda (s) my-completions)))

In neither case do you get to see the annotations. So how do you add annotations to completion items? I can’t be the only person who needs this feature, so perhaps there are third-party libraries implementing it? I searched on the Internet and found completion-help.el by Yuji Minejima. But this is not suitable for the p4.el use case because:

  1. It’s an extra library, which would complicate installation. (It’s released under the GPL, so in theory I could copy the parts I need, and rename them so that they don’t clash if a user has both p4.el and completion-help.el installed.)
  2. It provides a global user setting for turning on annotations for all completions, whereas I need a dynamic programmable setting for turning on annotations for the particular cases where I want it. (Again, in theory I could copy the parts I need and change the way the setting works.)
  3. It has a hairy implementation using defadvice to modify the behaviour of display-completion-list, which is the kind of thing that is likely to be hard to make compatible between Emacs releases. Also, defadvice is deprecated in favour of advice-add.
  4. The implementation technique won’t work! The problem is that it chooses which annotations to add by looking at the values of minibuffer-completion-table and minibuffer-completion-predicate. These are dynamic variables that correspond to the collection and predicate arguments to completing-read. But in p4.el, the predicate argument is not used, and the collection argument is impossible to identify because it’s dynamically generated for each completion by querying the Perforce server.

However, this analysis prompted me to look in detail at the implementation of completion in minibuffer.el and this revealed the existence of the variable completion-extra-properties. This was added quite recently: under “Lisp Changes in Emacs 24.1” the NEWS says:

New variable completion-extra-properties used to specify extra properties of the current completion:

The section on Completion Variables in the Emacs Lisp Reference Manual documents the variable:

completion-extra-properties

This variable is used to specify extra properties of the current completion command. It is intended to be let-bound by specialized completion commands. Its value should be a list of property and value pairs. The following properties are supported:

:annotation-function

The value should be a function to add annotations in the completions buffer. This function must accept one argument, a completion, and should either return nil or a string to be displayed next to the completion.

The manual doesn’t have any example code showing how to use completion-extra-properties, so here’s a simple demonstration: 

(defun my-annotation-function (s)
  (let ((item (assoc s minibuffer-completion-table)))
    (when item (concat "  -- " (second item)))))

(defvar my-completions '(("a" "description of a") ("b" "b's description")))

(let ((completion-extra-properties '(:annotation-function my-annotation-function)))
  (completing-read "Prompt: " my-completions))

See the screenshot. Note that as a result of keeping the example as simple as possible, the code is inefficient in a couple of ways:

  1. The concat is redone for each item each time the list is displayed. It would make sense to do this just once if the same completions are going to be displayed many times.
  2. Looking up a key using assoc is an O(n) operation, and this needs to be done for each item, which makes displaying the completion buffer O(n2).

The first problem is not a big deal for p4.el (the completion tables are generated dynamically and typically used once each), but the second is. When completing on, say, all the jobs on a Perforce server, there are likely to be thousands or tens of thousands of items, and a long delay here (on top of the unavoidable delay in fetching the items from the server) would not be acceptable. So in my implementation (commit 87fc1bdb) I used a hash table instead of an association list. A hash table has O(1) lookup, but it can’t be used as a collection table, and therefore can’t be accessed through the dynamic variable minibuffer-completion-table. I worked around this by using my own dynamic variable, p4-completion-annotations, to pass the hash table from the caller of completing-read down to the :annotation-function. See the screenshot for the annotation feature being used by the p4-job command.

This work was prompted by a user whose Perforce workflow makes heavy use of numbered pending changelists in order to manage work in progress (see issue #187). I can’t say that I approve of this approach, because it’s risky to keep work in progress private for long periods of time. If your hard drive fails suddenly then you lose your work, and if you are off sick then your colleagues have to scratch around in your workspace to try to figure out what you were up to. I prefer to use the default changelist, submit as soon as the code is working, and to use branches if I need to keep a sequence of changes isolated. But I guess if you’re doing the numbered pending changelist thing then you must feel that you have a pretty good reason for it: maybe you don’t know how to branch? or maybe your organization doesn’t have a policy for naming branches so you’re not sure where to put it? or maybe your repository is so big that branching takes a long time? Whatever the reason, it seems like a use case that’s worth putting a little effort into supporting.

  1. ↩ This is incorrect: the property is actually named :annotation-function. See bug 19820.