Consult interface for dash-docs
A consult interface for a command is the usage of consult--read
for
completing read. This post is a list of issues encountered in the
initial development of consult-dash.
1. Dynamic completing read
Unlike emacs' standard completing-read
function, consult--read
does not support dynamic completion tables. Such completion tables are
completions generated on the fly in response to user input. As an
example, consider a hypothetical command find-synonyms
which
produces choices of synonyms. Suppose the minibuffer contains ca
and
the user types in the characters p
, e
and r
. The choices are
produced by a dynamic commpletion function that lists synonyms of
cap
, cape
, and caper
. Note that the list of synonyms must be
regenerated every time the input changes.
Libraries must use an asynchronous generator of completion results to emulate dynamic completion tables with consult. This approach is much nicer in terms of lag, but complicates library design significantly.
Edit: Daniel Mendler, the author of consult
, kindly and correctly
points out that the complexity is inherent in the usage of
asynchronous facilities, and is not required by the use of
consult--read
itself. Indeed, the use of asynchronous completion
makes the interface feel far snappier than the usage of dynamic
completion tables. Additionally, consult--read
brings with it
multiple goodies, e.g., a simple way to set the default initial
value to the symbol at point, without requiring a new wrapper
function.
The basic pipeline used by dash-docs to identify candidates is to
synchronously query dash SQLite databases each time a new character is
input. There is some clever caching to avoid some redundant queries,
but the fundamental operation is synchronous. When input changes, the
generated candidate list is presented to completing-read
using the
dynamic completion table interface.
Faking asynchronous behavior using timers with a synchronous process
does not work with consult--read
. The simplest example then to
follow for asynchronous command completions is consult-man
from
consult.el
. The asynchronous pipeline consists of two parts:
consult-dash--builder
: a command builder given user inputconsult-dash--format
: a formatter that converts the output of the command into candidates
With those parts in place, constructing a new async command is pretty straightforward:
(defun consult-dash (&optional initial) "Consult interface for dash documentation. INITIAL is the default value provided." (interactive) (dash-docs-create-common-connections) (dash-docs-create-buffer-connections) (setq consult-dash--current-docset nil) (when-let* ((builder (consult-dash--with-buffer-context #'consult-dash--builder)) (search-result (consult--read (consult--async-command builder (consult--async-transform consult-dash--format) (consult--async-highlight builder)) :prompt "Dash: " :require-match t :group #'consult-dash--group :lookup #'consult--lookup-cdr :category 'consult-dash-result :annotate #'consult-dash--annotate :initial (consult--async-split-initial initial) :add-history (consult--async-split-thingatpt 'symbol) :history '(:input consult-dash--history)))) (dash-docs-browse-url search-result)))
2. Asynchronous commands and buffers
Unlike consult-man
, and unlike every other consult interface known
to this author, consult-dash
needs a command-line builder that is
specific to the buffer from which it is called: the list of docsets
used in any buffer is specific to that buffer. However, when the
builder is called asynchronously by consult--read
, it is run with a
different buffer context. Hence the buffer itself needs to be saved
when passing the builder to consult--read
:
(defun consult-dash--with-buffer-context (func) "Ensure that FUNC is called with the correct buffer context." (let ((buf (current-buffer))) (lambda (&rest args) (with-current-buffer buf (apply func args)))))
Note that lexical binding is in effect above. In the invocation of
consult--read
, the appropriate function is constructed on the fly:
(when-let* ((builder (consult-dash--with-buffer-context #'consult-dash--builder)) (search-result (consult--read (consult--async-command builder (consult--async-transform consult-dash--format) (consult--async-highlight builder)) ...))) ...)
Similarly, the formatter is called with partial output (though always
complete lines), and hence may need to retain state between calls. In
this case, the state (consult-dash--current-docset
) is stored a
global variable. (Edit: The idea of using a let
-bound variable, as in the
builder, does not work, but can be fixed with a bit of effort.)
(when-let* ((builder (consult-dash--with-buffer-context #'consult-dash--builder)) (current-docset t) ; t due to when-let (formatter (some-func current-docset #'consult-dash--format)) (search-result (consult--read (consult--async-command builder (consult--async-transform formatter) (consult--async-highlight builder)) ...))) ...)
Edit: It appears that the closure approach should work; still need to figure out the bugs in my earlier failed implementation.
3. Multiple commands
Consult's support for asynchronous command builders assumes a single
command that is run. However, in the case of dash-docs
, we need to
run multiple commands, one for each docset. Ideally, we would have a
sequence of commands which should be launched in a chain, i.e., each
new command would be spawned when the previous one completes. However,
it is unclear how one would incorporate such a process chain into
consult's async functionality. Any ideas here would be greatly
appreciated.
The alternate mechanism that was followed in consult-dash
was the
construction of a long shell command line consisting of independent
statements querying each docset. Essentially the command run is
equivalent to the following run under /bin/sh
:
echo "DOCSET:" "DOCSET1" sqlite3 "some DOCSET1 query" echo "DOCSET:" "DOCSET2" sqlite3 "some DOCSET2 query" echo "DOCSET:" "DOCSET3" sqlite3 "some DOCSET3 query" ...
The markers from the "DOCSET:"
lines are used to identify the docset from which subsequent query results are found since the transformer processes the lines in sequence. This approach restricts consult-dash
to only those systems with a POSIX shell. Any ideas on ways to execute these command sequences with pure emacs facilities would also be greatly appreciated.
4. Text properties
Attempts to add support for embark and marginalia indicated another
problem: only the formatted string (shown as the candidate in the
minibuffer) was presented to those functions, and not the actual
non-formatted candidates themselves. consult-dash
uses text
properties attached to the candidate strings to transfer information
to embark and marginalia annotators. consult--read
itself returns
the original non-formatted candidate. Any ideas on better techniques
would also be greatly appreciated.
Edit: Daniel Mendler, the author of consult
, provided a definitive
answer that the completing read interface is the limitation, not the
lack of techniques.