The symbol-macrolet trick

Sometimes you want to write a macro that communicates information to macros inside of its compile-time “scope.”

Pre-ANSI Common Lisp actually had a special form for this purpose, compiler-let, but it was removed in ANSI CL, and for good reason: what it did can be done entirely through the use of local macro definitions.

This may be surprising: although the ANSI Common Lisp standard has no API for lexical environments, there is just enough support to make it possible, in a portable way, to attach information to a lexical environment and retrieve it later.

Environments are only seen in macro definitions. A macro can choose to capture its environment by supplying an &environment binding.

(defmacro my-macro (arg1 arg2 &environment env)
  ...)

You may see careful macro writers pass the environment as an argument to functions like subtypep or constantp.

(defmacro my-macro (arg1 arg2 &environment env)
  (if (constantp arg1 env)
      ...))

(A word of warning: the environment object you get from &environment is stack-allocated: it is stored on the stack, not the heap, so it can only be used inside the macro where it is bound.)

Here’s a problem for macro writers. How do you expand local macros? This is useful if, for example, you want to expand differently depending on whether an argument is a constant. Using bare macroexpand you always get the global definition:

(defmacro current-scope ()
  :global)
(defmacro query-scope ()
  (macroexpand '(current-scope)))
(macrolet ((current-scope () :local))
  (query-scope))
=> :GLOBAL

This is where the environment comes in. If you capture the environment, you can pass it as a second argument to macroexpand, and now local macro definitions in the surrounding scope become visible.

(defmacro query-scope (&environment env)
  (macroexpand '(current-scope) env))
(macrolet ((current-scope () :local))
  (query-scope))
=> :LOCAL

This also works for symbol macros:

(define-symbol-macro current-scope :global)
(defmacro query-scope (&environment env)
  (macroexpand 'current-scope env))
(symbol-macrolet ((query-scope :local))
  (query-scope))
=> :LOCAL

Now the trick comes in. A macro expansion can bind symbol macros that never get expanded – that are only present in the lexical environment – as a back channel for communication with other macros within its lexical extent. (This would also work with list macros, but using symbol macros is easier because you don’t have to quote intermediate values.)

We can use symbol macros to annotate the environment with arbitrary information. Our own macros, if they know where to look, can then read or add to that information.

Here’s a real example, and one that connects back to a previous post: how does dispatch-case know, for the purpose of signaling errors, when it dispatches on the type of one parameter, what the types of the previous parameters were?

This information is in fact passed through the environment:

(define-symbol-macro matched-types ())
(defmacro with-matched-type (type &body body &environment env)
  (let ((matched-types (macroexpand-1 'matched-types env)))
    `(symbol-macrolet ((matched-types ,(cons type matched-types)))
       ,@body)))
(defmacro dispatch-case-error (&key type datum &environment env)
  (let ((matched-types (macroexpand-1 'matched-types env)))
    `(error 'dispatch-case-error
            :expected-type ,type
            :datum ,datum
            :matched-types ',(butlast matched-types))))