Arc Forumnew | comments | leaders | submitlogin
Quasiquoting and understanding macroexpansion
3 points by zck 4128 days ago | 3 comments
I've never been quite comfortable with how macros work under the hood. That's the root cause of this bug.

I have a macro that macroexpands into code containing, among other things, another macro call. That's fine. One of the arguments the inner macro expects is a symbol. However, I want this symbol to be generated based on input to the outer macro. I've reduced my code to this test case to show what I want.

Here's a function that takes inputs and simply combines them into a symbol:

  (def generate-symbol (symbol1 symbol2)
       (sym (string symbol1 #\. symbol2)))
Cool, this works easily enough:

  arc> (generate-symbol 'a 'b)
  a.b
Here's the inner macro. I've made it really simple; obviously my real code does more than this. Its main purpose here is to take a bare unquoted symbol and return a symbol:

  (mac consume-symbol (symbol)
       `',symbol)
And its use:

  arc> (consume-symbol arst)
  arst
I want to write a macro that takes two unquoted symbols, combine them, then call consume-symbol on them. So this code:

  (fancy-combine a b)
should end up with this return value:

  'a.b
Here's my first cut:

  (mac fancy-combine (sym1 sym2)
       `(consume-symbol (generate-symbol ,sym1
                                         ,sym2)))
But it doesn't quite work:

  arc> (fancy-combine a b)
  (generate-symbol a b)
Here's why:

  arc> (macex1 '(fancy-combine a b))
  (consume-symbol (generate-symbol a b))
Ok, this makes sense: whatever we have as the argument to consume-symbol will be taken as the symbol, and not evaluated: that's how macros work.

So how can we get it to do what I want? We can't just throw on another unquote:

  (mac fancy-combine2 (sym1 sym2)
         `(consume-symbol ,(generate-symbol ,sym1
                                           ,sym2)))
  #(tagged mac #<procedure: fancy-combine2>)
  arc> (fancy-combine2 a b)
  Error: "_unquote: undefined;\n cannot reference undefined identifier"
Calculating it outside of the quasiquote works, but I'm uncomfortable, because I'm not 100% sure when exactly things are evaluated: generate-symbol is run at macroexpansion-time, right? But then how does that interact with having different values passed to this new fancy-combine

  arc> (mac fancy-combine3 (sym1 sym2)
       (let symbol-to-consume (generate-symbol sym1 sym2)
            `(consume-symbol ,symbol-to-consume)))
  #(tagged mac #<procedure: fancy-combine3>)
  arc> (fancy-combine3 a b)
  a.b
Similarly, we can edit fancy-combine2 slightly to have a nested quasiquote, and this works, but it feels hacky to me. I don't like having the nested quasiquotes right next to the symbols, but I can't quite explain why. It just feels like a code smell.

  arc> (mac fancy-combine4 (sym1 sym2)
           `(consume-symbol ,(generate-symbol `,sym1
                                             `,sym2)))
  #(tagged mac #<procedure: fancy-combine4>)
  arc> (fancy-combine4 a b)
  a.b
It feels better to move the inner quasiquote out, like this:

  (mac fancy-combine5 (sym1 sym2)
           `(consume-symbol ,`(generate-symbol ,sym1
                                             ,sym2)))
But that doesn't work:

  arc> (fancy-combine5 a b)
  (generate-symbol a b)
So, which way would you prefer to write this? What am I missing about macroexpansion and quasiquotation that would make this understandable?


3 points by fallintothis 4128 days ago | link

The error you're getting from fancy-combine2 is because you have a double-unquote. You neglected to remove the unquotes from sym1 and sym2. The correct form would be

  (mac fancy-combine (sym1 sym2)
    `(consume-symbol ,(generate-symbol sym1 sym2)))
It might help to have a user-land defined quasiquote macro, to see how it works. I implemented such a macro here: https://bitbucket.org/fallintothis/qq

Using qq.arc, we can see that my solution is to use this expression:

  arc> (macex1 '(quasiquote
                  (consume-symbol (unquote (generate-symbol sym1 sym2)))))
  (list (quote consume-symbol) (generate-symbol sym1 sym2))
On the other hand, your version is like this:

  arc> (macex1 '(quasiquote
                  (consume-symbol
                    (unquote (generate-symbol (unquote sym1)
                                              (unquote sym2))))))
  (list (quote consume-symbol) (generate-symbol (unquote sym1) (unquote sym2)))
Hence, you get an error about unquote being undefined, because Arc doesn't actually define quasiquote or unquote as macros. They're just special identifiers that are basically parsed away before evaluation (assuming quasiquotes/unquotes are balanced). However, loading qq.arc defines an unquote macro, so instead of the error you had, you'll get this:

  arc> (fancy-combine2 a b)
  Error: "unquote not allowed outside of a quasiquote: a"
which makes it a bit more obvious what the error is.

-----

4 points by zck 4127 days ago | link

Ah, fantastic! Thanks for the hint.

For some reason, I had in my head that, inside quasiquote in a macro, you need to unquote every variable you want to evaluate before returning the code to be executed. Note that all my examples have ,sym1 and ,sym2 (except fancy-combine4, but that's deliberately executing the code before the returned generated code).

I guess the moral of the story here is that ,(...) evaluates the entire sexp, and puts the value there. I'll have to do more reading about macros.

-----

4 points by fallintothis 4127 days ago | link

I guess the moral of the story here is that ,(...) evaluates the entire sexp, and puts the value there.

Yup! If your quasiquoted expression only contains unquotes, the expansion is simple. Basically,

  `(sexp1 sexp2 ... ,sexpn ...)
will expand into

  (list 'sexp1 'sexp2 ... sexpn ...)
This also applies recursively, so that

  `(sexp1 (subexpr1 ,subexpr2))
expands into

  (list 'sexp1 (list 'subexpr1 subexpr2))
With unquote-splicing, you can think of quasiquote expanding into cons, so

  `(sexp1 sexp2 ,@sexp3)
will expand into

  (cons (list 'sexp1 'sexp2) sexp3)
If you're interested, I encourage you to look over my implementation of quasiquote in Arc. It's actually pretty simple (175 lines with comments), and might help clarify how quasiquotation works.

I'll have to do more reading about macros.

I think you seem to have the mental model for macros down. And I've always said that macros aren't that magical, because they fall out as a natural consequence of the language. Hell, here's a simple Arc "interpreter" that shows how macros are handled differently from functions:

  (def interpret (expr)
    (if (acons expr)
        (let op (interpret (car expr))
          (if (isa op 'mac) (interpret (apply (rep op) (cdr expr)))
              (isa op 'fn)  (apply op (map interpret (cdr expr)))
                            (eval expr))) ; punt on special forms (fn, if, etc.)
        (maybe-lookup-name expr)))

  (def maybe-lookup-name (expr)
    (if (and (isa expr 'sym) (bound expr))
        (eval expr)
        expr))
Here, interpret is really just another name for eval. Macros are simply functions whose inputs are unevaluated code. They construct some other bit of unevaluated code, and that result gets evaluated.

Quasiquote is then just a nice way to construct lists, whether it's unevaluated code for a macro to return or whatever. Its syntax takes some getting used to, but it comes with practice. Happy hacking!

-----