Arc Forum | Unquote only takes one argument: bug or feature?

Arc Forum

Unquote only takes one argument: bug or feature?

3 points by fallintothis 5420 days ago | 17 comments

In Common Lisp:

  $ clisp
  [1]> (setf xs '(x1 x2) ys '(y1 y2) x1 'a x2 'b y1 'c y2 'd)
  D
  [2]> ``(list ,,@(mapcar #'(lambda (x y) ``(,,x ,,y)) xs ys))
  (LIST 'LIST (LIST X1 Y1) (LIST X2 Y2))
  [3]> (eval ``(list ,,@(mapcar #'(lambda (x y) ``(,,x ,,y)) xs ys)))
  (LIST (A C) (B D))

But in Arc:

  $ cd arc3/
  $ mzscheme -v
  Welcome to MzScheme version 360, Copyright (c) 2004-2006 PLT Scheme Inc.
  $ rlwrap mzscheme -m -f as.scm
  Use (quit) to quit, (tl) to return here after an interrupt.
  arc> (= xs '(x1 x2) ys '(y1 y2) x1 'a x2 'b y1 'c y2 'd)
  d
  arc> (eval ``(list ,,@(map (fn (x y) ``(,,x ,,y)) xs ys)))
  (list (a c))

Odd. The culprit?

  arc> ``(list ,,@(map (fn (x y) ``(,,x ,,y)) xs ys))
  (quasiquote
    (list
      (unquote (quasiquote ((unquote x1) (unquote y1)))
               (quasiquote ((unquote x2) (unquote y2))))))

Or, more simply

  arc> `(unquote ''x ''y)
  (quote x)

So, Arc's unquote only expects a single argument, ditching the remaining ones silently, whereas mzscheme makes this rather explicit:

  $ mzscheme
  Welcome to MzScheme version 360, Copyright (c) 2004-2006 PLT Scheme Inc.
  > `(unquote ''x ''y)
  stdin::0: unquote: expects exactly one expression at: (unquote (quote (quote
  x)) (quote (quote y))) in: (quasiquote (unquote (quote (quote x)) (quote
  (quote y))))
  
   === context ===
  qq
  repl-loop

Is there any compelling reason for unquote to only take one argument? Common Lisp seems to simply unquote every argument, so that forms like ,,@(...) will work. Though rare, nested quasiquotes do appear in actual code. For instance, I found this case by porting Peter Siebel's Common Lisp version of 'once-only to Arc:

  (defmacro once-only ((&rest names) &body body)
    (let ((gensyms (loop for n in names collect (gensym))))
      `(let (,@(loop for g in gensyms collect `(,g (gensym))))
        `(let (,,@(loop for g in gensyms for n in names collect ``(,,g ,,n)))
          ,(let (,@(loop for n in names for g in gensyms collect `(,n ,g)))
             ,@body)))))

I got as far as

  (mac once-only (names . body)
    (let gensyms (map1 [uniq] names)
      `(with ,(mappend (fn (g) `(,g (uniq))) gensyms)
        `(with ;?
           ,(with ,(mappend (fn (n g) `(,n ,g)) names gensyms)
             ,@body)))))

before discovering this little property of unquote. Of course, using 'mappend here wouldn't work so well, since it'll churn out some fun results with the nested quasiquotes:

  arc> (mappend (fn (g n) ``(,,g ,,n)) '(g1 g2 g3) '(n1 n2 n3))
  (quasiquote ((unquote g1) (unquote n1))
              quasiquote
              ((unquote g2) (unquote n2))
              quasiquote
              ((unquote g3) (unquote n3)))

My next inclination would be to try using a 'let to destructure the gensyms against the names, but I'm burnt out at the moment. Anyone have any suggestions on how to finish this macro?

As for unquote only taking one argument: Is it a bug? A feature? It seems like Arc should at least report an error, as mzscheme does (in version 360, anyways).

4 points by rntz 5419 days ago | link

IIRC, Scheme handles nested quasiquoting differently than Common Lisp - expands the unquotes in a different order or some such. I've been meaning to figure out precisely how they differ at some point, but I suspect that merely taking code involving nested quasiquoting from lisp and transplanting it to Arc will not work; Arc code involving quasiquoting translates down fairly directly to Scheme.

Arc doesn't report an error because the second argument to unquote gets tossed out during compilation:

    arc> :a
    > (ac '`(foo (unquote bar baz)) '())
    (quasiquote (foo (unquote _bar)))

(In case you're unfamiliar with what just happened: ':a at the arc interpreter drops you into scheme, at which point you can access the internals of arc's compiler. (ac code env) compiles the Arc code 'code to scheme, where 'env is a list of locally-bound identifiers in scope.)

This is actually a fairly common "bug" in arc's compiler. The same thing happens with coerce:

    arc> (coerce 2 'int (prn "this is ignored"))
    this is ignored
    2

This is because 'coerce underneath (well, unless you're using my programmable coercion hack, which fixes the above) is just one big function of the form (lambda (x type . args) ...), and unless args is used, it's ignored. args is used, for example, in (coerce "f" 'int 16), in which 16 stands for the base in which "f" is to be read as an integer.

-----

1 point by fallintothis 5419 days ago | link

Scheme handles nested quasiquoting differently than Common Lisp - expands the unquotes in a different order or some such.

Well, if we're to believe a single statement in an Alan Bawden paper from 1999: "I am not aware of the existence of a correct optimizing quasiquotation expander for Scheme. (None of the Scheme implementations that I tested implement nested splicing correctly.)" -- http://repository.readscheme.org/ftp/papers/pepm99/bawden.pd...

The entire semantic reasoning is what I'm hung up on: does it seem right that ,,@(...) isn't supposed to work? Not that I'm necessarily advocating one rationale over another, just that I've simply not heard any such rationale. (My Google-fu is upsettingly weak here.)

Arc doesn't report an error because the second argument to unquote gets tossed out during compilation

Ah, duh. Makes sense. I had tried combing through the compiler, but was rather aimless. Thanks.

This is actually a fairly common "bug" in arc's compiler.

I'd probably call it a bug in the case of 'coerce, since the arguments (being passed directly to a Scheme function) all get evaluated -- if the third argument was truly ignored (in the "compiled away" sense), then it shouldn't even do anything. Plus, this behavior also reaches functions like 'outfile and (to an extent) 'writec.

Granted, maybe it's wise to go against "protecting" the programmer and assume that they know what they're doing; maybe I'd happen to want to assign a variable in the same statement as an 'outfile, I don't know.

-----

2 points by CatDancer 5419 days ago | link

does it seem right that ,,@(...) isn't supposed to work?

Doing some googling, I see that this bug was reported against plt scheme in 2002: http://www.cs.brown.edu/pipermail/plt-scheme/2002-June/00004...

R6RS takes the approach of allowing (unquote 1 2 3). Section 11.17 of the standard has this example:

  (let ((name 'foo))
    `((unquote name name name)))
          => (foo foo foo)

-----

1 point by fallintothis 5419 days ago | link

Good find, thanks. Now it at least makes sense to call this a bug (well, others have at any rate), though not specifically in Arc.

-----

4 points by CatDancer 5418 days ago | link

Here's a patch that allows us to use 'declare to bootstrap into our own implementation of quasiquote written in Arc: http://hacks.catdancer.ws/disable-ac-qq0.patch

For example, from his paper, here's Bawden's inefficient but simple and correct implementation, ported to Arc: http://hacks.catdancer.ws/bawden-qq0.arc

(We could also look around to see if there's other working implementations that we could use... plt scheme's R6RS implementation of quasiquote or Common Lisp's implementation might be worth checking out).

To run your example, I'll load my port of Bawden's implementation:

  $ mzscheme -m -f as.scm
  Use (quit) to quit, (tl) to return here after an interrupt.
  arc> (load "bawden-qq0.arc")
  nil

note how I'm bootstrapping here: I'm using Arc's default implementation of quasiquote based on MzScheme to load Arc and Bawden's code, and then I can switch over to using Bawden's code to implement quasiquote.

Next I can use the patch to disable Arc's expansion of quasiquote:

  arc> (declare 'quasiquotation nil)
  #<void>

...though the existing 'declare assertions use a true value to change Arc's behavior, so maybe this should be named 'disable-quasiquote-expansion or something like that instead.

Now that I'm not being preempted by the Arc compiler, I can implement quasiquote with a macro, such as with Bawden's implementation:

  arc> (mac quasiquote (x)
         (qq-expand x))
  #3(tagged mac #<procedure: quasiquote>)

and your example runs like this:

  arc> (= xs '(x1 x2) ys '(y1 y2) x1 'a x2 'b y1 'c y2 'd)
  d
  arc> (eval ``(list ,,@(map (fn (x y) ``(,,x ,,y)) xs ys)))
  (list (a c) (b d))

Is that the right result?

-----

1 point by CatDancer 5418 days ago | link

Aha, here's a nicer version of the patch that doesn't need the 'declare to be used: http://hacks.catdancer.ws/preempt-qq0.patch

What it does is if the user has defined a 'quasiquote macro, it allows the macro to take precedence over Arc's internal qq implementation; much like any macro in arc.arc can be redefined by the user.

The example now looks like:

  $ mzscheme -m -f as.scm
  Use (quit) to quit, (tl) to return here after an interrupt.
  arc> (load "bawden-qq0.arc")
  nil
  arc> (mac quasiquote (x)
         (qq-expand x))
  #3(tagged mac #<procedure: quasiquote>)
  arc> (= xs '(x1 x2) ys '(y1 y2) x1 'a x2 'b y1 'c y2 'd)
  d
  arc> (eval ``(list ,,@(map (fn (x y) ``(,,x ,,y)) xs ys)))
  (list (a c) (b d))

-----

2 points by fallintothis 5416 days ago | link

here's a nicer version of the patch that doesn't need the 'declare to be used

Cool, thanks.

We could also look around to see if there's other working implementations that we could use... plt scheme's R6RS implementation of quasiquote or Common Lisp's implementation might be worth checking out

Just for reference, I posted my code for a port of a Common Lisp backquote: http://arclanguage.org/item?id=9962

I also tried looking at PLT's r6rs quasiquote (specifically collects/r6rs/private/qq-gen.ss), but I found it a bit daunting because I'm unfamiliar with syntax-case macros. It's probably better for quasiquote to be written in Scheme, though, so perhaps the code could easily be transplanted into ac.scm. I'll just leave it to someone more Scheme-savvy.

-----

1 point by CatDancer 5416 days ago | link

It's probably better for quasiquote to be written in Scheme

Why?

-----

1 point by fallintothis 5416 days ago | link

I was thinking in terms of bootstrapping. The quasiquote library I wrote uses things defined in arc.arc, which in turn are written for quasiquote, but using the one on the Scheme side.

It's entirely possible to rewrite either to fit it all together. For instance, someone could rewrite the quasiquote library in terms of basic operators -- 'assign, 'cons, 'fn (instead of 'let), and the like. Probably more bearable would be rewriting arc.arc to use 'list and 'cons and so forth to build expressions instead of using quasiquote, up until we have enough to define quasiquotation (which doesn't require much); then load qq.arc and continue defining arc.arc with the newfound utility.

Neither of these options sounds pleasant if, as an alternative, we could just have an easily-added Scheme-side quasiquote -- e.g., if mzscheme already implemented it correctly, we'd be done. Granted, there are advantages to having a user-defined quasiquote, not just in the interest of keeping Arc axiomatic but also in giving programmers the option to modify it.

Efficiency was another concern of mine, in that doing quasiquotation in Scheme might be more efficient than cranking 'ac across all of the Arc functions back-and-forth. But this is likely unfounded and more a symptom me worrying rather than actual testing -- the difference could be negligible or even nonexistent. Arc quasiquotes (as I implemented) still compile down to simple Scheme code (to the results of the macroexpansions), so it's probably silly to worry. Just a fleeting concern.

-----

1 point by CatDancer 5416 days ago | link

Well, on efficiency, the quasiquote expansion happens at read / macro expansion / compile time, am I right? So writing the qq expander in Arc vs. Scheme isn't going to affect the speed of the running program, yes?

As for bootstrapping, MzScheme's qq implementation works as long as nested quasiquotation isn't used, and today arc.arc doesn't use nested quasiquotation. So I don't think we need to do anything beyond what we're already doing: load arc.arc, then load a qq implementation that handles nested qq, and then we're free to write macro utilities that use nested qq.

-----

1 point by fallintothis 5416 days ago | link

Arc vs. Scheme isn't going to affect the speed of the running program, yes?

Yes, so far as I can see. I was just being paranoid.

As for bootstrapping, it's still cleaner to have a single implementation, rather than juggling them. Given we're currently only juggling 2 implementations, the problem's not intractable, just "improper".

-----

3 points by CatDancer 5416 days ago | link

I suspect that bootstrapping is almost always doing something improper if you look at it too closely :)

-----

1 point by rntz 5419 days ago | link

Here's a working version of 'once-only. It avoids nested quasiquotation, because I was unfamiliar with how it worked at the time I wrote it.

    ;; encapsulates idiom: (w/uniq o `(let ,o ,obj ...))
    (mac once-only (names exp)
      (withs (names (if atom.names (list names) names)
              names-temps (map [list (uniq) _] names))
        `(with ,(apply join names-temps)
           (w/uniq ,names
             (list 'with (list ,@(mappend (fn ((name tmp)) (list tmp name))
                                   names-temps))
               ,exp)))))

-----

1 point by fallintothis 5419 days ago | link

Ah, I hadn't thought to mangle around with explicit lists; probably because I find it uglier than quasiquotation.

Siebel's version then becomes:

  (mac once-only (names . body)
    (let gensyms (map1 [uniq] names)
      `(w/uniq ,gensyms
         (list 'with (list ,@(mappend list gensyms names))
           (with ,(mappend list names gensyms)
             ,@body)))))

Could be worse. And it's usable, e.g.:

   (mac for (v init max . body)
     (once-only (init max)
       `(with (,v nil ,max (+ ,max 1))
          (loop (assign ,v ,init) (< ,v ,max) (assign ,v (+ ,v 1))
            ,@body))))
  *** redefining for
  #3(tagged mac #<procedure: for>)
  arc> (for x 1 5 (prn x))
  1
  2
  3
  4
  5
  nil
  arc> (for init 1 5 (prn init))
  1
  2
  3
  4
  5
  nil
  arc> (let max 2 (for init 1 5 (when (> init max) (prn init))))
  3
  4
  5
  nil
  arc> (for x 1 5 (prn init))
  Error: "reference to undefined identifier: _init"

Thanks for the help!

-----

3 points by fallintothis 5418 days ago | link

Not that anyone cares about beating this dead horse, but after coming back to the code, I instantly realized how to mitigate the ,,@(...) snafu:

  (mac once-only (names . body)
    (let gensyms (map1 [uniq] names)
      `(w/uniq ,gensyms
        `(with ,(list ,@(mappend list gensyms names))
          ,(with ,(mappend list names gensyms)
            ,@body)))))

Seems obvious in retrospect. (Also, I've been spelling Peter Seibel's name wrong in this thread; oops.)

-----

1 point by CatDancer 5419 days ago | link

What does once-only do?

-----

1 point by rntz 5419 days ago | link

It encapsulates the notion of only evaluating a given set of expressions once. It's used for writing macros. An example is probably best:

    arc> (with (x '(x-expr) y '(y-expr)) (once-only (x y) `(+ ,x ,y ,y)))
    (with (g1838 (x-expr) g1839 (y-expr)) (+ g1838 g1839 g1839))

This is the equivalent of the handwritten:

    arc> (with (x '(x-expr) y '(y-expr))
           (w/uniq (gx gy) 
             `(with (,gx ,x ,gy ,y) (+ ,gx ,gy ,gy))))
    (with (g1840 (x-expr) g1841 (y-expr)) (+ g1840 g1841 g1841))

-----