Arc Forumnew | comments | leaders | submitlogin
Macros without quasiquotation
1 point by evanrmurphy 5022 days ago | 17 comments
My latest iteration of lisp->javascript is itself written in JavaScript [1]. Macros are a must, but I'm starting to doubt that this language can (or should) support full-fledged quasiquotation.

How can you have macros without quasiquotion? Well, there are a great deal of useful macros that follow this general pattern:

  (mac foo (x y . body)
    `(bar ,x ,y ,@body))
The entire definition body is backquoted with each occurrence of the parameters comma'd out. I've started implementing LavaScript's macro system with the macro itself serving as a sort of dilluted quasiquote, based on the above pattern. In this system, you would write that macro as follows:

  (mac foo (x y body...)
    (bar x y @body))
It assumes that the entire definition body is backquoted, and it assumes that every occurrence of the parameters is unquoted. So you don't need to notate all that explicitly in your macro definitions. On the other hand, you don't have a choice. You must sacrifice all the other interesting macrological possibilities, such as nested backquote expressions.

I don't know if this is going to be work. The approach apparently wouldn't support gensyms, which could be a showstopper. I just started going down this path tonight and wanted to share / seek some feedback.

If it wasn't clear, the body... is just different syntactic sugar for rest parameters. LavaScript doesn't have true conses at the moment, so I don't want to tease by invoking dotted pair notation.

---

[1] It's called LavaScript :P https://github.com/evanrmurphy/lava-script



4 points by rocketnia 5022 days ago | link

I've taken the liberty of implementing a LavaScript REPL-on-the-web for you, similar to your Try Arc[1]. :) Your code seems to target server-side JavaScript, but I rigged it up to work on the client side anyway.

I hotlink to everything just so that I can keep it down to one file; as a side effect, if you push a new lava.js (not lava.coffee) to GitHub, it'll automatically use the new one.

There are a few nifty REPL variables. The variable "thatexpr" refers to the previous compiled command string. The variables "that" and "thaterror" refer to the most recent normal result and the most recent exception respectively. Would "thatunparsedexpr" and "thatparsedexpr" be worthwhile additions to this?

I see no way to hack in a stdin or stdout, or to fix the copy-and-paste issue, thanks to the way jQuery-console works. Now I understand why Try Arc has that trouble. ^_^;

https://gist.github.com/837840

http://rocketnia.kodingen.com/af/try-lava-script/

[1] http://tryarc.org/

-----

1 point by evanrmurphy 5022 days ago | link

Wow, thank you so much!! This is great! I've linked to it in the README: https://github.com/evanrmurphy/lava-script/blob/9ee2b4552ff8... :D

'that' and 'thatexpr' are both great! I should include those in the command-line REPL! :)

-----

1 point by rocketnia 5020 days ago | link

Oops! I posted that Gist as Anonymous. XD I've reposted it now under my own username: https://gist.github.com/840809

-----

2 points by evanrmurphy 5020 days ago | link

Updated the link in LavaScript's readme. :)

-----

1 point by rocketnia 5020 days ago | link

Thanks much. XD

-----

1 point by evanrmurphy 5021 days ago | link

Nice choice of background color, by the way. Very lava-like. ^_^

-----

1 point by rocketnia 5021 days ago | link

It was surprisingly easy to make it any of several variants of a fruity pink-and-orange, an antique brown, or a garish red. Finally I came upon something that at least looked like it was supposed to look like lava, and yay, you noticed. XD

Come to think of it, subtle gradients might help to evoke more glowiness and dimensionality.

-----

1 point by evanrmurphy 5021 days ago | link

I wondered the same thing about gradients. ^_^ Maybe as a future refinement. :)

A couple of other things:

1. I like that you made it full screen.

2. I was pleasantly surprised you kept the same REPL pattern!

  > [input]
  [compiled]
  => [evaluated]
I tried several variations and found this to be most intuitive and readable. One thing I was on the fence about was using '>' for the input prompt instead of 'lava>'. The former just felt less cluttered to me.

-----

1 point by rocketnia 5020 days ago | link

1. I like that you made it full screen.

That part took a bunch of fiddling! XD I tried to use it on some Android browsers though, and it's not so friendly there.... Using an online REPL from a mobile device would be pretty nifty. ^_^

Hmm, jQuery-console's manual keypress checking makes it especially Android-unfriendly. A slide-out keyboard technically works, but I'd be lost without it. :/

2. I was pleasantly surprised you kept the same REPL pattern!

My original intent was to hack your REPL to work with the console. Actually, my original intent was to have an input box with "compile" and "now evaluate that" buttons, but I liked your REPL much better. :-p

One thing I was on the fence about was using '>' for the input prompt instead of 'lava>'. The former just felt less cluttered to me.

I think "lava>" would make more sense if it were possible to switch between REPLs, but ">" makes more sense now. :)

-----

4 points by aw 5022 days ago | link

I haven't looked at LavaScript, but I'm curious as to why you wouldn't be able to support quasiquotation?

If you don't have quasiquotation yet, but you do have Arc's mac working, macros can be written without quasiquotation by expanding the quasiquotes by hand:

  (mac foo (x y . body)
    `(bar ,x ,y ,@body))
=>

  (mac foo (x y . body)
    (cons 'bar (cons x (cons y body))))
This in turn is enough to implement quasiquotes. That is, if you already have eval, Arc macros without quasiquotation, and lists, then we can implement quasiquote on top of that.

I've done a couple of quasiquote implementations already, so I could probably be of some help, if you'd like.

-----

1 point by evanrmurphy 5022 days ago | link

> That is, if you already have eval

I think you've gotten to the central issue here, which I failed to explain in the OP. Not supporting quasiquote is actually just a consequence of not supporting eval.

The reason for not supporting eval is that this is a source-to-source compiler like CoffeeScript, not a run-time environment. I do not know how to support eval (and perhaps even a meaningful quote operator) without introducing run-time dependencies.

Perhaps an example can illustrate. Here's an expression with lisp on the left and javascript output on the right:

  (+ 1 2)          1+2
Now quote it. What should the output be?

  '(+ 1 2)         '1+2'

  '(+ 1 2)         ['+', 1, 2]
Both are quoted expressions in some sense; the first is something JS can eval natively, the second requires LavaScript's special eval. The second (if I'm not mistaken) is what you need for quasiquotation, but it would require introducing into the target environment a run-time dependency on LavaScript's eval. Am I making any sense?

What I was suggesting in the OP is that the lack of eval and quasiquotation doesn't actually rule out macros. Rather it forces your macro system to be of a more limited, basic templating variety, but this is still useful. For example, here's how you define def and let using the proposed system [1]:

  (mac def (name parms body...)
    (= name (fn parms @body)))

  (mac let (var val body...)
    ((fn (var) @body) val))
> I've done a couple of quasiquote implementations already, so I could probably be of some help, if you'd like.

Great, thanks! And you're already helping by talking with me about it. :)

---

[1] I actually have this working already, except for rest parameters and the @ unquote-splicing operator. Since these are used in so many macro definitions, it isn't very useful quite yet. But the proof-of-concept is there.

-----

2 points by rocketnia 5022 days ago | link

it would require introducing into the target environment a run-time dependency on LavaScript's eval

Ohhhh, nice limitation.

Technically, if your compile phase ran the code in a LavaScript-capable environment and you kept a running total of all the compiled code in order to output it at the end, then macros should be fine. They're almost never called by the compiled code (just by the thing that was compiling that code earlier), so they're just cruft. I'd be surprised if there weren't a minifier that could cut them out of the result automatically.

On the other hand, if you did that, it would mean running the code in order to set up any definitions the macros use, and thus you would have to have all your run time libraries loaded at compile time--perhaps including the DOM and such--so maybe that's not what you're going for.

Your templating macro system is something that doesn't take advantage of any run time definitions, and is therefore usable from a completely different compile-time environment. Macros that can't execute arbitrary code are pretty dull, though. Maybe what you need is a phase control macro:

  ## (This is just a sketch. Please don't bother crediting me.)
  
  realMacros = {}
  
  realMacros.atCompileTime = (body) ->
    each body, (expr) ->
      eval lc(expr)
    'null'  ## or whatever makes sense as an ignored result
  
  (->
    orig = lc
    lc = (s) ->
      if isList(s) and (s[0] of realMacros)
        realMacros[s[0]](s[1..])
      else orig(s)
  )()
From here, people oughta be able to use (atCompileTime ...) forms to work in compiler-space, where they can modify the realMacros table directly, perhaps to implement a more convenient macro definition macro.

Hope this helps. ^_^

-----

1 point by evanrmurphy 5021 days ago | link

I like your atCompileTime approach. It would seem to open up the kinds of things the compiler can do while still not requiring anything special about the run-time environment. I'm leaning toward something like this.

> Technically, if your compile phase ran the code in a LavaScript-capable environment and you kept a running total of all the compiled code in order to output it at the end, then macros should be fine.

To be sure I understand, would this be kind of like if LavaScript both compiled every expression to JavaScript (as it does now) and ran it through the atCompileTime evaluator, so as to make it available for use in the compiler space?

-----

1 point by rocketnia 5021 days ago | link

To be sure I understand, would this be kind of like if LavaScript both compiled every expression to JavaScript (as it does now) and ran it through the atCompileTime evaluator, so as to make it available for use in the compiler space?

That and atCompileTime were almost independent trains of thought, but yeah, I think you understand it. The compiler would compile a command, write the compiled command to the output file, evaluate the command itself, then repeat with the next command.

  for command in input
    let compiled = compile( command ) in
      output-file.write compiled
      execute compiled
Like I said, I don't know if this applies to your case very well, since the compiler's environment may not adequately simulate the actual application conditions.

-----

2 points by aw 5022 days ago | link

The second (if I'm not mistaken) is what you need for quasiquotation, but it would require introducing into the target environment a run-time dependency on LavaScript's eval. Am I making any sense?

Not sure :-) To put it in my own words, macros and quasiquotation are expanded at compile-time. Thus there is a compile-time dependency: in whatever language you write your macros in, you need to be able to be able to call functions written in that language from your compiler.

Take Arc as an example. A macro in Arc is an association between the macro name and a function that does the work of expanding the macro. Saying

  (mac foo ()
    `(+ 1 2))
is just a shortcut for this:

  (def expand-foo ()
    `(+ 1 2))
  (assign foo (annotate 'mac expand-foo))
now, I can write the "expand-foo" function in any language I want, as long as I can call it from the compiler. Here, I happen to have written it in Arc. But I could have written it in Scheme. Or, I could have written it in Javascript, if I had some way of calling Javascript from Arc, such as by shelling out to a Javascript interpreter. All "expand-foo" does is take one list and return another list, so I could write that in any language.

So, if you want to write full-strength macros in LavaScript, you need some way for your compiler to be able to call, during compilation, a function you've previously written in LavaScript.

Which, if LavaScript functions are compiled into Javascript, means if you can call Javascript functions from your compiler.

If you can do that, then you'll also get quasiquotation, because quasiquotation can be implemented as a macro.

If you can't call Javascript functions from your compiler, and if you want full-strength macros, then you'd need to write your macros in some language that you can call from your compiler.

-----

3 points by aw 5021 days ago | link

Oh, I was confused:

The reason for not supporting eval is that this is a source-to-source compiler like CoffeeScript, not a run-time environment.

I wasn't paying attention and thought you meant LavaScript was written in CoffeeScript...

but I'm starting to doubt that this language can (or should) support full-fledged quasiquotation

I think I'm starting to get it: it's not that you couldn't support full-fledged quasiquotation if you wanted to by writing out an implementation in Javascript, it's that it wouldn't be very useful without being able to write macros via functions written in LavaScript, which in turn would mean that you'd have to be able to support loading LavaScript programs incrementally, which would mean having the LavaScript compiler in the runtime environment.

-----

1 point by evanrmurphy 5021 days ago | link

Yes, I think that's right.

-----