Arc Forumnew | comments | leaders | submitlogin
Simplifying arc-like implementation atop SBCL
11 points by akkartik 1431 days ago | discuss
http://github.com/akkartik/wart

When I started using arc I was surprised by two things: how ergonomic it was to use, and how short the implementation was. Over time I've gradually added a third source of surprise: how much shorter it could be. It was converting between () and nil, arc variable names and scheme variable names. Was all this really necessary? It's unnecessary, and much else besides. It turns out you can build out all the important parts of arc without a conventional interpreter cond, or without keeping track of variable bindings.

== The mountain

Accomplishing this required a common lisp (the mountain) and a few tricks (the wart atop the mountain). Arc is similar to common lisp in so many ways, perhaps it's only implemented in scheme so that it can be a lisp-1. Using common lisp lets us avoid implementing many primitives. Especially macros, perhaps the biggest reason arc needs a full cond with mutually-recursive calls across 600 lines of ac.scm, and env tracking every. single. variable binding.

== Tricks

I want a simple implementation that doesn't go against the grain of the underlying lisp; just let common lisp do everything it can. Most things can be macros except three features[1]:

  - overriding lisp primitives like if and let to their 
  ergonomic arc equivalents
  - ssyntax
  - allowing structures like lists and hash tables to be in
  functional position
To add these wart uses not an interpreter but a transformer -- anything it doesn't understand goes straight through to common lisp. The stages of the REPL now look like this:

  read transform macroexpand eval print
and each feature that needs more than a macro can be an independent transformer. Overriding lisp primitives: https://github.com/akkartik/wart/blob/master/005defover.lisp. ssyntax: https://github.com/akkartik/wart/blob/master/003ssyntax.lisp. Data structures can't go into functional position because we're on a lisp-2[2], but we can pass them to call, a thin layer to funcall that uses an extensible coerce[3] to convert to 'function: https://github.com/akkartik/wart/blob/master/007call.lisp.

I'm irrationally pleased with the directory organization[4] and the loader setup. Wart doesn't track filenames at all, just loads everything in the directory that starts with a digit, in well-defined order. Tests run similarly. Programming in wart? Just start adding code into filenames with the appropriate numeral prefix and it'll load up. Want to install a lisp library? Just keep the .lisp extension and it won't go through the wart transformer. The fact that the transformer is always around reminds me that I'm programming on common lisp with some syntactic sugar rather than a new language[5].

Independent transformers and the loader setup result in a really nice separation of concerns. Each (short) file in wart can be read in isolation, with just a sense of how primitives in previous files behave.

[1] See http://arclanguage.org/item?id=11505 for some initial lists.

[2] I'm unhappy to end up with a lisp-2; it feels like the last onion left (http://www.paulgraham.com/arcll1.html). It helps a bit that ssyntax like hash.key expands to (call hash key) and so on.

[3] Based on rntz's http://arclanguage.org/item?id=9828

[4] Original idea: http://arclanguage.org/item?id=11880

[5] Repeating myself from http://arclanguage.org/item?id=12661

== Credits

I started out with waterhouse's lisp implementation (http://arclanguage.org/item?id=11529), and stole aw's recent idea of generating the interpreter cond one step at a time (look at new-ac in https://github.com/awwx/ar/blob/c7c8f38bc966c3ff57c2a53a249bda099ad842f5/ar.ss; it's also described halfway down http://awwx.posterous.com/how-to-future-proof-your-code).

== Administrivia

Major incompatibilities: it's a lisp-2, so you need to use call (funcall) at places. And the ssyntax for compose is ^ because : is reserved for package lookup. Finally, it provides pervasive keyword args and uses ? to delimit optional args, like I've been fond of (http://arclanguage.org/item?id=12657)

To try it out:

  $ wart
  wart> (quit)
  $
You'll need a fairly recent sbcl (you can go farther back if you get rid of redefinition-warning suppression in synonyms.lisp)

To run the test cases:

  $ wart test
  $ # success
It isn't done yet, but I hope to soon start building my next webapp around 100init.wart. In the meantime, I hope you'll find the code entertaining to read. Let me know what you think.

https://github.com/akkartik/wart/blob/master/boot.lisp