Arc Forumnew | comments | leaders | submitlogin
Lexically scoped macros?
4 points by aw 83 days ago | 8 comments
Suppose I wanted to use some tests from Anarki:

    (suite cut
           (test finds-element-in-list
                 (assert-same '(3 4 5) (cut '(1 2 3 4 5) 2)))
           ...)
I'm already using "test" for something else, but I'd like to be able to use the Anarki tests as they are without having to rename all the "test" macros.

I could use something like:

    (anarki-test-suite  ; or e.g. zck-unit-test
      (suite cut
             (test finds-element-in-list
                   (assert-same '(3 4 5) (cut '(1 2 3 4 5) 2)))
             ...))
Which would say that within the lexical scope of "anarki-test-suite", "suite" and "test" would refer to Anarki's versions of these macros.

The Arc compiler keeps track of lexical variables introduced by "fn", and similarly could keep track of lexical macros introduced by some "lexical-macro" form.

   (lexical-macro test anarki-test
     (test finds-element-in-list ...))
And then "anarki-test" would be evaluated at compile time? Hmm...

Meanwhile, anarki-test-suite" would be a plain macro that expanded into the lexical-macro form.

Has anyone tried something like this? Any preferred way of being able to introduce lexically scoped macros that you like?



3 points by rocketnia 82 days ago | link

A situation where Common Lisp's `macrolet` or Racket's `let-syntax` would be handy came up here a few months ago. I dropped in with some thoughts on how we could add it to Arc: http://arclanguage.org/item?id=20561

I think what I describe there pretty much meshes with what you and waterhouse are describing here. :)

---

"And then "anarki-test" would be evaluated at compile time? Hmm..."

Yeah, I think basically every instance of local macros I've seen involves evaluating the expression at compile time.

That's even what Racket's `let-syntax` does, although in Racket's case it involves a little more detail since Racket enforces a strict separation between compile-time and run-time side effects. When Racket evaluates the expression at compile time (usually, in phase level 1) it first expands that expression in the phase level corresponding to the compile time of the compile time (phase level 2), and if that expression contains another `let-syntax`, then it starts expanding an expression in phase level 3 and so on.

Arc evaluates and expands everything in one phase, as far as Racket is concerned. It would make sense for `lexical-macro` to do its evaluation in the same phase too.

I notice Common Lisp's `macrolet` allows[1] inner `macrolet` macros to depend on outer ones, like this:

  (lexical-macro (foo) ''world
    (lexical-macro (bar) `(sym:+ "hello-" ,(foo))
      (bar)))
  ; could return the symbol 'hello-world
To support that, when `lexical-macro` evaluates the expression, it should expand that expression in the same macro binding environment the `lexical-macro` call was expanded in.

[1] http://www.lispworks.com/documentation/HyperSpec/Body/s_flet...

---

Here's a different take on the local macro concept by almkglor 10 years ago: http://arclanguage.org/item?id=3085

That implementation, `macwith`, is still around on Anarki's arc2.master branch, in the file lib/macrolet.arc: https://github.com/arclanguage/anarki/blob/af5b756e983807ba6...

It works by traversing the body s-expressions and expanding all occurrences it finds, leaving anything else alone. I think this means it tends to break if the code ever contains a list that looks like a call to that macro but isn't supposed to be, such as a quoted occurrence like '(my-macro), a binding occurrence like (fn (my-macro) ...), or even another `macwith` binding occurrence of the same macro name.

I don't prefer this to the other approach we're talking about, since after all it's within easy reach of the macro system to support local macros itself rather than with an error-prone code-walker like this.

However, `macwith` is a macro that makes sense in its own right. It's easy to work around many of the places the code walker runs into false positives, and if `macwith` came with support for an escape sequence, there would be easy workarounds in many other cases too.

If I tried to propose a particular escape sequence design for `macwith` right here and now, I could be here for a while. I've been working for two years to make a macro system suitable for factoring out escape sequence syntaxes into libraries, and my favorite designs for `macwith` escape sequences would be the ones that solved all the same problems I'm building that system for.

-----

5 points by zck 82 days ago | link

Another solution to this would be to have an actual namespace mechanism. For example, this would be the Clojure for a similar test file:

    (ns anarki.foo-tests
      (:require [zck.unit-test :as t]))
    
    (t/suite cut
             (t/test finds-element-in-list
                     (t/assert-same '(3 4 5) (cut '(1 2 3 4 5) 2))))
It's another benefit we'd get by having namespacing.

-----

3 points by akkartik 82 days ago | link

I don't think this has come up before. An f-expr based Lisp would automatically have this property (at much performance cost). But we haven't discussed lexical macros as a separate construct.

I feel like something kinda related has come up a couple of times before: allowing local variables to override macros. I think there was even a one-line patch to ac.scm at some point. But it must have had some issue since it's not in Anarki :)

Maybe this thread brings up some ideas? http://arclanguage.org/item?id=3056

-----

2 points by aw 82 days ago | link

> allowing local variables to override macros

Yes, that's easy to do. At the top of ac-call in ac.scm where it checks ac-macro?, also check to see that "fn" isn't a symbol and in env.

-----

3 points by waterhouse 82 days ago | link

Yup, I have that change.

(define (ac-call fn args env) (let ((macfn (ac-macro? fn))) - (cond (macfn + (cond ((and macfn (not (lex? fn env))) (ac-mac-call macfn args env))

Now, if you wanted to define the macro locally, like this...

  (let-macro my-def (name args . body) `(= ,name (fn ,args ,@body))
    ... (my-def ...))
Then, well, you could conceivably change the semantics of "env" as it's passed around in ac.scm. Currently it's just a list of variables that are bound, and things test for whether a variable is present in that list. You could change it to a list of (variable-name macro-it's-bound-to-if-any), and have the special form (let-macro name arglist bodexpr . body) insert `(,name (fn ,arglist ,bodexpr) into env, while everything else puts in (variable nil), and change all the existing tests on "env" to search for "a list whose car is x" rather than "x", and lastly make ac-call call the macro-function on the expression if it finds one in the lexenv.

In theory, one could put arbitrarily complicated information, such as about deduced types of variables, into this "env" mapping, and implement some amount of compiler optimization that way.

First-class macros, of course, are the semantically nicest approach, but more difficult to compile.

-----

3 points by waterhouse 82 days ago | link

Gack, formatting got messed up on the first snippet. Too late to edit. Should be:

   (define (ac-call fn args env)
     (let ((macfn (ac-macro? fn)))
  -    (cond (macfn
  +    (cond ((and macfn (not (lex? fn env)))
              (ac-mac-call macfn args env))

-----

4 points by shawn 81 days ago | link

Actually, here's an Arc solution.

When I implemented Arc in Lumen (more info on Lumen: http://arclanguage.org/item?id=20935), this technique was ported over naturally. You can play with it here:

  git clone https://github.com/lumen-language/lumen
  git checkout 2018-10-29/arc
  npm i
  rlwrap bin/lumen-node # rlwrap is optional
  > (load "arc.l")
  function
  > (mac awhen (cond . body)
      `(let it ,cond
         (when it ,@body)))
  > (awhen 'hi (print it))
  hi
  > (awhen nil (print 'hi))
  > (awhen 1 (print it))
  1
  > (awhen 1 (print it)
      (let-macro ((awhen (cond . body)
                    `(let it ,cond
                        (unless it ,@body))))
        (awhen false (print it)))
      (awhen 42 (print it)))
  1
  false
  42
  >
It's not quite a complete port, but it's pretty close. It also runs on node, so you can use any npm library in Arc.

-----

3 points by shawn 82 days ago | link

Yes, Lumen has this. https://docs.ycombinator.lol/tutorial/macros

https://imgur.com/a/CUccgz1

  (define-macro when (condition rest: body)
    `(if ,condition (do ,@body)))
  (when true
    (list 42
    (let-macro ((when (condition rest: body)
                  `(if (not ,condition) (do ,@body))))
      (when false
        21))
     (when true
       21)))

This gives a list of '(42 21 21).

If there is interest, I can port the technique to arc3.2 and let the community take it from there. We're using arc3.2 over at https://www.laarc.io

https://github.com/shawwn/arc3.2/tree/ln

-----