Arc Forumnew | comments | leaders | submitlogin
5 points by rocketnia 2930 days ago | link | parent

The essential trouble you'll have with this macro is that Arc macros don't know what variables are in their caller's lexical environment.

  (let a 1
    (w/tab (obj b 2)
      (+ a b)))
Since the place `b` will be looked up isn't determined unil run time, it has to use `eval`. But a call to `eval` always uses the global scope, so we lose `a`.

We could work around this by changing the way Arc's macroexpander worked. For now, let's not worry about that. I'll return to this later.

---

I see some particular bugs that can be fixed in your code.

First, you're inserting the bound values as flattened expressions in your generated code, so they'll actually be flattened executed. That is, if you write (w/tab (obj x '(+ (+ 2 y) 4)) ...), then I think `x` will be bound to the function `+`, `+` will be bound to 2, and `y` will be bound to 4.

If that's not what you want, we can make this change:

  -    `(let ,bindings (flat (tablist ,ht))
  +    `(let ,bindings (mappend [let (k v) _ `(,k ',v)] (tablist ,ht))
Second, you're assigning a nonlocal variable there, `res`. I don't know if you were doing that on purpose, but here's a version that avoids doing that:

   (mac w/tab (ht . body)
  -  (w/uniq (bindings k)
  +  (w/uniq (bindings res k)
       `(let ,bindings (mappend [let (k v) _ `(,k ',v)] (tablist ,ht))
          (eval `(with ,,bindings
  -                (= res (do ,@',body))
  -                (each ,',k (keys ,',ht)
  -                  (= (,',ht ,',k) (eval ,',k)))
  -               res)))))
  +                (let ,',res (do ,@',body)
  +                  (each ,',k (keys ,',ht)
  +                    (= (,',ht ,',k) (eval ,',k)))
  +                  ,',res))))))
Third, as you observed, the (eval ...) inside the (each ...) throws an error. What's going on is that `eval` evaluates things in the global scope, and you're trying to get a local variable. I recommend taking out the (each ...) loop and replacing it with a sequence of assignments. This means the keys you're assigning to will be determined based on the initial state of the table, rather than the final state, but that corresponds to the local variables that have been made anyway.

   (mac w/tab (ht . body)
  -  (w/uniq (bindings res k)
  +  (w/uniq (ht-list res)
  -    `(let ,bindings (mappend [let (k v) _ `(,k ',v)] (tablist ,ht))
  +    `(let ,ht-list (tablist ,ht)
  -       (eval `(with ,,bindings
  +       (eval `(with ,(mappend [let (k v) _ `(,k ',v)] ,ht-list)
                   (let ,',res (do ,@',body)
  -                  (each ,',k (keys ,',ht)
  -                    (= (,',ht ,',k) (eval ,',k)))
  +                  ,@(map [let (k v) _ `(= (,',ht ',k) ,k)] ,ht-list)
                     ,',res))))))
Fourth, this code evaluates the `ht` expression each and every time it does an assignment. This might be okay, except it's evaluating the expression in the local scope the first time and the global scope all the other times, which will probably lead to annoying errors. Let's evaluate it only once, in the local scope:

   (mac w/tab (ht . body)
  -  (w/uniq (ht-list res)
  +  (w/uniq (ght ht-list res)
  -    `(let ,ht-list (tablist ,ht)
  +    `(withs (,ght ,ht ,ht-list (tablist ,ght))
          (eval `(with ,(mappend [let (k v) _ `(,k ',v)] ,ht-list)
                   (let ,',res (do ,@',body)
  -                  ,@(map [let (k v) _ `(= (,',ht ',k) ,k)] ,ht-list)
  +                  ,@(map [let (k v) _ `(= (',,ght ',k) ,k)] ,ht-list)
                     ,',res))))))
I'm doing something tricky there: I'm inserting the table itself as a quoted value. It's possible to avoid this with a pattern like ((eval `(fn (x) (... x ...))) x), but Arc lets us just do (eval `(... ',x ...)).

Er, actually, something's making that code not work on Anarki, even though it does work on Arc 3.1. Embedding a function (instead of a table) works on Anarki, so let's do that:

  -                  ,@(map [let (k v) _ `(= (',,ght ',k) ,k)] ,ht-list)
  +                  ,@(map [let (k v) _ `(',[= (,ght k) _] ,k)] ,ht-list)
Here's the resulting code:

  (mac w/tab (ht . body)
    (w/uniq (ght ht-list res)
      `(withs (,ght ,ht ,ht-list (tablist ,ght))
         (eval `(with ,(mappend [let (k v) _ `(,k ',v)] ,ht-list)
                  (let ,',res (do ,@',body)
                    ,@(map [let (k v) _ `(',[= (,ght k) _] ,k)] ,ht-list)
                    ,',res))))))
As akkartik was saying, nested quasiquotation can be hard to get right. We can avoid nested quasiquotation if we factor the code a bit differently:

  (def w/tab-fn (ht body)
    (let ht-list tablist.ht
      (eval:w/uniq res
        `(with ,(mappend [let (k v) _ `(,k ',v)] ht-list)
           (let ,res (do ,@body)
             ,@(map [let (k v) _ `(',[= ht.k _] ,k)] ht-list)
             ,res)))))
  
  (mac w/tab (ht . body)
    `(w/tab-fn ,ht ',body))
Here it is in action:

  arc> (let a 1 (w/tab (obj b 2) (+ b b)))
  4
  arc> (let a (obj b 2) (w/tab a (= b 4)) a)
  #hash((b . 4))
  arc> (let a 1 (w/tab (obj b '(+ (+ 2 y) 3)) (join b b)))
  (+ (+ 2 y) 3 + (+ 2 y) 3)
---

As I mentioned above, there's one case that's sadly impossible to support without hacking on the language a bit:

  arc> (let a 1 (w/tab (obj b 2) (+ a b)))
  Error: "reference to undefined identifier: _a"
Okay, here's a really hackish way to add this capability to the language.

We can make this six-line modification to ac.scm:

  +(define latest-macex-env* 'nil)
  +
  +(xdef get-latest-macex-env (lambda () latest-macex-env*))
  +
   (define (ac-call fn args env)
  +  (set! latest-macex-env* env)
     (let ((macfn (ac-macro? fn)))
       ...))
  
   ...
  
   (define (ac-macex e . once)
  +  (set! latest-macex-env* 'nil)
     (if (pair? e)
       ...))
Now we can update the macro:

  (def w/tab-fn (caller-env ht body)
    (let ht-list tablist.ht
      (eval:w/uniq res
        `(with ,(mappend [let (k v) _ `(,k ',v)] tablist.caller-env)
           (with ,(mappend [let (k v) _ `(,k ',v)] ht-list)
             (let ,res (do ,@body)
               ,@(map [let (k v) _ `(',[= ht.k _] ,k)] ht-list)
               ,res))))))
  
  (mac w/tab (ht . body)
    `(w/tab-fn (obj ,@(mappend [do `(,_ ,_)] (get-latest-macex-env)))
       ,ht ',body))
Ta-da!

  arc> (let a 1 (w/tab (obj b 2) (+ a b)))
  3
We have effectively called `eval` in a local scope. We constructed the local scope table explicitly and then explicitly set it up again in the generated code.

---

If anyone would like to incorporate that six-line hack I demonstrated into Anarki, I think it will be a bit nicer if (get-latest-macex-env) were dynamically scoped rather than permanently mutated on each call. This will only ever matter if a macro invokes `eval`, `macex`, or `macex1` during its own macroexpansion, but hey, it could happen!

(This is effectively very similar to Kernel fexprs. The similarity would be even stronger if the environment were a table that mapped each variable to its local macroexpander. Arc doesn't have locally scoped macros, but I think this is a good way to go if we want them.)



4 points by mpr 2929 days ago | link

Wow great post, thanks for the reply. I didn't know eval was always executed at global scope, and I was not thinking about the ht variable being executed multiple times at different scope, so thanks for pointing that out. The rest of your explanation about how to get it working is very clear and I will be a better arc hacker for it!

The occasion for this macro is an object system built around closures and hash tables. I will post more about it when I've made more progress.

-----

3 points by mpr 2928 days ago | link

Hey, could you expand on what we might do to hack local macros into arc? I'm really keen on writing macrolet

-----

2 points by rocketnia 2928 days ago | link

It would take a bit of refactoring: Changing the `env` list to a table, adding it as a parameter of `ac-macro?` (which should now look things up from that table first), and finally adding an `env` parameter to Arc's `eval`.

-----