Arc Forumnew | comments | leaders | submitlogin
New in wart: def and mac are now open to overloading
3 points by akkartik 1284 days ago | 12 comments

  $ git clone https://github.com/akkartik/wart.git # needs SBCL
  $ cd wart
  $ git checkout 7b5109581b3c5d56c767e618f903c12cdd5173f7
  $ wart
  wart> (def fact(n)
          (* n (fact (- n 1))))
  fact
  wart> (def fact(n) :case (iso 0 n)
          1)
  fact
  wart> (fact 8)
  40320
While this style of definition with guarded clauses may be reminiscent of functional languages[1], I plan to use guards only where I may want to add more in the future. Some use cases:

a) Currying:

  (def foo(a) (fn(b) (foo a b)))
  (def foo(a b) (+ a b))
  foo.3.4 ; 7
b) Overloading macros. To allow let to define recursive local functions (http://arclanguage.org/item?id=13763):

  (mac let(f . body) :case (match f '(def . _))
    `(labels (,(cdr f))
       ,@body))
c) I'm going to replace my old defgeneric/defmethod setup to just use def.[2]

---

[1] In haskell, for example:

  fac 0 = 1
  fac n = n * fac (n-1)
[2] In acknowledgement that the approach of http://awwx.ws/extend is superior :)


2 points by rocketnia 1283 days ago | link

In acknowledgement that the approach of http://awwx.ws/extend is superior :)

Great to see you convert at last. ^_^

Some places I'd go from here, if it were up to me:

- Individual rule names. If I mess up entering a method implementation, I like to be able to edit that command and re-paste it, overwriting the old one. One of aw's older versions of 'extend used names to allow this, and Inform 7 has named rules too. Your 'defgeneric actually had this already, with the types themselves acting as names.

- Explicit rule precedence management, like I have in Lathe (but not in core Penknife). This powers Lathe's inheritance system, for instance.

- A way to do something if a method call runs out of cases, like I've mentioned having in my Penknife draft (but not in the current Penknife or Lathe). This lets someone extend one utility and automatically gain access to a bunch of other utilities. The way I plan to accomplish this--an extra "fail" parameter for every function--would totally conflict with the "thin layer" aspect of Wart, but maybe there's another way. Hmm, I already have half an idea, so you should see another post from me soon. :)

-----

2 points by rocketnia 1283 days ago | link

A way to do something if a method call runs out of cases ... Hmm, I already have half an idea, so you should see another post from me soon. :)

Well, here's the idea, which I've sketched up as a blob of untested and sorta-idealistic Arc. I'm using dynamic variables to simulate a "fail" parameter, but it's not quite as simple as it might sound. Until now, I didn't know it was possible; in fact, the strategy I'm using wouldn't cooperate with similar parameter-simulating frameworks. There are two tricks:

Since the surrounding language is full of fail-unaware "muggle" functions, I have to set up those variables inside the function implementation (for when a failure parameter isn't provided), as well as sometimes outside the function whenever the fail parameter is supposed to be passed.

If we stopped there, then if we failcalled a muggle function, any failure-aware functions it called would think we were failcalling them. To avoid that, we use our second trick: In 'failcall, we test the function to see whether it's fail-aware first, and if it's not, we don't set up the dynamic parameters. The catch is that it may not be easy to "test the function" this way; it depends on whether the underlying language supports 'defcall or weak sets. Here, I'll pretend the language has weak set support. (Using this framework probably invalidates 'defcall anyway, since ideally, you'd be able to 'defcall something so that it was fail-aware. That said, this weak set approach may work quite well with a 'defcall that coerces to 'fn.... Wow, unexpected 'coerce niftiness.)

There's one other thing to note: In my Penknife draft, [failfn fail [a b c] ...] and [failfn* fail [a b c] rest ...] are the most basic kinds of function syntax, but one of these "fail" parameters isn't actually the fail parameter the function was called with; instead, it's an escape continuation that calls the given fail parameter instead of continuing with the function. In my experience with Lathe's rulebooks, this continuation-handling is less efficient than explicit branching in each rule, but I'm glad to sacrifice a little efficiency here. :)

Okay, here's the code. (Remember, it's untested!) Insert your own varieties of dynamic parameters and weak sets:

  ; This branches to a different strategy when a continuation is called.
  ; Note that it's a pretty significant performance bottleneck.
  (def fn-onpoint (alternate body)
    (let (called result)
           (catch:list nil (body:fn args (throw:list t args)))
      (if called
        result
        (apply alternate result))))
  
  (mac onpoint (point alternate . body)
    `(fn-onpoint ,alternate (fn (,point) ,main)))
  
  
  (defdynamic fail* nil)
  (defdynamic failcalling* nil)
  (= fail-awares* (make-weak-set))
  
  (def failcall (func fail . args)
    (if (mem fail fail-awares*)
      (onpoint throw fail
        (w/dynamic (failcalling* t fail* throw)
          (apply func args)))))
  
  (def failable-wrapper (inner-func)
    (ret result (afn args
                  (if failcalling*
                    (w/dynamic (failcalling* nil)
                      (apply inner-func args))
                    ; The default failure is 'err.
                    (apply failcall self err args)))
      (push result fail-awares*)))
  
  
  ; Make partial functions using this.
  ; TODO: Make number-of-argument errors into failures.
  (mac failfn (fail parms . body)
    (w/uniq g-return
      `(failable-wrapper:fn ,parms
         ; We wrap the failures up in a value that should be easy to
         ; pretty-print (whenever we get around to writing such a
         ; pretty-printer).
         (onpoint ,fail [fail*:annotate 'function-failure
                          (list '(failfn ,fail ,parms ,@body) ,_)]
           ; The value of 'fail* shouldn't be used after this point,
           ; but we don't enforce that.
           ,@body))))
  
  ; This is an exception. I don't think it can be implemented without
  ; using 'failable-wrapper directly.
  (= fail-aware-apply (failable-wrapper:fn (func . args)
                        (apply apply func args)))
  
  
  (def fn-ifsuccess (func args then else)
    (failcall else (failfn fail ()
                     (then:apply failcall func fail args))))
  
  ; TODO: See if ssyntax should be supported.
  (mac ifsuccess (success failure (func . args) then . elses)
    `(fn-ifsuccess ,func (list ,@args) (fn (,success) ,then)
                                       (fn (,failure) (if ,@elses))
  
  (def failcall-cases (cases fail wrap-details collected-details . args)
    (ifdecap (case . rest) cases
      (ifsuccess success failure (fail-aware-apply case args)
        success
        (apply failcall-cases
          rest fail (cons failure collected-details) args))
      (fail wrap-details.collect-details)))
  
  
  ; Without further ado, here's one way to explicitly set up a generic
  ; function.
  
  (= fact-cases* nil)
  (= fact (failfn fail args
            (apply failcall-cases fact-cases*
              ; We wrap the failures up in a value that should be easy
              ; to pretty-print (whenever we get around to writing such
              ; a pretty-printer).
              [annotate 'rulebook-failure (list 'fact _)]
              fail nil args)))
  
  (push (failable-wrapper:failfn fail (n)
          (* n (fact:- n 1)))
        fact-cases*)
  
  (push (failable-wrapper:failfn fail (n)
          (unless (is n 0)
            (fail "The number wasn't 0."))
          1)
        fact-cases*)
Since Lathe already has support for dynamic parameters on every platform but Rainbow, and since Racket has support for weak tables, I should be able to get this in Lathe and tested soon enough. I think I've been able to use WeakHashTables from Jarc too, so this will hopefully cover three out of Lathe's four target platforms. (WeakHashTables are a bit of a stretch since they compare with equals(), but all my keys would be procedures, so that's probably close enough.)

-----

2 points by rocketnia 1282 days ago | link

It's in Lathe now! It works on Arc 3.1, Anarki, Jarc, and Rainbow, but only as long as you don't reenter continuations in Rainbow. (That's the same scope of support as Lathe's dynamic boxes.)

https://github.com/rocketnia/lathe/blob/master/arc/failcall....

There were an awful lot of embarrassing bugs to work out, like parameters out of order and stuff, and the fact that it was easy to implement 'fail-aware-apply using 'failfn after all, but it they're squashed. Woo! I even put the 'fact example in there, but I'll probably move that into a separate unit test.

One thing this means is that I should be able to port a lot of my Penknife draft's utilities into Arc. It won't have Penknife's hygiene support, and the module system will still be horribly limited, but Lathe has the advantage of already having a self-ordering rule library and, you know, actually existing. ^_^

Actually, it also means I might be able to reuse my existing Penknife core and hack this onto it. Nahh, the core in my draft is already far better designed for extensibility, even if it doesn't work....

I don't see any difficulty in translating fail parameters to Wart, only complexity. :-p What do you think?

-----

1 point by rocketnia 1282 days ago | link

For what it's worth, I've abstracted away the parameter simulation and put it in dyn.arc, which is where Lathe's dynamic box API is kept. Now you can define your own "secretargs" with default values and have it automatically seem like they're passed to every function in the language. It's not necessarily pretty; the point is to build things like 'failcall on top of it.

  ; Surprise, this code has been tested!
  
  (= lathe-dir* "your/path/to/lathe/arc/")
  (load:+ lathe-dir* "loadfirst.arc")
  (use-rels-as dy (+ lathe-dir* "dyn.arc"))
  
  (= secret1 (dy.secretarg 4))
  (= secret2 (dy.secretarg 5))
  (= secret3 (dy.secretarg 6))
  (dy:call-w/secrets (dy:secretarg-fn (a b c)
                         d secret1
                         e secret2
                       (list a b c d e (dy.secretargs)))
    (list (list secret1 9)
          (list secret3 200))
    1 2 3)
  
  => (1 2 3 9 5 ((#(tagged ...) 9) (#(tagged ...) 200)))
There's a catch, though, which is that general-purpose function-calling functions like 'apply and 'memo don't propagate secretargs, and there's probably nothing I can do about that aside from hacking on the language core. This is the same kind of compatibility-breaking I brought up when we were talking about keyword arguments.

Fortunately, secretargs are pretty general-purpose--they're pretty much just keyword arguments with non-symbol keys--so things like keyword argument systems can be implemented on top of them. Just make a secretarg that stores a special-purpose keyword map.

I'm not sure if that'll come up in Lathe for a long time, though. For now the only benefit is that it's a little easier to explain 'failcall and 'failfn: they use a non-symbol keyword argument (a secretarg) to communicate, and 'failfn makes that argument into a more natural failsafe by chaining it to an escape continuation.

-----

1 point by evanrmurphy 1282 days ago | link

You seem very excited about this feature, but I'm afraid I'm having trouble understanding what it's about. :-/

Is it a sort of base case upon which all functions' extend layers can be built up? What you've written about it so far reminds me of Ruby's method_missing. Is that a related idea?

-----

2 points by rocketnia 1282 days ago | link

You seem very excited about this feature, but I'm afraid I'm having trouble understanding what it's about. :-/

I expected that. ^_^;; I kept trying to find ways to emphasize the motive, but I kept getting swept up in minutia.

---

Is it a sort of base case upon which all functions' extend layers can be built up? What you've written about it so far reminds me of Ruby's method_missing. Is that a related idea?

I think you're on the right track. The point of my fail parameter is to explicitly describe cases where a function is open to extension. Sometimes a function call has a well-defined return value, and sometimes it has a well-defined error value. If it doesn't end in either of those ways and instead fails--by calling its fail continuation--then it's undefined there, and you're free to replace it with your own function that's well-defined in all the same cases and more.

That was the original idea, anyway. It's a little flawed: If another function ever actually catches a failure and does something with it, then its behavior relies on the callee being undefined in certain cases. Take 'testify for instance; if 'testify checks a 'testify-extensions function first but acts as [fn (x) (iso x _)] on failure, then you can't actually extend 'testify-extensions without changing the behavior of 'testify. In fact, if you replace a function with another function that calls the first one ('extend as we usually know it), then you can no longer extend the first one without impinging upon the new one. But in practice, I think it'll work out well enough.

The alternatives ways to deal with extension I see are a) encoding defined-ness in the return value using a wrapper type, and b) throwing an exception. In a language where almost everything's extensible, to have to unwrap every return value would be madness. As for exceptions, it's hard to tell whether the exception came from the function you called or some function it called. In fact, I was actually trying figure out how to have function calls take ownership of exceptions when I finally realized there needed to be a per-function-call ID, generated on the caller's side; this idea refined a little to become fail parameters.

I know, I digressed a bit again. XD Did it work for you this time?

-----

1 point by rocketnia 1282 days ago | link

Also, in my Penknife draft, I'm finding myself much more comfortable using failures rather than exceptions, letting the failures become exceptions when something's called without 'failcall. Failures form a natural tree of blame; this rulebook failed because all these functions failed, and this function failed because this other function failed....

The need to write natural-language error strings diminishes to just the base cases; the rest is pretty-printing and perhaps even a bit of SHRDLU-style "Why did you decide that?" interactive tree-drilling. ^_^

Arc's known for having rotten errors 'cause it shirks the error-checking code; with fail parameters, awesomely comprehensive error reports can be made in almost a single token.

-----

1 point by evanrmurphy 1284 days ago | link

So you've baked extend into def and mac? This is very interesting!

Should = gets the same benefits? Since wart is a lisp-2 I guess it doesn't really apply [1], but if it were a lisp-1 would you treat (= fact (fn(n) ...)) the same way?

---

[1] Or does it? I'm not yet well-versed in the ways of lisp-2-ness...

-----

1 point by akkartik 1284 days ago | link

That's a good question. My immediate reaction is to make = clear the overload table..

And yes, in a lisp-2 assigning a function is different from defining one. But you can assign to (symbol-function f) in common lisp, which would have the same effect as def.

-----

1 point by akkartik 1283 days ago | link

Update: wart now has no defgeneric.

Switching to def :case costed[1] me 28 LoC.

(e3cda3b487eb4280215bc322101e7844f7f7656c to 3d58df8ba744ff6e8c182b49206448addb9273ed)

[1] http://akkartik.name/blog/28672493

-----

2 points by rocketnia 1283 days ago | link

I thought maybe a compare view would help visualize which parts of the code grew and shrank:

https://github.com/akkartik/wart/compare/e3cda3b487eb4280215...

Now I'm not so sure. :-p It would take some effort to comprehend all those changes at once.

-----

1 point by akkartik 1283 days ago | link

:) Yeah, there was a lot of reorg. I try really hard to make each commit well-behaved, though, so the changelog should be useful.

-----