This is a tricky one! Turns out you're absolutely right, actually, this is sort of a bug (if you expect a different behavior of the function insert-sorted).
You're right in thinking that x and y share the same reference, like
The thing is, the only real way to get at pointers in Arc is to use sref, scar and scdr. Those will actually mutate the cons cell sitting in memory. We can verify this, plus the fact that x and y share the same reference, by using (say) scar:
arc> (= y (= x '(1 3 5)))
(1 3 5)
arc> x
(1 3 5)
arc> y
(1 3 5)
arc> (macex '(= (car x) 2)) ; '= expands into a form that uses 'scar
((fn () (atwith (gs1731 x gs1732 2) ((fn (val) (scar gs1731 val)) gs1732))))
arc> (= (car x) 2)
2
arc> x
(2 3 5)
arc> y
(2 3 5)
The problem is that insort doesn't truly mutate the cons cell of its argument. It's a lazy-man's "destructive", because it uses the zap macro to reassign x to a separate, insorted version of the list:
arc> (= y (= x '(1 3 5)))
(1 3 5)
arc> (macex '(insort < 2 x))
(atomic-invoke
(fn ()
(withs (gs1736 x
gs1735 [insert-sorted < 2 _])
((fn (gs1737) (assign x gs1737))
(gs1735 gs1736)))))
Notice how the expansion calls insert-sorted on x, then uses (assign x ...) to change where x points. insert-sorted is just a normal, FP-style function that doesn't mutate its arguments, instead consing together a new list in memory:
Then the free-floating cons-cell from the insert-sorted, the old car (the 1), and the old cdr (the (3 5)) would all be garbage-collected eventually. (Modulo implementation details, of course, like the numbers actually sharing the same spots in memory instead of being duplicated. But it makes the diagram cleaner.)
On line 5, he mentions that hours_to_minutes has its return value named minutes. And I can see how time.minutes would be "named" minutes. But he's really baking a lot of tacit semantics into the + operator that aren't actually obvious...
In a Python-like language, there's a distinction between the built-in "plus" statement and a "plus" function. Since + isn't a function, I'm assuming the "return value" doesn't need its own name. However, he's clearly under the impression that the expression "inherits" the names of its arguments somehow. But exactly how is unclear.
Variables obviously have names (name(var) = var). But what are the names of literals? Presumably it's kind of just "no name" (the _ wildcard he mentions?), so no errors arise from something like
any_way_you_want_it = "that's the way" + " " + "you need it"
Or does he propose to use names for literals to accomplish strong-typing? Like the name of any string is str and the name of any integer is int so that 10 + "1" would somehow be an error? If that were the case, wouldn't you be forced to use the name str instead of the above any_way_you_want_it? As far as I can tell, strong-typing would be relegated to something outside of the strong-naming system. Which might be an acceptable amount of complexity, as long as we're under no illusions about the universality of the strong-naming system.
What about mixing literals with named values in expressions? The result of an expression seems to take on the name of the named operands, as in his example where minutes needs the <- annotation:
minutes<-hours = hours * 60
These semantics are feasible. Really, the problem the compiler would need to solve would just be a simple dataflow analysis across a "names" lattice, similar to constant propagation (http://ecee.colorado.edu/~waite/Darmstadt/data.html#cprop). The lattice would have top = bad name, bottom = no name, and the "flat" elements in the middle tier (as in constant propagation) would be every possible name. Then the join operator would go
join(bottom, bottom) = bottom
join(name, bottom) = name
join(bottom, name) = name
join(name, name) = name
join(name1, name2) = top (where name1 != name2)
join(top, _) = top
join(_, top) = top
OK, so this makes sense. But it's overly restrictive. In particular, I don't know that join(name1, name2) = top is right:
hello = "hi, "
world = "planet"
hello_world = hello + world # error, since join(hello, world) = top
# and yet...
hello_world = "hi, " + "planet" # not an error: join(bottom, bottom) = bottom
hello and world individually did nothing to warrant their names: they've just been assigned to literals. But they are named (right?), and now hello + world is improperly-named, even though doing constant propagation would resolve the problem.
Could we coerce the name of hello_world to allow the mixing of hello and world? The <- notation seems unsuitable, because it only allows the renaming of a single name, as in the minutes<-hours example. Unless maybe you use a wildcard to switch off...well, really the whole strong-naming system, I guess:
hello_world<-_ = hello + world
This example isn't entirely silly, either. I've had plenty of times in code where I name intermediate values instead of just altering the same variable repeatedly. E.g., total_amount = amount_saved + amount_deleted.
At any rate, traditional copy propagation certainly becomes a nightmare. Of course, this is probably deemed a worthwhile trade-off in such a strong-naming system.
On another note, I think there's an error of omission. While he addresses the input parameters whose names you don't care about, what about functions that have no meaningful name for their return values? I think it would be appropriate to have something like
def _:reciprocal(_:n):
return 1.0 / n
(Unless you want to use a Pascal-like system where function return values are named the same thing as the function.)
It's weird though, because it further increases the distance between syntactic expressions and user-defined functions. If + were a function, it'd be one of those declared with a _ return name, because there's no meaningful way to predict the names of what you'll be adding. You'd have something like
def _:+(_:n, _:m):
while n > 0:
n = n - 1
m = succ(m) # primitive "plus one" function...again, a _ return name
return m
But if this were the case, you'd no longer have the "name propagation" in +-expressions, so his example
You're back to square one. So this strong-naming system encourages the sort of function/statement separation Python gets criticized for!
I dunno. I'm thinking the strong-naming system happens to work well for his example just because his example has variables named after units. I'm skeptical about how well the system would scale to other code bases. Good names are important, yes, but building a type system around them? Plus, it kind of puts names on a pedestal, which I think encourages mutation, which is runs somewhat counter to my normal mode of thinking.
You're absolutely right in that I've glossed over quite a few details, and clearly I've screwed up in some ways that were immediately obvious to you :P
I guess the first thing I'd have to introduce for the whole issue of basic operations destroying the entire system is to introduce something like:
def _foo:+(_foo:n, _foo:m):
...
i.e. that "underscore something" is a meta-name.
Then we have:
hello = "hi, "
world = "planet"
hello_world = hello + world # error, since join(hello, world) = top
# and yet...
hello_world = "hi, " + "planet" # not an error: join(bottom, bottom) = bottom
yes, this is exactly the point. The idea is that if you're giving it a name, that actually means something pretty significant, much more than just a way to access some specific bit of RAM with a bunch of ascii characters as reference.
I don't know how it'd scale but I imagine that many users of such a theoretical system would find it really annoying to write in the beginning without good tooling (kind of like Java with all those getters and setters :P). I do believe though that this is also the case in Objective-C and that in ObjC this tradeoff is appreciated later when maintaining big code bases. Maybe my rather draconian idea is unfeasible because of this reason alone!
I would like to have the short forms from my examples in ObjC though.. writing "x:x y:y width:width height:height" is actually something that happens quite a bit ^_-
I'm honored! (And I didn't even have to sign up for GitHub!) How'd you find your way over to this forum?
You're absolutely right in that I've glossed over quite a few details, and clearly I've screwed up in some ways that were immediately obvious to you :P
If it seems like I found it immediately obvious, it's only because I took a long time writing & discovering these thoughts before committing them to one big post. :)
yes, this is exactly the point. The idea is that if you're giving it a name, that actually means something pretty significant, much more than just a way to access some specific bit of RAM with a bunch of ascii characters as reference.
I would like to have the short forms from my examples in ObjC though.. writing "x:x y:y width:width height:height" is actually something that happens quite a bit ^_-
I do lack perspective on this because I've never used Objective-C. I mean, apparently other people already commit to more-verbose ways of having strict "name enforcement" (though I'm not sure how many people use Objective-C by choice...). It's certainly an interesting idea, and I hope to see how it develops.
(Note, this explanation is for the standard Arc 3.1, not Anarki.)
They're not defined in Arc itself. They're handled by ac.scm, the Arc "compiler" written in Scheme. Specifically, the top-level function ac takes an Arc expression and transforms it into equivalent Scheme code. The function takes the form of a big cond, like:
(define (ac s env)
(cond ((string? s) (ac-string s env))
((literal? s) s)
((eqv? s 'nil) (list 'quote 'nil))
((ssyntax? s) (ac (expand-ssyntax s) env))
...))
The ! and . operators are part of what's called ssyntax, which you can see is involved in the last clause I included in the cond snippet above. I've always thought it's ambiguous what the first "s" stands for: probably "special" or "symbol". At any rate, we can see further down that ssyntax? is defined like so:
(define (ssyntax? x)
(and (symbol? x)
(not (or (eqv? x '+) (eqv? x '++) (eqv? x '_)))
(let ((name (symbol->string x)))
(has-ssyntax-char? name (- (string-length name) 1)))))
(define (has-ssyntax-char? string i)
(and (>= i 0)
(or (let ((c (string-ref string i)))
(or (eqv? c #\:) (eqv? c #\~)
(eqv? c #\&)
;(eqv? c #\_)
(eqv? c #\.) (eqv? c #\!)))
(has-ssyntax-char? string (- i 1)))))
First, note that + and ++ never get interpreted as ssyntax. This has to do with an earlier version of Arc that used + as a ssyntax operator (it got changed later to &). Also, _ is an exclusion; it's the special variable used by the bracket function syntax, where [+ _ 1] = (fn (_) (+ _ 1)). _ is not currently used as an ssyntax operator, but there's a lingering idea for stuff like +_1 to expand into [+ _ 1].
Otherwise, a symbol is considered to have ssyntax if it has any of the special characters: :, ~, &, ., or !.
The actual meaning of each ssyntax operator is a little bit complicated. It all goes through expand-ssyntax, defined by
As you can see, this dispatches to the different functions expand-compose, expand-sexpr, expand-and, and experimentally to expand-curry. Each of these functions will actually recursively call expand-ssyntax, so it turns out that you can have multiple ssyntax operators in the same symbol. This also means that the order of the cond clauses here is important, as they establish the precedence of each operator: : and ~ have higher precedence than . and !, which have higher precedence than &.
You can peruse the Scheme code further to find out how specifically each ssyntax operator works. But, you can also just use the ssexpand function in Arc, which will give you the result of calling the Scheme function expand-ssyntax, because ac.scm has the line
Can't believe I didn't see this anywhere in the 'newest' queue, because it (a) is incredibly relevant to Arc and (b) has been on the LtU front page since last month---although, it was apparently a repost. See http://lambda-the-ultimate.org/node/4691 and the previous thread linked therein.
Surprisingly insightful paper, for being such an easy read. Starts out a little humdrum, but really picks up in section 3. Definitely food for thought.
1. I like your final solution! But no reason why either of ours couldn't be done in vanilla arc.
2. (whenlet var expr ...) could also be written:
(let var expr
(when var ...))
So I don't think letwhen is strictly necessary.
3. Another alternative formulation is:
(let var expr :when (f var)
...)
All that's really saving us is a couple of parens compared to the original pattern. Maybe we don't need a macro, just a different indentation convention:
(let var expr (when (f var)
...))
But both these make the conditional nature of the block harder to see. My original solution and yours don't have that drawback.
Ah, there's an idea. Never thought to use the technique for avoiding slashed names. It makes sense. When you start writing Lisp macros, you quickly notice that you can use uninterpreted symbols as syntax. I remember way back when (I think probably when I first read Comprehending Monads by Philip Wadler), I toyed with writing a list macro to support not just the usual (list x), but also list comprehensions, like
(list (+ x 1) for x in xs if (> x 5))
That was a fun one. I don't think I have the code for it lying around anywhere. Not that it'd be hard to write again---it translates straightforwardly into maps & keeps. I suppose the end of this slippery slope would be the loop macro (http://clisp.hg.sourceforge.net/hgweb/clisp/clisp/file/tip/s...). Not necessarily a bad thing, if you aren't eschewing specialized syntax anyway.
Although, if the sole purpose here is to do away with slashed names, the only real place they're used in Arc is to shorten with- to w/.
So, though the mechanism by which it works is more general, you could stay disciplined by focusing on with instead of mac2 stuff. Something like the following, only in wart instead of Arc. :)
; Make 'let not rely on 'with, so it won't dispatch on special keywords by
; mistake
(mac let (var val . body)
`((fn (,var) ,@body) ,val))
(= special-with* (table))
(mac with args
(aif (special-with* (car args))
(apply it (cdr args))
(let (parms . body) args
`((fn ,(map1 car (pair parms))
,@body)
,@(map1 cadr (pair parms))))))
(mac withify (keyword parms . body) ; for want of a better name...
`(= (special-with* ',keyword) (fn ,parms ,@body)))
(withify uniq (vars . body)
`((fn ,vars ,@body) ,@(map1 [uniq] vars)))
(withify tmpfile (f . body)
`(let ,f (tmpfile)
(before (system:join "rm " ,f)
,@body)))
arc> (macex '(with (x 1 y 2) (+ x y)))
((fn (x y) (+ x y)) 1 2)
arc> (macex '(with uniq (x y) (do-something-with x y)))
((fn (x y) (do-something-with x y)) gs1740 gs1741)
arc> (macex '(with tmpfile foo (writefile 'stuff foo)))
((fn (foo) (before (system:join "rm " foo) (writefile (quote stuff) foo))) (tmpfile))
I'm not sure if that alleviates your keyword argument woes. It makes the definitions of with variants explicit without needing sigils or whatnot (because what else are you going to pass as the first argument to withify?). But, it also doesn't get along so well (aesthetically) with Scheme-style name-goes-in-arglist definitions.
irb(main):001:0> class A
irb(main):002:1> end
=> nil
irb(main):003:0> a = A.new
=> #<A:0x8f6dd3c>
irb(main):004:0> a == a
=> true
irb(main):005:0> a == a.clone
=> false
irb(main):006:0> x = "abc"
=> "abc"
irb(main):007:0> x == x
=> true
irb(main):008:0> x == x.clone
=> true
irb(main):009:0> (1..10) == 10
=> false
irb(main):010:0> (1..10) === 10
=> true
- Perl (near as I can tell) doesn't have anything in the way of user-definable comparison, but still has separate operators. Granted, I don't think the separation is related to the reference-vs-value thing---I don't do Perl much, so the semantics confuse me:
It'll be hard to appeal to the authority/popularity of most languages, because equality is a tricky concept with many meanings---whether or not it's specifically the address-vs-value (eq-vs-equal) thing.
Interesting, passing in one of exactly two procedures. I tried to come up with a way to use compare, but it doesn't work.
I did find it a little ugly, but I thought it would be better than hard coding one over the other. At least the data structure itself is simple to understand, those details notwithstanding. :)
Each bucket seems to act as a stack rather than a queue; elements come off in reverse order that they go in.
Probably because each bucket is a stack, here. :P I favored -push and -pop over -enq and -deq names for that reason. It'd of course be easy to use Arc's queues instead; we'd get the same complexities.
I suppose "priority queue" is a misnomer we got stuck with. At least, Wikipedia thinks FILO v FIFO is secondary to the "priority" part: https://en.wikipedia.org/wiki/Priority_queue.
You're right about the consing, though. It seems like there should be a lazy way of doing keys. But then, best hard-codes calls to both car and cdr upon the sequence. So, it looks like we'd be resigned to doing it by hand, and I'm not sure it's any better:
Ah, but this has made me notice an issue with the best keys approach to begin with: once we empty out the queue, !top will be nil, which won't compare correctly on the next bhpq-push.
OK, tried writing a program in wart so I could give my proper impressions / bug reports. I ported https://bitbucket.org/fallintothis/qq to http://pastebin.com/zfZhQ0tS. I think I have it right, except that testing the optimization code-paths is impossible due to certain bugs (see point #9 below). Does it look idiomatic?
1) The slowest part of Wart I'm finding is just starting the REPL. Running programs hasn't been an issue yet. However, it takes a really long time to start up, which wouldn't be nearly as bad if it didn't die at innocuous errors, like unbound symbols. Could more errors be captured by the REPL so I don't have to keep restarting slow things and losing my place?
2) Bug report: loading non-existent filenames causes wart to hang. Luckily, I had an inkling that was the case when I tried loading a path that used a tilde instead of the proper path to my home directory. As it turns out, it will hang just in general.
3) Your vimrc.vim really isn't the right way to distribute language-specific features. See
:h syn-files
:h 44.11
:h ftplugin
4) Typically, generics dispatch on the type of an argument, not an arbitrary predicate. Your generics are interesting and certainly in the spirit of other generics, but I wonder if their implementation is too broad. Combined with the way that predicates are tested in the reverse order of their declarations, it can create some unintuitive behavior:
wart> def g(x) (prn "default")
def g(x) :case (x = 1) (prn "one")
(object function {body, sig, })
def g(x) :case (odd? x) (prn "odd")
(object function {env, body, sig, })
(object function {env, body, sig, })
wart> g 3
odd
"odd"
wart> g 2
default
"default"
wart> g 1
odd
"odd"
6) Indentation sensitivity makes sense in a "line based" language (e.g., Python), but I find it horribly confusing in an "expression based" language (e.g., Lisps). While your auto-parentheses rule is simple to state, it's an extra mental burden for me to separate where parentheses should appear, when they don't, how to group everything (when I have few/mixed visual cues), etc. That it's optional means I could get by in principle, but I'm compelled to follow conventions, which seem largely aligned on the "no parens" side. It's a muddled blend of both worlds, and I honestly don't like the way it turns out. But we already knew that I'm a stick in the mud about this: http://arclanguage.org/item?id=16769
7) Despite the lack of parentheses, I felt like I was hitting the shift key a lot to write wart code. Chiefly for underscores and capitalization. (Did I mention I like having hyphens for identifiers? :P) By the actual count of shift-characters in qq.wart vs qq.arc, it's actually about the same or a little more in wart. But there are extenuating circumstances: extra comments & functions, plus some longer identifier names I changed when it became a pain to type (e.g., I changed unquote_splicing to unqs). But it's probably more-or-less a wash.
8) The numbered filenames really threw a wrench into my workflow. Without fuzzy matching, it's obnoxious to try to open a file only to find I'm unable to tab-complete the name because I just see the mnemonic part, like "generic.wart" instead of "[arbitrary number]generic.wart". I can't really tell what they're good for other than having nostalgia for BASIC (surely a good model to emulate :P) or overlaying some sort of narrative on your source code. And if there's a narrative, I (evidently) am not going to follow it anyway. I'd rather (at this point) grep for specific things than sit down and read the source code straight through like a book. The narrative could live independently of the filenames, like a separate load-order list. If the need for a specific load-order is technical, why not enforce it on the compiler level with a def-before-use policy? I suppose that breaks the "top down" approach from point "e" in 000preface, though.
9) Long-form quote is broken:
wart> 'x
x
wart> '1
1
wart> (quote x)
013object.cc:53 can't coerce symbol ' to function
021eval.cc:59 Not a call: (quote x)
Perhaps you need to split the line in two.
dying
wart> (quote 1)
013object.cc:53 can't coerce symbol ' to function
021eval.cc:59 Not a call: (quote 1)
Perhaps you need to split the line in two.
dying
Thanks a lot for the comments! In particular you've got me thinking about the numeric filename prefixes. The fatal errors have been getting onerous for me as well, thanks for drawing my conscious attention to them.
Thanks for the crazy load bug. I'll fix it later today.
Yes, loading wart is painfully slow. It's gotten 2x slower since the infix changes. Startup isn't doing anything special, it's just that the wart 'prelude' is a lot more code (since it's most of the language) than we typically type into the repl. I'll work on this some and get back to you.
You're right that relying on load order for generic dispatch makes some cases weird. On the flip side, it's powerful enough to permit multiple dispatch. Like the regex idea in http://arclanguage.org/item?id=16833 that permits either arg to be a regex. When I experimented with generic functions in the past I was always unsure whether to dispatch on the first or last arg. Things like keep like the last arg to be generic, for example. I think the problem isn't predicate dispatch, but checking predicates in load order. Is there a way to make the clauses load-order independent? Or an elegant way to influence load order without moving code around? Hmm, I'm going to think about this.
Regarding 6: I really don't intend to make fully-parenthesized lisp some sort of second class citizen. Feel absolutely free to stick with it for code you write. And I'd love to hear if it makes reading existing code more painful.
Regarding 5: I intended to show that construct_macro_call is an internal detail. I would move it later if I could, but it's used when extending def later in the file. I could a) just unindent it where it is, b) move it after mac and unindent it, c) move it to the bottom of the file, and not use the new mac to extend def (because that wouldn't work yet). Somehow I don't like any of those choices. What do you think?
Regarding 7: It didn't occur to me to watch keypresses on shift. That wasn't a motivator at all for eliminating parens or anything else. (I've got parens swapped with square brackets in my local environment for lisp.) In general, wart is more concerned with reading than with writing. But I'm going to think about it.
Regarding 9: Yes, this is a little weird when you come from other lisps. Wart doesn't consider ' to be an abbreviation for quote. It's the other way around; ' is part of the core syntax (it's processed during tokenization) along with backquote and unquote. The name quote is just for manipulating code. Things like if (car.form = quote), etc. I used to use it in my inliner/partial evaluator, but the inliner is not even in there anymore, and there's really no reason for code-manipulation stuff to be available so early on. I'm just going to take the long forms out. [Edit: done]
Another difference with traditional lisp: 'a is read as (' . a) rather than (' a). Similarly for backquote and unquote. This allows me to sidestep complexities like what happens in (' a b) or ,@,@x that I haven't found a use for yet.
Is there a way to make the clauses load-order independent?
The Magpie stuff Pauan linked is really interesting. From my layman's intuition, though, it still only seems to be able to linearize clauses because they're limited. As long as we know what sort of clauses we're working with, we can define a partial order on them. The same also happens to hold true for "typical" generic methods that dispatch based on class, because classes readily form a poset (using the subclass relationship). I have a feeling that the answer would be somewhere around "intractable" for arbitrary code, especially if the code didn't have formal semantics we could reason about. I mean, arbitrary code might never halt, so it'd be impossible to linearize those cases to the "end of the line" just because (in general) the Halting Problem is undecidable. But hey, I'm not a doctor.
At least the order that definitions appear in the code easily linearizes the clauses, despite the elephant-in-the-room of loading multiple files.
Regarding 6: I really don't intend to make fully-parenthesized lisp some sort of second class citizen.
That's good. I didn't mean to mischaracterize. I think that might inadvertently be the impression (or even outcome?) from so much as having a no-parens rule: the thought goes from "let's do Lisp with fewer parentheses" to "let's do Lisp with no parentheses", slippery slope fallacy though that is. Plus, at a glance, their elision seems common in your source.
And I'd love to hear if it makes reading existing code more painful.
I think that's a big reason I don't like removing them. There are fewer visual cues for grouping expressions. Not as big a deal in something where indentation is all you have, like Python. A little weirder when my head's trying to fill in where parentheses should rightly be. Blah blah, curmudgeon, blah.
Regarding 5: What do you think?
It only has the one call site; could you just inline it using afn?
Regarding 7: It didn't occur to me to watch keypresses on shift.
It's more of a heuristic than a solid metric, but I definitely notice it. E.g., I get annoyed working in camelCase languages.
Regarding 9: I'm just going to take the long forms out.
So, how do I manipulate quotes in macroexpansions, as needed in qq.wart?
"how do I manipulate quotes in macroexpansions, as needed in qq.wart?"
Yes, you're right that I need to bring back my definitions of quote, etc for qq.wart. But there was more to it than that. Here is -- I think -- a relatively faithful port of qq-expand including the optimizer: http://gist.github.com/3971932#file_080qq.wart. I say 'relatively' because wart doesn't support multiple args to quote/unquote/quasiquote, so a lot of your hardest tests become moot. Also, the backquote doesn't expand to a call to quasiquote in wart.
Sorry this took so long. Even when you provided such thorough tests it was hard to make myself focus on this exercise; there's always alternatives with more instant gratification. But in the end it was a fun exercise in recreating the 'theory' of a (tiny) codebase (http://alistair.cockburn.us/ASD+book+extract%3A+%22Naur,+Ehn...)
---
Rather than start with your port I went back to your arc sources and tried to track my process in an effort to generalize lessons. I was able to split your recursive functions into clauses without much effort[1]. For example:
The key was to peel the cases off in reverse order.
As in the past, wart is more verbose than arc, but in exchange you get to move the cases around. For example, all but the first clause can be in a separate file so you don't care about them when Optimize_cons is unset.
---
But there was more to this than a simple transliterated port, because quote/unquote/backquote were represented differently in wart than other lisps. I had to track down and update all the places in your version that made this foundational assumption.
Since the backquote is baked into wart's reader rather than expanding to quasiquote like in other lisps, I traced through all top-level calls to qq-expand as a baseline to compare against:
; change in original arc version
(mac quasiquote (expr)
(ret ans qq-expand.expr
(ero expr " => " ans)))
Once I had this output in hand I could start porting your tests. I started with a pass at just visually catching as many cases of treating the cdr of quote, quasiquote, etc. as a list as I could[2]; you might enjoy seeing what I was missing at that point, the 5 tokens that took a week of debugging :)
Two forms turned out to be quite useful in my debugging:
# http://arclanguage.org/item?id=11140
after_fn qq_expand_list(expr)
(prn "qq_expand_list " expr "
=> " result)
# ..and so on for qq_transform, qq_cons, qq_append.
# Easily turn debug prints on and off.
def xprn args
nil
"As long as we know what sort of clauses we're working with, we can define a partial order on them."
You could require that the user provide an order when defining a new pattern.
---
My idea with Nulan is that functions shouldn't change: they should be static algorithms. Thus, Nulan doesn't allow you to overload functions like you can in Arc or Wart. Instead, if you wish to overload a function, you define a new gensym which can be stuck into any object to overload it:
Now, if you call the "foo" function with an object that has a %foo key, it'll call the first clause. Otherwise it'll call the second clause.
This gives complete control to the function to decide whether it can be overloaded or not, and exactly how the overloading happens. I think this is the most flexible approach I've ever seen.
As a more concrete example, objects can have a %len key to define a custom (len ...) behavior, an %eval key for custom evaluation behavior, %call for custom call behavior, %pattern-match for custom pattern match behavior, etc.
And because of the way pattern matching works, you can even match against multiple gensyms at once:
$def %foo (uniq)
$def %bar (uniq)
$def foo
{ %foo F
%bar G } -> ...
{ %foo F } -> ...
{ %bar G } -> ...
... -> ...
The above defines different behavior when an object has both the %foo and %bar keys, and also when it has just %foo or just %bar.
---
"So, how do I manipulate quotes in macroexpansions, as needed in qq.wart?"
I think hardcoding symbols (like in quasiquote) is a terrible idea, so I would just not define quasiquote. Instead, I'd use Arc/Nu quasisyntax, which doesn't use quote at all:
`(foo bar qux) -> (list foo bar qux)
`(foo (bar) qux) -> (list foo (list bar) qux)
If you want quote, just use unquote:
`(foo ,'bar qux) -> (list foo (quote bar) qux)
Also, I wouldn't define quasiquote as a macro, I'd have it be a part of the reader.
You could require that the user provide an order when defining a new pattern.
Interesting idea, but the proper mechanism for it eludes me.
- You could have the user give a precedence level, since integers are totally ordered, but that's a terrible idea---magic constants in every generic declaration. Any more restricted domain (e.g., specifying high, medium, or low precedence) gets a little vague.
- You could have a simpler mechanism where each new rule is added to a known, fixed location in the linearization. So basically the order is a double-ended queue and generic declarations have keywords for "push me to the front" or "push me to the back". But that's probably a bit too basic, and still relies on declaration order (in fact, complicating it).
- You could have the user specify a previously-declared chunk of code to execute first, like
But that's too tightly-coupled and requires code duplication. Plus, if you're already duplicating the code that's close by a definition, why not instead reorder the definitions so a simpler order-they're-read mechanism works?
- I'm really scraping the bottom of the barrel now...Have the user supply their own predicate that determines which generic to apply first? Which means the user basically has to implement their own customized partial-order. I really don't see that happening.
When using generics myself, I don't want to think about these sorts of things too hard. That's what I like about class-based generic dispatch: I just have to think about the function one class at a time, and the right function will be applied to the right instances using their simple, implicit order. For general predicates, instead of hard-coding an order I'd rather use a big if/case/pattern-match that I at least know is ordered the way I wrote it.
"You could have the user specify a previously-declared chunk of code to execute first"
I think this is similar to Inform 7's approach, where rules can be referred to by name. Generally, every rule in a library has a name, but individual stories omit them unless necessary.
---
"- I'm really scraping the bottom of the barrel now...Have the user supply their own predicate that determines which generic to apply first? Which means the user basically has to implement their own customized partial-order. I really don't see that happening."
That's the approach I took in Lathe a long time ago.[1] Under my approach, the partial order is itself extensible, and it even determines the ordering of its own extensions. I quite like the expressiveness of this approach, but this expressiveness is barely meaningful for in-the-large programming: In order for the precedence rule programmer to have enough information to make judgments by, they need to require all other programmers to annotate their extensions with appropriate metadata. That or they need to maintain their own up-to-date metadata describing common libraries! Instead of programming language battles, we get framework battles full of boilerplate.
Since finishing that system, I've been pondering the "framework" conventions I'd like to use myself, and I've been trying to fold those decisions into the core of a language design.
Whereas Magpie and many other multimethod systems make a big deal about subclasses, I try to avoid any kind of programming where almost all X's do Xfoo, but some X's are also Z's so they do Zfoo instead. By the same principle, I'd avoid the [odd? _] vs. [= 1 _] case altogether. If fact, as long as I succeed in formulating in the non-overlapping designs I like, I avoid the need for precedence rules altogether... but it's still an option I keep in mind just in case.
Currently, I favor supporting extensibility by way of sealer/unsealer pairs and first-class (multi)sets.
Sealer/unsealer pairs work when each extension is an all new range of possibilities. In Arc, I'd just represent these as tagged values, and I wouldn't bother to pursue the secure encapsulation associated with sealer/unsealer pairs. In linear logic, the additive operators describe this kind of combination.
First-class (multi)sets work for when each extension is a new participant in a single computation. In Arc, a list is a good representation for this. In linear logic, the multiplicative operators describe this kind of combination.
When precedence is necessary, it can be modeled explicitly as a transformation of a (multi)set into a more structured model. I think any programmer who makes an extensible tool should carefully choose a transformation that makes sense for their own purposes--whether it's really "precedence" or something else.
---
"For general predicates, instead of hard-coding an order I'd rather use a big if/case/pattern-match that I at least know is ordered the way I wrote it."
That's my take on it, too. My examples of multi-rule functions have included factorial and exponentiation-by-squaring, but those are unrealistic. There's no reason for an extension to come interfere in that process, so it might as well be the body of a single declaration.
When I was using predicate dispatch more often, I often discovered that I could satisfy most of my use cases with just two extension annotations:
- This is the real meaning of the function, and all the other cases are just for auto-coercion of parameters into the expected format. Use this extension first. (This came up when implementing 'iso in terms of 'is, and it also makes sense for coercion functions themselves, such as 'testify.)
- This extension is the last resort. Use it last. (Maybe it throws an informative error, or maybe it returns false when everything else returns true. This also works for all-accepting coercion functions like 'testify, but I'm suspicious of this kind of design. A call to 'testify always does Xfoo, except when it does Zfoo.)
---
[1] Unfortunately, right now arc-Lathe's version of the precedence system isn't something I maintain, and Lathe.js's version has no easy-looking examples because I no longer store extensions in mutable global state. Lathe.js's "hackable" utilities are now higher-order utilities that take all their once-global dependencies as explicit parameters.
"they need to require all other programmers to annotate their extensions with appropriate metadata."
If Nulan had multimethods, I'd probably just require programmers to add additional metadata to the object in addition to the %pattern-match gensym. But Nulan doesn't have linearization or multimethods, so I avoid that whole mess!
---
"Whereas Magpie and many other multimethod systems make a big deal about subclasses"
And I have to agree: objects are amazing, but classes and subclasses are terrible. In fact, I'm of the opinion that probably all hierarchial relationships are too restrictive. Hence Nulan's object system which is completely flat, but has pretty much all the benefits of classes/prototypes.
Basically, the behavior that is common to all objects (of a particular kind) is put into a function, and behavior that is specific to a particular object is put into the object itself. And immutability gives you wonderfully clean semantics for prototype/class behavior, without all the complication and baggage.
Well, I was thinking in terms of Nulan, where you define new patterns with a custom "%pattern-match" property. It wouldn't be hard to add in a "%pattern-match-order" property or such, though I admit I haven't really thought through how such a property would work...
Obviously that kind of system wouldn't work in wart where predicate dispatch is based on arbitrary function calls. Hence my idea in Nulan of not allowing function mutation. I would write the above code like this:
The above is basically exactly the same as Arc, except it uses an array rather than a cons, and it has a custom %len property. If you don't want the different parts of the queue to be public, you could use gensyms like this:
(w/uniq (len left right)
(def queue ()
(dict %len (fn (x) (x len))
len 0
left nil
right nil)))
Rather than returning an array of 3 elements, it returns an object, which has a "len", "left", and "right" property.
In either case, the object that is returned has a %len property, which is a function that computes the length. The "len" function would then be defined like this:
(def len (x)
((x %len) x))
That is, it first looks up the %len property in the object, and then calls it with itself as the first argument.
---
* [1]: You might be wondering what's going on here... well, in Nulan, a "list" is just like a JavaScript array: it's an ordinary object which uses numeric keys and has a custom %len property. In particular, that means that all the tools that work on objects work on arrays too.
The @ syntax is used for splicing. So, for instance, to merge 3 objects together, you'd say { @foo @bar @qux }. And because arrays are objects too, you can splice them.
So what we're doing is, we first create an array of 3 elements, and we then splice in a new object. This new object has a custom %len property, which overrides the %len property of the array.
Alternatively, we could have done it like this:
(set (array nil nil 0) %len ...)
But I think it's better to use the splicing notation, especially when you use [] for arrays and {} for objects. By the way, these two are actually equivalent:
{ @[[] [] 0]
%len ... }
[[] [] 0
@{ %len ... }]
In the first case we take an empty object, splice in an array, and then assign a %len property. In the second case, we take an array and splice in an object that has a %len property.
In either case, the object that is returned has the same keys and values.
Sorry to hijack your wart thread, but... I just realized something. I wanted to allow for iterating over the keys of an object, but that caused issues, as I discussed with rocketnia earlier (http://arclanguage.org/item?id=16823)
Anyways, JavaScript gets around this problem by allowing you to define a property on an object that is "non-enumerable". But any kind of system that I add in that lets you define "non-enumerable" properties is going to be big and complicated.
Instead, I found a very simple way to create enumerable objects in a way that is completely customizable, and doesn't even need to be built-in:
(= %keys (uniq))
(def iterable-object ()
(let keys []
{ %set (fn (x k v)
(pushnew k keys))
%rem (fn (x k)
(pull k v))
%keys (fn (x) keys) }))
Here's what's going on. Firstly, we got the %keys gensym, which is supposed to be a function that returns a list of keys in the object.
The function "iterable-object" returns a new object that has custom %set, %rem, and %keys properties:
%set is called when assigning a property to the object. It takes the key and pushes it into the "keys" array.
%rem is called when deleting a property from the object. It takes the key and removes it from the "keys" array.
%keys just returns the "keys" array.
Now, these objects are capable of being enumerated, which means they could be passed to "map", for instance. But here's the fun part: you can completely control which properties are enumerated and which aren't.
In this case, the object will only enumerate properties that are added or removed after the object is created. So any properties defined previously are still hidden. At least, depending on how I implement splicing and %set...
---
What the above basically means is... "every computer problem can be solved by the addition of more properties on an object" :P
After fidgeting with the syntax, here's what I got:
$def iterable-object ->
[ %set -> x k o n
[ @x %keys -> ~ (pushnew (keys x) k) ]
%rem -> x k o
[ @x %keys -> ~ (pull (keys x) k) ]
%keys -> ~ {} ]
I actually realized that swapping [] and {} is way better, meaning that [ foo bar ] is (dict foo bar) and { foo bar } is (array foo bar). There's two reasons for this:
1) {} is closer to () than [] is, which is really nice in macros:
$mac $accessor -> n v
$uniq %a
{$def n -> %a
{{%a v} %a}}
2) I found that {} didn't stand out enough, but [] does.
---
By the way, in case you're curious about the Nulan syntax... $ is prefixed to vau/macros, which is why it's "$def" rather than "def"
-> is the function macro, which means (-> x y z ...) is equivalent to (fn (x y z) ...) in Arc
~ is the "wildcard syntax" which matches anything, just like _ in Haskell
[ foo bar ] translates to (dict foo bar), and { 1 2 3 } translates to (array 1 2 3)
@ is for splicing. Which means that [ @foo @bar @qux ] merges three objects into one. If you want to update an object with new properties, it's idiomatic to say [ @foo ... ]
Which, if translated into JavaScript, would look something like this...
var iterableObject = function () {
var a = {};
a.set = function (x, k, o, n) {
var a = Object.create(x);
a.keys = function () {
return pushnew(keys(x), k)
}
};
a.rem = function (x, k, o) {
var a = Object.create(x);
a.keys = function () {
return pull(keys(x), k)
}
};
a.keys = function () {
return []
};
return a
}
I also managed to return performance to pre-infix levels by the simple expedient of turning on compiler optimizations; on my machine it yields a 2x speedup. Startup time is now 2.6s, and tests take 35s to run.
Finally, the interpreter no longer dies when it encounters unbound vars, or when trying to invoke a non-function.
Update 9 hours later: After a little more optimization, wart will now start up in less than a second if the binary is up to date. The first run after downloading (build+run time) takes 20s. Tests run in 10s (excluding build time).
I tried to be clever and assume frequent building if running tests, but it turns out wart now does enough work that optimizations always improve build+tests time.
1) I would use alias to ensure that atom? automatically picks up extensions to cons?, etc.
2) I've been experimenting with keeping prefix <- at the top-level, just to make globals salient. But this 'idiom' is less than 48 hours old; what do you think of it?
<- carif (cons? & car)
3) I tend to often wrap stuff in parens even when I don't have to. I only skip parens for def/mac/if/while, etc., and for imperatives like err and prn. So:
6) There's a gotcha with multi-branch ifs: beware if you rely on paren insertion in test expressions:
if nil
car '(34 35)
no nil
cdr '(34 35)
You might expect this to return (35), but it doesn't because it's interpreted as:
(if nil
(car '(34 35))
(no nil (cdr '(34 35))))
I've been guarding against this by always adding parens for things that should act as expressions, and by always explicitly adding parens in multi-branch ifs:
I think this adds to your case against indent-sensitivity :) Multi-branch if is the one place I find myself wanting more syntax. Curlies for all their flaws really help to separate test expressions from actions in C or Java. I experimented with a 'comment token' in the past (http://arclanguage.org/item?id=16495) but that feature was lost in the move to infix.
1) Interesting! I assumed alias was an alias for <-. Clever, making it define a macro.
2) I liked being able to line up the infix <- for different-length variable names assigned in succession:
Optimize_cons <- nil
Optimize_append <- nil
For carif, I was using it more like an "anonymous" def where I didn't have to name the arguments. Perhaps this purpose could use its own top-level form?
By the way, I like the Global naming scheme, which I picked up from Coercions; it's a good, surprisingly readable alternative to mashing shift for GLOBAL names.
3) Ah, ern is what I was looking for! Should've looked instead of guessing names from Arc, haha.
4) A bit much for my tastes, but then so is infix. ;)
5) I noticed and tried to use it throughout. Still missed a few instances, I gather! :)
6) Yeah, I ran into that a bit. Might've missed some cases, though.
Your vimrc.vim really isn't the right way to distribute language-specific features.
Yes. It was laziness more than ignorance. I don't fully grok the vimscript mental model, and there's always the fear that somebody else's settings might interact poorly with these. In my experience, inlining settings in the vimrc has been the most simple and reliable way to ensure they work out of the box and don't get accidentally overridden by something else. At the cost of being a bad citizen :)
But I've taken the trouble to think things through now, and this should follow best practices: http://github.com/akkartik/wart/commit/deabc891e8#diff-1. Now that it's done, it's nice to see all the rules laid out cohesively rather than as tweaks on the much more complex rules for lisp.
I've been thinking harder about the numbered filenames. I had three goals in doing them.
a) To be able to add new files, split files, merge files, more easily. To not need to hack on the build system as much.
b) To make it dead obvious where a new user should start reading. But this only needs the early files to be numbered; past a point the precise number isn't important.
c) To not need to mess with include files as much. This overlaps with a) above.
Still thinking. I'm mulling some sort of symlink-based approach to make navigation more convenient.. Or maybe I can stop numbering past a point and ensure that the remaining files are loadable in any order. That might interact with your point 4 about generics, though.
Excluding parts of "03 utils.arc", the above is roughly the minimum needed to define Arc, the "import" macro, and the REPL.
They are loaded in numeric order, but that's controlled by the "arc" executable, so there's nothing stopping you from changing the order, or adding new non-numeric files[1], etc.
Everything else is put into either the "app" or "lib" folder, and none of the files in either folder are numbered.
This means the numbers are purely for documentation purposes, to help people see which parts go where. In particular, changing the numbers doesn't change the order in which things are loaded: you need to make the changes in the "arc" executable.
I think this is a good blend between not being too cumbersome, improving the visibility of order (which matters), and allowing fine-grained control of order.
---
* [1]: As an example of that, the "arc" executable also loads "lib/strings.arc" and "lib/re.arc" in addition to the above.
Interesting. It doesn't address fallintothis's pain point though -- he still needs to remember the numeric prefix to navigate between the files.
As an experiment I've added a script that creates more mnemonic symlinks to all the files in the repo:
$ git clone http://github.com/akkartik/wart.git
$ cd wart
$ git checkout 8332909aec # Unnecessary except for visitors in the far future
$ ./denum # New helper
$ vim -O assign.wart mutate.wart # etc.
I'm going to play with keeping these symlinks around and see how much I use them.
