I thought about it some more. Here's why I think the wrapping approach is superior to a new arg. Let's say you define a function on lists:
(def foo (x)
(... x))
Later you want to use foo, but traversing nested lists like it's a tree. Ok, so we'll add a new version:
(defextend foo (x y) (is y :tree)
(... x y))
But now calls to the base version will stop working; defextend passes both args to the base version which is only expecting one arg. To make things work you have to modify the base case, and teach it about an arg it doesn't need to know about. That seems unnecessarily ugly.
Another problem is the need to sequence args. If you create a new version with yet another arg:
(defextend foo (x z) (...)
(... x z))
Now you have to worry about the second arg being used for wildly different uses, possibly interfering with each other and causing bugs, or having to provide unnecessary args:
(foo a nil :widget) ; don't traverse a like a tree
Edit: ah, I just thought of one counter-argument. If you have multiple tree arguments you have to wrap each of them in my approach. This is more apparent when trying to support dotted lists in map (fallintothis's original complaint: http://arclanguage.org/item?id=18247)
(map + (dotted '(1 2 3)) (dotted '(1 2 . 3)))
---
Now, arc could be more flexible. My wart language (http://akkartik.name/post/wart) is more like javascript: excess args are silently ignored, and missing args are silently passed nil. Also, it has keyword args, so different versions might be called as:
(foo a :traverse-as tree)
(foo a :like widget) ; don't traverse like a tree
In such a situation the difference seems more minor.
The thing is, I don't want a new arg. I think we would be better of using keyword arguments, similar to ones as you mentioned in wart[1]. The reason I'm making such a big deal out of this is why create a new solution (tagging the arguments as they are passed in) when we can already can do this with keywords which are much more flexible. With keywords we can call them with anything we want: a list, a symbol, or even some sort of complex data structure.
(foo :a 'tree)
(bar :b '(1 2 3))
(baz :c (obj a 'b c 'd))
We can use keywords to pass in many different kinds of arguments. If we were to just tag the args, we could pretty much only pass in different types, not much else.
(foo (tree x)) ; how would you change this from a type to a number?
We can also implement them in such a way that when a function is called with extra keyword args, it won't change the behavior of the program (we would probably want it to log some kind warning). This way when we extend foo, it doesn't matter if we pass the original the keyword arg. As far as the original foo knows, the keyword arg doesn't even exist.
Sorry if I confused you. I don't want extra args, I want keyword args.
That makes sense! I've long been a proponent of keyword args as you can see. My love for them goes back all the way to one of my first posts here: http://arclanguage.org/item?id=10692. I think I've implemented them three times so far. I even have a version of arc with them: http://github.com/akkartik/arc. But I haven't updated it lately. I'll spend a few hours and merge in all the new stuff in anarki over the past year.
That's great! I just googled "Paul Graham arc no keywords" and found in one of his early writings on arc, he had a plan to replace keywords with something called "get parameters".
Ok, I've updated http://github.com/akkartik/arc. It was a useful exercise, because it turns out it has more tests than anarki, and I found several bugs in my work yesterday :) I've copied the tests to anarki as well.
It doesn't really need to know about a new arg, but there's no way to pass it in automatically because there's already a rest arg: body. Ok, so let's try that:
(mac each (var expr ? as nil . body)
`(walk ,expr :as as (fn (,var) ,@body)))
The problem with this is that you need to explicitly provide the optional arg, or it gets taken out of body.
I had this old idea that rest params should take precedence over optionals, but it turns out to have fallen by the wayside: http://www.arclanguage.org/item?id=13083. Time to bring it back here.
Well, the problem is you are trying to make keyword args the same as optional args. It might be better if you make make keyword args and optional args two completely separate kinds of arguments (similar to how common lisp has both &optional and &key). This way you would define each as:
(def each (var expr : as nil . body) ; I used : to represent keyword args
`(walk ,expr :as as (fn (,var) ,@body)))
This way calling each like
(each x '(1 2 3) x)
will expand correctly because keywords only take a value when they are passed in explicitly. It would have to be called as
(each x '(1 2 3) :as x)
to get the error you had. If you need to define a function that takes both optional args and keyword args:
(def foo (a b c ? g h i : j k l . rest)
...)
This way a-c are required, g-i are optional, j-l are keyword, and rest is everything else.
As for the problem you had with needing to redefine each (it is also mentioned by rocketnia in the post you linked[1]), each is just an interface for walk, so if a new feature is added to walk, each need to be redesigned. There might be some way that we can define some sort of macro, interface, which can be used to create a function/macro which is a very thin layer over another function/macro. As for how interface is to be designed, I'm not sure.
Well, it seems more general to let any arg be acceptable as a keyword.
rocketnia's concern I addressed by passing through unknown keyword args unchanged:
(def foo (a b)
(list a b))
(def bar args
(apply foo args))
arc> (bar 3 :a 4)
(4 3)
But that doesn't work in this case because each needs to treat as specially. I don't think that's avoidable.
We _could_ include a new category of 'required' keyword args like you suggest, yes. Let me know if you think of a situation where just giving rest args precedence isn't enough. Things are already pretty complicated with rest, destructured, optional, and keyword args, so I'm reluctant to add another delimiter and category.
"rocketnia's concern I addressed by passing through unknown keyword args unchanged"
The reason I'm okay with this approach is because it localizes the complexity to each function. This way I don't dislike the language; I just dislike the idioms currently being used in it.
Consider the use of flat text formats for shell scripts, HTTP headers, CSS properties, XML attributes, etc. They're usually convenient enough to type out manually, and that also makes them convenient to generate in simple cases, but writing a parser for them can be tough due to the same syntactic sugars that made them convenient to generate in the first place. Writing a full-featured generator or parser can also be tough, because sometimes future versions of the text format may allow it to encode more information.
With keyword arguments, the syntax for a function call is more complicated and sugary, putting programmers like me in the position of wondering what kind of headaches they should go through to parse or generate this format. If the headaches are unclear, combining programs becomes hard, because each developer might opt for different headaches, at which point I think they overlap in the form of complex glue code. :)
I have a greatly-enhanced respect for this perspective over the past month. Look at my travails over the past day trying to get optional, keyword and destructured args to play well together: https://github.com/akkartik/arc/compare/342ebf57a8...e347973.... Or my travails over the past month trying to build macex in wart in the presence of optional, keyword and destructured args AND quoted AND aliased params AND already-eval'd args[1] AND incompletely-eval'd args[2]: https://github.com/akkartik/wart/compare/30025ee25c...cabaa0.... Clearly there's some point at which I jump the shark :)
[2] related to partial eval: https://github.com/akkartik/wart/commit/8239ed9d21. I've been struggling intermittently with partial eval for two years now. This now year-long attempt might well be utterly useless.
---
Perhaps the easiest to understand and most serious extant example of the knots I end up tying myself into:
arc> (iso :x :x)
nil
This is because x is one of the params of iso. Even more serious, this happens even if you try to compare a variable that evaluates to :x. Or a variable that evaluates to a large structure containing :x somewhere in it. Ugh! The reason for this is that I'm extracting keyword args at function-call time inside each function. This:
It took me three years to notice yesterday that I'm extracting keywords from post-evaluated arguments. Ugh! Urrrgggh!
Weird thing is, even wart has this problem. Even though I would superficially seem to be extracting keyword args pre-evaluation. I still need to debug this. Or perhaps I shouldn't debug this. Perhaps I should just throw up my hands and delete all my repos and become a hermit.
We should be able to specify the type of the function as part of the function. We shouldn't need to specify it in the arguments. I just think of it as we are walking over a tree as opposed to calling walk on an object that is a tree.
What I'm trying to say is the keyword arg shouldn't apply to a specific argument. It should apply to the function call as a whole.
That doesn't seem too bad, IMO: if your sym uses ssyntax chars you can't use it in ssyntax. There's probably worse corner cases I haven't considered, though :/
Holy crap, how have I never heard of noparen.arc? I'm gonna see how PG's design choices were different from mine in http://akkartik.name/post/wart. Did I do something he didn't consider, or did he dismiss my approach for aesthetic reasons?
In the past I've tried to think about ways to handle it automatically, but I usually get hung up on macros and their pesky propensity for undoing assumptions. E.g., for my Vim ftplugin (https://bitbucket.org/fallintothis/arc-vim), I used the heuristic "is the macro's rest parameter named body?" to figure out the indentation of so-called "bodops"...except that I had to manually maintain white/blacklists of troublesome macros with code similar to http://arclanguage.org/item?id=11288
Edit 40 minutes later: Everything looks great![1] The unit tests in arc.arc.t all pass, and the HN server comes up fine. I tried submitting, editing, commenting, everything works.
Edit 75 minutes later: I've added your tree-subst suggestion[2]. Good idea, thanks!
I realized the tree-subst is going to have to be a little more complicated. If the function isn't expecting a list it's going to throw an error.
(tree-subst even [+ _ 1] '((1 . 2) . (3 . 4)))
This code throws an error even though it is very clear what the result should be. I'm not sure what the best way to fix it is. I managed to write a version that does work but it looks pretty ugly.
(def tree-subst (old new tree)
(with (test (testify old)
f (if (isa new 'fn) new (const new)))
(if (no tree) '()
(atom&test tree) (f tree)
(atom tree) tree
t (cons (tree-subst test f (car tree))
(tree-subst test f (cdr tree))))))
I just realized there probably is a better way to write it using treewise. Something like:
(def tree-subst (old new tree)
(with (test (testify old)
f (if (isa new 'fn) new (const new)))
(treewise cons [if (test _) (f _) _] tree)))
But on the other hand tree-subst compared subtrees using is until my change today, and there are no calls to tree-subst in the repo. So I suppose there's no way to see what the original authors had in mind. Anybody else have use cases to share?
On the third hand, it's easy to use tree-subst the way you want with a slight change to the call, like I showed in my tests:
If you want to do substitution using a function, it seems to me like the operation is more of a roundabout map. Particularly if it's just over the atoms of a tree, it's easy to think of the higher-order functions from a different angle, instead of trying to make tree-subst do everything:
Other name ideas: treemap, maptree, hyphenated versions of any of those, map-leaves.
Digression: the issue I've been more annoyed by in Arc's suite of "tree" utilities is sloppiness with direct cdr recursion. Functions have no way of discerning whether some list (a b c) they're looking at is a "top-level" form or if it's just from recursing on cdrs of (x y z a b c). E.g., using ontree in http://arclanguage.org/item?id=18217:
arc> (let if 2 (+ if (do 2)))
4
arc> (count-ifs-with-do (list '(let if 2 (+ if (do 2)))))
1
arc> (ifs-with-do (list '(let if 2 (+ if (do 2)))))
((if (do 2)))
arc> (ifs (list '(let if 2 (+ if (do 2)))))
((if 2 (+ if (do 2))) (if (do 2)))
I usually want the recursion to instead occur more like
(def ontree (f tree)
(f tree)
(unless (atom tree)
(each subexpr tree
(ontree f subexpr))))
Or even just have the above as a separate function, like on-exprs or something.
I'm pretty sure we have to do the car/cdr recursion because functions like map and each don't work on pairs, they only work on proper lists and most trees are not proper lists. Code such as the following wouldn't work mainly because of the use of map/each.
(treemap [cons 'a _] '(a . a))
I'm pretty sure we would also only want treemap to work only on atoms (your first example), it doesn't make much sense to apply the function then recur on the new tree (I find it very hard to figure out what is going happen). The problem with this is we can no longer call functions on subtrees. I do like the idea of treemap which could be written more cleanly as:
(def treemap (f tree)
(treewise cons f tree))
I'm pretty sure we should stick to akkartik's implementation of tree-subst (we can actually remove the first clause because it is covered in the second clause) because it works just like all of the other higher order functions where they testify their input and it works on entire subtrees.
Note: you can use iff instead of writing a new function whenf
I'm pretty sure we have to do the car/cdr recursion because functions like map and each don't work on pairs, they only work on proper lists and most trees are not proper lists.
I think the obvious problem with that reasoning is it reinforces the notion that map, each, etc. "should" break on dotted lists. Even if that's the case, it's not like ontree & pals couldn't open-code a recursive function that worked on dotted tails; I merely used each/map in my example definitions as a convenience. If you prefer:
; Again, not that this has to replace the standard 'ontree...just can't think
; of a better name.
(def ontree (f expr)
(f expr)
(unless (atom expr)
((afn (subexpr)
(if (atom subexpr)
(and subexpr (ontree f subexpr))
(do (ontree f (car subexpr))
(self (cdr subexpr)))))
expr)))
arc> (ontree prn '(def f (x . xs) (cons nil xs)))
(def f (x . xs) (cons nil xs))
def
f
(x . xs)
x
xs
(cons nil xs)
cons
nil
xs
This versus the standard ontree:
arc> (ontree prn '(def f (x . xs) (cons nil xs)))
(def f (x . xs) (cons nil xs))
def
(f (x . xs) (cons nil xs)) ; <-- this potentially looks like a call to f!
f
((x . xs) (cons nil xs))
(x . xs)
x
xs
((cons nil xs))
(cons nil xs)
cons
(nil xs)
nil
(xs)
xs
nil
nil
On that note, I certainly find myself working around Arc's rampant disregard for dotted lists in my projects:
Note: you can use iff instead of writing a new function whenf
Not in vanilla Arc, I'm afraid. :) And I've always thought Anarki's iff is poorly named, due to the English collision with the "if and only if" abbreviation.
It does seem like a good idea, abstracting as a list of lists instead of a binary tree. The best name I can think of is onlol. I really don't think that we should extend map and each to work on dotted lists, instead we should create other functions treemap and treeach since dotted lists are actually trees and no longer lists.
Hmm, it sounds like we need three sets of primitives: for operating on lists, trees and code (like trees, but aware of call vs arg positions). Perhaps we could use the same names like map and count but first tag the arg as tree or code..?
I didn't notice that your version could check against subtrees. Another possibility would be to use errsafe along with the function (but I'm worried that might be too dangerous). Other than that I would have to agree with your design. I played around with the common lisp design of subst-if and found it does the exact same thing as yours but it also does not test nil. This can easily be done in arc in a similar way to atom&... with idfn&... I would have to go with your design for now but we might want to think about other designs such as whether or not to test nil.
Yeah, I too like not relying on an implicit 'else'. In fact, I use :else instead of t to be even more clear.
This is all most excellent. Don't be afraid to commit and push! Feel free to override the default mac if there aren't any obvious problems. There aren't very many people using anarki, and we can always roll it back later if necessary.
Welcome! I can see how lispers wouldn't like the arc approach, but I don't see how it makes arc more like non-lisps. What traditional language requires counting positions? I usually associate that with lisps.
You're right about blocks, so let's just focus on single-line 'if's. I can't think of a single non-lisp that allows adjacent condition and action without an intervening token. C and Java require parens, Go and Perl require curlies, Python requires the colon, Ruby requires a newline or semi-colon.
if (condition) action; # C, Java
if condition { action; } # Go, Perl
if condition: action # Python
if condition; action; end # Ruby
The 'else' token is a further separator. Never in these languages will you ever have two either-or expressions side by side. They're separated by either 'else' or 'else if' or 'elif'. Perl and Ruby even sometimes use 'if' as a separator.
action if condition # Perl, Ruby
Am I missing any counter-examples? I think the biggest error in this article is to blame Arc, when it's just extending the logic all lisps have always had. If you permit such adjacencies as (if condition action) and (cond ((condition action))), then Arc's choices don't seem so unreasonable.
(Though Clojure's mixture of Arc 'if' and optional types is a whole new level of unholiness.)
"Though Clojure's mixture of Arc 'if' and optional types is a whole new level of unholiness."
Is this what you meant instead? "Though Clojure's 'let', which mixes Arc's 'withs' with type hints, is a whole new level of unholiness."
Even so, I think Clojure's 'let doesn't actually have any[1] added complexity when it comes to type hints. It might look like the list is bunched into groups of either two or three depending on whether a type hint is present...
(let [^String x "x string"
y 2]
(body-goes-here))
...but that's not a quality of 'let. That's a quality of the ^ syntax. An occurrence of ^ consumes the next two s-expressions, just like ' consumes the next one s-expression:
So the bindings of a 'let are consistently bunched into groups of two s-expressions, just like Arc's 'withs.
[1] Of course, the type hints are actually used for optimization at some point, so the complexity of parsing them has to go somewhere. This is exactly as complex as destructuring: Arc's 'withs syntax supports destructuring, but it doesn't need special-case destructuring logic because it just translates down to 'fn. As it happens, Clojure's 'let syntax also supports destructuring, and it probably uses the same general technique.
Yes! Turns out I didn't notice the switch from if to let.
I have to say, though, I have no sympathy for the argument that it's still groups of two s-expressions. As a reader it's still more onerous to have to mentally group:
^a b
compared to:
'a
So the presence or absence of parsing complexity feels irrelevant.
I can't think of a single non-lisp that allows adjacent condition and action without an intervening token.
I mean, isn't this requirement mostly because those languages are infix anyway? Parsing gets easier with explicit ways of separating things. I could easily imagine shift/reduce conflicts or what-have-you cropping up when you try to eliminate the requirement for, say, parentheses around conditionals in Java.
For example, in some hypothetical infix language that doesn't require conditional separators (parens, braces, then, etc.), would
if x==10-x ...
be
if (x == 10) -x ... // maybe this language doesn't use the "return" keyword,
// so the conditional is returning negative ten.
or
if (x == (10 - x)) ...
?
Because Lisps use s-expressions, "intervening tokens" (per se) are unnecessary. As you say, Arc's choices don't seem so unreasonable, considering that.
Yeah, that's a good point. Non-lisps use keywords and punctuation for readability and to make parsing tractable, and the two reasons are often hard to separate in any single design decision.
To summarize my position: I have some sympathy for the specific argument that multi-branch 'if's are harder to read in lisp than in traditional languages[1]. But this affects any arrangement of parens, whether traditional 'cond' or arc 'if'.
