I meant an unquote that wasn't balanced by an enclosing quasiquote. e.g., `(,,x) has one too many unquotes. This is also true of unquote-splicing, like `(,,@x). clisp handles this in the reader:
$ clisp
[1]> `(,,x)
*** - READ: more commas out than backquotes in, is illegal
The following restarts are available:
ABORT :R1 Abort main loop
Break 1 [2]>
The Arc implementation can handle some cases, because the top-level 'unquote and 'unquote-splicing (outside of quasiquotes) will gripe, but it happens at macroexpansion:
$ rlwrap mzscheme -m -f as.scm
Use (quit) to quit, (tl) to return here after an interrupt.
arc> (load "qq.arc")
nil
arc> `(,,x)
Error: "reference to undefined identifier: _x"
arc> (= x 1)
1
arc> `(,,x)
Error: "unquote not allowed outside of a quasiquote: 1"
...and it now occurs to me that I'm not sure of any test cases that fail to signal an error. The error message isn't very nice, but as long as it does signal one, it's livable.
Extra input on this would be great. I'll try to hunt for ways to break it. Even if errors are signaled properly, it's better to have them at read-time, I think.
Extra input on this would be great. I'll try to hunt for ways to break it. Even if errors are signaled properly, it's better to have them at read-time, I think.
Hmm, you'd be rather ahead of much of Arc then. As you noticed with unquote, there are parts of Arc which currently don't catch errors at all, much less as early as possible.
I'm wondering if any read-time checks should be able to be turned off? Maybe I might want to be defining my own unusual quasiquote macro which does something with (quasiquote ((unquote (unquote x))))? I can't think of what that might be though.
What quasiquote expression expands into a dotted list so that you had to make your own append instead of using Arc's join? Is (join '(a) 'b) the kind of expression that would need to work? Do you see any disadvantage to fixing join so that it could do that?
What quasiquote expression expands into a dotted list so that you had to make your own append instead of using Arc's join?
It's predominantly because that's how the clisp version was implemented -- it's basically a straight port without much design consideration. But 'append does have some sort of impact (though probably rare in practice, and with debatable semantics):
$ rlwrap mzscheme -m -f as.scm
Use (quit) to quit, (tl) to return here after an interrupt.
arc> (load "qq.arc")
nil
arc> (toggle-optimize)
nil
arc> (defs foo () 'x bar () 'y)
#<procedure: bar>
arc> (macex1 '`(,(foo) ,@(bar)))
(append (list (foo)) (bar))
arc> `(,(foo) ,@(bar))
(x . y)
arc> (toggle-optimize)
t
arc> (macex1 '`(,(foo) ,@(bar)))
(cons (foo) (bar))
arc> `(,(foo) ,@(bar))
(x . y)
Optimizations or no, the expression should still mean the same thing, but as you can see, without optimizations it would break if 'join was used instead.
Incidentally, these are the same results you get from clisp. I don't have the interaction here because I just loaded the file and waded through repeated "continue"s until it finally yielded the damned result; if anyone's interested, I can reproduce it. Also, it's the result you get after fixing 'toggle-optimize, which I added but didn't use, so didn't have the chance to see it was broken. It should be:
Is (join '(a) 'b) the kind of expression that would need to work? Do you see any disadvantage to fixing join so that it could do that?
Well, arc seems to pitch on dotted lists a lot, since they aren't used too often. But it seems an odd case to miss, if only for the sake of completeness. Even functions like 'len are broken on dotted lists.
Have you ever written a program where you used append with a non-list in the last argument to create a dotted list? Or called length on a dotted list? Aside from unit testing?
The reason why I ask is that there's lots of places where Arc isn't "complete" (notice there's a caar, a cadr, and a cddr, but no cdar function in arc.arc, for example), but the question is what would actually get used in a program? And what will help programs be written more succinctly?
Perhaps 'len is a bad example, since some languages (e.g., Common Lisp) like to consider it an error to ask for the length of an improper list. There's a difference between incidental errors ("oh, right, didn't bother with dotted lists because I don't really use them") and intentional errors ("dotted lists are flat-out wrong to use here"). But the point is orthogonal to the one example, I think.
Inasmuch as Arc isn't "complete", you wind up needing to re-invent the wheel, losing succinctness in the process. Your example of car/cdr compositions is such an example, albeit an uncompelling case since (= cdar cdr:car) is easy to write. The reimplementation of 'len for ppr.arc is another, more compelling instance.
That's not to say that a language should be designed to cater to every possibility that might crop up in code. That leads to bloat. I'm just saying that things like handling dotted lists seem more like incidental "didn't happen to get around to it" bugs than "this should be an error". Indeed, the most striking example of this distinction I can think of is the entire reason that I wound up porting clisp's backquote: "never really used ,,@(...) before" vs. "you shouldn't use ,,@(...)". It's not so much a matter of commonality as it is about what makes sense.
Does that mean that 'append is right to use here? I don't know. I was just copying Common Lisp. If there's no logical reason to include it, going with 'join would be cleaner. On the other hand, if there's no logical reason to exclude it, which do we go for? I'd opt for the one that gives the most freedom: since 'append can be used just like 'join except in an edge case that might be useful, 'append offers you more utility -- rather than giving you an error.
(This sentiment could be phrased much more strongly if I actually had a useful case for choosing 'append over 'join. Note that I didn't try to provide a "real" example, just a demonstrative one.)
Here's what I suggest: publish a fixed 'join that is able to correctly create dotted lists. Now anyone who wants that doesn't have to reinvent the wheel.
Next, publish a version of your qq port that uses 'join. Now qq is simpler because it doesn't have to implement its own 'append; and you can point out that if we've using the old join then some qq expressions won't produce dotted lists, but if we use the new join they will.
I'm certainly open to using a fixed 'join, though my aim was to keep qq working as closely to vanilla Arc as possible, instead of modifying arc.arc functionality. Using Arc's current 'join gives way to some incorrect optimizations, as ported from the Common Lisp code -- e.g., (join nil form2) is not necessarily equivalent to just form2. So, I'll have to inspect that code, if there's the possibility of using arc.arc 'join (though I wonder if giving both options would be any "cleaner" than just keeping 'append around until/if 'join is "fixed").
On another note, a more appropriate place for the code and especially an altered 'join would probably be the Anarki repository (if only for keeping it a little more centralized). But I'm not sure I understand the protocol there, since I don't see any 'lib' directory in the arc3 branches. Unless this counts as a fix, I guess. Anyone care to give an executive summary / crash course? (The last I read about it was http://arclanguage.org/item?id=9777) Does it matter to anyone if this gets pushed to the git repo?
Using Arc's current 'join gives way to some incorrect optimizations
For any expression that doesn't have a dotted list as its final result?
my aim was to keep qq working as closely to vanilla Arc as possible, instead of modifying arc.arc functionality
The goal of Arc is to make it easy to write succinct programs. So, if:
A) someone finds that being able to use `(1 ,@2) helps them write a succinct program; or,
B) you have some natural optimizations that are easy to write if intermediate forms can use dotted lists, but would need more difficult or cumbersome code to work around a join / append that didn't work with (join ... atom), or even be conceptually more difficult to think about its correctness,
then I'd say let's fix Arc so that 'join is able to produce dotted lists.
On the other hand, if the only reason that you're implementing your own 'append is so that the code will pendantically pass Common Lisp unit tests for expressions that no one actually uses in their programs, then I'd question why we need that functionality :-)
What does 'list* do?
;; arc doesn't like macro definitions inside of 'do blocks (bug?), so resort to
;; defining them here.
What didn't work? In arc3:
arc> (do (mac a () 33))
#3(tagged mac #<procedure: a>)
arc> (a)
33
For any expression that doesn't have a dotted list as its final result?
Yes.
The way the optimizations work is to merely streamline the things that quasiquote itself introduces. It doesn't require or even expect user code to use 'append. e.g.,
Note that the top-level 'append wasn't touched -- it gets turned into (list (quote append)). But also note that the naive macroexpansion without optimizations produces (append (list a) (list b)), which is obviously excessive. So,
arc> (toggle-optimize)
t
arc> (macex1 '`(append ,a ,b))
(list (quote append) a b)
We still keep the top-level 'append, but a and b are no longer nested in a bunch of conses; the optimizer only changed the 'appends that quasiquote introduced haphazardly.
Then, if the optimizer tries to get rid of the extra 'appends, it must know that the final result will be the same. e.g., (append nil form2) == form2. Meaning that
The (append nil (f1)) that quasiquote introduced in the first case is turned into just (f1) in the second. Note again that the top-level 'append isn't touched; it doesn't matter if the programmer uses 'append.
Now, if quasiquote was changed to use 'join, so that any 'join it introduced might be optimized, there are fewer invariants because, for instance, (join nil form2) != form2 if form2 is not a list.
This wouldn't matter in optimized code, because the meaning of (join nil form2) could be taken to mean the same as (append nil form2). That is, we could use any ol' sentinel in lieu of 'append and treat its rules the same, like
thus leading to the same optimized code. However, when optimizations are off, if quasiquote produces (join nil form2) it might lead to an error where (append nil form2) would not. That is, if the above OPTIMIZE-ME were 'join, evaluating it as such dies when (f1) is not a list. Moreover,
(iso (join (list 'append) (join nil (f1)))
(cons 'append (f1)))
wouldn't necessarily be true.
So! If 'join were used instead of 'append, optimized code wouldn't care. But the invariant is supposed to be that optimized code results == non-optimized code results. (join nil form2) != form2, so the optimizer shouldn't touch 'join in this case, like it does with 'append currently.
All this means is that, if we s/append/join/g, the optimizer would need to use a different set of rules. This isn't so bad; it's not an insurmountable task by any stretch. But in so doing, it would change quasiquote semantics vs. using an append-style join (as discussed previously). So, giving both options would need to cover these altered semantics. Again, doable, just a note of concern when it doesn't seem a bad thing that 'append can be used (or 'join could be modified to work like CL 'append -- not against that, except when working with vanilla Arc).
In a similar vein, 'list* is introduced by the optimizer in attempts to cons less. Quoth CLTL:
list* is like list except that the last cons of the constructed list is
"dotted." The last argument to list* is used as the cdr of the last cons
constructed; this need not be an atom. If it is not an atom, then the effect
is to add several new elements to the front of a list. For example:
(list* 'a 'b 'c 'd) => (a b c . d)
This is like
(cons 'a (cons 'b (cons 'c 'd)))
Also:
(list* 'a 'b 'c '(d e f)) => (a b c d e f)
(list* x) == x
The most glaring issue with this being that it relies on the efficiency of the implementation, of course. So, I'm not even sure that mine conses any less than using just 'cons. This is fixed by a not-dumb implementation, of course!
What didn't work?
arc> (let b nil (mac b () 33) (b))
Error: "Function call on inappropriate object #3(tagged mac #<procedure: b>) ()"
arc> (do (mac b () 33) (b))
Error: "Function call on inappropriate object #3(tagged mac #<procedure: b>) ()"
(By the way, is this right-margin crowded enough for you? Yay, nesting!)
thus leading to the same optimized code. However, when optimizations are off, if quasiquote produces (join nil form2) it might lead to an error where (append nil form2) would not. That is, if the above OPTIMIZE-ME were 'join, evaluating it as such dies when (f1) is not a list
Yes, but your example was ",@(f1)", so naturally that wouldn't work without 'append when (f1) isn't a list.
I can think of three cases:
Case A: we want to use ",@X" when X is not a list. This isn't supported by Bawden's implementation, so it's a new feature. Clearly we need to use 'append then.
Case B: we require X to be a list when we use ",@X", but the qq expander (optimizing or not) relies on intermediate forms that call 'append with non-list arguments, even though neither the input nor the eventual final output has dotted pairs. Then we'd either need to use 'append, or rewrite the expander not to produce those intermediate forms.
Case C: When X is a list in any use of ",@X", the expander calls append and produces code that calls append with only lists. Then we could use Arc's current 'join instead of 'append.
Maybe I'm just up too late, but I'm not seeing anything in your examples that would indicate that B is the case?
If I always only use ",@" with lists, does the current code (with optimization turned on or off) ever produce the wrong result if 'join is used instead of 'append? (It's OK if the answer is you wouldn't know without looking into the question further).
(do (mac b () 33) (b))
oh, were you hoping that the macro would be staticly scoped?
I seem to have misunderstood your question. Apologies. The cases will make it easier for me to not talk past you:
Case A is what I've been defending (and is currently how the port works). I don't have any direct examples of its usefulness, but think that it's semantically clearer (a point which I don't think I've made clear):
$ rlwrap mzscheme -m -f as.scm
Use (quit) to quit, (tl) to return here after an interrupt.
arc> (load "qq.arc")
nil
arc> (= tail '(b c d))
(b c d)
arc> `(a ,@tail)
(a b c d)
arc> (cons 'a tail)
(a b c d)
arc> (= tail 'b)
b
arc> `(a ,@tail)
(a . b)
arc> (cons 'a tail)
(a . b)
I also consider it an added bonus: dotted lists exist, so it seems presumptuous to think that macros might not stand to gain from using them, splicing into them, whatever. (Just as it seems presumptuous to think that ,,@(...) should be an error.)
Case B is incorrect, so far as I can tell. Indeed, the clisp code was very careful about this -- making sure not to pass the cdr of a dotted list as the first argument to 'append, for instance.
Case C is correct, I think, but would run against case A, since using 'join when the splice is not a list will result in an error, as in the examples I've belabored.
As an example, the above definition implies that
`((,a b) ,c ,@d)
will be interpreted as if it were
(append (list (append (list a) (list 'b) 'nil)) (list c) d 'nil)
but it could also be legitimately interpreted to mean any of the following.
(append (list (append (list a) (list 'b))) (list c) d)
(append (list (append (list a) '(b))) (list c) d)
(append (list (cons a '(b))) (list c) d)
(list* (cons a '(b)) c d)
(list* (cons a (list 'b)) c d)
(list* (cons a '(b)) c (copy-list d))
(There is no good reason why copy-list should be performed, but it is not prohibited.)
I just happen to like the idea, is all. And, admittedly, for wholly impractical reasons!
If I always only use ",@" with lists, does the current code (with optimization turned on or off) ever produce the wrong result if 'join is used instead of 'append? (It's OK if the answer is you wouldn't know without looking into the question further).
Sorry if I come off like that. I don't mean to seem like I'm trying to answer without knowing anything ("going from the gut" or whatever).
For what it's worth, I strongly think that, if unquote-splicing is used strictly with lists, 'join will do the same job as 'append.
Of course, I wouldn't know without looking into the question further. ;)
So, have I finally managed to address what you asked?
oh, were you hoping that the macro would be staticly scoped?
Not even that. I just wanted it to be done in a single sexp on the last unit test (since it's 1 sexp per test), as I note in the source:
"local" macros don't work in arc:
arc> (let a 12
(let b nil
(mac b ()
(let c 19
``(,a ,@',@(list c))))
(b)))
Error: "Function call on inappropriate object #3(tagged mac #<procedure: b>) ()"
Even using a 'do block doesn't work (for some reason):
arc> (do
(mac m ()
(let c 19
``(,a ,@',@(list c))))
(let a 12
(m)))
Error: "Function call on inappropriate object #3(tagged mac #<procedure: m>) ()"
arc> (do
(mac m ()
(let c 19
``(,a ,@',@(list c))))
(let a 12
(m)))
*** redefining m
(12 . 19)
Unsure whether this is an Arc bug, the problem was mitigated by using a (mac ...) outside of the 'do block.
So, have I finally managed to address what you asked?
Yup!
OK, here's what I think.
One principle of Arc is to find the minimum combination of code that implements the language. So we shouldn't end up with both a 'join and a 'append; either we decide we want 'join to be able to produce a dotted list, or else we should just use the old 'join.
One way of getting succinct code is not to implement features that no one uses. So if the only reason for using 'append (or extending 'join to be able to produce a dotted list) was to pendantically pass some Common Lisp unit tests, then I be inclined to go with case (A) and just use Arc's current 'join.
However, another principle of Arc is not to forbid things to programmers unless there's no way to provide them with logical consistency. I can't think of any reason to tell you not to use
arc> `(a ,@'b)
(a . b)
if you want to... so why should I advocate for restricting splicing to lists?
It's shorter than your append2/append, at the cost of not providing an explicit error message if a non-list is passed in some argument not the last.
I just wanted it to be done in a single sexp... Unsure whether this is an Arc bug
In Arc, the entire expression is macro expanded and compiled, and then evaluated. So in
arc> (do (mac a () 33)
(a))
Error: "Function call on inappropriate object #3(tagged mac #<procedure: a>) ()"
what's happening is that the 'mac form, when evaluated, will create a new global macro 'a. However, when the 'do form is being macro expanded / compiled, the 'a macro hasn't been defined yet. So "(a)" compiles into a function call on 'a. Then, when the 'do form is evaluated, 'a is set to be a macro, and then 'a is function called. Which produces an error, since a macro object isn't callable.
I've called length on a dotted list in ppr.arc, and to do that I needed to redefine 'len to support it. I can't really think of an application outside of messing with argument lists.
here's a nicer version of the patch that doesn't need the 'declare to be used
Cool, thanks.
We could also look around to see if there's other working implementations that we could use... plt scheme's R6RS implementation of quasiquote or Common Lisp's implementation might be worth checking out
I also tried looking at PLT's r6rs quasiquote (specifically collects/r6rs/private/qq-gen.ss), but I found it a bit daunting because I'm unfamiliar with syntax-case macros. It's probably better for quasiquote to be written in Scheme, though, so perhaps the code could easily be transplanted into ac.scm. I'll just leave it to someone more Scheme-savvy.
I was thinking in terms of bootstrapping. The quasiquote library I wrote uses things defined in arc.arc, which in turn are written for quasiquote, but using the one on the Scheme side.
It's entirely possible to rewrite either to fit it all together. For instance, someone could rewrite the quasiquote library in terms of basic operators -- 'assign, 'cons, 'fn (instead of 'let), and the like. Probably more bearable would be rewriting arc.arc to use 'list and 'cons and so forth to build expressions instead of using quasiquote, up until we have enough to define quasiquotation (which doesn't require much); then load qq.arc and continue defining arc.arc with the newfound utility.
Neither of these options sounds pleasant if, as an alternative, we could just have an easily-added Scheme-side quasiquote -- e.g., if mzscheme already implemented it correctly, we'd be done. Granted, there are advantages to having a user-defined quasiquote, not just in the interest of keeping Arc axiomatic but also in giving programmers the option to modify it.
Efficiency was another concern of mine, in that doing quasiquotation in Scheme might be more efficient than cranking 'ac across all of the Arc functions back-and-forth. But this is likely unfounded and more a symptom me worrying rather than actual testing -- the difference could be negligible or even nonexistent. Arc quasiquotes (as I implemented) still compile down to simple Scheme code (to the results of the macroexpansions), so it's probably silly to worry. Just a fleeting concern.
Well, on efficiency, the quasiquote expansion happens at read / macro expansion / compile time, am I right? So writing the qq expander in Arc vs. Scheme isn't going to affect the speed of the running program, yes?
As for bootstrapping, MzScheme's qq implementation works as long as nested quasiquotation isn't used, and today arc.arc doesn't use nested quasiquotation. So I don't think we need to do anything beyond what we're already doing: load arc.arc, then load a qq implementation that handles nested qq, and then we're free to write macro utilities that use nested qq.
Arc vs. Scheme isn't going to affect the speed of the running program, yes?
Yes, so far as I can see. I was just being paranoid.
As for bootstrapping, it's still cleaner to have a single implementation, rather than juggling them. Given we're currently only juggling 2 implementations, the problem's not intractable, just "improper".
But ^ is the symbol for conjunction, i.e. "and"-ing (well, not the literal caret, but the upwards-pointing symbol -- $\wedge$ in Latex). Did you mean to suggest ^ instead of +?
Scheme handles nested quasiquoting differently than Common Lisp - expands the unquotes in a different order or some such.
Well, if we're to believe a single statement in an Alan Bawden paper from 1999: "I am not aware of the existence of a correct optimizing quasiquotation expander for Scheme. (None of the Scheme implementations that I tested implement nested splicing correctly.)" -- http://repository.readscheme.org/ftp/papers/pepm99/bawden.pd...
The entire semantic reasoning is what I'm hung up on: does it seem right that ,,@(...) isn't supposed to work? Not that I'm necessarily advocating one rationale over another, just that I've simply not heard any such rationale. (My Google-fu is upsettingly weak here.)
Arc doesn't report an error because the second argument to unquote gets tossed out during compilation
Ah, duh. Makes sense. I had tried combing through the compiler, but was rather aimless. Thanks.
This is actually a fairly common "bug" in arc's compiler.
I'd probably call it a bug in the case of 'coerce, since the arguments (being passed directly to a Scheme function) all get evaluated -- if the third argument was truly ignored (in the "compiled away" sense), then it shouldn't even do anything. Plus, this behavior also reaches functions like 'outfile and (to an extent) 'writec.
Granted, maybe it's wise to go against "protecting" the programmer and assume that they know what they're doing; maybe I'd happen to want to assign a variable in the same statement as an 'outfile, I don't know.
Accepting the hypothetical that there's not a clear-cut way to simplify the data structures (hey, it happens), I can't think of much to do. I know what it is you want to do, as the problem does crop up from time to time -- trying to make use of the expression instead of the value of an expression. An obvious way to solve this would be to introduce a macro, e.g.
But this isn't a very palatable solution -- you'd need to define macros for each field, and even then they're only shortcuts, really. It'd be easier if there were a shorter way to do nested indexing (which there will be, come arc3: http://arclanguage.org/item?id=9163).
Upon writing this and finding someone else replying, I notice that conanite's post http://arclanguage.org/item?id=9326 contains probably the best solutions, though.
I like the mac idea too. if the concepts like "budget" and "forecast" are the main data concepts of the program; then I can see being able to re-use the macro a lot. Also I have to read up on 'deftem' and 'inst'.
My Perl knowledge is wanting, but presuming that that snippet reads from standard input, you could do the same with Arc:
(counts:tokens:drain:readc:stdin)
However, I am of the mind that string-based reading is so common that there really should be equivalent functions. Things like 'readfile use 'read to collect all the s-expressions in a file, but not the text -- there's not a 'readc variant of 'readfile. If 'readfile used 'readc instead (or you defined such a function), you could reduce the problem to something like
(counts:tokens:readfile "filename")
which is far more readable, and shorter besides.
These snippets, of course, don't exercise the same sorts of syntactic compactness that the Perl demonstrates, instead relying more on existing library functions. So it goes with these sorts of comparisons.
This is not to say Arc's exclusion of keyword parameters is without merit. e.g., Arc's (rreduce ...) is easier to read than Common Lisp's (reduce ... :from-end t). Maybe you could consider breaking your problem function up into several functions? This isn't always applicable, though, and can result in code duplication. You could resort to passing an alist or table as the last parameter. This is still clunky as all hell, so you could instead macro the process out a bit. For inspiration (and, moreover, code) on such a macro, check http://arclanguage.org/item?id=4247
1. Install version 352 of MzScheme. (This is the version we use. Don't use the latest version. There is said to be some bug/feature in it that breaks Arc.)
Arc3F has a launcher script for Linux which is supposed to work with any sufficiently Unix-like system (it's tested and works with GNU/Linux, specifically Ubuntu Hardy Heron; AFAIK the others here don't have problems, so it'll probably work with a large enough percentage of GNU/Linux systems). It also has a launcher batch file which is supposed to work with Windows; eds and b0R_ report that they have managed to run Arc3F with that batch file.
You are not supposed to use mzscheme -m -f with Arc3F, although I haven't actually tried; it might work.
