Yeah I want it to splice it into the list. That's what it does right now. But it takes a special-case. I'm having trouble figuring out how it would work if I removed the special-case from tokenize, parse, and eval. I'm not sure that the behavior of ,@ can be boiled down to just , and @. It should be independent of my implementation.
I think it's kinda intuitive for `(foo ,@(list 1 2)) to expand to (quasiquote (foo (unquote 1 2))). If unquote takes multiple subexpressions, as it does in Common Lisp (http://arclanguage.org/item?id=9912), this can evaluate to (foo 1 2).
However, I really think this should be considered part of the behavior of 'quasiquote. Just like 'unquote has no meaning except as interpreted by 'quasiquote, it's fine for @ to have no meaning except as interpreted by function calls and 'quasiquote.
Hmm... if you say `@(list 1 2), does that pass (splicing (list 1 2)) or something to 'quasiquote, or does it actually pass 1 and 2? If I were trying to accomplish @ with fexprs, I think I'd prefer the former, and I'd treat regular functions/applicatives as though they were fexprs that expanded (splicing ...) manually.
And even more fixes. It now works correctly enough that I have switched Arubic[1] over to use import.arc. This allowed me to safely make the changes I described here: http://arclanguage.org/item?id=15147
Now all you need to do is put (use arubic) at the top of your file and you'll be able to use nifty stuff like this:
Speed seems acceptable to me, though it is slower than using Arc directly (no namespaces, no Arubic, etc).
I've also been working on fexprs, writing a (much) better ssyntax parser, and a reader written in Arc. I plan for at least the ssyntax part to be included in Arubic, and probably the reader as well. I'm undecided about fexprs, mostly because of performance concerns.
I just had a fun time tracking down what I thought was a bug, but actually wasn't. If you decide to use Arubic and then try to import something, you may end up with an error similar to this:
What's happening is that it encountered something like the following:
(map (fn (e) (prn) (ppr e (+ col 2)))
(nthcdr (1+ n) expr))
But keep in mind Arubic changes the meaning of "map", "all", "some", etc. which then causes the error. The correct thing to do then is to import it in Arc's namespace, rather than Arubic's namespace:
The reason I'm pushing for this is because I need it. With my work on Arubic (and other things), I've found myself needing namespaces more and more and more.
One example is that I changed it so [1 2 3] is expanded into (list 1 2 3) rather than creating an anonymous function. This works okay, until I start using Arc code/libraries that rely on the [] syntax, then things break.
Another example is that I want to change it so map, some, all, etc. are renamed into mapfn, somefn, allfn. And then there would be macros that would expand into those functions, like so:
(each x foo
(prn x))
(eachfn prn x)
(map x foo
(+ x 5))
(mapfn car foo)
(some x foo
(some x x
(is x 5))
(somefn whitec foo)
This removes 90%+ of the need for the [] syntax, which means I can then use it for something else, like `list`. But, once again, changing stuff like that breaks things. I could safely change it with proper namespaces.
And sorry, no, using ar's runtimes won't help, because they just create a new copy and then reload arc.arc into it; but the changes I'm describing break arc.arc itself. I need namespace inheritance, or something similar.
I made quite a few major changes to get things to the point where I would consider using import.arc in my own code. In particular, I got macros working in a way that I'm satisfied with.
Same basic idea as before. You have a namespace, which contains mappings between variables/values, and also contains a reference to the parent namespace. When a variable isn't found, it checks the parent namespace, such forth and so on.
---
Biggest difference now is that I'm no longer using object.arc. Not because object.arc isn't useful, quite the contrary. Rewriting it to not use object.arc showed me just how useful of an abstraction it is. Using extend feels like using low-level byte code in comparison.
No, I changed it to not use object.arc because I plan for import to be a part of Arubic's core, so I need it to both be fast and have few dependencies. Then again, I might put objects in Arubic's core as well... so perhaps the dependency isn't so bad. I'm still undecided about that, though.
---
The second biggest difference is the way that macros are handled. Before, I simply evaluated the macro in the current namespace. I didn't like this, it felt hacky. In fact, I wanted to get rid of the call to eval completely, but of course you can't do that.
So instead, here's what I did. Macros now contain two bits of information: the function that does the code transformation, and the namespace the macro was created in. When executing the macro, it's always evaluated in the namespace it was created in, no matter what.
This will probably break all macros created before the change, though, as I can't make it retroactive. I can, however, put in code to guard against that, so it should be okay.
---
Another big change is how I actually represent namespaces. As said, I removed the dependency on object.arc, which means I now need to manually create a representation. I chose a flat list of namespaces: it will check each namespace in order until the variable is found, or it runs out of namespaces.
A flat list has many advantages, such as being quite fast, very little extra overhead, it's mutable so you can change the order of namespaces, etc.
Downside is it's clunky to implement, since I have no choice but to go in and extend multiple functions... in any case, it uses Racket namespaces now, rather than hash tables, so it should be faster, hopefully.
---
Oh yeah, another thing. With the previous version, if you didn't use w/namespace or similar, it wouldn't execute the namespace code, it just did whatever Racket did. But now it always executes the namespace code. One consequence is that before, if you tried to access an undefined variable, it would say "reference to undefined identifier: foo" but now it says "undefined variable: foo"
The only major consequence of this is speed: it'll probably run slower. Hopefully it should be fast enough for normal usage, though.
---
Lastly, I changed it so (import foo) will behave just like (use foo) except with namespace isolation. If you want to explicitly specify a path, you can use (import-as foo "/path/to/foo")
"When executing the macro, it's always evaluated in the namespace it was created in, no matter what."
I'll note that this basically makes macros pseudo-hygienic. Within a single namespace, they are unhygienic. But, if you import a macro from a different namespace, then they are hygienic. Thus, if people usually import code (rather than using "use" or "load"), then that minimizes the effect of macro hygiene.
As far as implicit gensyms go, it is certainly nicer than having to explicitly use w/uniq. I can see how it can get rather tedious though, and starts to make your elegant code look a lot like Perl.
Have you considered moving the emphasis away from macros and toward fexprs instead? I suspect if you primarily used fexprs, then the problem with implicit gensyms would go away. Of course, then you end up with new problems... like this:
(fexpr foo (x)
(eval x))
(let x 5
(foo x)) -> would this return x or 5?
I attempted to add fexprs to ar, and in doing so had to face this problem. I "solved" it by making eval implicitly evaluate in the outer (caller) environment, ignoring the fexpr's environment. You can, however, explicitly evaluate in the current environment by passing a second argument to eval.
So.. let me see if I understand this correctly. Please correct me if I'm wrong. What you are saying is that wart will have lexical scope, just like Arc. But, when a variable does not exist in the lexical scope (it is a global variable), it will then be treated as a dynamic variable, and thus take it from the caller's scope? What about this?
(def bar (x) x)
(def foo (x)
(bar x))
(foo 5) -> returns 5, as expected
(let bar nil
(foo 5)) -> ???
Would the above return 5, or would it treat "bar" as a dynamic variable?
You could work around that by saying that a global variable is only treated as dynamic if it is undefined. In other words, rather than throwing an "undefined variable" error like Racket does, it would instead just grab the value from the caller's scope. That would help a lot, but I predict it would still be somewhat risky.
However, I think there is an inherent benefit/cost tradeoff in all things, and so I suspect that this could actually work out okay in practice. I myself have desired a Lisp where all global variables are dynamic, but I never thought to make all undefined variables dynamic.
My suggestion would be to try it out and see how it works in practice.
Psst, there's already at least two Arc to JS compilers out there. You could try one of them, like mine[1]. On the other hand, an interpreter can do things that a compiler cannot. So, I think it would be good to have a de facto Arc to JS compiler, and a de facto Arc to JS interpreter. Both are useful, for different situations.
---
P.S. What do you mean when you say the ssyntax 'a:b~c is supported? ~ isn't an infix operator.
---
P.P.S. If you do decide to use my compiler, and you find a bug, or something that is missing that you need, by all means tell me about it and I'll try to patch it up.
Oh, I just looked at your brainstorm file. I would like to note that arcfn is out of date. I believe it references Arc 2, not Arc 3.1. I'm sure that myself or others can help you out if you have any questions. One of the changes between Arc 2 and Arc 3 is that Arc 3 uses "assign" whereas Arc 2 uses "set".
---
I would also like to comment on "var" vs. "let". My arc2js compiler provides "let", just like Arc, and also provides a "var" special form. I have found both to be useful, so I think both should be provided. And because they behave differently, it's somewhat tricky (and pointless) to implement one in terms of the other.
If you do implement a "var" form (I think you should), I recommend the following:
1) There should be no var hoisting. This means that in the following code:
(fn ()
(foo a)
(var a 10)
a)
It should call "foo" with the global value of "a", then bind the local variable "a" to 10, then return the local binding. This is different from how "var" behaves in JS, where all "var"s are "hoisted" to the top.
---
2) It should behave like "let", allowing you to capture global variables. Consider this code:
(fn ()
(var a a)
a)
In JS, that will set the variable "a" to undefined. I think this is incorrect behavior. Instead, it should take the global value of "a" and then bind it to the local variable "a". This is how "let" behaves.
---
3) This is somewhat of a nobrainer, but "var" should accept more than two arguments, just like "=":
(fn ()
(var a 10
b 20)
(list a b))
The above binds the local variable "a" to 10, and the local variable "b" to 20. It should also accept an odd number of arguments, with the default being nil.
---
4) It should be possible to use "var" as an expression. Everything in Arc is an expression, and I find that simplicity to be a wonderful property. "var" should follow it:
(fn ()
(if a (var b 10))
b)
The above will return either "10" or the global value of "b" depending on whether "a" is truthy or not. Note that it should only bind the local variable "b" when the "if" expression is actually evaluated.
---
My compiler handles the 1st, 2nd, and 3rd points correctly, but cannot handle the 4th correctly, because it is a compiler, not an interpreter. To be more specific, you can use "var" as an expression, but this...
(fn ()
(if a (var b 10))
b)
...is rewritten into this:
(fn ()
(var b)
(if a (= b 10))
b)
That is the best I can do, with a compiler, unless I start getting into some really insane code transformation rules.
I would also like to answer some questions you had in the brainstorm file.
---
"besides quasiquoting, a big challenge will be implementing closures"
If you have first-class environments, then implementing closures is a piece of cake. And since you're writing an interpreter, first-class environments make an awful lot of sense.
How it works is that every function has an environment. This environment contains two bits of information: a mapping between variables and values, and a reference to the outer environment.
Then, when looking up a variable, you start in the current environment, and if it's not found, you check in the outer environment. You then repeat this process as many times as necessary until you either find the variable, or reach the global scope.
I wrote an interpreter in Python that implements a language that is almost identical to Arc, so I can help out if you like.
---
"maybe there's no need to "declare" local variables; if you set a variable that doesn't exist, it will just be local (like coffee script)"
Probably a bad idea. Paul Graham already experimented with that with a (very) early version of Arc[1], but then realized that implicit bindings causes problems with macros. It's not a technical problem, but rather a human one: with implicit binding, it's very easy to accidentally create local bindings when you didn't want to.
You can, of course, experiment with it yourself, and perhaps you'll find a way to make it work. But personally, I like keeping binding/assignment separate. I find that it makes working with closures a lot easier. Consider this:
(let x nil
(def foo ()
(map [= x _] '(1 2 3))))
As you can see, we're creating a local variable "x", and then creating a global function "foo" that has access to "x". Then we're mapping over 3 numbers, and assigning each one to the outer variable "x".
Python gets around it by having a "nonlocal" and "global" keyword. If you're wondering, here's how you would do that in Python:
def foo():
x = None
def bar():
nonlocal x
for _ in [1, 2, 3]:
x = _
return bar
foo = foo()
Thus, you must either make bindings implicit, or assignment implicit. Python chose to make bindings implicit, which works out just fine in Python, but Python doesn't have macros.
The brainstorm file is never updated. I keep putting some brain dump there without cleaning the previous brain dumps.
I think closures already work; I just haven't tested them enough.
> How it works is that every function has an environment. This environment contains two bits of information: a mapping between variables and values, and a reference to the outer environment.
> Then, when looking up a variable, you start in the current environment, and if it's not found, you check in the outer environment. You then repeat this process as many times as necessary until you either find the variable, or reach the global scope.
> I would like to note that arcfn is out of date. I believe it references Arc 2, not Arc 3.1. I'm sure that myself or others can help you out if you have any questions. One of the changes between Arc 2 and Arc 3 is that Arc 3 uses "assign" whereas Arc 2 uses "set".
Thanks. I did notice ac.scm uses assign where as arcfn talks about 'set'. Anyway, I didn't implement neither assign nor set, not scar not scdr. I just implemented '=' as a special form.
Thanks for #3 btw, my '=' only worked with one variable before; now fixed.
I'm probably harping on this too much, but... I would like to note that "compatibility with Arc/3.1" is a result, not a goal. I personally feel you should drop the whole "backwards compatibility" thing. You already did to a certain degree, but only a little, with things like using stdin rather than (stdin), etc.
There are many ways to achieve backwards compatibility, like a compatibility layer, or having separate runtimes, etc. I think that one of ar's goals should be to create a better Arc. Arc has stagnated, it lacks pg's raw desire to make it better. We're fretting over backwards compatibility when we should be pushing Arc further.
There's tons of languages out there that deal with backwards compatibility, but not too many that try to become better by throwing out backwards compat completely. That's the language I fell in love with: one that tries to be perfect, to a fault. I don't want to see Arc stagnate.
What we're looking at here is the base case of a dotted list:
(a b . rest)
(a . rest)
rest
The syntax for a single cons cell is (a . b). The (a b c . d) syntax is a shorthand for (a . (b . (c . d))), and the (a b c) syntax is a shorthand for (a b c . ()), where () is nil. Here's a bit of a demonstration:
