Arc Forumnew | comments | leaders | submitlogin
Nu Arc compiler (github.com)
5 points by Pauan 4707 days ago | 76 comments


4 points by Pauan 4693 days ago | link

I implemented the Boyer–Moore–Horspool substring-search algorithm in Arc[1].

I then plugged it into my playlist program, and, well... see for yourself:

  ;; Arc posmatch:          38,236 ms
  ;; Boyer–Moore–Horspool:   6,396 ms
Wow. I think this confirms that the biggest bottleneck in my program was posmatch[2].

Keep in mind, though, Boyer-Moore isn't always faster than posmatch[3]: due to the preprocessing time, long pattern strings will cause the algorithm to perform slower.

This only applies when the string is preprocessed multiple times: if it's only used once, then the overhead should be negligible. And if you want to use the same pattern string multiple times, you can preprocess it once and then reuse it:

  (let x (boyer-moore-process "foobarquxcorgenou")
    (boyer-moore-search x "adadadadadadadadadadadad")
    (boyer-moore-search x "adadadadadadadadadadadad")
    ...
    (boyer-moore-search x "adadadadadadadadadadadad"))
In general, though, Boyer-Moore is faster. Especially for multi-string searches, which is what my playlist program does.

---

* [1]: https://github.com/Pauan/ar/blob/nu/lib/boyer-moore.arc

* [2]: http://arclanguage.org/item?id=14204

* [3]: https://github.com/Pauan/ar/blob/nu/notes/timing

-----

2 points by Pauan 4693 days ago | link

New timing link: https://github.com/Pauan/ar/blob/nu/notes/timing.md

-----

3 points by Pauan 4701 days ago | link

I just added in keyword args:

  > (def foo (:a :b)
      (list a b))

  > (foo :a 1)
  (1 nil)

  > (foo :a 1 :b 2)
  (1 2)
As a side note, keyword arguments are currently almost twice as slow as normal args:

  > (let foo (fn (:a :b) (list a b))
      (timeit (foo :a 1 :b 2)))
  time: 17.521  gc: 0.408  mem: 2303.568

  > (let foo (fn (a b) (list a b))
      (timeit (foo 1 2)))
  time:  9.515  gc: 0.3    mem: -22736.128
So if you really want performance at any cost... then why are you using Arc in the first place?! :P

---

I also (mostly) added in alist key support:

  > (= foo '((b 2) (a 1) (c 3)))
  ((b 2) (a 1) (c 3))

  > foo!a
  1

  > (= foo!a 10)
  10

  > foo
  ((b 2) (a 10) (c 3))
Which means alists now work with keyword destructuring as well:

  > (let (:a :b :c) '((b 2) (a 1) (c 3))
      (list a b c))
  (1 2 3)

-----

2 points by Pauan 4689 days ago | link

I added in Arubic-style namespace support, and also implemented dynamic and parameter variables better. If you're interested in reading more, check out these two links:

https://github.com/Pauan/ar/blob/nu/notes/dynamic.md

https://github.com/Pauan/ar/blob/nu/notes/namespaces.md

---

I also discovered that it's possible to implement `quote` as a macro:

  (mac quote (x)
    (list (fn () x)))

-----

1 point by Pauan 4688 days ago | link

I finally added in Arubic to Nu. Now all you have to do is `./arc -i arubic` and you'll get a REPL with Arubic functionality. You can also use `(import arubic)` to write a file using Arubic.

Why would you want to do that? Well, here are the current changes between Arc and Arubic, though I plan to add more over time:

  (map x foo ...)  ;; Arubic
  (map [...] foo)  ;; Arc 3.1
Ditto for mappend, some, all, and keep. This is especially convenient when using destructuring:

  (map (x y) foo ...)        ;; Arubic
  (map (fn ((x y)) ...) foo) ;; Arc 3.1
I changed the [] and {} syntax as well:

  [1 2 3]   -> (list 1 2 3)
  {a 1 b 2} -> (obj a 1 b 2)
Because of the changes above, you no longer need the [] syntax for functions most of the time, so I'm repurposing them to create lists, which I've found to be far more common.

`isa` accepts multiple arguments:

  (isa x 'foo 'bar 'qux)      ;; Arubic
  (in type.x 'foo 'bar 'qux)  ;; Arc 3.1
`prn` prints spaces between each argument:

  (prn "foo" "bar" "qux") -> "foo bar qux\n"
You can also use `str` and `str?` as aliases for `string` and `string?`.

Small changes, but they do make writing code more pleasant.

-----

2 points by Pauan 4688 days ago | link

I just removed `setforms` from every place in arc.arc except for `rotate` and `=`[1]. Why? Well, there's two reasons:

1) It truly does not make any sense to me why zap is defined like this:

  (mac zap (op place . args)
    (with (gop    (uniq)
           gargs  (map [uniq] args)
           mix    (afn seqs
                    (if (some no seqs)
                        nil
                        (+ (map car seqs)
                           (apply self (map cdr seqs))))))
      (let (binds val setter) (setforms place)
        `(atwiths ,(+ binds (list gop op) (mix gargs args))
           (,setter (,gop ,val ,@gargs))))))
Rather than this:

  (mac zap (f x . args)
    `(= ,x (,f ,x ,@args)))
2) This helps out a ton with arc2js. In arc2js, now all I need to do is provide an `=` macro and I'll get zap, or=, push, pull, swap, etc. all for free. Without this change, I'd have to define my own custom versions of zap, or=, etc...

---

* [1]: https://github.com/Pauan/ar/commit/e58a46fb47802394031dc4afe...

-----

3 points by rocketnia 4687 days ago | link

"It truly does not make any sense to me why zap is defined like this"

'zap is the only use I typically have for the 'setforms "binds" list (which ensures the subexpressions of 'place are only evaluated once).

Still, 'zap doesn't need to work that way: as long as I'm using a language where I know 'zap evaluates its place twice, I'm pretty much okay with it. It's a wart, but it's not an impediment.

To explore Arc-3.1-like options for a bit, here's a cleanup of Arc 3.1's definition of 'zap:

  (mac zap (op place . args)
    (with (gop                 (uniq)
           gargs               (map [uniq] args)
           (binds val setter)  setforms.place)
      `(atwiths (,@binds ,gop ,op ,@(mappend list gargs args))
         (,setter (,gop ,val ,@gargs)))))
If we allow 'setter and 'val to compile and evaluate before 'op and 'args, it gets shorter:

  (mac zap (op place . args)
    (let (binds val setter) setforms.place
      `(atwiths ,binds
         (,setter (,op ,val ,@args)))))
I prefer to arrange the compilation and evaluation orders from left to right ('op, 'place, 'args), using a technique like this:

  (mac place (place)
    (let (binds val setter) setforms.place
      `(withs ,binds
         (list (fn () ,val) ,setter))))
  
  (mac zap (op place . args)
    `(atomic:fn-zap ,op (place ,place) (list ,@args)))
  
  (def fn-zap (op (getter setter) args)
    (setter:apply op (getter) args))

-----

1 point by Pauan 4687 days ago | link

"'zap is the only use I typically have for the 'setforms "binds" list (which ensures the subexpressions of 'place are only evaluated once)."

Hm... yes, you're right, `(zap + (foo (bar qux)) 1)` evaluates `(bar qux)` twice, and I don't see an easy/obvious way to fix that in `=`. I'll need to think about this.

-----

1 point by Pauan 4688 days ago | link

And I just removed setforms from rotate too[1]. An improvement? You be the judge:

  ;; Arc 3.1
  (mac rotate places
    (with (vars (map [uniq] places)
           forms (map setforms places))
      `(atwiths ,(mappend (fn (g (binds val setter))
                            (+ binds (list g val)))
                          vars
                          forms)
         ,@(map (fn (g (binds val setter))
                  (list setter g))
                (+ (cdr vars) (list (car vars)))
                forms))))

  ;; Nu
  (mac rotate places
    (w/uniq u
      (let shift (join (cdr places) (list u))
        `(let ,u ,(car places)
           (= ,@(mappend list places shift))))))
---

* [1]: https://github.com/Pauan/ar/commit/890209c76356da2993a35ed4b...

-----

1 point by Pauan 4688 days ago | link

And now `=` no longer calls `atomic-invoke` for complex assignments. Based on this information here:

http://docs.racket-lang.org/reference/threads.html

It would appear that the `=` operator is already thread-safe even without `atomic-invoke`. In any case, if you're terribly worried, you can always wrap it yourself.

-----

1 point by Pauan 4688 days ago | link

I just got rid of `setforms` completely[1]: not even `=` uses it anymore. I then reimplemented `=` in a much shorter and clearer way[2].

Arc 3.1 takes 80 lines to implement expand=, but Nu takes only 31. And Nu is much clearer and easier to understand as well. In addition, Nu's output is much shorter and is faster:

  ;; Nu
  > (macex1 '(= foo!bar 5))
  (do (sref foo 5 (quote bar)))

  ;; Arc 3.1
  > (macex1 '(= foo!bar 5))
  (do (atwith (g1 foo g3 (quote bar) g4 5) ((fn (g2) (sref g1 g2 g3)) g4)))
---

* [1]: setforms is included in compat.arc for backwards compatibility with Arc 3.1, but it's not actually used anywhere.

* [2]: https://github.com/Pauan/ar/blob/c835e67d919d7a555a1c856812a...

-----

1 point by rocketnia 4687 days ago | link

I can't find whatever information you're talking about. Would you mind quoting it and/or elaborating?

Don't put too much work into the explanation, 'cause I'm likely to come in at the end and say "but what about X?" :-p It sounds too good to be true.

-----

1 point by Pauan 4686 days ago | link

It's right near the top of the link:

  All constant-time procedures and operations provided by Racket are
  thread-safe because they are atomic. For example, set! assigns to a variable
  as an atomic action with respect to all threads, so that no thread can see a
  “half-assigned” variable. Similarly, vector-set! assigns to a vector
  atomically. The hash-set! procedure is not atomic, but the table is
  protected by a lock; see Hash Tables for more information. Port operations
  are generally not atomic, but they are thread-safe in the sense that a byte
  consumed by one thread from an input port will not be returned also to
  another thread, and procedures like port-commit-peeked and write-bytes-avail
  offer specific concurrency guarantees.
It mentions that hash table assignment is not thread-safe, however if you then go to the hash table page[1], it says this:

  A mutable hash table can be manipulated with hash-ref, hash-set!, and
  hash-remove! concurrently by multiple threads, and the operations are
  protected by a table-specific semaphore as needed. Three caveats apply,
  however [...]
In other words, Racket already handles everything, according to the docs. If you're ever worried enough, or run into any problems, it's not hard to wrap it in `atomic` yourself. I'd rather not have the cost of `atomic-invoke` for every assignment, especially if you run all your code in one thread (like I do).

By the way, Nu doesn't use `set!` for global assignment, so I'm not sure if global assignment in Nu is thread-safe or not. But I'd assume it is, since I think `namespace-set-variable-value!` is constant-time.

---

* [1]: http://docs.racket-lang.org/reference/hashtables.html

-----

2 points by rocketnia 4686 days ago | link

In pg-Arc, '= on a variable is 'assign without 'atomic. Where 'atomic comes in is when there's a setforms thing to worry about.

And in that case, this...

  (= (car car.foo) (bar))
...turns into something like this:

  (atomic:with (gs1 car.foo gs2 (bar))
    ( (fn (val) (scar gs1 val))
      gs2))
This ensures that car.foo, (bar), and (scar gs1 val) all happen without interference in between. I suspect Racket at most protects those on an individual basis.

That said, I don't care about 'atomic myself. :-p

-----

1 point by Pauan 4686 days ago | link

"In pg-Arc, '= on a variable is 'assign without 'atomic. Where 'atomic comes in is when there's a setforms thing to worry about."

I am aware. It still seems to me that if you're dealing with threads, you should wrap assignment in atomic yourself if you're worried about such things. Code that doesn't deal with threads shouldn't have to use atomic.

Perhaps there should be an `a=` macro that's just like `=` but it calls `atomic`. Hm... I wonder... would it be possible to detect whether code is running in the default thread and if not, automatically wrap it in atomic...? May be more trouble than it's worth, though.

-----

2 points by rocketnia 4686 days ago | link

"May be more trouble than it's worth, though."

That's what I think. Anyone who cares can say (atomic:= ...) or (atomic:zap ...), so I only see a couple of reasons why we'd want to have the 'atomic implicit:

- We want to use it all the time anyway. (I doubt it, but it's hard to tell. I haven't used threads, and therefore I've never bothered to find a way to squeeze utility out of it.)

- There are people who do care, and they'd be better off if the people who didn't care still used 'atomic by accident. (Again, it's hard for me to tell if this is true.)

-----

1 point by Pauan 4688 days ago | link

I just implemented `eachfn` and then changed Arc so the `each` macro just calls `eachfn`. This had a significant speedup, with no cost at all in functionality. At this rate, Nu will end up being just as fast as Arc 3.1, with all the shiny extra features and bug fixes.

So, the rule of thumb is: don't write macros that do a lot of work. Instead, write a function that does the work, and then write a macro that just calls the function. This goes double for macros like `each` that do a type-check at runtime.

-----

1 point by akkartik 4688 days ago | link

  (map x foo ...)  ;; Arubic
  (map [...] foo)  ;; Arc 3.1
I don't follow. Can you provide an example?

-----

1 point by Pauan 4688 days ago | link

Basically, map, mappend, some, etc. behave like the "each" macro:

  (= foo '(1 2 3 4 5))

  ;; Arubic
  (map x foo (prn x)) -> (1 2 3 4 5)

  ;; Arc 3.1
  (map [prn _] foo)   -> (1 2 3 4 5)

-----

1 point by akkartik 4687 days ago | link

Hmm. Can x also be a function?

-----

1 point by Pauan 4687 days ago | link

No.. it can't be. If you want that, you should use mapfn:

  (mapfn x foo)
Which behaves exactly like how map currently behaves in Arc 3.1.

-----

1 point by akkartik 4687 days ago | link

Why not just use each?

map has a pretty consistent semantic over the decades; I wouldn't mess with what it means.

-----

1 point by Pauan 4687 days ago | link

...because each does not map. Nor does it some. Nor does it mappend. Nor does it all. Nor does it keep. Nor does it rem.

I don't think you understand what I'm talking about. In Arubic, map is just a macro that expands into mapfn. The semantics are exactly the same as Arc 3.1, it's just easier to say...

  (map x foo ...)
...rather than:

  (map (fn (x) ...) foo)
The only point of it is to remove the (fn ...) bit, resulting in shorter and easier to read code.

-----

1 point by akkartik 4687 days ago | link

...because each does not map. Nor does it some. Nor does it mappend. Nor does it all. Nor does it keep. Nor does it rem.

Sorry, that makes no sense to me.

Focus on just map. Are you saying it doesn't return a transformed list? Ok. But can we call it something different? You'll have to pry

  (map car xs)
from my cold dead hands :)

-----

1 point by Pauan 4687 days ago | link

"Focus on just map. Are you saying it doesn't return a transformed list?"

No. You are still misunderstanding me. I will be as clear and precise as I possibly can. In Arc 3.1, you call `(map x foo)` where `x` is a function and `foo` is a list.

In Arubic, you would say that as `(mapfn x foo)`. It's exactly the same, except you use the name `mapfn` rather than `map`. Now, Arubic also provides a macro called `map`, which expands into `mapfn`. In other words:

  (mac map (var x . body)
    `(mapfn (fn (,var) ,@body) ,x))
The sole purpose of this macro is so that you don't need to type out the `(fn ...)` part in `(mapfn (fn (x) ...) foo)`. That's it. The meaning of `map` is the same, it's just that now that it's a macro, it's a lot more convenient to use. But you can still pass in a function directly by using `mapfn`.

---

"You'll have to pry (map car xs) from my cold dead hands :)"

If you dislike any parts of Arubic (whether misunderstood or not), simply do not use those parts. The point of the namespace system is that you can change what you want and use what you want, rather than being tied down to one semantic/syntax/meaning/language. Thus, it is possible (and relatively easy) to use some of the things from Arubic that you like, while not using the parts you dislike.

In this particular case, however, you are misunderstanding me. Your example would be `(mapfn car foo)` in Arubic.

-----

1 point by akkartik 4687 days ago | link

In this particular case, however, you are misunderstanding me. Your example would be `(mapfn car foo)` in Arubic.

No, I'm now pretty sure I'm not misunderstanding you.

My entire objection is to the name switch. Of course I know I can change the default. It's not a sensible default, it's like defining 1 to be 0.99. You're fragmenting the semantics of a name that has thus far had a pretty clear meaning. You're welcome to do this, but it's going to hinder users with any programming background.

I don't understand why the thing you want must be called map. Why not mapmc, for example? Or some other 3-letter name?

-----

1 point by Pauan 4687 days ago | link

"My entire objection is to the name switch. Of course I know I can change the default. It's not a sensible default, it's like defining 1 to be 0.99."

It's not the default. The default is Arc 3.1. You have to opt-into Arubic. In any case, how is it not sensible? It provides the shortest, easiest to read, and fastest code.

---

"You're fragmenting the semantics of a name that has thus far had a pretty clear meaning."

And this is why I think you're still misunderstanding me. The meaning of map is the same. It's just that now you can write (map x foo ...) rather than (map (fn (x) ...) foo), which is shorter and easier to read.

If you're worried that it'll mess up Arc programmers who are used to the naming of map, then as already said, Arc 3.1 is the default. You only get the new syntax if you import Arubic.

I do not see how changing the meaning of names in a new language is a bad thing. That's like me saying that wart should use exactly the same names as Arc, which is absurd: they're different languages. If you want to use a different name for something in wart, then go for it.

Perhaps you still do not grasp that Arubic is a different language from Arc, implemented in a separate namespace so as not to mess up Arc's namespace.

---

"I don't understand why the thing you want must be called map. Why not mapmc, for example? Or some other 3-letter name?"

....

I'm done arguing. Goodbye.

-----

1 point by akkartik 4687 days ago | link

"I'm done arguing. Goodbye."

Suit yourself. I don't understand what I have said to merit hostility.

I know we can all do what we want. I also am aware of lots of things I can try if I don't like something you do. I don't understand these defensive reactions. Was I rude somewhere?

All this while you keep insisting you know what I am misunderstanding. Is that not rude?

Arubic's map is a macro, not a function. Until it grows fexprs the meaning is by definition not the same.

"This is how I like it" is a perfectly valid reason (and would have ended this conversation eons ago). Just don't insist it's all still the same when it isn't.

-----

1 point by akkartik 4687 days ago | link

I do not see how changing the meaning of names in a new language is a bad thing.

That's an equally good response to any feedback you get.

Do you seriously think changing the meaning of 42 is a 'good' thing?

I don't even feel that strongly about this conversation. It's only gone on so long because:

a) You kept insisting I am too dense to comprehend your subtle creations.

b) You kept insisting you're changing nothing.

-----

1 point by akkartik 4687 days ago | link

It's not the default. The default is Arc 3.1.

No, the default is racket. No, it's machine code. Or is it cosmic background radiation? WTF, are you seriously saying that creating a new language isn't an act of choosing defaults?

-----

2 points by rocketnia 4686 days ago | link

In Nu, the default namespace is the one corresponding closely with Arc 3.1. And in this namespace, the default definition of 'map is pretty much Arc 3.1's. Therefore, "it's not the default" is true in Nu.

Another namespace in Nu is Arubic. Using this namespace is like using a language other than Arc 3.1, where 'map is a macro. If your problem is with Arubic itself, you're free to ignore the parts of Arubic you don't like (even if you want to use an library written in full Arubic). If the point is you think people shouldn't dilute the overall meaning of "map" with the meaning Pauan gives it in Arubic, that's a moral position, with Pauan as an unintentional villain!

---

What I think is that the meaning of "map" isn't tainted here, at least not any more than usual. In fact, I wanted 'map to be an 'each -like macro before I knew Pauan had the same thing in mind.

If I don't know what language we're talking about and I hear "map," I assume it'll be an abstraction that applies a given transformation to elements of a given data structure (whatever "abstraction," "transformation," "data structure," and "given" mean), along with some accidental complexity suitable to the language, such as the timing of computation, side effects permitted in the transformation, or late-bound dependencies (like 'map calling out to 'cons, such that rebinding 'cons changes what happens).

In a language where macros like 'after and 'each are more convenient to use than higher-order functions like 'protect and Anarki's 'trav, a macro for map is to be expected.

In Arc, almkglor[1] calls this macro "mapeach" in Anarki, and I call it "maplet" in Lathe. Pauan and I call it "map" as long as we don't have naming conflicts to worry about. As long as this macro is the primary way we use the map concept, using that name keeps our programs brief and frank.

[1] I'm not sure about this credit, but I found it at https://github.com/nex3/arc/blame/arc2.master/arc.arc and verified it with a Web search.

-----

2 points by rocketnia 4686 days ago | link

Whoops, Anarki's 'trav is a macro. I meant 'walk. ^_^;

-----

1 point by Pauan 4689 days ago | link

By the way, just to show how easy it is to use the namespace system, I'll demonstrate how it's possible to load and use an Arubic library from Arc.

First off, Arubic is implemented as a namespace on top of Arc's namespace, so assuming the Arubic library is called "foo.arc", when you do this:

  > (import foo)
  nil

  > namespace
  #<namespace (len 2)>
Notice that the namespace is now length 2, because it inherits from Arc's namespace. But you don't want to use Arubic, you want to use Arc! But because of the malleability and inspectability of Nu namespaces, it's easy to fix that. Just do this:

  > (zap namespace-fn namespace rev)
  #<namespace (len 2)>
The above just reversed the order of the namespace, so now the Arc namespace inherits from the Arubic namespace, rather than vice versa.

This effectively "undoes" the changes that Arubic made, which is exactly what you want: now you can use the functions and macros defined in the Arubic library from within Arc, and everything works perfectly.

-----

1 point by Pauan 4689 days ago | link

I just did some timing tests. As I suspected, the namespace system is quite costly in terms of performance:

  1   2      3    4    6    6    7
  0%  1732%  31%  22%  20%  17%  12%
What the above means is that when going from 1 namespace to 2, you lose 1732% of speed, meaning your code will now run ~17 times slower. But going from 2 namespaces to 3 has only a 31% drop in speed, and 3 to 4 only has 22%, etc.

What this means is that the initial hurdle of namespaces is quite large, but adding more namespaces after the 2nd is quite decent in terms of performance.

The reason for this is (ironically) speed optimizations. When executing code in the Arc namespace, I actually completely bypass the namespace system entirely, so it's relying on Racket to do global lookups, and Racket is far faster at looking up global variables than Nu is.

That's why going from 1 to 2 namespaces is so slow: Racket is so fast that when it switches to the Nu code, Nu looks horribly slow by comparison. But when going from 2 to 3 namespaces, we're already using the Nu code, so the difference in speed is tiny.

In other words, Racket is so fast it makes Nu look bad.

---

By the way, this only applies to code that is evaluated in a Nu namespace: if you evaluate everything in Arc's namespace, there's no cost in speed at all. So this drop in speed only applies while using things like Arubic, and not Arc itself.

-----

1 point by Pauan 4689 days ago | link

I just added in an interesting feature: automatic namespace creation.

When using the `(import ...)` form, if the file tries to overwrite an already-existing variable, it will automatically create a new namespace. Otherwise, it won't.

This is mostly a performance optimization, since it's entirely possible to load every single file into a different namespace, but using this technique I avoid creating namespaces except when absolutely necessary, while still providing a reasonable amount of library isolation.

-----

2 points by akkartik 4689 days ago | link

I enjoyed the writeups!

I'm still anti-namespaces -- I'd rather work on codebases that don't need them, and I'd rather just rely on tests to catch collisions. But your article makes an eloquent case for them that has had more effect than anything else I've read.

-----

1 point by Pauan 4689 days ago | link

I would like to note something. If all you want to do is write programs and get stuff done then namespaces are indeed a luxury. It is not particularly difficult to use a language (including Arc) to write useful programs, despite the lack of namespaces.

Where namespaces start to become necessary is when you want to change the language itself beyond what can be easily done by adding macros. Then you are in the predicament that you want to write your code in the new (and presumably better) language that you created, but now you have to give up libraries written in the old language.

With namespaces, it's no longer an either/or thing: you can have your cake and eat it too (at the cost of some performance). So, for writing code, namespaces aren't important. But for language experimentation, namespaces are so incredibly useful that I consider them necessary.

The only reason I put namespace support into Nu's core is because I really want to use Arubic, but doing so practically requires decent namespace support. I invented this system out of necessity, which should (hopefully) demonstrate its usefulness.

-----

1 point by akkartik 4689 days ago | link

That's a good point. Arc permits lifting scheme primitives. Would it be hard for Nu to permit lifting arc primitives?

-----

2 points by Pauan 4689 days ago | link

I'm going to go back and re-answer this question:

"Would it be hard for Nu to permit lifting arc primitives?"

Not at all. Arc lifting happens automatically and transparently with namespace inheritance. It works right now. In fact, that's the entire point of the namespace system.

And if you ever want to directly evaluate something in Arc (bypassing the inheritance system), you can call `(w/arc3 ...)`

-----

2 points by akkartik 4689 days ago | link

Right, makes sense.

In going from racket to arc, nothing is passed across by default, and in going from arc to Nu[1] everything is available by default.

[1] just like in going from common lisp to wart :)

-----

1 point by Pauan 4689 days ago | link

What do you mean by "lifting Scheme primitives"?

-----

2 points by akkartik 4689 days ago | link

I mean the $ primitive:

http://github.com/nex3/arc/blob/6132d33e09/ac.scm#L49

Example of its use:

http://github.com/nex3/arc/blob/6132d33e09/lib/re.arc

(The regexp- functions are from racket.)

-----

1 point by Pauan 4689 days ago | link

Okay, I'm still not sure what you mean by "lifting Arc primitives" but I'll do my best to explain how Nu works.

The compiler itself is written in Arc's namespace. There is no separation at all between Arc, the Arc compiler, and Racket: everything is cobbled into one giant namespace. So changing the compiler is as simple as writing an Arc function/macro. This is how ar works, and I liked it enough that I copied the idea for Nu.

That also means no need for the $ primitive because all Racket stuff is prefixed with `racket-`. Thus you would say `racket-regexp-match` rather than `$.regexp-match`.

Note: it's possible to import Racket modules with any prefix you like (including no prefix), but it's there mostly to avoid namespace collisions with Arc stuff.

---

As for doing the same thing with Arc primitives, I assume you mean that if you create a new namespace, you might sometimes want to use the definitions in the new namespace, and sometimes the definitions in Arc's namespace.

You can do that too, by calling eval-w/ with arc3-namespace:

  (eval-w/ arc3-namespace
    foo)
Because it's common enough to want to load things into Arc's namespace, the above can also be expressed as w/arc3:

  (w/arc3 foo)
That will give you the definition of "foo" from Arc's namespace. This shouldn't be necessary most of the time because it's possible to have a namespace inherit from another. In that case, if the variable isn't found, it'll search in the parent.

Thus, if you make a new namespace that inherits from Arc's namespace, you'll automatically get all the Arc stuff for free. That's how Arubic works: it makes a namespace that inherits from Arc, and then changes things. Anything that Arubic doesn't change is automatically taken from Arc.

-----

1 point by akkartik 4689 days ago | link

Ok perhaps your namespaces are the simplest way to 'lift' arc into Nu.

If you implement language A in language B, A is above B in the stack. It's meaningless to call A's code from B, but useful to call B from A. The latter is what I call lifting. But I realize this terminology doesn't work for anything but simple static compilation.

-----

2 points by Pauan 4689 days ago | link

"It's meaningless to call A's code from B"

If I wrote an awesome library in Arubic, especially a library that would be difficult to port to Arc, and you wanted to use the library, but didn't want to use Arubic, it would be very useful to write Arc code that used said Arubic library.

As another example, you may want to write a module in Racket that imports a library written in Arc. Even though Arc is implemented in Racket, I don't see how that counts as "meaningless". This example isn't even theoretical, there was somebody on this forum earlier who wanted to do exactly that:

http://arclanguage.org/item?id=14608

---

"but useful to call B from A."

That's handled by the w/arc3 form, which as already said is not needed most of the time, due to inheritance.

-----

2 points by Pauan 4707 days ago | link

Okay, some interesting info: Racket is apparently quite slow at applying a function at runtime. I ran this code in Arc 3.1, ar, and Nu:

  (time (repeat 1000000 (let (a b) (list 1 2) a)))
Here are my results:

  Nu      time: 3165 msec.
  Arc 3.1 time: 1830 msec.
  ar      time: 1391 msec.
Very surprising! Arc 3.1 expands into a Racket let* and Nu expands into a Racket lambda... but ar's code is implemented in Arc, so it expands into multiple nested Arc lambdas... yet somehow manages to be faster than Arc 3.1 and Nu.

I am not sure how plain lambdas are faster than Racket's built-in let* . In any case, I changed my code so it expands into let* and now Nu gets 1649 msec, which is comparable to ar. Thus the problem is clearly applying a function at runtime.

Just to remove some more variables, I changed the function so it didn't use rest or optional args, but it ended up being almost as slow, so I know it's not that.

-----

1 point by Pauan 4707 days ago | link

Hm... perhaps it's not the fact that it's applying, per se. Perhaps it's because Racket has to create a function every single time. In other words, if Racket sees this:

  ((fn (x) ...) ...)
It can try and optimize it away so a function is never actually created. But when Racket sees this:

  (apply (fn (x) ...) ...)
It can't do that optimization... thus it's forced to create a function. Creating and destroying a function every time the let is called could indeed cause a performance problem.

---

EDIT: to test my hypothesis, I used this code:

  (let foo (fn ((o a) (o b) . rest) a)
    (time (repeat 1000000 (let x (list 1 2) (apply foo x)))))
...unfortunately, it ended up getting 2966 msec, so it seems the problem is indeed Racket's apply, or perhaps my compiler's apply. I'll see if I can optimize it a bit.

-----

2 points by waterhouse 4700 days ago | link

Racket seems fairly good at "lambda lifting", if that is the correct term. To demonstrate with adding the numbers from 1 to n: version 0 should pretty obviously be compiled into a loop; versions 1 and 2 are less obvious. In 64-bit "racket", versions 1 and 2 seem to take twice as long as the loop in version 0, but that's still much better than allocating lambdas; in 32-bit, the "twice as long" difference is smothered by the cost of allocating/GCing bignums. The last version is actually written in Arc, and 25x slower in arc3.1; though in 32-bit, the cost of handling bignums makes arc3.1 only 2x slower. The results of the 64-bit version seem to demonstrate that Racket successfully avoided allocating closures at runtime in all cases.

  (define (sum0 n)
    (let loop ((n n) (tt 0))
      (if (zero? n)
          tt
          (loop (- n 1) (+ n tt)))))
  (define sum1
    (λ (n)
      ((λ (f n tt)
         (if (zero? n)
             tt
             (f f (- n 1) (+ n tt))))
       (λ (f n tt)
         (if (zero? n)
             tt
             (f f (- n 1) (+ n tt))))
       n
       0)))
  (define sum2
    (λ (n)
      ((λ (f) (f f n 0))
       (λ (f n tt)
         (if (zero? n)
             tt
             (f f (- n 1) (+ n tt)))))))
  (= sum3
     (fn (n)
       ((fn (f n tt)
          (if (is n 0)
              tt
              (f f (- n 1) (+ n tt))))
        (fn (f n tt)
          (if (is n 0)
              tt
              (f f (- n 1) (+ n tt))))
        n
        0)))
  
  ;Paste this command in, but copy the above to clipboard before running:
  arc> (let xs (readall:pbpaste) (map [eval (list '$ _)] butlast.xs) (eval last.xs)
  (each x '(sum0 sum1 sum2 _sum3) (repeat 2 (time:eval `(($ ,x) 10000000)))))

  ;64-bit racket v5.2.0.3: no mallocing beyond initial compilation
  time: 41 cpu: 41 gc: 0 mem: 25720
  time: 40 cpu: 41 gc: 0 mem: 6096
  time: 80 cpu: 80 gc: 0 mem: 7576
  time: 80 cpu: 80 gc: 0 mem: 6136
  time: 81 cpu: 80 gc: 0 mem: 7576
  time: 80 cpu: 80 gc: 0 mem: 6096
  time: 1026 cpu: 1027 gc: 0 mem: 7408
  time: 1018 cpu: 1019 gc: 0 mem: 6112

  ;32-bit racket v5.1.3.10: runtime is dominated by consing bignums
  time: 894 cpu: 892 gc: 24 mem: 1478560
  time: 872 cpu: 872 gc: 16 mem: 1236500
  time: 841 cpu: 841 gc: 15 mem: 1238156
  time: 844 cpu: 843 gc: 17 mem: 1236476
  time: 839 cpu: 839 gc: 15 mem: 1237300
  time: 838 cpu: 837 gc: 15 mem: -15541124
  time: 1857 cpu: 1857 gc: 18 mem: 1237784
  time: 1864 cpu: 1864 gc: 17 mem: 1236436

-----

1 point by Pauan 4707 days ago | link

Update: I just tested this code:

  (time (repeat 1000000 (apply (fn (a) a) (list 1))))
And the times:

  Nu time:      2875 msec.
  ar time:      1776 msec.
  Arc 3.1 time: 1725 msec.
So it seems the problem is with Nu's apply, not with Racket's apply. That's good, since that means I should be able to fix the problem.

-----

2 points by Pauan 4706 days ago | link

I haven't fixed `apply` yet, but I did put in a workaround. Using Racket's `apply`, Nu is actually faster than Arc 3.1 and ar!

   (timeit (let a 1 a))
   
  ar      time: 8.101  gc: 0.268  mem:   973.008
  Nu      time: 6.923  gc: 0.0    mem:    88.736
  Arc 3.1 time: 4.77   gc: 0.28   mem: -3305.9


   (timeit (let a (list 1 2) (car a)))

  Arc 3.1 time: 17.3    gc: 0.86   mem: -7,759.58
  ar      time: 10.303  gc: 0.552  mem: 1258.64
  Nu      time:  8.158  gc: 0.196  mem: -515.648


   (timeit (let (a b) (list 1 2) a))

  Arc 3.1 time: 17.47   gc: 1.0    mem:  -6997.07
  ar      time: 13.166  gc: 0.696  mem: -16510.112
  Nu      time: 12.102  gc: 0.512  mem: -10028.488
  
So, it seems my idea of applying nested functions to implement destructuring is good in essentially every way: faster, shorter, and easier to implement.

Interestingly, judging by the data above, it would seem Arc 3.1 is very slow at creating lists, probably because `list` is implemented in arc.arc, whereas ar and Nu provide faster implementations.

---

Now let's test optional args:

   (timeit ((fn (a) a) 1))
  
  ar      time: 7.534  gc: 0.352  mem:   866.232
  Nu      time: 6.976  gc: 0.0    mem:    88.368
  Arc 3.1 time: 4.78   gc: 0.28   mem: -3295.58
  
  
   (timeit ((fn (a (o b)) a) 1))
  
  ar      time: 14.493  gc: 0.464  mem:  1639.648
  Nu      time:  7.903  gc: 0.248  mem: -1664.792
  Arc 3.1 time:  5.84   gc: 0.36   mem: -2097.19


   Overhead
  ar      - 6.959
  Arc 3.1 - 1.06
  Nu      - 0.927
  
As you can see, in Nu and Arc 3.1, there's very little overhead from optional args, but in ar, optional args are quite costly.

-----

1 point by Pauan 4706 days ago | link

Update: I didn't want to be unfair to Arc 3.1 because of its slow implementation of `list`, so I redid the tests using `quote` instead:

   (timeit (let a '(1 2) (car a)))
  
  Nu      time: 10.628  gc: 0.196  mem: -1747.08
  ar      time: 8.529   gc: 0.252  mem:   967.432
  Arc 3.1 time: 5.26    gc: 0.34   mem:  4952.98
  
  
   (timeit (let (a b) '(1 2) a))
   
  Nu      time: 14.066   gc: 0.52   mem:    90.504
  ar      time: 13.305   gc: 0.376  mem: -9236.904
  Arc 3.1 time: 6.79     gc: 0.35   mem: -2,093.93


   Overhead
  ar      - 4.776
  Nu      - 3.438
  Arc 3.1 - 1.53
  
As expected, Arc 3.1 is miles ahead of both ar and Nu. Interestingly, Nu is now listed as slower than ar... it would appear that either Nu has a faster implementation of `list`, a slower implementation of `quote`, or possibly both. In any case, this demonstrates that applying nested functions should be approximately the same as complex fns in terms of speed.

One thing I noticed is that Nu has drastically greater overhead than Arc 3.1, but less than ar.

-----

1 point by Pauan 4704 days ago | link

It seems the problem was that quote was slow in Nu. I've fixed that, so here's the new times:

   (timeit (let a '(1 2) (car a)))
  
  ar      time: 8.613  gc: 0.308  mem:  460.696
  Nu      time: 7.671  gc: 0.0    mem:   88.976
  Arc 3.1 time: 5.33   gc: 0.35   mem: 5050.25
  
  
   (timeit (let (a b) '(1 2) a))
   
  ar      time: 12.111  gc: 0.436  mem: -19278.128
  Nu      time: 11.438  gc: 0.324  mem:   1435.016  (apply fn)
  Nu      time: 8.96    gc: 0.0    mem:    125.352  (Racket let*)
  Arc 3.1 time: 7.0     gc: 0.35   mem:  -2124.82


   Overhead
  ar      - 3.498
  Arc 3.1 - 1.67
  Nu      - 1.289
Nu now has the lowest overhead out of the three...! Also note that Nu does not spend any time in garbage collection, unlike ar and Arc 3.1.

This seems to be a common trend: Nu either spending no time in garbage collection, or less time than ar and Arc 3.1. Not sure how important that is compared to raw speed, but it's nice.

Unfortunately, this also demonstrates that applying nested functions is slower than using a Racket let*. So the reason Nu won the speed contest earlier wasn't because of my destructuring idea: Nu was just plain faster than ar in general.

-----

2 points by Pauan 4707 days ago | link

And since I'm in a timing mood, here's the times for optional args:

  (time (repeat 1000000 ((fn (a (o b 3)) (list a b)) 1 2)))

  Arc 3.1 time: 1828 msec.
  ar time:      1814 msec.
  Nu time:      1554 msec.
So, it does make a difference that Nu uses plain lambdas, rather than complex fns! Now I just need to get apply to be faster.

---

On a related note: racket-set! is slow. Using racket-let or Arc's let is faster, by a fairly significant amount. So I'll be changing my compiler so it doesn't use mutation.

-----

2 points by rocketnia 4707 days ago | link

Awesome! I've been looking forward to seeing some development on ar. ^_^ I like what you're doing with the argument lists. Have you found your arc2js progress useful for this? (Part of me wonders how Racket's local (define ...) forms compare against its other options in terms of performance. If they're faster, that could pose the same compilation annoyances as arc2js deals with. ;) )

As far as the name "Nu" goes, it's actually been taken by another lisp (and I'm not even talking about NewLISP :-p ): https://github.com/timburks/nu. Early this year I did a couple of months of Objective-C programming for work, and I GitHub-followed Nu in envy. XD

-----

1 point by Pauan 4707 days ago | link

"I like what you're doing with the argument lists."

You mean the "destructuring is just lambdas" thing?

---

"Have you found your arc2js progress useful for this?"

No, except insofar as writing arc2js taught me some useful compiler things. For the most part, it's been pretty simple and easy, so I don't think arc2js helped much, if at all.

---

"Part of me wonders how Racket's local (define ...) forms compare against its other options in terms of performance."

  (let a nil (%nocompile (racket-define a 5)) a)
  (let a nil (= a 5) a)
  Time: 1500-1600 ms

  (let a nil (%nocompile (racket-let* ((a 5)) a)))
  (let a nil (%nocompile (racket-let ((a 5)) a)))
  (let a nil (let a 5 a))
  (let a nil a)
  Time: 1400-1500 ms
It would seem Racket optimizes lambdas very heavily.

---

"As far as the name "Nu" goes, it's actually been taken by another lisp (and I'm not even talking about NewLISP :-p )"

Darn. I still like the name "Nu" enough that I want to keep it, though.

-----

3 points by rocketnia 4705 days ago | link

It took me a while to reply because I was planning to do some of my own testing before posting, but I haven't found the time.

---

"You mean the "destructuring is just lambdas" thing?"

No, I don't know what to think about that yet. I like it if and only if this works:

  (let (a . b) '(1 . 2)
    ...)
Even if that works, I almost expect Racket to copy a mutable argument list into an immutable one for use inside the lambda, so here's a second test case:

  (withs (foo (list 1 2 3)
          (a . b) foo)
    (= b.0 4)
    (is foo.1 4))
What I definitely do like is the use of Racket's optional arg syntax where possible. ^_^

---

"It would seem Racket optimizes lambdas very heavily."

IMO, those examples aren't normal uses of (define ...), since they're contexts where the variable is already defined. I'd like to test these cases:

  ; Racket code
  
  (let ((a 5)) a)
  ((lambda (a) a) 5)
  (begin (define a 5) a)
    ; NOTE: Make sure this (begin ...) isn't at the top level, or the
    ; definition will be global.
  (let () (define a 5) a)
  ((lambda () (define a 5) a))
One reason it's on my mind is the Racket 5.2 changelog (http://lists.racket-lang.org/users/archive/2011-November/048...):

  * Internal-definition expansion has changed to use `let*' semantics
    for sequences that contain no back references.  This change
    removes a performance penalty for using internal definitions
    instead of `let' in common cases, and it only changes the meaning
    of programs that capture continuations in internal definitions.
    Internal definitions are now considered preferable in style to
    `let'.
Then again, if they're going to make more semantic changes like this one, it might be better to avoid compiling to an internal-definition style. :-p

Anyway, from the phrasing in the changelog, it sounds like internal definitions are implemented in ways the programmer can already control at a lower level with things like 'let*, so you'll probably continue to find that internal definitions are at least as slow as other options.

-----

2 points by Pauan 4705 days ago | link

"I like it if and only if this works:"

That doesn't, but this does:

  (let (a . b) '(1 2 3)
    ...)
The reason is because racket-mlist->list expects a proper list. I can fix that easily.

Update: I fixed racket-mlist->list, but Racket's apply absolutely positively requires a proper list, so it looks like I can't use nested lambdas if I want to fix that: I'll have to use a Racket let*

---

"I almost expect Racket to copy a mutable argument list into an immutable one for use inside the lambda"

It's true, it does copy the list, so that returns nil. I wonder if I can work around that...

---

"One reason it's on my mind is the Racket 5.2 changelog"

I'm not using Racket 5.2. And even if I were, the reason for the speed tests was to see how it would perform within the (fn ...) expansion, where it does indeed overwrite an already-existing variable. (Though I could change it to use a gensym...)

-----

3 points by Pauan 4702 days ago | link

I just added in keyword destructuring:

  > (let (:a :b :c) (obj a 1 b 2 c 3)
      (list a b c))
  (1 2 3)

-----

2 points by akkartik 4701 days ago | link

Here it is in wart:

  mac let(vars f . body) :case (and cons?.vars (keyword? car.vars))
    `(with ,(collect:each k vars
              (yield sym.k)
              (yield:f sym.k))
       ,@body)
It took me several hours to get to this; in the process I uncovered lots of things I'd forgotten to implement :) And bugs :/

http://github.com/akkartik/wart/compare/37d4a2d736...d38d477...

When I try to eval:

  let (:a :b) (table 'a 1 'b 2) (cons a b)
..it cranks for a good second and a half, and conses over a million cells. (It also leaks 24 cells; that's top priority to fix.)

-----

3 points by akkartik 4700 days ago | link

I've modified ssyntax precedence to follow rocketnia's suggestion[1] rather than arc. Now[2] it looks like this:

  mac let(vars f . body) :case (and cons?.vars keyword?:car.vars)
    `(with ,(collect:each k vars
              (yield sym.k)
              (yield f:sym.k))
       ,@body)
[1] http://arclanguage.org/item?id=13974

[2] git clone git://github.com/akkartik/wart.git && git checkout 2007e158c8

-----

1 point by Pauan 4700 days ago | link

"(with hash ...)"

This is very similar to a macro in Arubic-style namespaces:

  (eval-w/ foo ...)
The above evaluates the expression "..." in the "foo" namespace, which may potentially be a table. That's not quite right, though. Here's the correct version:

  (eval-w/ (new-namespace foo namespace)
    ...)
The above could then be easily wrapped in a macro like "w/table", or even overloading the "with" macro as you did.

---

"..it cranks for a good second and a half, and conses over a million cells."

I'm not entirely sure how such a simple expression could cause so much consing. What is wart doing with all those cells?

-----

2 points by akkartik 4700 days ago | link

"What is wart doing with all those cells?"

It's turtles all the way down, but I need to better understand the details.

-----

1 point by akkartik 4702 days ago | link

Interesting. Why not just destructure to (a b c) and call it table destructuring?

-----

3 points by Pauan 4702 days ago | link

Because that would require hash table's keys being sorted, and then the results would vary depending on what keys the hash table has. For instance:

  (let (a b c) (obj d 4 c 3 b 2 a 1)
    (list a b c))
Should that be (3 2 1) or (1 2 3)? Or maybe (4 3 2)? But, that's assuming hash table keys are sorted in the first place, when they aren't.

In addition, that would require a runtime check to determine whether the argument is a hash table or a list. With my method, the check is done at compile-time: no runtime costs.

Also, it works on more than just hash tables. In fact, it works on anything that accepts 1-2 arguments, where the first is a symbol. Thus, if you created a new data type, you could use keyword destructuring on it. Basically, this:

  (let (:a :b :c) (obj a 1 b 2 c 3)
    (list a b c))
Compiles into the equivalent of this:

  (withs (g1 (obj a 1 b 2 c 3)
          a  (g1 'a)
          b  (g1 'b)
          c  (g1 'c))
    (list a b c))
Compare that to the following:

  (let (a b c) (list 1 2 3)
    (list a b c))
Which compiles into the equivalent of this:

  (withs (g1 (list 1 2 3)
          a  (car g1)
          g1 (cdr g1)
          b  (car g1)
          g1 (cdr g1)
          c  (car g1))
    (list a b c))
Interestingly, that means you could create a data type that worked with both kinds of destructuring. For instance, an alist might support the `foo!bar` syntax, and then you would need a way to distinguish between "look this up based on the key" and "look this up based on the index".

As a final note, from a conceptual standpoint, I view keyword arguments as being implemented with an implicit hash table. Thus using keyword syntax to destructure a hash table makes lots of sense to me.

In other words, if these two are equivalent:

  (let (a b c) foo ...) 
  (apply (fn (a b c) ...) foo)
...then I view these two as equivalent as well:

  (let (:a :b :c) foo ...)
  (keyword-apply (fn (:a :b :c) ...) foo)

-----

2 points by akkartik 4702 days ago | link

"I view keyword arguments as being implemented with an implicit hash table. Thus using keyword syntax to destructure a hash table makes lots of sense to me."

Interesting. If you make keyword args pervasive and optional like I do, you need a sense of ordering as well in the table, which weakens that imagery.

"Should that be (3 2 1) or (1 2 3)?"

It should be whatever you define it to be :) Making the symbols keywords didn't actually help me avoid that question at first glance.

"With my method, the check is done at compile-time."

But how does it perform the check for this expression?

  (let (:a :b :c) h
    ..)
Are you using the presence of keywords to disambiguate whether or not to insert the g1 (cdr g1) pairs in the withs?

-----

1 point by Pauan 4702 days ago | link

"Interesting. If you make keyword args pervasive and optional like I do, you need a sense of ordering as well in the table, which weakens that imagery."

Oh I'd love to have pervasive keywords like Python (or wart), but it seems Racket doesn't support that. And I don't plan for Nu to become an interpreter. Keywords are optional in Nu, though (or will be, once I've implemented them).

---

"Making the symbols keywords didn't actually help me avoid that question at first glance."

I just like how there's a clean (and visible) separation between "lookup by key" and "lookup by index". It also avoids a runtime check as well, which is nice.

---

"Are you using the presence of keywords to disambiguate whether or not to insert the g1 (cdr g1) pairs in the withs?"

Yes. The code the compiler outputs depends on whether the argument is a keyword or not.

-----

1 point by akkartik 4702 days ago | link

I was playing around with the idea and thought of an alternative syntax:

  (with hash
     ..)
I even came up with a kinda icky implementation in wart:

  mac with(vars . body) :case (~cons? vars)
    `(with ,(collect:each (k v) eval.vars
              yield.k
              yield.v)
       ,@body)
Pros: it's shorter to use because I don't need to repeat the keys.

Cons: it only works with syms. You can't eval an expr that returns a hash. It replaces the runtime check you're concerned about with that super ugly eval-inside-unquote. You can't import just a few of the keys so it feels a little like 'using namespace std' in C++ -- you may not know what vars you're going to get, and you may end up overriding bindings.

Perhaps I should just make it a new term. That would address the first limitation:

  (w/bindings hash
    ..)
Meh, I'm not sure how I feel about this.

-----

1 point by rocketnia 4701 days ago | link

You're not going to be able to use (with hash ...) with a local variable 'hash unless you have at least one of these:

- Fexprs. An fexpr implementation of 'with can use the complete value of 'hash as it determines how to treat the unparsed body.

- Static typing with record types, so that a macro can use the type of 'hash as it determines how to treat the unparsed body.

- Some variant of JavaScript-style scope chain semantics, in the sense that a bare variable reference means (or can mean) a field lookup in general. IMO, this would be the most straightforward to add to an Arc-3.1-like compiler, since it's a matter of compiling foo to (scope 'foo), (with foo ...) to (let ((scope (shadow (scope 'foo) scope))) ...), and other scope-related things in their own analogous ways.

- Something else I haven't thought of. :)

-----

1 point by rocketnia 4701 days ago | link

"Oh I'd love to have pervasive keywords like Python (or wart), but it seems Racket doesn't support that."

Racket has pervasive keyword arguments though, right? What's in your way?

-----

2 points by Pauan 4701 days ago | link

I assumed akkartik was talking about this:

  (def foo (a b)
    (list a b))

  (foo 1 2)       -> (1 2)
  (foo :b 2 :a 1) -> (1 2)
That doesn't work in Racket. You need to explicitly say that the arguments are keywords:

  (def foo (:a :b)
    (list a b))
In other words, arguments are either positional or keyword-based, but not both at the same time. This is different from Python, which lets you treat an argument as either one:

  def foo(a, b):
      return [a, b]

  foo(1, 2)     -> [1, 2]
  foo(b=2, a=1) -> [1, 2]

-----

1 point by Pauan 4707 days ago | link

Nu should be feature-complete, which means it implements everything that Arc 3.1 does.

However, seeing as how I've only spent the past 4-5 days working on it, obviously it may have bugs, might be slow, and is missing some new additions I want to add to it.

The "Details" section of the README should cover the stuff you want to know, but in case it doesn't, feel free to ask me.

---

Thank you to awwx for writing ar, which was both an inspiration and the physical basis for large parts of Nu.

-----

2 points by akkartik 4707 days ago | link

Interesting approach to implement destructured args!

-----

1 point by Pauan 4707 days ago | link

Okay, we need to get it together, guys. Arc 3.1 is kicking our butts:

  (time (repeat 1000000 (+ 1 2)))

  Nu time:      1523 msec.
  ar time:      1173 msec.
  Arc 3.1 time:  745 msec. (direct-calls #f)
  Arc 3.1 time:  155 msec. (direct-calls #t)
What's strange is that I even optimized this so that Nu outputs the exact same code as Arc 3.1. Makes me wonder where the bottleneck is.

-----

2 points by Pauan 4706 days ago | link

I figured out why `(+ 1 2)` is so slow in Nu. It was calling Arc's `or` which is pretty slow (compared to not calling it, anyways). So here's the new numbers, this time using the time library[1]:

  (timeit (+ 1 2))

  ar      time: 11.904  gc: 0.404  mem: -19311.072
  Nu      time:  9.282  gc: 0.0    mem:     88.416  (direct-calls #f)
  Arc 3.1 time:  7.86   gc: 0.39   mem:  -9687.53   (direct-calls #f)
  Nu      time:  5.282  gc: 0.0    mem:     88.8    (direct-calls #t)
  Arc 3.1 time:  1.55   gc: 0.0    mem:     89.95   (direct-calls #t)
As you can see, Nu is now faster than ar, but still slower than Arc 3.1. On the other hand, Nu's + is implemented in terms of case-lambda, so it should be better on memory in the case of only 2 arguments.

---

* [1]: https://github.com/Pauan/ar/blob/nu/lib/time.arc

---

Random side note: zip is awesome.

-----

2 points by akkartik 4706 days ago | link

If it's any consolation:

  wart> time:repeat 1000 (+ 1 2)
  16050000/1000000
That's 16s for a thousandth of the work.

edit: the consing in wart is seriously out of control. This:

  repeat 1
    (+ 1 2)
conses 12330 new cells.

(There's no leak, so they're all freed by the end.)

-----

2 points by Pauan 4704 days ago | link

I've added in implicit variables and improved the REPL functionality substantially. You can see more details in the README.

-----