Arc Forumnew | comments | leaders | submit | Pauan's commentslogin
1 point by Pauan 5290 days ago | link | parent | on: Arcueid, a C implementation of Arc

"[...] not defining 'dynamic-wind [...]"

'protect is implemented with 'dynamic-wind, so the only functionality we lose is the ability to specify a pre-thunk. Are there any areas where that would be useful?

-----

2 points by rocketnia 5289 days ago | link

Dynamic-wind gives us most of the ability to implement parameters ourselves. We just mutate a box upon every entry and exit of the expression. Unfortunately, it might take some crazy trampolining to get the last expression of (parameterize ...) in tail position. I'm not even sure if tail position is possible....

I think the last missing piece is thread-friendliness. In the face of threads, we'd need the box to be thread-local like Racket's parameters. But my point here is just that the pre-thunk is useful for something. ^_^

-----

1 point by Pauan 5290 days ago | link | parent | on: Arcueid, a C implementation of Arc

"[...] they are essentially a way to do dynamic binding in a language like Arc that normally uses static binding. This is in large part exactly what call-w/stdin does with stdin: it sets up a dynamic binding for that function."

That is correct. In fact, in Arc 3.1, std{in,out,err} are Racket parameters[1], and call-w/std{in,out} use Racket's parameterize. My point was merely that it is useful to provide parameters to Arc programmers so they can define their own parameters beyond just stdin/stdout/stderr.

* [1]: That's why you need to use (stdin), (stdout), and (stderr) rather than stdin, stdout, and stderr.

---

"And no, while Arcueid's main goal is to produce a version of Arc compatible with at least Paul Graham's Arc3.1, I am of course not above introducing improvements and extensions, provided that they do not also break compatibility."

Glad to hear it. I would just like to note that any changes whatsoever will break compatibility. For instance, if you provide a "parameterize" form, a library written in Arc might also define a "parameterize" global, etc. My feeling on such things is that there should be a social convention for specifying implementation-specific global variables.

Something like, "if a global variable starts with % it is implementation-defined, so portable Arc libraries shouldn't use or define global variables starting with %".

Then your implementation could provide "%parameterize" to Arc and there would be no problems, because Arc libraries aren't supposed to use variables starting with %, so there's no conflict.

This should be solely a social convention, not enforced by the compiler. I may want to write an Arc library that does use/define implementation-specific globals, while understanding that such a library won't be portable and may break in the future.

-----

1 point by Pauan 5291 days ago | link | parent | on: Arcueid, a C implementation of Arc

"Any plans to generalize 'call-w/stdout and 'call-w/stdin to 'parameterize?"

As a side note to this, I think it would be very preferable to have a `parameterize` form which `call-w/stdin` and `call-w/stdout` would call. It should behave similarly to Racket's parameterize.

This isn't necessary for an implementation of Arc 3.1, but it's very useful in practice: you could provide a way for users to create their own parameters and then call parameterize on them. This is what ar and Nu do, and it's incredibly convenient, especially when you provide a way to make the parameters implicit[1].

It really does depend on your goals, though. Do you intend for this to be just an implementation of Arc 3.1 and nothing more? Or do you intend to provide convenient features that Arc 3.1 doesn't have? Your work on numerical functions seems to suggest that you're not entirely against extending your Arc runtime to do things that Arc 3.1 doesn't.

---

* [1]: By "implicit parameters" I mean parameters that you don't have to call to extract their value. In other words, you can just say `stdin` rather than `(stdin)` for instance.

-----


"ac.scm:1323:9: racket/unsafe/ops: standard-module-name-resolver: collection not found: #<path:racket/unsafe> in any of: (#<path:/home/user/.plt-scheme/4.2.1/collects> #<path:/usr/lib/plt/collects>) in: racket/unsafe/ops"

The solution is simple: use Racket. You see, mzscheme 4 doesn't support any Racket stuff. So when trying to load a Racket library (in this case, "unsafe") it won't be able to find it.

If Anarki wants to support people using mzscheme 4, this line should be changed:

https://github.com/nex3/arc/blob/76a61293bc7bc49ef8b57641ad5...

-----

1 point by darjeeling 5291 days ago | link

The debian package description of racket (http://packages.debian.org/sid/racket) says that the package was previously plt-scheme. So, I seem to be using it :(

which package provides racket in Ubuntu/Debian systems? I am running the October version on a Debian 6 system and it is running fine. apt-cache search racket returns no relevant package.

-----

2 points by rocketnia 5291 days ago | link

You mentioned you used the command "mzscheme -f as.scm". If you change that to "racket -f as.scm", does it make any difference?

-----


"So I suppose it would not do to just expand it into ((compose orig-cc fn) ...), and we have to actually make it a real function composition."

Not so. If you look at line 29 in ac.scm you'll see this:

  ; the next three clauses could be removed without changing semantics
  ; ... except that they work for macros (so prob should do this for
  ; every elt of s, not just the car)
  ((eq? (xcar (xcar s)) 'compose) (ac (decompose (cdar s) (cdr s)) env))
  ((eq? (xcar (xcar s)) 'complement)
   (ac (list 'no (cons (cadar s) (cdr s))) env))
  ((eq? (xcar (xcar s)) 'andf) (ac-andf s env))
For those not familiar with the Arc compiler, what it's doing is basically these transformations:

  ((compose foo bar) 1) -> (foo (bar 1))

  ((complement foo) 1)  -> (no (foo 1))

  ((andf foo bar) 1)    -> (let g1 1 (and (foo g1) (bar g1)))
If you wish for compose, complement, and andf to work on macros and special forms like fn, your compiler will need to do a similar transformation. The catch is that this transformation only works in functional position:

  (map no:do (list 1 2 3 nil nil)) ;; doesn't work
It's all very hacky and whatnot, macros aren't very clean at all in Arc. The other catch is that it hardcodes the symbols 'compose, 'complement, and 'andf, but my Nu compiler fixes that.

-----

3 points by dido 5294 days ago | link

I figured that out and my compiler now does that transformation for compose in:

https://github.com/dido/arcueid/commit/f998de0ba562103cee6a7...

Will see if we can't apply the same for complement and andf in the same way. :)

-----

2 points by Pauan 5330 days ago | link | parent | on: Arc program for generating .xspf playlists

Currently it only runs on Nu, so perhaps it should be called a "Nu program" rather than an "Arc program".

In any case, the README should cover everything you need to know. If it doesn't, let me know.

-----

4 points by Pauan 5330 days ago | link | parent | on: Nu Arc compiler

I implemented the Boyer–Moore–Horspool substring-search algorithm in Arc[1].

I then plugged it into my playlist program, and, well... see for yourself:

  ;; Arc posmatch:          38,236 ms
  ;; Boyer–Moore–Horspool:   6,396 ms
Wow. I think this confirms that the biggest bottleneck in my program was posmatch[2].

Keep in mind, though, Boyer-Moore isn't always faster than posmatch[3]: due to the preprocessing time, long pattern strings will cause the algorithm to perform slower.

This only applies when the string is preprocessed multiple times: if it's only used once, then the overhead should be negligible. And if you want to use the same pattern string multiple times, you can preprocess it once and then reuse it:

  (let x (boyer-moore-process "foobarquxcorgenou")
    (boyer-moore-search x "adadadadadadadadadadadad")
    (boyer-moore-search x "adadadadadadadadadadadad")
    ...
    (boyer-moore-search x "adadadadadadadadadadadad"))
In general, though, Boyer-Moore is faster. Especially for multi-string searches, which is what my playlist program does.

---

* [1]: https://github.com/Pauan/ar/blob/nu/lib/boyer-moore.arc

* [2]: http://arclanguage.org/item?id=14204

* [3]: https://github.com/Pauan/ar/blob/nu/notes/timing

-----

2 points by Pauan 5330 days ago | link

New timing link: https://github.com/Pauan/ar/blob/nu/notes/timing.md

-----

1 point by Pauan 5331 days ago | link | parent | on: Interesting approach to ssyntax

And if you want that behavior in Arc, here's a macro:

  (mac chain (x . body)
    (w/uniq u
      `(let ,u ,x
         ,@(map (fn ((f . args))
                  ;; could use list* but Arc 3.1 doesn't include it
                  `(,f ,u ,@args))
                body)
         ,u)))
And how to use it:

  (chain path
    (move-to 10 10)
    (stroke "red")
    (fill "blue")
    (ellipse 50 50))
Though... adding in syntax sugar for it would certainly be more convenient. Come to think of it, this would be quite convenient for arc2js:

  ;; Basic DOM              
  (chain (div)
    (set-attribute "bar" "qux")
    (append-child (div)))

  ;; jQuery
  (chain ($ "foo")
    (offset 10)
    (add-class "bar")
    (click (fn ...)))
Amusingly enough, this is similar to the JS "with" block, only without all the problems that "with" has.

-----

3 points by Pauan 5338 days ago | link | parent | on: Nu Arc compiler

I just added in keyword args:

  > (def foo (:a :b)
      (list a b))

  > (foo :a 1)
  (1 nil)

  > (foo :a 1 :b 2)
  (1 2)
As a side note, keyword arguments are currently almost twice as slow as normal args:

  > (let foo (fn (:a :b) (list a b))
      (timeit (foo :a 1 :b 2)))
  time: 17.521  gc: 0.408  mem: 2303.568

  > (let foo (fn (a b) (list a b))
      (timeit (foo 1 2)))
  time:  9.515  gc: 0.3    mem: -22736.128
So if you really want performance at any cost... then why are you using Arc in the first place?! :P

---

I also (mostly) added in alist key support:

  > (= foo '((b 2) (a 1) (c 3)))
  ((b 2) (a 1) (c 3))

  > foo!a
  1

  > (= foo!a 10)
  10

  > foo
  ((b 2) (a 10) (c 3))
Which means alists now work with keyword destructuring as well:

  > (let (:a :b :c) '((b 2) (a 1) (c 3))
      (list a b c))
  (1 2 3)

-----

2 points by Pauan 5326 days ago | link

I added in Arubic-style namespace support, and also implemented dynamic and parameter variables better. If you're interested in reading more, check out these two links:

https://github.com/Pauan/ar/blob/nu/notes/dynamic.md

https://github.com/Pauan/ar/blob/nu/notes/namespaces.md

---

I also discovered that it's possible to implement `quote` as a macro:

  (mac quote (x)
    (list (fn () x)))

-----

1 point by Pauan 5325 days ago | link

I finally added in Arubic to Nu. Now all you have to do is `./arc -i arubic` and you'll get a REPL with Arubic functionality. You can also use `(import arubic)` to write a file using Arubic.

Why would you want to do that? Well, here are the current changes between Arc and Arubic, though I plan to add more over time:

  (map x foo ...)  ;; Arubic
  (map [...] foo)  ;; Arc 3.1
Ditto for mappend, some, all, and keep. This is especially convenient when using destructuring:

  (map (x y) foo ...)        ;; Arubic
  (map (fn ((x y)) ...) foo) ;; Arc 3.1
I changed the [] and {} syntax as well:

  [1 2 3]   -> (list 1 2 3)
  {a 1 b 2} -> (obj a 1 b 2)
Because of the changes above, you no longer need the [] syntax for functions most of the time, so I'm repurposing them to create lists, which I've found to be far more common.

`isa` accepts multiple arguments:

  (isa x 'foo 'bar 'qux)      ;; Arubic
  (in type.x 'foo 'bar 'qux)  ;; Arc 3.1
`prn` prints spaces between each argument:

  (prn "foo" "bar" "qux") -> "foo bar qux\n"
You can also use `str` and `str?` as aliases for `string` and `string?`.

Small changes, but they do make writing code more pleasant.

-----

2 points by Pauan 5324 days ago | link

I just removed `setforms` from every place in arc.arc except for `rotate` and `=`[1]. Why? Well, there's two reasons:

1) It truly does not make any sense to me why zap is defined like this:

  (mac zap (op place . args)
    (with (gop    (uniq)
           gargs  (map [uniq] args)
           mix    (afn seqs
                    (if (some no seqs)
                        nil
                        (+ (map car seqs)
                           (apply self (map cdr seqs))))))
      (let (binds val setter) (setforms place)
        `(atwiths ,(+ binds (list gop op) (mix gargs args))
           (,setter (,gop ,val ,@gargs))))))
Rather than this:

  (mac zap (f x . args)
    `(= ,x (,f ,x ,@args)))
2) This helps out a ton with arc2js. In arc2js, now all I need to do is provide an `=` macro and I'll get zap, or=, push, pull, swap, etc. all for free. Without this change, I'd have to define my own custom versions of zap, or=, etc...

---

* [1]: https://github.com/Pauan/ar/commit/e58a46fb47802394031dc4afe...

-----

3 points by rocketnia 5324 days ago | link

"It truly does not make any sense to me why zap is defined like this"

'zap is the only use I typically have for the 'setforms "binds" list (which ensures the subexpressions of 'place are only evaluated once).

Still, 'zap doesn't need to work that way: as long as I'm using a language where I know 'zap evaluates its place twice, I'm pretty much okay with it. It's a wart, but it's not an impediment.

To explore Arc-3.1-like options for a bit, here's a cleanup of Arc 3.1's definition of 'zap:

  (mac zap (op place . args)
    (with (gop                 (uniq)
           gargs               (map [uniq] args)
           (binds val setter)  setforms.place)
      `(atwiths (,@binds ,gop ,op ,@(mappend list gargs args))
         (,setter (,gop ,val ,@gargs)))))
If we allow 'setter and 'val to compile and evaluate before 'op and 'args, it gets shorter:

  (mac zap (op place . args)
    (let (binds val setter) setforms.place
      `(atwiths ,binds
         (,setter (,op ,val ,@args)))))
I prefer to arrange the compilation and evaluation orders from left to right ('op, 'place, 'args), using a technique like this:

  (mac place (place)
    (let (binds val setter) setforms.place
      `(withs ,binds
         (list (fn () ,val) ,setter))))
  
  (mac zap (op place . args)
    `(atomic:fn-zap ,op (place ,place) (list ,@args)))
  
  (def fn-zap (op (getter setter) args)
    (setter:apply op (getter) args))

-----

1 point by Pauan 5324 days ago | link

"'zap is the only use I typically have for the 'setforms "binds" list (which ensures the subexpressions of 'place are only evaluated once)."

Hm... yes, you're right, `(zap + (foo (bar qux)) 1)` evaluates `(bar qux)` twice, and I don't see an easy/obvious way to fix that in `=`. I'll need to think about this.

-----

1 point by Pauan 5324 days ago | link

And I just removed setforms from rotate too[1]. An improvement? You be the judge:

  ;; Arc 3.1
  (mac rotate places
    (with (vars (map [uniq] places)
           forms (map setforms places))
      `(atwiths ,(mappend (fn (g (binds val setter))
                            (+ binds (list g val)))
                          vars
                          forms)
         ,@(map (fn (g (binds val setter))
                  (list setter g))
                (+ (cdr vars) (list (car vars)))
                forms))))

  ;; Nu
  (mac rotate places
    (w/uniq u
      (let shift (join (cdr places) (list u))
        `(let ,u ,(car places)
           (= ,@(mappend list places shift))))))
---

* [1]: https://github.com/Pauan/ar/commit/890209c76356da2993a35ed4b...

-----

1 point by Pauan 5324 days ago | link

And now `=` no longer calls `atomic-invoke` for complex assignments. Based on this information here:

http://docs.racket-lang.org/reference/threads.html

It would appear that the `=` operator is already thread-safe even without `atomic-invoke`. In any case, if you're terribly worried, you can always wrap it yourself.

-----

1 point by Pauan 5324 days ago | link

I just got rid of `setforms` completely[1]: not even `=` uses it anymore. I then reimplemented `=` in a much shorter and clearer way[2].

Arc 3.1 takes 80 lines to implement expand=, but Nu takes only 31. And Nu is much clearer and easier to understand as well. In addition, Nu's output is much shorter and is faster:

  ;; Nu
  > (macex1 '(= foo!bar 5))
  (do (sref foo 5 (quote bar)))

  ;; Arc 3.1
  > (macex1 '(= foo!bar 5))
  (do (atwith (g1 foo g3 (quote bar) g4 5) ((fn (g2) (sref g1 g2 g3)) g4)))
---

* [1]: setforms is included in compat.arc for backwards compatibility with Arc 3.1, but it's not actually used anywhere.

* [2]: https://github.com/Pauan/ar/blob/c835e67d919d7a555a1c856812a...

-----

1 point by rocketnia 5323 days ago | link

I can't find whatever information you're talking about. Would you mind quoting it and/or elaborating?

Don't put too much work into the explanation, 'cause I'm likely to come in at the end and say "but what about X?" :-p It sounds too good to be true.

-----

1 point by Pauan 5323 days ago | link

It's right near the top of the link:

  All constant-time procedures and operations provided by Racket are
  thread-safe because they are atomic. For example, set! assigns to a variable
  as an atomic action with respect to all threads, so that no thread can see a
  “half-assigned” variable. Similarly, vector-set! assigns to a vector
  atomically. The hash-set! procedure is not atomic, but the table is
  protected by a lock; see Hash Tables for more information. Port operations
  are generally not atomic, but they are thread-safe in the sense that a byte
  consumed by one thread from an input port will not be returned also to
  another thread, and procedures like port-commit-peeked and write-bytes-avail
  offer specific concurrency guarantees.
It mentions that hash table assignment is not thread-safe, however if you then go to the hash table page[1], it says this:

  A mutable hash table can be manipulated with hash-ref, hash-set!, and
  hash-remove! concurrently by multiple threads, and the operations are
  protected by a table-specific semaphore as needed. Three caveats apply,
  however [...]
In other words, Racket already handles everything, according to the docs. If you're ever worried enough, or run into any problems, it's not hard to wrap it in `atomic` yourself. I'd rather not have the cost of `atomic-invoke` for every assignment, especially if you run all your code in one thread (like I do).

By the way, Nu doesn't use `set!` for global assignment, so I'm not sure if global assignment in Nu is thread-safe or not. But I'd assume it is, since I think `namespace-set-variable-value!` is constant-time.

---

* [1]: http://docs.racket-lang.org/reference/hashtables.html

-----

2 points by rocketnia 5323 days ago | link

In pg-Arc, '= on a variable is 'assign without 'atomic. Where 'atomic comes in is when there's a setforms thing to worry about.

And in that case, this...

  (= (car car.foo) (bar))
...turns into something like this:

  (atomic:with (gs1 car.foo gs2 (bar))
    ( (fn (val) (scar gs1 val))
      gs2))
This ensures that car.foo, (bar), and (scar gs1 val) all happen without interference in between. I suspect Racket at most protects those on an individual basis.

That said, I don't care about 'atomic myself. :-p

-----

1 point by Pauan 5323 days ago | link

"In pg-Arc, '= on a variable is 'assign without 'atomic. Where 'atomic comes in is when there's a setforms thing to worry about."

I am aware. It still seems to me that if you're dealing with threads, you should wrap assignment in atomic yourself if you're worried about such things. Code that doesn't deal with threads shouldn't have to use atomic.

Perhaps there should be an `a=` macro that's just like `=` but it calls `atomic`. Hm... I wonder... would it be possible to detect whether code is running in the default thread and if not, automatically wrap it in atomic...? May be more trouble than it's worth, though.

-----

2 points by rocketnia 5323 days ago | link

"May be more trouble than it's worth, though."

That's what I think. Anyone who cares can say (atomic:= ...) or (atomic:zap ...), so I only see a couple of reasons why we'd want to have the 'atomic implicit:

- We want to use it all the time anyway. (I doubt it, but it's hard to tell. I haven't used threads, and therefore I've never bothered to find a way to squeeze utility out of it.)

- There are people who do care, and they'd be better off if the people who didn't care still used 'atomic by accident. (Again, it's hard for me to tell if this is true.)

-----

1 point by Pauan 5324 days ago | link

I just implemented `eachfn` and then changed Arc so the `each` macro just calls `eachfn`. This had a significant speedup, with no cost at all in functionality. At this rate, Nu will end up being just as fast as Arc 3.1, with all the shiny extra features and bug fixes.

So, the rule of thumb is: don't write macros that do a lot of work. Instead, write a function that does the work, and then write a macro that just calls the function. This goes double for macros like `each` that do a type-check at runtime.

-----

1 point by akkartik 5324 days ago | link

  (map x foo ...)  ;; Arubic
  (map [...] foo)  ;; Arc 3.1
I don't follow. Can you provide an example?

-----

1 point by Pauan 5324 days ago | link

Basically, map, mappend, some, etc. behave like the "each" macro:

  (= foo '(1 2 3 4 5))

  ;; Arubic
  (map x foo (prn x)) -> (1 2 3 4 5)

  ;; Arc 3.1
  (map [prn _] foo)   -> (1 2 3 4 5)

-----

1 point by akkartik 5324 days ago | link

Hmm. Can x also be a function?

-----

1 point by Pauan 5324 days ago | link

No.. it can't be. If you want that, you should use mapfn:

  (mapfn x foo)
Which behaves exactly like how map currently behaves in Arc 3.1.

-----

1 point by akkartik 5324 days ago | link

Why not just use each?

map has a pretty consistent semantic over the decades; I wouldn't mess with what it means.

-----

1 point by Pauan 5324 days ago | link

...because each does not map. Nor does it some. Nor does it mappend. Nor does it all. Nor does it keep. Nor does it rem.

I don't think you understand what I'm talking about. In Arubic, map is just a macro that expands into mapfn. The semantics are exactly the same as Arc 3.1, it's just easier to say...

  (map x foo ...)
...rather than:

  (map (fn (x) ...) foo)
The only point of it is to remove the (fn ...) bit, resulting in shorter and easier to read code.

-----

1 point by akkartik 5324 days ago | link

...because each does not map. Nor does it some. Nor does it mappend. Nor does it all. Nor does it keep. Nor does it rem.

Sorry, that makes no sense to me.

Focus on just map. Are you saying it doesn't return a transformed list? Ok. But can we call it something different? You'll have to pry

  (map car xs)
from my cold dead hands :)

-----

1 point by Pauan 5324 days ago | link

"Focus on just map. Are you saying it doesn't return a transformed list?"

No. You are still misunderstanding me. I will be as clear and precise as I possibly can. In Arc 3.1, you call `(map x foo)` where `x` is a function and `foo` is a list.

In Arubic, you would say that as `(mapfn x foo)`. It's exactly the same, except you use the name `mapfn` rather than `map`. Now, Arubic also provides a macro called `map`, which expands into `mapfn`. In other words:

  (mac map (var x . body)
    `(mapfn (fn (,var) ,@body) ,x))
The sole purpose of this macro is so that you don't need to type out the `(fn ...)` part in `(mapfn (fn (x) ...) foo)`. That's it. The meaning of `map` is the same, it's just that now that it's a macro, it's a lot more convenient to use. But you can still pass in a function directly by using `mapfn`.

---

"You'll have to pry (map car xs) from my cold dead hands :)"

If you dislike any parts of Arubic (whether misunderstood or not), simply do not use those parts. The point of the namespace system is that you can change what you want and use what you want, rather than being tied down to one semantic/syntax/meaning/language. Thus, it is possible (and relatively easy) to use some of the things from Arubic that you like, while not using the parts you dislike.

In this particular case, however, you are misunderstanding me. Your example would be `(mapfn car foo)` in Arubic.

-----

1 point by akkartik 5324 days ago | link

In this particular case, however, you are misunderstanding me. Your example would be `(mapfn car foo)` in Arubic.

No, I'm now pretty sure I'm not misunderstanding you.

My entire objection is to the name switch. Of course I know I can change the default. It's not a sensible default, it's like defining 1 to be 0.99. You're fragmenting the semantics of a name that has thus far had a pretty clear meaning. You're welcome to do this, but it's going to hinder users with any programming background.

I don't understand why the thing you want must be called map. Why not mapmc, for example? Or some other 3-letter name?

-----

1 point by Pauan 5324 days ago | link

"My entire objection is to the name switch. Of course I know I can change the default. It's not a sensible default, it's like defining 1 to be 0.99."

It's not the default. The default is Arc 3.1. You have to opt-into Arubic. In any case, how is it not sensible? It provides the shortest, easiest to read, and fastest code.

---

"You're fragmenting the semantics of a name that has thus far had a pretty clear meaning."

And this is why I think you're still misunderstanding me. The meaning of map is the same. It's just that now you can write (map x foo ...) rather than (map (fn (x) ...) foo), which is shorter and easier to read.

If you're worried that it'll mess up Arc programmers who are used to the naming of map, then as already said, Arc 3.1 is the default. You only get the new syntax if you import Arubic.

I do not see how changing the meaning of names in a new language is a bad thing. That's like me saying that wart should use exactly the same names as Arc, which is absurd: they're different languages. If you want to use a different name for something in wart, then go for it.

Perhaps you still do not grasp that Arubic is a different language from Arc, implemented in a separate namespace so as not to mess up Arc's namespace.

---

"I don't understand why the thing you want must be called map. Why not mapmc, for example? Or some other 3-letter name?"

....

I'm done arguing. Goodbye.

-----

1 point by akkartik 5324 days ago | link

"I'm done arguing. Goodbye."

Suit yourself. I don't understand what I have said to merit hostility.

I know we can all do what we want. I also am aware of lots of things I can try if I don't like something you do. I don't understand these defensive reactions. Was I rude somewhere?

All this while you keep insisting you know what I am misunderstanding. Is that not rude?

Arubic's map is a macro, not a function. Until it grows fexprs the meaning is by definition not the same.

"This is how I like it" is a perfectly valid reason (and would have ended this conversation eons ago). Just don't insist it's all still the same when it isn't.

-----

1 point by akkartik 5324 days ago | link

I do not see how changing the meaning of names in a new language is a bad thing.

That's an equally good response to any feedback you get.

Do you seriously think changing the meaning of 42 is a 'good' thing?

I don't even feel that strongly about this conversation. It's only gone on so long because:

a) You kept insisting I am too dense to comprehend your subtle creations.

b) You kept insisting you're changing nothing.

-----

1 point by akkartik 5324 days ago | link

It's not the default. The default is Arc 3.1.

No, the default is racket. No, it's machine code. Or is it cosmic background radiation? WTF, are you seriously saying that creating a new language isn't an act of choosing defaults?

-----

2 points by rocketnia 5323 days ago | link

In Nu, the default namespace is the one corresponding closely with Arc 3.1. And in this namespace, the default definition of 'map is pretty much Arc 3.1's. Therefore, "it's not the default" is true in Nu.

Another namespace in Nu is Arubic. Using this namespace is like using a language other than Arc 3.1, where 'map is a macro. If your problem is with Arubic itself, you're free to ignore the parts of Arubic you don't like (even if you want to use an library written in full Arubic). If the point is you think people shouldn't dilute the overall meaning of "map" with the meaning Pauan gives it in Arubic, that's a moral position, with Pauan as an unintentional villain!

---

What I think is that the meaning of "map" isn't tainted here, at least not any more than usual. In fact, I wanted 'map to be an 'each -like macro before I knew Pauan had the same thing in mind.

If I don't know what language we're talking about and I hear "map," I assume it'll be an abstraction that applies a given transformation to elements of a given data structure (whatever "abstraction," "transformation," "data structure," and "given" mean), along with some accidental complexity suitable to the language, such as the timing of computation, side effects permitted in the transformation, or late-bound dependencies (like 'map calling out to 'cons, such that rebinding 'cons changes what happens).

In a language where macros like 'after and 'each are more convenient to use than higher-order functions like 'protect and Anarki's 'trav, a macro for map is to be expected.

In Arc, almkglor[1] calls this macro "mapeach" in Anarki, and I call it "maplet" in Lathe. Pauan and I call it "map" as long as we don't have naming conflicts to worry about. As long as this macro is the primary way we use the map concept, using that name keeps our programs brief and frank.

[1] I'm not sure about this credit, but I found it at https://github.com/nex3/arc/blame/arc2.master/arc.arc and verified it with a Web search.

-----

2 points by rocketnia 5323 days ago | link

Whoops, Anarki's 'trav is a macro. I meant 'walk. ^_^;

-----

1 point by Pauan 5325 days ago | link

By the way, just to show how easy it is to use the namespace system, I'll demonstrate how it's possible to load and use an Arubic library from Arc.

First off, Arubic is implemented as a namespace on top of Arc's namespace, so assuming the Arubic library is called "foo.arc", when you do this:

  > (import foo)
  nil

  > namespace
  #<namespace (len 2)>
Notice that the namespace is now length 2, because it inherits from Arc's namespace. But you don't want to use Arubic, you want to use Arc! But because of the malleability and inspectability of Nu namespaces, it's easy to fix that. Just do this:

  > (zap namespace-fn namespace rev)
  #<namespace (len 2)>
The above just reversed the order of the namespace, so now the Arc namespace inherits from the Arubic namespace, rather than vice versa.

This effectively "undoes" the changes that Arubic made, which is exactly what you want: now you can use the functions and macros defined in the Arubic library from within Arc, and everything works perfectly.

-----

1 point by Pauan 5325 days ago | link

I just did some timing tests. As I suspected, the namespace system is quite costly in terms of performance:

  1   2      3    4    6    6    7
  0%  1732%  31%  22%  20%  17%  12%
What the above means is that when going from 1 namespace to 2, you lose 1732% of speed, meaning your code will now run ~17 times slower. But going from 2 namespaces to 3 has only a 31% drop in speed, and 3 to 4 only has 22%, etc.

What this means is that the initial hurdle of namespaces is quite large, but adding more namespaces after the 2nd is quite decent in terms of performance.

The reason for this is (ironically) speed optimizations. When executing code in the Arc namespace, I actually completely bypass the namespace system entirely, so it's relying on Racket to do global lookups, and Racket is far faster at looking up global variables than Nu is.

That's why going from 1 to 2 namespaces is so slow: Racket is so fast that when it switches to the Nu code, Nu looks horribly slow by comparison. But when going from 2 to 3 namespaces, we're already using the Nu code, so the difference in speed is tiny.

In other words, Racket is so fast it makes Nu look bad.

---

By the way, this only applies to code that is evaluated in a Nu namespace: if you evaluate everything in Arc's namespace, there's no cost in speed at all. So this drop in speed only applies while using things like Arubic, and not Arc itself.

-----

1 point by Pauan 5325 days ago | link

I just added in an interesting feature: automatic namespace creation.

When using the `(import ...)` form, if the file tries to overwrite an already-existing variable, it will automatically create a new namespace. Otherwise, it won't.

This is mostly a performance optimization, since it's entirely possible to load every single file into a different namespace, but using this technique I avoid creating namespaces except when absolutely necessary, while still providing a reasonable amount of library isolation.

-----

2 points by akkartik 5326 days ago | link

I enjoyed the writeups!

I'm still anti-namespaces -- I'd rather work on codebases that don't need them, and I'd rather just rely on tests to catch collisions. But your article makes an eloquent case for them that has had more effect than anything else I've read.

-----

1 point by Pauan 5325 days ago | link

I would like to note something. If all you want to do is write programs and get stuff done then namespaces are indeed a luxury. It is not particularly difficult to use a language (including Arc) to write useful programs, despite the lack of namespaces.

Where namespaces start to become necessary is when you want to change the language itself beyond what can be easily done by adding macros. Then you are in the predicament that you want to write your code in the new (and presumably better) language that you created, but now you have to give up libraries written in the old language.

With namespaces, it's no longer an either/or thing: you can have your cake and eat it too (at the cost of some performance). So, for writing code, namespaces aren't important. But for language experimentation, namespaces are so incredibly useful that I consider them necessary.

The only reason I put namespace support into Nu's core is because I really want to use Arubic, but doing so practically requires decent namespace support. I invented this system out of necessity, which should (hopefully) demonstrate its usefulness.

-----

1 point by akkartik 5325 days ago | link

That's a good point. Arc permits lifting scheme primitives. Would it be hard for Nu to permit lifting arc primitives?

-----

2 points by Pauan 5325 days ago | link

I'm going to go back and re-answer this question:

"Would it be hard for Nu to permit lifting arc primitives?"

Not at all. Arc lifting happens automatically and transparently with namespace inheritance. It works right now. In fact, that's the entire point of the namespace system.

And if you ever want to directly evaluate something in Arc (bypassing the inheritance system), you can call `(w/arc3 ...)`

-----

2 points by akkartik 5325 days ago | link

Right, makes sense.

In going from racket to arc, nothing is passed across by default, and in going from arc to Nu[1] everything is available by default.

[1] just like in going from common lisp to wart :)

-----

1 point by Pauan 5325 days ago | link

What do you mean by "lifting Scheme primitives"?

-----

2 points by akkartik 5325 days ago | link

I mean the $ primitive:

http://github.com/nex3/arc/blob/6132d33e09/ac.scm#L49

Example of its use:

http://github.com/nex3/arc/blob/6132d33e09/lib/re.arc

(The regexp- functions are from racket.)

-----

1 point by Pauan 5325 days ago | link

Okay, I'm still not sure what you mean by "lifting Arc primitives" but I'll do my best to explain how Nu works.

The compiler itself is written in Arc's namespace. There is no separation at all between Arc, the Arc compiler, and Racket: everything is cobbled into one giant namespace. So changing the compiler is as simple as writing an Arc function/macro. This is how ar works, and I liked it enough that I copied the idea for Nu.

That also means no need for the $ primitive because all Racket stuff is prefixed with `racket-`. Thus you would say `racket-regexp-match` rather than `$.regexp-match`.

Note: it's possible to import Racket modules with any prefix you like (including no prefix), but it's there mostly to avoid namespace collisions with Arc stuff.

---

As for doing the same thing with Arc primitives, I assume you mean that if you create a new namespace, you might sometimes want to use the definitions in the new namespace, and sometimes the definitions in Arc's namespace.

You can do that too, by calling eval-w/ with arc3-namespace:

  (eval-w/ arc3-namespace
    foo)
Because it's common enough to want to load things into Arc's namespace, the above can also be expressed as w/arc3:

  (w/arc3 foo)
That will give you the definition of "foo" from Arc's namespace. This shouldn't be necessary most of the time because it's possible to have a namespace inherit from another. In that case, if the variable isn't found, it'll search in the parent.

Thus, if you make a new namespace that inherits from Arc's namespace, you'll automatically get all the Arc stuff for free. That's how Arubic works: it makes a namespace that inherits from Arc, and then changes things. Anything that Arubic doesn't change is automatically taken from Arc.

-----

1 point by akkartik 5325 days ago | link

Ok perhaps your namespaces are the simplest way to 'lift' arc into Nu.

If you implement language A in language B, A is above B in the stack. It's meaningless to call A's code from B, but useful to call B from A. The latter is what I call lifting. But I realize this terminology doesn't work for anything but simple static compilation.

-----

2 points by Pauan 5325 days ago | link

"It's meaningless to call A's code from B"

If I wrote an awesome library in Arubic, especially a library that would be difficult to port to Arc, and you wanted to use the library, but didn't want to use Arubic, it would be very useful to write Arc code that used said Arubic library.

As another example, you may want to write a module in Racket that imports a library written in Arc. Even though Arc is implemented in Racket, I don't see how that counts as "meaningless". This example isn't even theoretical, there was somebody on this forum earlier who wanted to do exactly that:

http://arclanguage.org/item?id=14608

---

"but useful to call B from A."

That's handled by the w/arc3 form, which as already said is not needed most of the time, due to inheritance.

-----

3 points by Pauan 5339 days ago | link | parent | on: Nu Arc compiler

I just added in keyword destructuring:

  > (let (:a :b :c) (obj a 1 b 2 c 3)
      (list a b c))
  (1 2 3)

-----

2 points by akkartik 5338 days ago | link

Here it is in wart:

  mac let(vars f . body) :case (and cons?.vars (keyword? car.vars))
    `(with ,(collect:each k vars
              (yield sym.k)
              (yield:f sym.k))
       ,@body)
It took me several hours to get to this; in the process I uncovered lots of things I'd forgotten to implement :) And bugs :/

http://github.com/akkartik/wart/compare/37d4a2d736...d38d477...

When I try to eval:

  let (:a :b) (table 'a 1 'b 2) (cons a b)
..it cranks for a good second and a half, and conses over a million cells. (It also leaks 24 cells; that's top priority to fix.)

-----

3 points by akkartik 5337 days ago | link

I've modified ssyntax precedence to follow rocketnia's suggestion[1] rather than arc. Now[2] it looks like this:

  mac let(vars f . body) :case (and cons?.vars keyword?:car.vars)
    `(with ,(collect:each k vars
              (yield sym.k)
              (yield f:sym.k))
       ,@body)
[1] http://arclanguage.org/item?id=13974

[2] git clone git://github.com/akkartik/wart.git && git checkout 2007e158c8

-----

1 point by Pauan 5336 days ago | link

"(with hash ...)"

This is very similar to a macro in Arubic-style namespaces:

  (eval-w/ foo ...)
The above evaluates the expression "..." in the "foo" namespace, which may potentially be a table. That's not quite right, though. Here's the correct version:

  (eval-w/ (new-namespace foo namespace)
    ...)
The above could then be easily wrapped in a macro like "w/table", or even overloading the "with" macro as you did.

---

"..it cranks for a good second and a half, and conses over a million cells."

I'm not entirely sure how such a simple expression could cause so much consing. What is wart doing with all those cells?

-----

2 points by akkartik 5336 days ago | link

"What is wart doing with all those cells?"

It's turtles all the way down, but I need to better understand the details.

-----

1 point by akkartik 5339 days ago | link

Interesting. Why not just destructure to (a b c) and call it table destructuring?

-----

3 points by Pauan 5338 days ago | link

Because that would require hash table's keys being sorted, and then the results would vary depending on what keys the hash table has. For instance:

  (let (a b c) (obj d 4 c 3 b 2 a 1)
    (list a b c))
Should that be (3 2 1) or (1 2 3)? Or maybe (4 3 2)? But, that's assuming hash table keys are sorted in the first place, when they aren't.

In addition, that would require a runtime check to determine whether the argument is a hash table or a list. With my method, the check is done at compile-time: no runtime costs.

Also, it works on more than just hash tables. In fact, it works on anything that accepts 1-2 arguments, where the first is a symbol. Thus, if you created a new data type, you could use keyword destructuring on it. Basically, this:

  (let (:a :b :c) (obj a 1 b 2 c 3)
    (list a b c))
Compiles into the equivalent of this:

  (withs (g1 (obj a 1 b 2 c 3)
          a  (g1 'a)
          b  (g1 'b)
          c  (g1 'c))
    (list a b c))
Compare that to the following:

  (let (a b c) (list 1 2 3)
    (list a b c))
Which compiles into the equivalent of this:

  (withs (g1 (list 1 2 3)
          a  (car g1)
          g1 (cdr g1)
          b  (car g1)
          g1 (cdr g1)
          c  (car g1))
    (list a b c))
Interestingly, that means you could create a data type that worked with both kinds of destructuring. For instance, an alist might support the `foo!bar` syntax, and then you would need a way to distinguish between "look this up based on the key" and "look this up based on the index".

As a final note, from a conceptual standpoint, I view keyword arguments as being implemented with an implicit hash table. Thus using keyword syntax to destructure a hash table makes lots of sense to me.

In other words, if these two are equivalent:

  (let (a b c) foo ...) 
  (apply (fn (a b c) ...) foo)
...then I view these two as equivalent as well:

  (let (:a :b :c) foo ...)
  (keyword-apply (fn (:a :b :c) ...) foo)

-----

2 points by akkartik 5338 days ago | link

"I view keyword arguments as being implemented with an implicit hash table. Thus using keyword syntax to destructure a hash table makes lots of sense to me."

Interesting. If you make keyword args pervasive and optional like I do, you need a sense of ordering as well in the table, which weakens that imagery.

"Should that be (3 2 1) or (1 2 3)?"

It should be whatever you define it to be :) Making the symbols keywords didn't actually help me avoid that question at first glance.

"With my method, the check is done at compile-time."

But how does it perform the check for this expression?

  (let (:a :b :c) h
    ..)
Are you using the presence of keywords to disambiguate whether or not to insert the g1 (cdr g1) pairs in the withs?

-----

1 point by Pauan 5338 days ago | link

"Interesting. If you make keyword args pervasive and optional like I do, you need a sense of ordering as well in the table, which weakens that imagery."

Oh I'd love to have pervasive keywords like Python (or wart), but it seems Racket doesn't support that. And I don't plan for Nu to become an interpreter. Keywords are optional in Nu, though (or will be, once I've implemented them).

---

"Making the symbols keywords didn't actually help me avoid that question at first glance."

I just like how there's a clean (and visible) separation between "lookup by key" and "lookup by index". It also avoids a runtime check as well, which is nice.

---

"Are you using the presence of keywords to disambiguate whether or not to insert the g1 (cdr g1) pairs in the withs?"

Yes. The code the compiler outputs depends on whether the argument is a keyword or not.

-----

1 point by akkartik 5338 days ago | link

I was playing around with the idea and thought of an alternative syntax:

  (with hash
     ..)
I even came up with a kinda icky implementation in wart:

  mac with(vars . body) :case (~cons? vars)
    `(with ,(collect:each (k v) eval.vars
              yield.k
              yield.v)
       ,@body)
Pros: it's shorter to use because I don't need to repeat the keys.

Cons: it only works with syms. You can't eval an expr that returns a hash. It replaces the runtime check you're concerned about with that super ugly eval-inside-unquote. You can't import just a few of the keys so it feels a little like 'using namespace std' in C++ -- you may not know what vars you're going to get, and you may end up overriding bindings.

Perhaps I should just make it a new term. That would address the first limitation:

  (w/bindings hash
    ..)
Meh, I'm not sure how I feel about this.

-----

1 point by rocketnia 5338 days ago | link

You're not going to be able to use (with hash ...) with a local variable 'hash unless you have at least one of these:

- Fexprs. An fexpr implementation of 'with can use the complete value of 'hash as it determines how to treat the unparsed body.

- Static typing with record types, so that a macro can use the type of 'hash as it determines how to treat the unparsed body.

- Some variant of JavaScript-style scope chain semantics, in the sense that a bare variable reference means (or can mean) a field lookup in general. IMO, this would be the most straightforward to add to an Arc-3.1-like compiler, since it's a matter of compiling foo to (scope 'foo), (with foo ...) to (let ((scope (shadow (scope 'foo) scope))) ...), and other scope-related things in their own analogous ways.

- Something else I haven't thought of. :)

-----

1 point by rocketnia 5338 days ago | link

"Oh I'd love to have pervasive keywords like Python (or wart), but it seems Racket doesn't support that."

Racket has pervasive keyword arguments though, right? What's in your way?

-----

2 points by Pauan 5338 days ago | link

I assumed akkartik was talking about this:

  (def foo (a b)
    (list a b))

  (foo 1 2)       -> (1 2)
  (foo :b 2 :a 1) -> (1 2)
That doesn't work in Racket. You need to explicitly say that the arguments are keywords:

  (def foo (:a :b)
    (list a b))
In other words, arguments are either positional or keyword-based, but not both at the same time. This is different from Python, which lets you treat an argument as either one:

  def foo(a, b):
      return [a, b]

  foo(1, 2)     -> [1, 2]
  foo(b=2, a=1) -> [1, 2]

-----

More