(perform + * :over q.q 2.p.q)

  mac perform(op1 op2 . list-of-args/over)
    `(,op1 ,@(map (fn(args) `(,op2 ,@flatten.args))
                  list-of-args))

Meh. The prefix version reads fine, to me. I think infix notation is too much of a sacred cow ("but math!"). People bend over backwards to try to selectively write infix notation in both prefix & postfix languages, but not much ever seems to get saved in the process. Elegant parsing algorithms are typically general, so why not stick with them? Prefix math isn't that bad.

At the same time, I'm not sure that perform is general enough beyond this problem. Wouldn't it be more general to have syntax for infix expressions (prefix be damned)? You could use a macro for rudimentary infix parsing, e.g.,

  (= priority* (obj + 1 - 1 * 2 / 2 expt 3))

  (def operand (tok) (~priority* tok))

  (def priority>= (a b)
    (>= (priority* a) (priority* b)))

  (mac infix toks
    (let (ops prefix) nil
      (each tok toks
        (if (operand tok)
            (push tok prefix)
            (do (while (and ops (priority>= (car ops) tok))
                  (push (list (pop ops) (pop (cdr prefix)) (pop prefix))
                        prefix))
                (push tok ops))))
      (while ops
        (push (list (pop ops) (pop (cdr prefix)) (pop prefix))
              prefix))
      (pop prefix)))

  (def zero (n) (is n 0))

  (def sq (n) (* n n))

  (def fib-iter (a b p q n)
    (if (zero n) b
        (even n) (fib-iter a
                           b
                           (infix sq.p + sq.q)
                           (infix sq.q + 2 * p * q)
                           (/ n 2))
                 (fib-iter (infix b * q + a * q + a * p)
                           (infix b * p + a * q)
                           p
                           q
                           (- n 1))))

What does leap out at me, though, is the if where each clause is a function repeatedly applied to the same argument. I know I've seen versions around on the forum before with different names, but it's essentially just a variant of case:

  (mac where (expr . args)
    (withs (var (uniq)
            ex  (afn (args)
                  (if (no (cdr args)) 
                      (car args)
                      `(if (,(car args) ,var)
                           ,(cadr args)
                           ,(self (cddr args))))))
      `(let ,var ,expr ,(ex args))))

  (def fib-iter (a b p q n)
    (where n
      zero b
      even (fib-iter a
                     b
                     (infix sq.p + sq.q)
                     (infix sq.q + 2 * p * q)
                     (/ n 2))
           (fib-iter (infix b * q + a * q + a * p)
                     (infix b * p + a * q)
                     p
                     q
                     (- n 1))))

Also, I think you underestimate how generally useful perform can be :) What if I called it poly for polynomial? I'll keep an eye out for more examples. Adding precedence doesn't change how uniform complex expressions have to look if you can't group them. It isn't as big a deal here, but I think more complex expressions with the infix macro would get less readable. What I really want is to use juxtaposition to indicate higher precedence:

  (+ b.p a.q a.p)

Whoa, whacky idea alert. What if we use whitespace to indicate operator precedence?

  (fib-iter {b * q   +   a * q   +   a * p}
            {b * p   +   a * q}
            p
            q
            {n - 1})

Ha! File that under "never would've thought of it". Still think prefix does just fine, but I like that idea for its quirkiness.

Also, I think you underestimate how generally useful perform can be :)

Superficially contemplating perform (namely the fact that it's a macro) made me think of one thing I wish I had while writing infix. pop is a macro, so I couldn't pass it to map, proper. What I needed was a macro to map macros:

  ; Goofy-looking name, but bear with me
  (mac macmap (m . exprs)
    `(list ,@(map [list m _] exprs)))

  (push (list (pop ops) (pop (cdr prefix)) (pop prefix)) prefix)
  

  (push (macmap pop ops (cdr prefix) prefix) prefix)

  (sum [apply * _] (list (q q) (2 p q)))

Then I actually double-checked your original definition of perform and realized (macmap pop ...) = (perform list pop ...). So I've found my own example! But I think we might could do one better.

What if I called it poly for polynomial?

poly is too specific, if this operator hopes to be general. The operation itself smacks more of distributivity---almost like distributing an operator over cons. Emphasis on "an" operator, not a pair of them (lest it read like "distribute addition over multiplication"). But because (macmap ...) = (perform list ...), maybe it's just a better name for macmap:

  (mac distribute (op . exprs)
    `(list ,@(map [list op _] exprs)))
  
  ; Thus
  (distribute pop ops (cdr prefix) prefix)

  (mac distribute (op . exprs)
    `(list ,@(map [cons op _] exprs))) ; cons vs list!

  ; Thus
  (apply + (distribute * (q q) (2 p q)))

1. I didn't pay attention to your implementation earlier, but this is insane:

  (list (pop ops) (pop (cdr prefix)) (pop prefix))

  arc> (= x '(1 2 3))
  (1 2 3)
  arc> (pop cdr.x)
  2
  arc> x
  (1 3)

Still, way too clever for my taste to rely on arg eval order. I'd rather make the ordering of mutations explicit:

  (withs (b pop.prefix
          a pop.prefix)
    (push (list pop.ops a b) prefix))

2. "Still think prefix does just fine.."

Absolutely. I'm not sure infix is worthwhile, just mildly dissatisfied by how arithmetic expressions in lisp turn out.

3. It turns out David Wheeler has thoroughly thought through the various scenarios for infix precedence: http://sourceforge.net/p/readable/wiki/Precedence. I haven't digested it yet.

4. "poly is too specific."

Yes, I was being lazy. I imagined poly without operator parameters:

  (poly q.q 2.p.q)

That was incredibly useful in following your comment.

Well, birds of a feather...

1. Still, way too clever for my taste

Oh, certainly. It was a result of me golfing an earlier version. :)

6. Can I prevail on you to switch from zero to zero??

I think I first heard of that idea from the Gel syntax: http://www.cs.utexas.edu/~wcook/Drafts/2008/gel.pdf

The discussion at http://c2.com/cgi/wiki?SyntacticallySignificantWhitespaceCon... leads to the stillborn Merd language project from 2001-2003, which calls this "horizontal layout": http://merd.sourceforge.net/#features

---

"Whoa, whacky idea alert. What if we use whitespace to indicate operator precedence?"

The Merd site mentions that case, too: "experimentation is needed to know if [horizontal layout] could work for more than one space" http://merd.sourceforge.net/choices_syntax.html#6

I've doodled this idea independently a couple of times, but I see a few issues:

1. Line breaks throw a monkey wrench in the works (though it might be reasonable to prohibit line breaks within { ... }).

2. In many contexts, it can be tough to count spaces. :)

3. Sometimes it's nice to tabulate similar lines of code using extra spaces, and in those cases this feature would be a minor annoyance, since it would force some subexpressions to bail out to parentheses.

  // Horizontal layout interferes:
  var wargle         =   2 * x + 0.03  * y +  200;
  var tricennarious  =  13 * x + 0.4   * y +   50;
  
  // Fix:
  var wargle         =   2 * x + (0.03  * y) +  200;
  var tricennarious  =  13 * x + (0.4   * y) +   50;

  var wargle           =     2 * x   +   0.03 * y   +   200;
  var tricennarious    =    13 * x   +   0.4  * y   +    50;

---

"The programmers did not handle the exception because the assumption was made that the program was correct until proved at fault, apparently a feature of the programming culture for this system, (this observation is worth an article in itself)."

Unit tests and type checkers are both part of this culture. They prove a program is at fault, and if the program passes, they have nothing further to say.

What's missing is mathematical proof and, for the remaining informal properties, unbiased empirical testing (including empirical testing of the proof checker).

The best way I know to integrate mathematical proof throughout a codebase is total functional programming with dependent typing. Unfortunately, precise dependent types are notable for being inhibitive to code reuse. (But don't take my word for it; I'm just passing along rumor here, lol.)

A straightforward approach to bring reusability to mathematics would be to study the possible faithful translations from one mathematical discipline to another. This field of study already exists: It's category theory.

I've been thinking about this sentence for several days now. You're right, a forum shouldn't be flat. And a forum about code perhaps needs a dynamic, fluid topology.

An HN-style forum like this one can be traversed in a couple of ways. You could start with item?id=1 and continually increment[1]. You can also navigate the stories in reverse order from /newest and thence to clusters of comments belonging to them. Old-style forums like http://forums.linuxmint.com permit a third dimension/ordering for traversal: time of last post.

A dimension has the property that you can start at one well-defined end and travel in one direction and be sure you saw everything. This is a useful property, not because we expect users to do this very often (though some will) but because it hooks into our spatial metaphors for information in an intuitive way. We tend to have a sense of the neighborhood of a post[2], to place a mental bookmark on how far we've read. Above all, they help in browsing in search of a post.

Can we come up with new dimensions for a forum? Sometimes I find myself wishing we could see posts here ordered by the place in the arc codebase they are referring to, suggesting changes to, etc. Anarki we can actually make changes to, so perhaps there's a possible 2-D space of revisions + file + line number. I don't want to go too far off the deep end with the topological metaphors, but there's likely a useful 2D ordering for a forum.

Some dimensions can be more useful than others. File+line number isn't ideal because the ordering of functions is often arbitrary. Just space+time isn't ideal either because the same change may show up in many ways, and because changes close together in time may be utterly unrelated.

I'm not sure what the ideal dimensions are for our forum, but this idea bears experimentation. Imagine a forum where you can define new dimensions as 2 functions: one to convert a new post to a coordinate, and another to render a neighborhood[3] of coordinates. Now anybody could download the posts and sources, and experiment with different ways of browsing them. You could order the functions alphabetically or by types or other categories[4]. Or you could seek out an equivalent of the dewey system[5].

[1] rocketnia, you implicitly used this dimension in http://arclanguage.org/item?id=16675 :)

[2] Sometimes pseudo-neighborhoods can lead us astray; HN comments mentioning another story on the front page have a way of decaying in usefulness over time.

[3] http://en.wikipedia.org/wiki/Neighbourhood_%28topology%29

[4] Compare http://arcfn.com/doc/fnindex.html and http://arcfn.com/doc.

I thought I was making more of an observation than a suggestion. A forum has acyclic relationships that give it structure:

- "___ is a reply to ___" (establishes a forest of stars[1] or shallow trees, depending on whether people can reply to replies)

- "___ was authored after the typically expected period of editing for ___" (establishes a rather dense partial order)

Module systems are sometimes encumbered with the need to resolve cyclic dependencies. On a forum, with its already acyclic structure, it should be possible to avoid that complexity in the common case.

[1] http://en.wikipedia.org/wiki/Star_(graph_theory)

---

"A dimension has the property that you can start at one well-defined end and travel in one direction and be sure you saw everything. This is a useful property, not because we expect users to do this very often (though some will) but because it hooks into our spatial metaphors for information in an intuitive way."

While I see some value in that kind of pursuit, I'd like to even the field of discussion a bit. Humans may live in a low-dimensional world, but they move around, manage a changing environment, and maintain different levels of human-to-human association even within a single population[citation needed]. I think it's plausible that humans evolved to reason about both spaces and networks.

Maybe an interesting scientific-ish approach would be to look at what parts of the brain are involved in forum use. If there isn't enough activity on the right side of the brain, we could work on spatial metaphors. Once there isn't enough activity on the left side, we can go back to language-like metaphors. :-p

---

"Imagine a forum where you can define new dimensions as 2 functions: one to convert a new post to a coordinate, and another to render a neighborhood of coordinates."

I don't think it needs to be formalized that well yet. If the forum software is itself an open source project like Anarki, that should be stable enough for a small, dedicated community to experiment.

I bet once people can browse source code repositories with forum topic annotations on the side, it'll be a while before other features begin to appear necessary. If nothing else, the UI design and module system integration of these annotations would be finicky enough that the developers would have to chew on it for a while before they take on yet another revolutionary change. :-p

For LtU, I did the selection a few days ago, and I didn't pay close attention.

For Arc Forum, I took exactly 10 random numbers from the range 1..16610:

http://arclanguage.org/item?id=12662

http://arclanguage.org/item?id=9559

http://arclanguage.org/item?id=12887

http://arclanguage.org/item?id=1797

http://arclanguage.org/item?id=14375

http://arclanguage.org/item?id=2434

http://arclanguage.org/item?id=7467

http://arclanguage.org/item?id=4338

http://arclanguage.org/item?id=10167

"Would you rather we spin our own cloth, etc.?"

There's multiple ways to answer this:

a) There's a distinction between an end-user's view of the world and a developer/manufacturer's. From that perspective the analogy with spinning cloth might be misleading.

b) You're absolutely right that there always must be some set of abstractions we rely on, and that it can't be turtles all the way down. I'm not claiming everyone needs to fork every library they use. I'm claiming that even if they do so it shouldn't be cause for worry to anyone (besides themselves).

c) There is in fact lots of different ways to spin cloth. I don't think people in the cloth-spinning industry worry that somebody else in the world might use a different way to spin cloth. Think about all the competing kinds of wrenches, on metric vs imperial scales, etc. I think I might have found this disquieting and kinda ugly if I thought about it a few years ago. Now I have no problem with it. The tower of babel seems like a relatively minor issue to me now. If you're manufacturing a car or a crane you just decide what kind of wrench you're going to rely on and keep on cranking. Just avoid mixing multiple wrenches in the same assembly line.

---

"For me there's no distinction between adapting an existing library and creating a new one. Perhaps the distinction is whether you try to move your patch upstream. You're trading some pain now for mitigating merging pains later. Depending on the context it might be worthwhile. However, it is valuable to think about this tradeoff rather than just flinch instinctively away from making changes to any libraries and try to patch them from the outside."

Was Arc/Nu[1] really the first, not the link he referred to (http://matt.might.net/articles/metacircular-evaluation-and-f...), or something else?

We're all stumbling around here, discovering and rediscovering things.

I was merely pointing out that 1) the technique isn't new, and 2) it doesn't require first-class macros, so it's usable in Arc 3.1, etc. Sorry if I gave the impression that Arc/Nu was the first to come up with the idea.

When multiple people independently come up with the same idea, I see that as evidence that the idea is good, not that people are a bunch of thieving stealers. I don't believe in the idea of "stealing an idea", since ideas can only be copied, not stolen. Unfortunately, in the past I've used the word "stealing" to mean "copying". I'll stop doing that.

In any case, I don't think people should be penalized in any way shape or form for copying ideas from others. What matters is whether the idea is good or not, not who came up with it first. At the same time, pointing out that an idea has been invented in the past means that:

1) It can give extra validation to it, based on the assumption that if an idea is independently invented multiple times, it's probably a good idea

2) It suggests further research that you can do, to contrast how these other people tweaked the idea for their own uses. Whereas, if an idea is original, you're completely on your own, without much help.

While developing Nulan, it's been a lot of help for me to research how Arc, Clojure, Racket, etc. handle things like immutability, types, laziness, etc. In that case, having existing ideas to draw from is a plus not a minus.

---

"[...] not the link he referred to [...]"

At a quick glance, that link doesn't seem much like the technique that Kernel or Arc/Nu uses... it's more complicated and seems to be more like rocketnia's latemac: http://arclanguage.org/item?id=14521

---

"We're all stumbling around here, discovering and rediscovering things."

I would just like to point out that apply does work with vaus in Kernel, you just have to wrap them first:

  (apply (wrap $and) ...)

  (apply foo ...)
  (eval (list* (unwrap foo) ...))

This works because "wrap" in Kernel takes a vau and returns a function, and "unwrap" takes a function and returns a vau. So, yes, you're technically right: you can only apply functions in Kernel... but it's so easy to convert back and forth between functions/vaus that I consider this a non-issue.

But I'm going to assume that when you said that Kernel "can't handle non-functions", what you really meant is that Kernel can't automagically figure out your intent... well, yeah. I'm unaware of any macro system that does any better.

My above link actually refers to the entire discussion.

> "..apply does work with vaus in Kernel, you just have to wrap them first.."

You're saying the same thing I am: apply in Kernel requires a function/applicative.

> "if you mean "Kernel can't automagically figure out your intent when applying a vau", well... yeah, duh."

Yes, we are again in agreement, but I find it less incendiary to phrase this as, "apply is not intuitive (in Kernel or other fexpr lisps)".

It's not just about throwing in an extra unwrap. Summarizing for other readers, the entire discussion revolves around two examples:

  (mac foo(a b) `(cons ,a ,b))
  (apply foo '(1 (2 3)))

  (apply = '(x 3))

It's pretty clear that this problem is ill-posed. But we still want apply, for all its misshapen form in our heads. I spent some time trying to do without it, and did not have a good time. In cases like this I believe the first allegiance of a language is to the intuition of the humans using it, not to some abstract platonic ideal.

> "I'm unaware of any macro system that does any better."

I think wart handles many of the easy cases more intuitively (and continues to flag hard cases outside its ken, just like Kernel). My original link makes the case that it is strictly better. I've been using it and haven't run into bugs in a while. But you haven't commented on it yet; I'd love to hear counter-examples and criticism.

Sure, but there's still some vaus that do work. So it's a question of how many vaus work, not whether they work at all. In any case, we both agree, I was just clarifying for the sake of Patient0.

---

"I think wart handles many of the easy cases more intuitively (and continues to flag hard cases outside its ken, just like Kernel). My original link makes the case that it is strictly better. I've been using it for a while and haven't run into bugs. But you haven't commented on it yet; I'd love to hear counter-examples and criticism."

I'm not really fond of hardcoding things. To me, it smells like a limitation in the language. So I'd rather find a way that works well with both vaus and functions. My plan for Nulan is like so:

  ($def qux [1 2 3 4 5])

  ($foo bar @qux) -> ($foo bar (splice? qux)) -> ($foo bar 1 2 3 4 5)

  [1 2 3 @qux] -> (list? 1 2 3 (splice? qux)) -> (list* 1 2 3 qux)

Interesting. I'm going to think about this.

Sure, otherwise all of us here would be using Kernel, right? I probably went a bit overboard in my nitpicking, sorry.

---

This actually brings me to something I wanted to discuss... I've decided to significantly simplify the way types behave in Nulan. Here's how it works:

There's a function called "type" which when called with 0 arguments will return a special function:

  $def foo: type

  $def bar: foo 1 2 3

  $let (foo A B C) bar
    # in here, A is 1, B is 2, and C is 3

And then there's a small twist... you can pass two callable things to "type":

  $def foo; type
    $fn ...
    $fn ...

That's it. That should be enough to implement everything: lists, dictionaries, vau, fn, special data types that have different behavior when splicing on construction or destruction...

Yeah, but in retrospect it was an awful idea. My latest ideas for Nulan are soooo much better than my old ideas. I still think my old idea would have been better than Arc's annotate/type/rep, but...

1) It was too complicated: something that should be in a library, not in the core

2) It used hardcoded strings/symbols for the keys... something I've really tried to avoid lately. My new idea for Nulan has zero hardcoding: it's all done based on values and positions

3) It was a bit too OOP for my tastes... even back then. But at the time it was the best idea I had for extensible data types

---

"rocketnia hates isa too, iirc :)"

Well, I'm not rocketnia, but I think that's because:

1) It uses symbols for types, at least in Arc 3.1

2) It plays poorly with ssyntax, since you can say foo?.x but you have to say (isa x foo)

3) If you want to add new types that behave like existing types, you need to always extend isa (or type), but with proper foo? functions you only need to extend those individual functions

This suggests that having string?, cons?, etc. rather than (isa ... 'string) and (isa ... 'cons) is a good idea.

But I take it a step further in Nulan and also say that annotate/rep are poor because if you want to tag something with multiple types, you need to wrap it repeatedly: (annotate 'foo (annotate 'bar ...))[1] and then you need to unwrap it repeatedly: (rep (rep ...))

This also means that annotate is not transparent: an annotated thing doesn't behave the same as a non-annotated thing... quite a while back waterhouse talked about having the base primitives like car/cdr/etc. automatically call rep: that way you could annotate things and they'd transparently work with existing stuff.

But as rocketnia pointed out, sometimes you don't want that. So there are times where you want something to behave transparently and other times where you don't want that. I think my type system can accomodate that easily.

---

In any case, I've become pretty convinced that Paul Graham was right at least in theory: you only need the theoretical equivalents of annotate/type/rep to implement any type system you want. The problem is that they were implemented poorly in Arc 3.1, not that the idea itself is bad. Nulan's (current) type system is theoretically equivalent to Arc's annotate/type/rep but I believe it's far superior in implementation.

---

You're a pretty good representative of... my past self. ^_^

---

"1) It uses symbols for types, at least in Arc 3.1"

Yep, there's no need to have first-class type tags of any sort when first-class wrapper and unwrapper functions could do that job.

---

"2) It plays poorly with ssyntax, since you can say foo?.x but you have to say (isa x foo)"

Nowadays I'd rather design the technical parts of the system first and then design the syntax to support it the best it can.

---

"3) If you want to add new types that behave like existing types, you need to always extend isa (or type), but with proper foo? functions you only need to extend those individual functions"

These days I would rather give library writers the ability to say what's extensible and what isn't. If it isn't extensible enough to support a new type, too bad. Fork the library.

This applies to languages too. If the language isn't extensible enough, fork it. I once wanted to design languages such that nobody would need to fork them, but I don't put much faith in that anymore, so I see it as a lower priority.

Anyway, I don't like Arc's 'type and 'rep because even though they can be used on any value, that misses the point. If I want to make a value that anyone can take apart, I'll use a table or a cons cell. Type wrappers are good for setting aside new areas of value space that _no_ existing utility knows how to handle yet.

--

"But I take it a step further in Nulan and also say that annotate/rep are poor because if you want to tag something with multiple types, you need to wrap it repeatedly: (annotate 'foo (annotate 'bar ...)) and then you need to unwrap it repeatedly: (rep (rep ...))"

In that particular case, I would consider making an 'unwrap-bar function and extending it to handle both 'foo and 'bar.

I would probably name 'unwrap-bar more descriptively, reducing its association with the concrete implementation details of the 'bar type, if only these weren't metasyntactic variables. :-p

But reuse in what context? Reuse all across the universe? I think the notion of reuse is often ill-posed. The only reuse that matters is in the context of a codebase. And as long as a codebase only has one fork of any software you can have unlimited forking in the world without reducing reuse.

Well, I only care about libraries because it means that (theoretically) one guy can write the code and a bunch of people can use it. The same is true of languages.

This is nice for things that are pretty stable, like regexps. Everybody knows regexps. The syntax is reasonably standardized across implementations. It doesn't make sense to have everybody write their own custom regexp implementation: just write one solid one and use it as a library.

But that only works when there really is a Single Right Way to do it. As soon as there's different goals and priorities, you just end up with a lot of little custom libraries (or lots of forks of libraries), in which case they might as well not be libraries to begin with, since they're only really useful to their original author.

This means you should only write libraries when there's some sort of standard or consensus. If there isn't, just write your code in a very flat simple style, no libraries needed.

---

As for namespaces... I don't actually care about those for code reuse. Libraries handle code reuse just fine with or without namespaces: look at Emacs Lisp as an example of a language with no namespaces and dynamic scope yet they seem to manage okay. C also has lots of libraries and code reuse yet doesn't have namespaces.

The reason I care about namespaces is that it makes certain things easier to reason about, that is, it reduces the cognitive load needed to design and understand a program. It also makes your programs shorter because when conflicts do arise, you don't need to do things like prefixing all your global variables, like they do in C/Emacs/Python/JavaScript/etc.

Well, that's the theory, anyways... in practice, I agree with you that traditional namespaces are overrated and only mildly helpful while requiring a lot of infrastructure to support them. An overall net loss that can only be recouped by writing many libraries over a long period of time that make use of the namespace system.

I also just realized it might have some interesting uses as an alternative to exceptions. Rather than throwing an exception, you would instead return an object that has been tagged with an "exception" type.

By default this exception type would be propagated upwards until it reaches the top of the scope, but you can catch it by using pattern matching:

  $let (exception? X) ...
    ...

Yep! There's a pretty close connection to comonads, the opposite of monads. Comonads are for structures that can be unwrapped into a value at any time, but which can also be transformed without unwrapping by... confusingly, sending them a function that would unwrap them.

  extract :: MyWrap x -> x
  extend :: (MyWrap x -> y) -> MyWrap x -> MyWrap y

To illustrate, a pair of a Kernel expression and a Kernel environment can be used as a comonad, as long as we design Kernel expressions so that they can reify arbitrary values using 'quote. The extract method is eval, and the extend method bundles up the result as a literal expression together with the original environment.

This is monadic. Generally if you build a program out of a lot of functions of type (x -> MyWrap y) you can probably treat MyWrap as a monad, whereas if you build it out of a lot of functions of type (MyWrap x -> y) you can probably treat it as a comonad.

---

"By default this exception type would be propagated upwards until it reaches the top of the scope, but you can catch it by using pattern matching"

In this case you might be implementing it in a monadic way, but for most purposes the language user might as well treat it as a regular side effect.