I've been playing for some time now with writing all my code in a style of stepwise refinement. For example, here's how I extend (http://arclanguage.org/item?id=13790) len to understand arc's queues: def len(x) :case (isa x queue)
rep.x.2
This can be a really elegant style for separating concerns. All my code about queues is in one place, even if it updates primitives. But there are costs:a) Performance. What used to be a simple macro call can now need to be expanded multiple times. Especially if you build the language from the ground up in this style, the increase in calls has a compounding effect (http://arclanguage.org/item?id=15392). It's all very well to say 'design the language without regard to performance', but it's five orders of magnitude slower than anything else out there. I've been working on a partial-evaluator to 'inline' a lot of this redundant work, but it's been slow going. Check out http://github.com/akkartik/wart/blob/00f18c9235/070inline.test. I'm not yet at the point where (if a b) uses the compiled if rather than the macro to support more than 2 branches. b) Infinite loops. Especially when dealing with core language features like assignment and conditions and equality, it's easy to overload too much. See, for example, how I implement isa: http://github.com/akkartik/wart/blob/00f18c9235/030.wart#L49. So far I've been rationalizing these problems as transients that I have to write a test case for and debug once, and never deal with again. However it's getting to the point where it's really hard to spot the infinite loop, to insert a print in just the right place to see the repetition. Most recently, I found myself trying to overload if to support pattern matching, and realized that if the :case clause ever calls if, even indirectly, I end up with an infinite loop (http://github.com/akkartik/wart/commit/67129351d3). And the language doesn't even do all that much yet! One idea to deal with this: I've been mulling changing the semantics so that new :case clauses on existing functions aren't used in code already loaded. It would improve performance and also hugely mitigate the headache of debugging infinite loops. But it feels like a compromise of precisely the sort that I've been railing against.[1] Anyways, just wanted to toss this out there to get your reactions. --- [1] Rather surprisingly, I can't find a decent link for my rant. Was it all in my head? The best I can find is http://www.arclanguage.org/item?id=13263. Or maybe http://arclanguage.org/item?id=12057. Hmm, let me try to state it. A lot of the benefit of lisp has traditionally come from the repl. And a lot of the benefit of the repl comes from the fact that you can type incorrect code in, and you won't be bothered by the toolchain until you try to run it. Functions can be defined in any order, called in other function bodies before they're defined, and lisp won't complain. Dynamic types and late binding are specialcases of this ethos. The value of this approach isn't just that you can be disorganized when sketching out a new program. Eliminating machine-driven constraints affords freedom to organize code. Organizing code for readability is hard. So hard that most complex programs in any language are poorly organized. They start out with structure following behavior, and then that mapping gradually rots. Axes of organization that made sense before cease to do so, or are swamped by the needs of a new aspect. Definitions in a file are more intimately connected to distant definitions than to nearby ones, and the reorganization never quite catches up. Existing programmers can handle this gradual semantic drift, but it increases the barrier to new programmers contributing to the project. Given how reluctant most of us are to even try, every constraint added by our tools makes it less likely that we'll reorganize everytime the emphasis of the codebase switches directions. Many's the time I've tried to reorganize the header files in a C codebase and just given up after a few hours because it was just too hard to manually solve the graph constraint satisfaction problem that would make the compiler happy. So the fact that lisp started out with very few constraints is super super key to my mind. But most of the world doesn't seem to think so. Both lisp and scheme have over time added features that constrain code organization. Languages ahead of their time, they have stolen package and module systems from the static languages around them (partly to simplify the lives of compiler writers). This has happened so gradually that I think there isn't a realization of just what has been lost. Wart employs an init-style convention for deciding what files to load, and in what order (http://arclanguage.org/item?id=11869). That idea stems from the same considerations. Renaming a file should require: step 1, renaming the file. There ought to be no step 2. That's how easy I want reorganization to be. And I want it to be as true as possible at all levels of granularity. Richard Gabriel: "If I look at any small part of it, I can see what is going on -- I don’t need to refer to other parts to understand what something is doing. This tells me that the abstractions make sense by themselves -- they are whole. If I look at any large part in overview, I can see what is going on -- I don’t need to know all the details to get it. It is like a fractal, in which every level of detail is as locally coherent and as well thought-out as any other level." ("The quality without a name," http://dreamsongs.com/Files/PatternsOfSoftware.pdf) --- Oh, another variant of this rant, if you're really bored: http://news.ycombinator.com/item?id=2329680. Does it even seem like the same rant? Maybe it really is all in my mind. | 
