I have to say I'm not sure why '= is atomic. It does mean that modifications to complex data structures are more-or-less guaranteed not to put them into an inconsistent state, at least at the granularity level of a single assignment. However:
1) As a general principle, IMO, it should be the programmer's responsibility to avoid problems arising from concurrent use of objects by explicitly using concurrency primitives (mzscheme provides a host of these, and it's not too hard to write a wrapper which lifts them into arc, if someone hasn't already put one into Anarki). If they find this too much of a burden, then they can write some convenience macros.
2) Adding the overhead of 'atomic, which is implemented (and pretty much has to be implemented) using a global mutex, to every single assignment is just not okay.
3) It's not safe to assume that the granularity which the programmer needs in concurrency is precisely assignment-wide. Admittedly, the programmer can wrap whatever they want in 'atomic if the granularity is coarser, but then they're explicitly dealing with concurrency, as suggested in (1), and the atomicity of '= is moot.
I'm also thinking that having Arc make things atomic for us may be a bad idea, though perhaps for different reasons than you enumerate.
1) As a general principle, IMO, it should be the programmer's responsibility
I myself am quite happy to have responsibility taken off my hands, if the resulting code actually works. However...
2) Adding the overhead of 'atomic
If it worked, I wouldn't mind the overhead. (I haven't measured it, so I don't know if things being made atomic by Arc adds 10% to my program's execution time or 0.01% or 0.00001%) But the way to make a program fast is to profile it and fix the parts that are slow, not to leave out useful features in order to do global, unmeasured optimizations.
However...
I don't think it does work, which is probably part of what you're saying as well. Consider
(readfile "foo.txt")
Reading a file might happen instantaneously if the operating system already has the file cached in memory; it might be slow if we're waiting for the hard disk to rotate into place to read it off of disk; it might take forever if the file is on a network filesystem.
Should I have the misfortune to say
(obj a (readfile "foo.txt"))
this will work fine most of the time when reading the file is fast, but if the reading the file happens to be slow then I'll bring my entire web server to a halt waiting for the file to be read.
Yeah, it's reasons like that that lead me to point (1). When stuff is done for the programmer for nonobvious reasons, it leads to problems like this - there's nothing in the conceptual mechanics of (obj ...) that would make you think "oh, it's wrapped in a global mutex", and yet it is because assignment itself uses a global mutex. It's not so much that the language is "doing it for me" that's the issue with (1); that's fine. It's "explicitness" that's the problem: I want to know when I'm using a concurrency-controlling form like 'atomic. Otherwise, it's hard to reason about how my program is going to behave.
Also, this stuff about profiling to fix hotspots is mostly true, but hotspots come in many forms: tough algorithmic problems, folding or mapping some operation over a large data structure, etc. One form of hotspot is a common operation that isn't particularly localized in use. For example, some people have experimented with adding infix math to arc by having numbers cause infix parsing when they're called; they found this caused a slowdown of ~10%. I strongly suspect assignment falls into this category: it's used everywhere, it's a fundamental language feature, and making it faster will reap noticeable (albeit not overwhelming) benefits. However, I must admit I haven't tested this, so feel free to prove me wrong.
For the second, well, I wouldn't be surprised if someone who worked at optimizing Arc could make it run twice or maybe even ten times as fast, while still implementing the full specification. So if a particular language feature is actually useful, and happens to cause a 10% slowdown, I mostly don't worry about it myself... I suspect greater gains would be found from a systematic optimization approach.
Besides, most of the programs I write already run fast enough, so I wouldn't mind a 10% slowdown for a feature that was actually useful. Which, unless I'm presented with new information that changes my mind, I agree with you that Arc's implicit locking is not.
5. Macro bodies are written in Arc; of course they're Turing complete! Even using macros and some non-turing-complete subset of the rest of Arc, it should be possible to write a Turing-complete language using recursive macros, albeit not one that you'd really want to use. It's true that recursion without a base case won't terminate, but a Turing complete language whose evaluation always terminates is a contradiction.
Actually, I apologize. My earlier statement about macros plus some turing-incomplete subset of arc being turing-complete makes no sense. There is no such thing as "macros on their own". Macros are just Arc code that gets evaluated before what we like to think of as "runtime". Arc macros sans the rest of arc are nothing. The "difference" here is not that macros are (or rather, macroexpansion is) turing-incomplete. The difference is that Arc delineates macroexpansion from normal evaluation, whereas Pure doesn't have a distinction; everything is term rewriting.
Although I aimed for clarity and modularity when writing the above, rather than lines of code or speed, it actually does quite well in both departments. It takes about three seconds on a 20x20 board, and about ten for a 30x30. I suspect that using a hashtable to keep track of visited squares rather than a simple list would speed it up on higher board sizes.
Otherwise, the code will generate incorrect tours. This increases runtime. However, I've also tested a version which uses hashtables, and as I suspected, this significantly decreases runtime. To use hashtables, replace 'valid and 'knights-tour in the original with the following:
(def valid (pos visited)
(and
(< -1 car.pos board-size)
(< -1 cdr.pos board-size)
(no visited.pos)))
(def trace (k e)
(cons k (let v e.k (if (isnt t v) (trace v e)))))
(def knights-tour (start (o prev t) (o visited (table)))
(= visited.start prev)
(do1 (if (is len.visited (* board-size board-size)) (rev:trace start visited)
(ormap [knights-tour _ start visited] (succs start visited)))
(= visited.start nil)))
Fully hygienic macros have now been implemented and pushed to the hygiene branch - w/uniq is no longer necessary. Moreover, this hygiene can be avoided, which makes possible the writing of anaphoric and other macros which deliberately make use of variable capture. All that is necessary is to unsyntax (#,) the variable which is to be captured. 'iflet neatly demonstrates both automatic avoidance of variable capture and the ability to deliberately capture:
Admittedly, in the case of 'aif at least, the "hygienic" way of doing things doesn't give you any more safety than doing it with normal quasiquotes would. However, nothing prevents you from using normal quasiquoting in this case; arc now has the best of both worlds, as it were.
It's true, I can override this. But this would change the language for any consumer of my library, as well; and however bad it may be to have absent keys map to nil, it's even worse to face the possibility of breaking the language for someone else. Sure, it's an optional parameter here... but what happens if someone else has their own idea about adding an extra parameter and what it should do? Your idea is probably the right design (though 'len and 'keys and the setter for tables would have to be changed as well to fit it), but it's not the way arc currently works. Hopefully pg will pick it up and run with it, though.
If arc is continuing by momentum, it's a very different kind of momentum than the momentum languages like Java, Python and Perl survive on. They have momentum in that they have many large and complex libraries and programs written in them which are being actively used to the point where rewriting said software in another language is no small task; and hence most people are interested, not in making a new and better language, but in making the old language better.
Arc has none of this. The reason why it continues is because the people using it are enamored of it for its own sake, not because they depend on it. This is both a blessing and a curse. A blessing, because it means Arc is free to change and improve; a curse, because it means Arc is incomplete and a moving target.
Moore's law has actually been failing as of late. I suspect that exponential increase in computer capability is a feature of the yet-nascent state of computer technology. Assuming that it will continue into the foreseeable future is a bad idea. Performance is important; it's just not as important as some think, and in particular, it's not so much all-around performance as the ability to avoid bottlenecks that's important. This is why a profiler for arc would be nice, as would some easy and standard way to hook into a lower level language for speed boosts.
If you generalize Moore's law to cost of computation per second, it isn't failing. In 100 years, network availability, speed, and capacity will lead to yet-unimagined changes.
How will programming languages take advantage of a thousand or a million CPU's?
I've got to say I agree with pg on this. It's more important that Arc turn out well in the long run than in the short run. Also, although Arc isn't moving very fast, I have to say that the more I actually program in it, the more I appreciate its design. I still find myself missing the lack of a proper library/module/namespace system, and of course given that some actual libraries would be useful, but that's about it for truly major issues.
That said, I do think pg could do with talking to the community a little more, and vice-versa.