"Apparently, in my attempts at implementing this functionality I've also independently kludged up a special-purpose version of what Racket calls a continuation mark, and obviously doing such a thing bothers me to no end."
Why? Are you worried your runtime will be exactly like Racket but less mature? When I looked at your call-w/std(out|in) commits, I liked your approach exactly because I noticed it was in the same vein as continuation marks. :-p
As you've noticed with complex numbers, Arc exposes lots of accidental complexity that it inherits from Racket. In fact, speaking of accidents, Arc 3.1 without modification exposes pretty much all of Racket: http://arclanguage.org/item?id=11838
When it comes to threads and parameters, I'd say Arc pretty much specifies nothing and leaves it up to Racket to provide the meaning and implementation. Arc literally defines 'call-w/stdin and 'call-w/stdout in terms of Racket's 'parameterize. If Arcueid doesn't end up with (internal) functionality equivalent to thread cells and continuation marks, there's a good chance it'll have certain corner-case inconsistencies with Arc, even if there aren't enough inconsistencies to break the programs we actually care about.
But even so, I wouldn't worry about it too much. I personally consider Arc to have shoddy support for threads (just exposing a tiny subset of Racket's functionality and imposing a GIL) and also for reentrant continuations (not defining 'dynamic-wind, implementing loops with mutation), so I don't really blame an Arc implementation for being incompatible in these areas. In some cases, full compatibility might be more harmful than not trying!
---
"I had for a time considered using real POSIX or other OS-level threads to be able to take advantage of multiple cores but soon realized that this would introduce quite a bit of complication. Using real threads affects just about every aspect of the implementation."
If you want to give an Arc program power to take advantage of those, but you're having trouble with multithreaded allocation, an alternate path might be to have the Arc namespace and most data structures be local to an OS thread but then to have other tools to write and read manually-managed shared memory. I dunno, maybe that's not very inspiring. :-p
---
"The only thing directly shared by all threads is the global environment, and then I'd also have to make available a flattened version of the structures I used to store the call-w/std(out|in) bindings from the thread's creator, and the more general dynamic bindings created by parameterize as well."
Er, local scopes and first-class data structures might need to be shared too, right?
(let foo (list nil nil)
(for n 1 10
(thread (push n foo.0) (push n foo.1)))
(def get-foo ()
foo))
"Why? Are you worried your runtime will be exactly like Racket but less mature?"
Not in the slightest. It just bothered me that I had to embed a special-purpose data structure inside Arcueid's continuations just to support one language feature. Now that I see that there is a natural generalization to this feature, that makes me feel a lot better. :)
'protect is implemented with 'dynamic-wind, so the only functionality we lose is the ability to specify a pre-thunk. Are there any areas where that would be useful?
Dynamic-wind gives us most of the ability to implement parameters ourselves. We just mutate a box upon every entry and exit of the expression. Unfortunately, it might take some crazy trampolining to get the last expression of (parameterize ...) in tail position. I'm not even sure if tail position is possible....
I think the last missing piece is thread-friendliness. In the face of threads, we'd need the box to be thread-local like Racket's parameters. But my point here is just that the pre-thunk is useful for something. ^_^
