Could one cluster news.arc as is? It persists everything in files. Could that be a problem whilst saving an item that's modified by two users at the same time? (If the users happen to be in different runtime instances that is). I guess I should just try it, but any feedback will be appreciated.
You've asked a good question. pg could probably answer off the top of his head; otherwise, it may take some digging into the source. If I had to guess, I'd say that you'd have problems clustering it as is, but that's an easy guess, because clustering usually has problems :)
Ruby on Rails (and many others) use a shared nothing approach, so each node in the cluster has to read everything it needs from the database (or other store) for each request. It's very easy to scale until you overload your database, but not very efficient.
Another approach is to have the nodes in the cluster communicate information amongst themselves.
I would think using something like memcached would be pretty effective in helping to cluster news.yc
Thanks for the reply. I played with it a little bit, and I think there could be some contention. If two users created kids for the same item, the kids entry in the 'item template has to be updated to reflect the two new articles (ie, the item has to be serialized again). If the users are on different instances I believe you could lose information ... I think :)
Also, it doesn't seem save-table is safe in the sense that it just overwrites the file that's already there (as opposed to writing it with a different name, and just 'mv' over which is supposedly atomic).
