Here's an example page:

At this point I'm willing to put in the time to implement a solution that works the way I want it to.

However I wouldn't discourage anyone from setting up a GitHub Pages for Arc today. I hope that my site will turn out to better than GitHub Pages (for this particular purpose), but that doesn't mean that you need to wait around for me to implement something, or even necessarily that you will in fact find my site better than GitHub Pages.

As some of you know the first prog. lang. I learned was Arc, but since then I've been working with Clojure and have been really happy to have namespaces.

For starters:

* I don't need to spend brain power coming up with stupid hacked names to prevent collisions. I can use the ones that flow naturally.

* I'm not a fan of having gobs of documentation, so for me a defvar or fn name should be meaningful yet as short as possible, I can achieve this with namespaces and lose this otherwise.

* I think having the option for a few lines at the top of your files to represent namespace management is far less a problem than making your code less readable.

Also, for example, using namespaces one could normalize[1] much of the code between Arc and Clojure, such that converting programs over becomes a cinch:

i.e. You could create an arc library in Clojure that overrides Clojure names:

  (ns arc.core
    (:refer-clojure :exclude [remove]))

  (defn remove [x y]
     (code))

  (ns myprogram.core
    (:refer-clojure :exclude [remove])
    (:use arc.core))

  (defn myfn []
    (do-stuff
      (remove list1 list2)))

  (arc.core/remove list1 list2)

[1] Obviously some arc sugar will not covert over, but you could write a program in arc knowing in advance what semantics to avoid, making the conversion work, or at least lessening the impact.

:) I noticed that as well. Thanks for the writeup.

"..having the option for a few lines at the top of your files.."

Oh, I'd love the option. It's good to have the ability to rename declarations, but I don't need it all the time. The common case isn't collisions. The common case is just one definition to a name, and when I use the name I want the definition. It seems a false economy to foist verbosity on the common case.

More concretely, I want to phrase your examples as:

  (use arc.core :exclude [remove])

Or an interactive session like this:

  > (load "a")
  > (load "b")
  warning: b.foo shadows a.foo ; now a.foo isn't available
  > (rename b.foo :to b_foo) ; a.foo becomes available again

And you can choose to incorporate name-spacing only when you need to.

That being said, once you start crafting projects having more than one file you will end up adding a bare min namespace, since it's just actually easier than load-file:

  i.e.
    (ns myprog.core) 
  does the same as: 
    (load-file "/myprog/core.arc")
  only it creates the names in the namespace 'myprog.core'.

  user=>(ns mynamespace)
  mynamespace=>

Since Clojure core libraries are already loaded, my example actually had to exclude the 'remove' causing a little bit of boilerplate:

  (ns arc.core
    (:refer-clojure :exclude [remove]))

  .i.e  to add more items to exclude...

  (ns arc.core
    (:refer-clojure :exclude [remove find others]))

  (ns myprog.core
   (:use [somelibrary.core :only (every re-sub pull]))

  (ns myprog.core
   (:use somelibrary.core))

> 'No need for a namespace declaration inside arc.core, or in the caller'.

None, but as stated above - you typically have one since it's easier.

[edit: None, assuming you choose to use load-file (load-file "arc/core.clj") which loads the code into the default namespace, or where ever you ran load-file)

I'm not too concerned about the verbosity when you need to exclude something. What clojure does seems fine.

I didn't realize that ns is like load, and not like PLT's module. It goes in the caller, not the callee. That's cool. But once I use:

  (ns arc.core) ; provides say find

I'm not sure if Clojure is doing this for me or leiningen.

https://github.com/technomancy/leiningen

As I started right off using it.

[edit: yeah lein is doing the load for me, but you can still just use load if you like. To run as a script Clojure uses 'java -cp ....', so your files need to be on your classpath location. It's been a long time, since I bothered with that way.]

> Can I use all its declarations as just find and not arc.find?

  HTML Layout:

  <html>
  <head>
    <script type="text/javascript">
      (pr (get+xmlhttp (string "chatlog?room=" room "&max=" maxm))
          (checkif+xmlhttp (string "checkchat?room=" room))
    </script>
  </head>
  <body onLoad="javascript: GetData();">
     <div id='chatlog'  ....>
     </div>
   </body>
  <html>

  Services:

2. checkchat?room=1, this service checks to see if there's been an update in the chatlog since the last time the user checked. So you need to store the last time checked, and compare that to the timestamp of when the chatlog, for that room, was last updated. Comparing the two will return either Y or N. Y will trigger the update. There's javascript noted below that will auto check this for you.

  Javascript: 

  (def get+xmlhttp (url) 
     (string 
	"function GetData(){
	  if (window.XMLHttpRequest){
          var xmlHttp = new XMLHttpRequest ();}
          if (window.ActiveXObject){
          var xmlHttp = new ActiveXObject(\"Microsoft.XMLHTTP\");}
          xmlHttp.open (\"GET\",\"" url "\", false);
          xmlHttp.send (null);
          var targetNode = document.getElementById (\"chatlog\");
          targetNode.innerHTML = xmlHttp.responseText;};"))

  (def checkif+xmlhttp (url)
    (string 
	   "function CheckChatLog(){
	      if (window.XMLHttpRequest){
                var textHttp = new XMLHttpRequest ();}
                if (window.ActiveXObject){
                var textHttp = new ActiveXObject(\"Microsoft.XMLHTTP\");}
                textHttp.open (\"GET\",\"" url "\", false);
                textHttp.send (null);
                if (textHttp.responseText == \"Y\") GetData();};	     
           window.setInterval(\"CheckChatLog()\",5000);"))

Notice 'checkif+xmlhttp' has 'window.setInterval', so the js will periodically run 'CheckChatLog' for you - you may want to tweak the 5000 time interval.

Suppose you have a list that you know shouldn't be changed. If you have immutability in the language, you can make the list immutable, and then if some buggy code tries to change the list you'll get an error.

However, bug-free correct code won't try to modify the list. (If it did, it would get an error, and it wouldn't be correct code). So for correct code, it doesn't matter whether the list is actually enforced to the immutable or not, it will be immutable in practice because the correct code won't try to change it.

So being able to enforce immutability doesn't help code that is already correct. What it does do is maybe help you find bugs quicker. If you have some buggy code that is modifying a list when it shouldn't be, you'll find out right away instead of getting some other error or problem later.

So, you go to this extra effort to mark certain lists as immutable (or maybe have things immutable by default and then make particular things mutable), and in return for this extra effort some kinds of bugs are easier to figure out.

Is it worth it?

Well, for me, just in my own personal experience when I've been tracking down my multithreading bugs, I haven't had a problem with accidentally mutating things. The kind of bugs I run into are typically things like not doing enough locking or doing too much locking. So I could run around marking all kinds of things immutable in my own code and it wouldn't help me find the bugs that I actually have.

Of course, that doesn't mean that there might not be some future situation in which I'd find immutability helpful, such as where I was accidentally modifying some list I didn't want to modify and I wasn't sure where I was doing it. Then I'd happily make the list immutable, so I could quickly find out where my bug was.

Which would be easy to do. Even in Arc 3.1 with PLT 4, which makes Scheme's "immutable" pairs mutable, just add a datatype to the Arc runtime, have "car" and "cdr" work on the datatype, but don't add setters. Since this would be so easy to add, that no one has done it suggests to me that maybe no one has found it particularly useful yet.

Another situation in which I suspect people like enforced immutability when they're working in a team of programmers and they don't trust the skills other the other programmers very much. When the other programmers introduce bugs it's easy to see that it's their fault when they get an error.

---

When you just load a series of files in PLT you can declare your functions in any order and in any file. Nice.

But turn those files into modules and now you have errors.

  compile: unbound identifier in module

---

According to R5RS (http://www.schemers.org/Documents/Standards/R5RS/HTML/r5rs-Z...):

  "In Scheme, combinations must have at least one subexpression, so that
  () is not a syntactically valid expression."

(And to decompose arc into arbitrary files I need to come up with a weird language that accepts () like mzscheme mode, but doesn't use modules.)

---

The arc implementation is far more verbose than it could be because of PLT's phase separation rules -- it must implement its own macro system (http://arclanguage.org/item?id=11529)

---

I view it as a convenience. Similar to HN or this forum app, many of my apps keep data in-memory (to improve performance) however unlike news.arc my apps tend to allow concurrent data changes among multiple users. News.arc doesn't need to worry about this 99% of the time (where 1 user, 1 story) and I'm thinking arc is very news.arc centric.

Using Clojure I just find it easier to wrap my statement in a dosync, walk away and not worry about writing more code to manage the transaction to shared in-memory data.

Arc (without transaction code):

  (= data!1 (value-of-function-taking-a-variable-amount-of-time))

  (dosync 
     (alter data assoc 1 (value-of-function-taking-a-variable-amount-of-time))) 

I could start writing a bunch of code to show the equivalent of dosync in arc, but it would be implementation specific and I believe the point is shown that I don't have a coding exercise in Clojure it's part of the language, therefore a convenience.

Addtionally I can keep two data structures in sync at all times.

  (dosync 
     (let [x (value-of-function-taking-a-variable-amount-of-time)]
       (alter data assoc 1 x)
       (do-something-else)
       (alter data2 assoc 1 x)))

So... Bob triggers a process at a time-stamp of 1. The process takes some input arguments and the current value of a record in a data-structure, as additional input arguments, then builds upon it. The process takes a time-length of 4. Afterwards it stores the result back into the record.

Alice triggers the same process on the same record at time-stamp 2 only her process takes a time-length of 1.

Using Arc the end data structure will contain Bobs change, an incorrect value. It should have done Bobs process, then using it's result as an input argument for Alice's process to produce the final result.

The reason why I am not writing a function for you is that these are common problems/scenarios that require complex setup. They are common enough that we have databases requiring ACID compliance to overcome them.

There are a variety of business examples that would expose these problems. ie ROFR's (right of first refusal algorithms): where processes grab data as input args then allow user checks/processes, yet rights are granted at the initial time of execution.

I know these things could be accomplished in arc, but I believe you're going to be writing a lot implementation specific code, that would just get in your way. You're going to write much more than 'dosync' and you're likely going to have to troubleshoot that code more often than you would like. No one writes 'correct' code all the time.

I also wonder why arc avoids it. It seems like arc is going out of it's way to bypass immutable data-structures rather than build upon them and progress to offer convenient features like built in transaction semantics.

"..convenient features like builtin transaction semantics."

To my jaundiced eye a feature that requires a lot of cycles of setup doesn't seem very convenient ;) It may well be impossible to do without, and transaction semantics may be useful because they make your life easier, but they aren't convenient.

I agree. I was never intending to use arc for doing so, I was just asking about the rationale of going against scheme and other functional programming languages evolution. Just seeing what others think.

> ... feature that requires a lot of cycles of setup doesn't seem very convenient.

I was referring to crafting an example problem using Arc, to show the value of the feature - not the feature itself. The feature implementation is dead simple and convenient.

> It's hard to be a hacker's language with such advanced/complicated/complex features. Arc's approach seems to be, "if it's complicated throw it out."

Depends on how you view it. I'm not a language junkie, but I value languages that have a good balance between power, efficiency, and usability. Arc provides this. So does Clojure. Clojure has more features. You see power in arc, you understand all of its' minimalistic feature sets. I see feature sets provide power simply because they exist - and they still they have a very low complexity requirement to achieve a given result. (ie . wrapping an update statement in a 'dosync' is pretty much free of any complexity yet you get a lot in return). That, to me, is valuable. That to me is power.

It's funny, I'm starting to notice some rosy glasses around here. No offense and this isn't just from your comment, but I get the impression if a feature gets implemented in arc, then arc users see the power arc provides and say "look how cool arc is - its powerful". Yet if someone says language "x" has a cool feature that arc doesn't have then all of the sudden it's a "framework", "cruft" or extra complexity that's not needed anyway....Really? I don't buy it. These other languages are doing great things too.

> Hackers will find a way to live within its limitations. And we often do.

But the implementation is irrelevant! What matters is how easy the feature is to use. [Insert rant about how 37signals confuses interface complexity with implementation complexity. Implementations should be complicated. http://www.joelonsoftware.com/items/2010/08/19.html]

---

I don't think I have rose-colored glasses at all. I didn't mean to imply that transactions aren't useful, or that convenience is all. We all use other languages here. My point is this: if you want to get stuff done, don't go dreaming of concurrency and transactional semantics in arc. It's just too hard, PLT's foundations are too crappy from that perspective. It would be a waste of your time.

Arc is a place to experiment on how far we can go with a minimal software stack. I wouldn't use it for other kinds of experiments.

---

Before I worked on improving the port of Arc to PLT 4 (the bit about using MzScheme's mutable pairs to implement Arc lists instead of whacking immutable pairs with pointers), I spent some time looking around to see if there was another runtime I'd want to use instead. Because the Arc runtime is less than a thousand lines of code, it would be easy to move it to something else if it were better, and I wouldn't want to take the time improving the port to PLT 4 if I were going to use something else instead.

I never got many chances to use such exotic features, but I believe the combination of message passing and serializable continuations would allow you to do fun stuff like easily transfer code execution between different computers.

My limited experience suggested it was fast. It's also compilable to C. I suspect tail calls are properly handled - doesn't seem like something Feeley would let slide. I'm not sure about reliability.

Which means what exactly? Slow? Crashes a lot? Doesn't sparkle?

Do you have a single shred of evidence for these claims, whatever they are? Do those "people"? Or is this just all blabble?

You seem to be saying reputations are irrelevant. They're a shortcut. They're for when the program I'd have to write to test some property would require a week to months of work. If you want to ignore reputations, be my guest. But reputation is how you found out about PLT in the first place. You didn't run some idealized experiment comparing every single platform in existence for precisely the nuanced characteristics of the program you were planning to write.

I hear lots of people online touting Erlang as high performance at hundreds of cores, and highly fault tolerant. Have you not heard this? Do I need to cite chapter and verse for things I assume people are aware of? I don't hear anybody saying similar things for PLT. Am I just not listening in the right places? Feel free to cite evidence yourself.

I use PLT and arc because of programming convenience. I don't spend much time thinking about how good the runtime quality is because it's frankly not that important to me. It doesn't give me much confidence when I see 12 segfaults a day on my readwarp server[1]. So yes, it does 'crash a lot'[2]. In this very thread you're switching away from using immutable cons pairs because they seem to be not working as advertised. Is that not 'evidence'?

You've said things like "if there's anything better than PLT.." so it sounds like you're thinking in terms of an absolute ordering that you can stack language/platform combinations on. Surely the ordering is in the context of your use case. Or would you write the linux kernel in PLT scheme? Based on what you know now? How would you scale news.arc to two servers? I don't care that PLT 'has message passing'; if I wrote something that required lots of machines I know I would switch to Erlang or C, and put up with crappy syntax. You can't have the best programming environment and the best runtime - because no single community focuses equally on both.

I was careless in my phrasing of 'some people'. Sorry. I didn't expect to get pounced on, and I didn't mean for the conversation to get yet more noisy. It takes two to do that.

[1] http://arclanguage.org/item?id=11355

The other day my auto-update routine didn't run for whatever reason. I figured this was no big deal, happened once before and I only had to trigger my handy-dandy web service which started a dataload process in a new thread. Everything worked perfect. Only this time I triggered it then logged out right away. It seems using arcs existing srv framework is less flexible than I would like.

I believe implementing a RESTful service that is stateless will likely make the service more robust. Opinions anyone?

This time I was really lucky that I checked a few minutes later and restarted the service.... My 40 users were almost in tears - lol :)

Thanks to palsecam for laying out a path....

http://arclanguage.org/item?id=11337

I've moved 3 of my 4 domains over to the new server (Ubuntu10.4+Nginx). While I don't have any real evidence or even enough traffic to prove nginx is better than Apache, I am quit happy with the move. I didn't want to start spewing benchmarks, since the configuration settings are different[1], but I will say this: My first run at setting up Apache, as a total newb, took 3+ days enduring far too much complexity and troubleshooting ending in poor results. I have accomplished so much more with nginx in 3-4 hours.I know I am spewing a bunch stuff with out much real evidence, but for anyone else using Apache as a front end server I highly recommend considering nginx.

As for my original problem, I stopped using the http-get library and instead use wget with one other change (getting all the files downloaded first) I've cut the process time from 2 hours to 48 minutes. So far have received no errors. My CPU still churns at 96%.

[1] I hadn't applied re-write rules with my Apache set up, but nginx was dead simple to separate what I needed relayed to Arc vs. what to pull direct:

  location / {
  http://127.0.0.1:9000;
  }

  location /css {
  root home/arc/static/;
  expire max;}

  location /js {
  root home/arc/static/;
  expire max;}

  location /images {
  root home/arc/static/;
  expire max;}

  I'm sure I could have done something like:

  location /css /js /images{
  root home/arc/static/;
  expire max;}

  but I've only used it a little bit.

http://github.com/nex3/arc/tree/arc2.master/lib/http-get/

I'm using the save-page function.

[Edit - Sorry, in retrospect, I don't know why I didn't reference it to begin with, I just naturally Assume everyone knows what I am talking about Lol!)

..... I suppose I could just use wget or curl which would at least rule in/out that library!

> System fork error raised inside the arc process?

hmmmm good question I am pretty sure it dropped out of arc and then provided a system error.

> I don't understand in (2) how slowing down the routine results in increased traffic.

It's not that I think it will increase traffic, just that if I have to slow the requests down to somehow release connections or memory, then it's not really a solution since I would like to be capable of handling future increased traffic(to much larger degrees than my 2000 connections for stocks :).

I think it's just my poor understanding of how threading/memory allocation/networking & gc work. Somehow I have it in my head that the process is not gc'ing memory or releasing connections from previous iterations before it moves on to the next, or not releasing the underlying OS thread for the get request before moving on to the next. That's probably all wrong, correct?

The process is pretty simple: download a file, load data from file to memory (assign to variable), parse the data and do the math, write the results to file, write the progress to file, wipe the variable. Rinse, Wash, & Repeat :)

I am just moving over to my new Ubuntu 10.4 Linode with Nginx. Setup is complete, data is copying over. Then I plan to run some benchmarks with out changing any code. Do a before and after to see the difference.