; function calls inside macros?
  wart> (def f(x) x)
  wart> (mac foo(a b) `(cons ,a ,f.b))
  wart> (foo @'(1 (2 3)))
  (1 2 3)

  ; splice friendly macros?
  wart> (foo @(foo @'(1 ((2 3)))))
  (1 2 3)

  ; splice friendly macros?
  wart> (mac bar(a b) `(cons 3 ,@(foo a b)))
  wart> (bar @'(1 (2 3)))
  (3 . 1)

So if x is a variable containing ''abc, when we say (bar x x) we wind up evaluating (cons 3 ''(cons ''x ''x))?

When we evaluate (cons ''x ''x), it strips off the '' to get (cons foo foo), and then we end up with (''abc . ''abc)?

Then, altogether, we end up with a result of (3 . (''abc . ''abc)), which appears at the REPL complete with ''?

  wart> (mac foo(a b) `(cons ,a ,b))
  wart> (mac bar(a b) (prn `(cons 3 ,@(foo a b)))) ; prn for debugging
  wart> ((fn(x) (bar x x)) 'abc)
  (cons 3 x . x)
  004cell.cc:158 car of non-cons: x
  004cell.cc:158 car of non-cons: x
  (3 . abc)

  wart> (mac foo(a b) `(cons ,a ,b))
  wart> (mac bar(a b) `(cons 3 ,(foo a b)))
  wart> ((fn(x) (bar x x)) 'abc)
  005types.cc:263 can't coerce symbol abc to function

  wart> (mac foo(a b) `(cons ,a ,b))
  wart> (mac bar(a b) `(cons 3 ,@(foo a b)))
  wart> ((fn(x) (bar x x)) @'(abc))
  (3 . abc)

  wart> (mac foo(a b) `(cons ,a ,b))
  wart> (mac bar(a b) `(cons 3 ,(foo a b)))
  wart> ((fn(x) (bar x x)) @'(abc))
  005types.cc:263 can't coerce symbol abc to function   ; no change

---

Another potential concern: I'm using a single boolean to track whether we encounter a '', and I reset it before unquote's eval and check it after. I think it might need to be a stack of booleans to handle nested expressions, but I've not been able to come up with a failing test case.

  (all present? '((1) (2 4 6)))

  def all(f seq)
    (and @(map f seq))

  wart> `,(write `,'foo)
  foo005types.cc:263 can't coerce symbol foo to function

" '' is an internal detail. The reader provides no way to type it in. If you ever see it printed at the REPL, that's a bug."

If unquote adds a '', then `,'foo should return ''foo, right? Then 'write should write ''foo at the REPL.

In order to weasel out of anything that stripped '' from the arguments to a function, I tried putting `, around the whole command, in the hopes that it would somehow manipulate the state of the don't-strip-'' flag(s) for the duration of the command.

In any case, there's no reason foo should be called, is there? O_o

Unquote doesn't add '', @ does. Perhaps that changes your example?

"In any case, there's no reason foo should be called, is there? O_o"

:) Yeah that's a bug with my paren inference; it was implicitly wrapping the expression in parens. I'd never seen backquote-comma before. Here's a temporary workaround:

  wart> (list `,(write `,'foo))
  foo(foo)  ; ignore the (foo) -- it's the return value

  wart> `,(write `,'foo)
  foofoo

One bug: you don't need the parens inside the [].

Update: it turns out support for multipart/form-data in anarki is incomplete. I'll see if I can fix this; in the meantime you'll have to POST to a static defop which is then responsible for reading and parsing the multipart body from req!in. webupload.arc shows how to read, but it punts on parsing.

I get in trouble when needing to update Anarki. Some patches like this one don't apply on the Anarki version I use and it seems only a brand new git clone gives the latest Anarki version. Which then requires I move around data, stuff in static, etc.

Is there a better way? How can I separate my app from the Anarki changes?

You can run anarki from anywhere:

  $ racket -f $ARC/as.scm

  reference to undefined identifier: _ranked-stories*

Update: a few bugfixes later you can see all the different kinds of forms in action here: http://github.com/nex3/arc/blob/cc0f448538/lib/form-tests.ar...

There's still one open question regarding filenames. So far we've been writing uploaded files to random filenames, but the uploaded filename is actually present in the POST request. I see 2 ways to provide it to the programmer (say when uploading a file in a field called file):

a) (arg req "file") returns just the contents of the uploaded file; the filename is in (arg req "filename"). "filename" is the multipart header, so I could just inline all the various headers into the args table.

b) (arg req "file") returns an alist or table with the various headers for that part of the multipart POST. File contents are in ((arg req "file") 'contents).

c) (arg req "file") returns file contents like in a, and other metadata like filename or creation-date is in some other field of req, say req!multipart.

Does the fnid multipart form test (the second test) and the static multipart test (the third test) work for you?

They do not work for me. I get a "srv thread took too long for <ip address>". I suspect this could be related to nginx again -- though, I started the webserver on a different port and run form-tests through that different port.

When testing multipart stuff, I start the app in a custom port -- not port 80. So it should not be talking to nginx at all.

Looks like the static multipart form hangs and times out while the fnid multipart form immediately disconnects. Is that what you see?

Update: The bug has to do with reading in bytes vs characters. Earlier, srv would readc from the POST body unless type was multipart, and your code (like the webupload example) would readb from the body otherwise. Now srv is always the one reading body, and it always reads characters using readc. As a result it gets confused by binary uploads that don't translate to characters.

  $ ls -l bintest
  -rw-r--r-- 1 akkartik akkartik 145974 Jun  5 12:39 bintest
  $ racket -f as.scm
  arc> (w/infile f "bintest" (len:readbytes 200000 f))
  145974
  arc> (w/infile f "bintest" (len:readchars 200000 f))
  141878

http://github.com/nex3/arc/commit/b3d97ffc91

Your existing code won't work as is. Since it's meaningless to try to convert possibly-binary data to a string, file contents are now returned as a list of bytes. There's a helper called bytes-string for when you're sure you have just ascii data. Otherwise you'll need to know the encoding of text uploads and convert the bytes appropriately.

I see the following error:

  Can't coerce  (98 98 83 116 88 110 82 115 98 103 . nil) sym

http://github.com/nex3/arc/commit/46e3820a6b

  (mac usform (n s f . body)
   (w/uniq ga
     `(tag (form method 'post action fnurl* class (string ,s) id ,n name ,n
                enctype "multipart/form-data")
       (fnid-field (fnid (fn (,ga)
                           (prn)
                           (,f ,ga))))
       ,@body)))

  (defop || req
    (usform "this-is-a-bug" "" [ (pr "args are " _) ]
          (tab (row "price: " (input "price"))
               (submit))))

  (def main ()
     (asv))

Update: Ok, I finally found a way to check when it's safe to convert to string, so now all fields (including fnid) will auto-convert to string when possible. If you get a list of bytes you know it has some sort of non-ascii data.

http://github.com/nex3/arc/commit/d5331ac897

I've also made it faster; 4x faster in my experiments. http://github.com/nex3/arc/commit/16173ec1ff

---

It's actually kinda interesting to explore why this is. I have a site as well, but most people don't come there. I need to go where the audience lives: facebook, twitter, G+, arc forum. The readers throng where everybody else is, and where useability is great. RSS promised a collectively-owned common area, but its useability was inadequate. Facebook and G+ have great useability, but it's concerning to give a single entity so much power over our conversations. Is there a way to incent people to build a great experience without owning the venue of conversation?

And why do I not go to Google Reader as much? In the beginning it was because I subscribed to too much stuff[1]. http://readwarp.com has solved that problem for me. Now I only subscribe to low-volume feeds on Google reader; the rest goes into Readwarp so I can sip from it at random. But there's a new problem I'm just starting to grow aware of: even if I have only 10 posts a day on google reader they're all over the map, and the brain sometimes shrinks from the prospect. To go to Google Reader I need to be in an open enough mood. Whereas when I come here (or even when I go to HN) I have a better sense of what to expect.

Yes, part of what FredBrach did with scopes was eerily similar to what I was thinking about at the time. ^_^ I didn't talk about it much in response, but I did consider it promising.

I was also thinking about this topic a couple of days before http://arclanguage.org/item?id=15831, where I wrote "Suppose there were a special form (varscope <var> <...body...>) which scoped (<var> <name> <val>) as a special form in the body, which both defines and assigns to a variable and returns the new value. One can say (var foo foo) to define a variable without modifying it."

I think this kind of generalized scope jives well with a linear system. Linearity is about assuming every value is produced once and consumed once by default. Everyday lambda calculus lets you introduce a variable at the acquisition site and then (nonlinearly) use it any number of times in the body. For the reverse of that, when we're dealing with values we use once but define in any number of places (e.g. for set accumulation or nondeterministic choice), maybe it can make more sense to "introduce" the variable at the usage site.

My blog post's "Aside" expressions cater to many-to-many, zero-to-many, many-to-zero, and zero-to-zero operations, where there's no obvious definition or usage site to single out for the variable introduction.

---

"I haven't wrapped my head around it all (are linear variables different from linear types?)."

Hmm, when I said "linear variable," I just meant a variable with a linear value or a linear type. This kind of variable appears in the code at exactly one definition site and one usage site, unless the code uses conditionals or something.[1]

If you're in the midst of changing your question to "are linear values different from linear types?" the answer is no, I'm talking about the same concept. :) However, it is possible to have dynamically checked linear values in an untyped language. Likewise, I bet it's even possible to have linear typing without linear values, in the sense of fostering a programming subculture which considers the types important and the values unimportant.

If you're in the midst of changing your question again, no, I'm not sure why it's called "linear." :-p But in order to rationalize the name myself, I look at the life story of a single value. For an intuitionistic value, this timeline can split apart across multiple parts of the program at once. A linear value can't typically branch, merge, begin, or end, unless it does so by transferring its content into new linear values. This makes its life story linear.

I find many of the interactive features to be over-engineered and/or under-documented. I've never been able to use the condition system to do anything useful. It's not apparent what compiler errors are recoverable and how to recover from them.

I dislike its package management. But this is a general criticism of most languages -- no matter how late-bound the language, there's a consistent temptation toward early binding for performance reasons (see the latter half of http://arclanguage.org/item?id=15587, ie footnote 1. Also http://arclanguage.org/item?id=12057)

Bottomline: while lispers like to say the world hasn't fully understood the power of 70's lisp, it's equally true that Common Lisp is stuck in the 70's and hasn't benefited from ergonomic advances of the past 20 years. It still uses UPPERCASE. These issues are certainly minor compared to having macros. But they carry some weight.

I don't have empirical evidence, but I think that the advantage of 80s basic for learning is that there are no hidden semantics that need to be known -- what you see is exactly what you get. No parameter passing, no local variables, no call/return (except in gosub, which is presented later), no blocks...Basic is - in a sense - like assembly language with math expressions and graphics.

I think this kind of technical simplicity can also be found in concatenative programming (no parameter passing, no local variables, no blocks, no math expressions) or point-free functional programming (no local variables, no blocks, nothing but math expressions).

On the other hand, people who want to try teaching those styles can probably fashion their own library in Factor or Haskell that sets up the foundation they want to teach from. Those languages don't impose a lot of syntactic cruft.

---

"...they make the computer seem mysterious and complex, since the child knows that if they write that code so and so will happen, but don't understand how it happens."

I'd say real understanding isn't so easy to find among programmers.[1] Between JIT, predictive branching, optimizing compilers, and other such supercharging platform features, application programmers have long been willing to sacrifice an easy-to-understand code-to-mechanism mapping in favor of performance, even for imperative code.

Most of our high-level languages do leave things mysterious at some point, and in doing so, they save the programmer from peering into the insanity-inducing complexity underneath. :-p

Imperative languages have mystery as much as other languages, but sometimes their mystery matches up with the "how" questions we raise: "How does the computer begin to show a graphic on the screen" might be taken care of a single "begin to show a graphic on the screen" statement.

I think the "how" questions could be less imperative and more reactive: "How is the computer showing a graphic on the screen?" "How does the system know where the mouse is pointing?" In a reactive programming model, these questions could have corresponding mysterious primitives: "Sustain a graphic on the screen." "Watch where the mouse is pointing." When a programmer builds compound demands in terms of these, what else could it mean but to hold more than one demand at the same time? Concurrency is the intuitive default.

I think a programmer who can't say "how" a program works... simply hasn't built up a significant enough program to be worth explaining. They're still using mainly the primitives, whose "how" is mysteriously unanswered by the language.

Don't get me wrong, I'm happy you're making a language to help people teach programming, and I even think you're on a fine path sticking with the more mainstream and classic kinds of programming. I'm just hoping to debunk some assumptions and expand your horizons, either so you can change your direction or be even more confident where you are. :)

When I was talking about "less mysterious and complex" I wasn't thinking about how a real CPU works, but how a programmable machine works.

My idea is that a language too high-level would make the child think of a computer as something having human-like behavior, that the computer just "knows" what to do when you talk to it. On the other hand a programming language with a well-known execution model and few assumptions would mean the child can know early how to map a language's syntax to its semantics.

Let's compare two programming languages for children, Logo and Kalimat. To print "hello" ten times in Logo:

repeat 10 [print "hello]

Here the child has to either assume the computer 'just knows'/be told "you'll understand later", or otherwise has to take some conceptual leaps in order to understand what's really happening:

* The stuff between [] is code not to be executed yet, a description of a future action to be applied

* This code is given to 'repeat'

* 'repeat' would apply this code as many times as needed

Now in Kalimat:

label top

x = 0

print "hello"

x = x + 1

if x<10 : goto top

Assuming the child already knows assignment 'if', and 'goto', the code here can be analyzed, traced, and adapted, without having to know little more than what is already known.

When the student becomes more confident, they can learn about 'for' and 'while', while being told that under the hood, those high-level constructs are pretty much the same as the previous version.

  label top
  x = 0

(By the way, here are some terse formatting instructions: http://arclanguage.org/formatdoc. You can edit a post for about an hour after making it if you need to experiment.)

---

"When I was talking about "less mysterious and complex" I wasn't thinking about how a real CPU works, but how a programmable machine works."

Where do you make the distinction? These days I like to think of a machine as being any tool at all, and programming as being a way to control that tool by writing specification documents of what it should do.

If someone begins with no understanding of how a computer could work, maybe it makes things plausible to think there's an underlying mechanical device like a player piano reading a sequence of triggers off of a tape. But rather than focusing on that tape, I'd like to think it's also plausible to ground understanding in terms of writing the blueprints to that machine (parts of which may be tapes if necessary).

http://en.wikipedia.org/wiki/Analogue_computer

http://en.wikipedia.org/wiki/Fluidics

http://en.wikipedia.org/wiki/Electronic_circuit

I won't claim my own intuition works this way, but obviously some people work with this stuff. :-p

---

"Let's compare two programming languages for children, Logo and Kalimat."

I actually think you have apples and apples there and you're not comparing them as such. In Logo, if you want to solve 'repeat 10 [print "hello]' in a more complicated way, you can:

  to greet :numberoftimes
    if :numberoftimes > 0 [
      print "hello
      greet difference :numberoftimes 1
    ]
  end
  
  greet 10

I think both this and your Kalimat example involve more conceptual leaps than the "repeat" version. Here are just a few of them:

* The stuff after Kalimat's : or inside Logo's [] is code not to be executed yet.

* This code is given to "if".

* "if" applies this code if appropriate.

Beyond this, the student is challenged to understand user-defined names, arithmetic simplification, operator precedence, sequential execution, and bounded scope (in Kalimat's case, the kind of dynamic scope that arises from variable assignment).

* How many concepts must be taught to the child before writing useful programs?

* How many of those concepts are right there, and how many are hidden?

I intuitively feel that Kalimat's example is just more simple and direct than the Logo equivalent, but "intuitively feel" and "just more simple..." are not very scientific; perhaps when Kalimat is more experimented with, we could have more empirical results.

But at least I can try to justify my feelings a little:

* The Logo example seems more nested while the Kalimat one is flatter. "Do this task, then that one" seems easier to keep inside one's head than do this task, which is made out of so and so.

* The Kalimat example completely avoids the need to teach function definition and invocation.

* The Logo code needs a discussion about variable scope and function activations; how :numberoftimes has a value in this greet different from that greet.

Yes, in Kalimat we'd have a related discussion about mutable variables, but a variable is an isolated concept that doesn't need to be explained alongside invocation and scope (at this level).

Surprisingly, I actually see your point. :-p For languages where "stepping" even makes sense, you can focus on explaining a very narrow window of the program, and then explaining another very narrow window of the program, following those steps. If the language is based on GOTO and simple variables, then the state of the program at any window is of a constant size, rather than structured into a stack or a tree.

Concatenative programming would pretty much lead to a stack-like growing and shrinking state right away (regardless of whether it takes shape as an actual stack).

The kind of programming I guess I'd call combinatoric programming (point-free functional, such as arithmetic expressions) would lead to a tree of partial results.

These systems could probably be broken down into even simpler subsets to limit the complexity, but probably not in a way that feels as open-ended as GOTO.

---

"The Kalimat example completely avoids the need to teach function definition and invocation."

The Logo example avoids the need to teach label definition and jumping. :-p

There's a little hiccup in understanding functions when GOTO's around. When I studied C after Applesoft BASIC, I remember being confused and curious about what would happen if I did a goto from one function to another. That's just an erroneous use of C's goto, and I think when I found that out, I got a little frustrated that anyone would program in a language with such arbitrary limitations. :-p

---

"Yes, in Kalimat we'd have a related discussion about mutable variables, but a variable is an isolated concept that doesn't need to be explained alongside invocation and scope (at this level)."

In case you missed it, the converse is also true: Invocation and scope could be introduced without introducing mutability. I can't say it's easier to do that, but it's at least independent.

Once again, I think there's some negative interplay between concepts here. Classmates from my intro programming classes found it nontrivial to grasp that passing a value to a function wasn't just the same thing as assignment to its parameter. That flawed interpretation worked pretty well until recursion came along. Even then, it was easy to repair the understanding by assuming it's a temporary assignment that reset itself after the function ended. Then they'd run into trouble when lexical scope came along, but... you know, that never came along. XD

Once I was out of the intro programming classes, they switched the intro curriculum from Java to Python. That means lexical scope might actually become relevant to the students before they learn another language... so I'm actually very glad in hindsight.

Eep, I'm not sounding like a history lesson of some sort, so for reference, my college days were 2005-2009. >.>; Pretty recent... but not as recent as I'd like, lol.

---

"But again, you've made me look again at my assumptions and those of the Logo creators, and ask myself again and again about those assumptions; and that's definitely a good thing :)"

It reminds me of Logo, which was also targeted at youth, and included graphics primitives. I used it myself to create some drawings to help us hang pictures :=)

This way, the student can go directly into writing interesting programs using only if/goto/expressions. Later they can learn more disciplined forms like procedures or OOP, also included.

I know I've seen comments here about integrating it with a db. Anyone remember the article # or URL?

Option 1. Use Fleetdb.

http://fleetdb.org/. Here's some rudimentary code to make this work with anarki: http://blackstag.com/fleetdb-client.arc

Option 2. Use any database that supports an HTTP interface.

You can use the anarki http library here: https://github.com/nex3/arc/blob/master/lib/http.arc along with something like Riak or Couchdb, etc.

Option 3. MySql over HTTP:

http://code.nytimes.com/projects/dbslayer

Option 4. Get the FFI working to access MySql and other more traditional database systems.

I had actually come up with an interview question: take two strings and return the index of the first difference. I posted some arc code, and wanted to have people run it.

My code:

  (def first-difference (seq1 seq2 (o comparer is))
       "Returns the index of the first difference between seq1 and seq2, or nil if
        they are the same. This function uses 'comparer as the function to compare
        elements of the sequences."
       (withs (len1 (len seq1) ;; 'withs ([var1 val1]* ) binds each var to its val
               len2 (len seq2)
               helper (afn (seq1 seq2 pos) ;; 'afn makes an anonymous function that
                                           ;; can recurse by calling 'self
                           (if (is len1
                                   len2
                                   pos) ;; at end of both sequences
                               nil
                             (or (is len1
                                     pos)
                                 (is len2
                                     pos)) ;; at end of only one sequence
                             pos
                             (~comparer seq1.pos
                                        seq2.pos) ;; found a difference
                             pos
                             (self seq1 seq2 (+ pos 1)))))
               (helper seq1 seq2 0)))

  (def first-difference(seq1 seq2)
    (if (or no.seq1 no.seq2
            (~iso car.seq1 car.seq2))
      0
      (+ 1 (first-difference cdr.seq1 cdr.seq2))))

  (def first-difference(seq1 seq2 (o pos 0))
    (if
      (and no.seq1 no.seq2)
        nil
      (or no.seq1 no.seq2 (~iso car.seq1 car.seq2))
        pos
      'else
        (first-difference cdr.seq1 cdr.seq2 (+ pos 1))))

  (def first-difference(seq1 seq2)
    (if
      (and no.seq1 no.seq2)
        nil
      (or no.seq1 no.seq2 (~iso car.seq1 car.seq2))
        0
      'else
        (only.+ (first-difference cdr.seq1 cdr.seq2) 1)))
 
  (test-iso "" 0 (first-difference nil '(34)))
  (test-iso "" nil (first-difference '(1) '(1)))
  (test-iso "" 1 (first-difference '(1) '(1 2)))
  (test-iso "" 3 (first-difference '(1 2 3 4) '(1 2 3)))

  arc> (def 1st-diff (str1 str2 i)
    (if (or (is str1 "") (is str2 ""))
      0
      (or (> i (- (len str1) 1)) (> i (- (len str2) 1)))
      i
      (no (is (str1 i) (str2 i)))
      i
      (1st-diff str1 str2 (+ i 1))))
  #<procedure: 1st-diff>
  arc> (1st-diff "" "" 0)
  0
  arc> (1st-diff "" "" 0)
  0
  arc> (= astring "abcde")
  "abcde"
  arc> (= bstring "abced")
  "abced"
  arc> (1st-diff "" "" 0)
  0
  arc> (1st-diff astring "" 0)
  0
  arc> (1st-diff "" astring 0)
  0
  arc> (1st-diff astring bstring 0)
  3
  arc> (1st-diff bstring astring 0)
  3
  arc> (1st-diff astring "abcdef" 0)
  5
  arc> (1st-diff "abcdef" astring 0)
  5
  arc>

I'm used to seeing it as the empty string

Here is the expanded M version (! = LF):

  IF X=Y WRITE !,0 ; No difference
  ELSE  FOR I=1:1 IF $EXTRACT(X,I)'=$EXTRACT(Y,I) WRITE !,I QUIT

Otherwise, go through each character of the strings and at the point where they differ, print that index and quit looping.

  I X=Y W !,0 ; No difference
  E  F I=1:1 I $E(X,I)'=$E(Y,I) W !,I Q