> 2) It would be extremely bad to write code that depends on unquoted symbols getting treated as quoted symbols when they happen to be unbound.

Yes. It would be a sort of inadvertent variable capture and could be the cause of some insidious bugs. Not any more insidious than the bugs you can get from an abuse of unhygienic macros or mutation though.

> (It'd probably be a good idea to make a global-assignment thing called something like "var" warn when assigning to an already-bound variable; currently this is called "safeset", and "def" and "mac" are implemented with it.)

That is a good idea! How about warning you when you assign to a variable that existing code depends on? For example:

  > (def foo () (a b c))
  #<procedure: foo>
  > (= a 5)
  Warning: Oh no, what are you doing? You're either
  being really clever or you forgot that your previous
  code depends on this variable being unbound. Why
  didn't you just use the ' operator? It's only one
  character, for god's sake.

> If, say, you use (a b c) to mean the list '(a b c) in your code, and then you test things out at the REPL, you'd better be careful not to give "a" a value, or it'll break your code. I think the cognitive overhead of worrying about that far outweighs the cost of using the ' operator.

I think the warning we just talked about could go a long way toward eliminating this cognitive overhead. Don't worry about it, just deal with it if you get the warning. Or don't use it in the first place because...

The proposed changes are backwards-compatible with Arc 3.1, since all they attempt to do is provide sensible defaults for things that presently raise errors. I want to emphasize that the goal here is not to eliminate or replace quote and quasiquote. Rather it's to enhance these operators by giving you the ability to use them implicitly sometimes, instead of requiring that you be explicit in cases where it's the only useful meaning possible.

Sometimes you do that on purpose, though. E.g. you define a function that relies on a second function, and define the second function later. (rocketnia gives an example, but I'll proceed with this one anyway.)

  > (def mod-expt (a n m)
      (fast-expt-* a n 1 (fn (x y) (mod (* x y) m))))
  #<procedure: mod-expt>
  > (def fast-expt-* (a n one *)
      (xloop (a a n n tt one)
        (if (is n 0)
            tt
            even.n
            (next (* a a) (/ n 2) tt)
            (next a dec.n (* a tt)))))
  Warning: ...

> The proposed changes are backwards-compatible with Arc 3.1, since all they attempt to do is provide sensible defaults for things that presently raise errors.

It is nice when you can introduce something without breaking old things. However, I think this thing is bad: it's fragile and shallow, and I think most programmers would just not use it and resent the time it took to understand it.

Imagine if, say, whenever mathematical operations (e.g. sqrt) were called with a list argument, then, instead of throwing an error, the function was instead applied to the car of the list. (Mapping it over the list is more likely to be useful; applying it to the average is also possible.) Or if the expression (a < b) evaluated to (< a b) when "a" evaluated to a number or anything else passable to the < function. Or if, whenever you used the variable "it" inside a then-expression in a call to "if", and "it" was otherwise unbound, it bound "it" to the if-expression (as in "aif")? Or why not all of these at once, and more?

In principle, you might be able to ignore extra little "features" like this. I think it'd annoy me, though--in the case of "sqrt" et al. being applied to lists, I'd probably think about it every time I dealt with math and lists (which I do a lot). It adds one more case to deal with to mentally evaluate any mathematical function call. The best thing that could happen is that I'd never use it, or encounter it in anyone else's code, and my mind would freed of the impulse to worry. But even if it was never used in correct code, I'd still have to think about it whenever I made a mistake and had to diagnose a problem.

By the way, perhaps you are just looking for something you'd use at the REPL, instead of a new language feature. Maybe a REPL with capabilities like Clisp's:

  [1]> (+ 1 achtung)
  *** - SYSTEM::READ-EVAL-PRINT: variable ACHTUNG has no value
  The following restarts are available:
  USE-VALUE      :R1      Input a value to be used instead of ACHTUNG.
  STORE-VALUE    :R2      Input a new value for ACHTUNG.
  ABORT          :R3      Abort main loop
  Break 1 [2]> 

> Not any more insidious than the bugs you can get from an abuse of unhygienic macros or mutation though.

> Sometimes you do that on purpose, though. E.g. you define a function that relies on a second function, and define the second function later. (rocketnia gives an example, but I'll proceed with this one anyway.)

  > (def mod-expt (a n m)
      (fast-expt-* a n 1 (fn (x y) (mod (* x y) m))))
  #<procedure: mod-expt>
  > (def fast-expt-* (a n one *)

I can't remember the last time I intentionally wrote code that didn't conform to this principle. And if you're willing to write your code this way, then it eliminates the largest problems you all have identified with quote inference. But you and rocketnia seem to place value on being able reference functions before they've been defined. So while I would be willing to trade that ability for implicit quotation, it seems you would prefer the reverse.

Have I misrepresented your views here?

---

[1] http://awwx.posterous.com/how-to-future-proof-your-code

> But you and rocketnia seem to place value on being able reference functions before they've been defined. So while I would be willing to trade that ability for implicit quotation, it seems you would prefer the reverse.

It is true that I'd prefer being able to permute my definitions over having implicit quotation, but that's by far not my only reason for disliking the idea. As I said, it's a fragile, shallow add-on feature that would confuse me (by giving me weird results for erroneous code) and that would make code harder for me to reason about (whenever I see a variable reference, either that refers to something that's been defined, OR it refers to a symbol! and I can establish the latter only by ensuring that it's not defined anywhere).

By the way, why do you think this feature is a good idea? I find two quotes that seem to suggest your reasoning:

1. "My sense is that something like this would rate highly in both complexity of implementation and convenience for programming." How do you get this sense? Do you write a lot of code that uses quoted symbols? Let's take a count in my big fat Arc file:

  $ grep -o "'" a | wc -l
     107
  $ egrep -o "[^' ()]" a | wc -l
   31362
  $ egrep -o "[a-zA-Z-+$*/]+" a | wc -l
    8261

You can do it by extending one of them:

  (def foo ())
  (def bar () (foo))
  (extend foo () t (bar))

> add-on feature

Not sure what you mean by "add-on". Is xs.0 list access an add-on? This is whatever that is.

> why do you think this feature is a good idea?

It could unify the "." and "!" ssyntaxes to some degree by allowing table access with h.k instead of h!k in most cases. alists that you presently have to express with '((x 1) (y 2)) could be contracted to (x.1 y.2).

I haven't worked out all the useful applications of this yet but I'm finding it interesting and think there's some potential.

> fragile, shallow [...] Tacking things on just because you can isn't a good idea.

I'm sorry, my intent wasn't to insult you. (I'm glad you explained that, though.) I thought my words were clear. Let me explain:

I called the idea "fragile" because it would be easy to break code that depended on it--just by defining a new variable. You suggested a thing that would warn upon defining a previously-used-as-unquoted-symbol variable, but rocketnia and I brought up cases where the warning would be a false positive. I considered a more sophisticated warning system--one that had some kind of mechanical procedure for guessing whether an unbound variable was supposed to be an unquoted symbol or a function to be defined later--one that required the programmer to follow one naming scheme for unquoted symbols and another for functions to be defined later. My conclusion was that for this system to work, the programmer would basically have to tiptoe around it and be very careful, or else things would break. Hence, I thought the word "fragile" was appropriate, and used it.

I called it "shallow" because the maximum benefit, in the best possible case, is that you don't have to type the ' character most of the time. Contrast this with, say, learning to use lists when you're used to naming every single variable. Not only does it become easier to, say, compute the sum of the cubes of five numbers, but you can write a single function to compute the sum of the cubes of any number of numbers! And then you can make matrices--again, referencing them with a single variable--and writing a single function to deal with n x n matrices for all n! Without lists or other compound data structures, it'd seem really hard and annoying just to deal with 2 x 2 matrices, and 10 x 10 would seem impossible. There are deep and rich benefits to using compound data structures. But the only thing this unquoted symbols idea can possibly be good for is letting you omit the ' character; I therefore thought the word "shallow" was a good descriptor.

Regarding "Tacking things on just because you can isn't a good idea". Your motivation seemed like it might be, at least partially, something like this: "We can get strictly greater functionality out of Arc if we take some things that are errors and assign them a meaning in the language. Therefore, we should do it. Let's start looking for error cases we can replace with functionality!" Your comment "I just think we're missing out on such valuable real estate here!" added weight to this interpretation. And so I attacked it with a reductio ad absurdum, giving several examples of how one might "add functionality" in this way. I hoped to show that the line of reasoning "You get strictly greater functionality, therefore it can't be a bad idea" was wrong. I summed it up by saying "Tacking things on just because you can isn't a good idea."

> I'm not saying this is a better way to define mutually recursive functions. Just pointing out that it's possible.

And there are other ways to do it[0]. But it would be impossible to do it by just writing (def foo () (bar)) and (def bar () (foo)) and putting them in the right order. Hence, this idea would make such programs more complex/verbose. (Eh, perhaps you could set up a warning system and teach it to recognize mutual recursion. I think learning about this would distract the programmer somewhat, which isn't by itself a deal-breaker but is an undesirable aspect. I suspect there are more cases yet to be covered; and you'd still have to order your definitions properly--I don't think a compiler without psychic abilities could always tell what you were going to define later; and even if you were warned when you made a function with a conflicting name, it'd be annoying to have to give your function a new name, or to change the code that used that name.)

> alists that you presently have to express with '((x 1) (y 2)) could be contracted to (x.1 y.2).

Incidentally, I do find it annoying to type out a lot of such things, and I have a routine for dealing with that. Perhaps you'd find it sufficient? So whenever I want to make a big alist, I do something like this:

  (tuples 2 '(Jan 1 Feb 2 Mar 3 Apr 4 May 5 Jun 6
              Jul 7 Aug 8 Sep 9 Oct 10 Nov 11 Dec 12))
  ;instead of '((Jan 1) (Feb 2) ...)

Oh, and, by the way, if ssyntax were implemented at the reader level--which I think it should be; I think the current situation is just a hack that will be changed eventually--you could write '((x 1) (y 2)) as '(x.1 y.2). [I see you address this in a sister post.]

And this example suggests that your intent goes beyond merely having unbound symbols evaluate to themselves. In my post I cite at the top of this thread, I addressed problems with trying to have ((x 1) (y 2)) evaluate as '((x 1) (y 2)).

> make arc programs shorter.

That is a worthy goal, one I'd forgotten about. It's good that you brought it up. I suppose the shortness of a program is kind of a good static measure, whereas objections like "It'd confuse me" are usually only temporary, and the programmer gets used to it.

But I do believe that a) using it would create either horrible risks of breaking things or annoying false-positive compiler warnings, b) therefore I'd never use it, so it wouldn't actually make my programs any shorter, and c) it would, inevitably, make debugging harder--instead of UNBOUND-VARIABLE errors I'd get diverse results, depending on precisely what happens when a symbol is put in the place of the unbound variable.

Now, (c) also applies to having xs.0 list/table/string reference work. But (a) and (b) don't. I do use it[1], and relying on it doesn't cause any problems like the fragility I've described. And the payoffs are pretty good. Many things are significantly shorter--e.g. m.i.j for reaching into a matrix, instead of (aref m i j) or, worse, (vector-ref (vector-ref m i) j).[2] The error-obfuscation objection still applies, but I think the benefits override the objection.

[0] I was thinking you could define them in the same lexical context:

  (with (foo-val nil bar-val nil)
    (= foo-val (fn () (bar-val))
       bar-val (fn () (foo-val))
       foo     foo-val
       bar     bar-val))

Is there a general interest in moving ssyntax functionality to the reader? I've found some past discussions about it but wanted to know the present sentiment.

In education, lots of the time new topics are built upon the foundations the old topics provided, but sometimes they're built upon established motivations and provide all-new foundations, like an analysis course justifying the calculus courses that came before it, or a mechanics course casting Newton's laws in a new light.

One reason to favor the dependencies-first approach in Arc is that we have mutability.

If you're only doing single assignment and side effect-free programming, then your code doesn't have a significant order [1]. But insofar as your program is imperative and performing mutations, the order is significant.

A consequence of this is that if you want to be able to take advantage of imperative features, you're making it harder by ordering your code any other way. I say this because even if your code is purely functional right now, when you try to insert some imperative code later, the order is going to start mattering more. And it's going to start seeming tangled and confused if it doesn't build up in the order of execution (at least it does for me).

So dependencies-first programming plays especially well with imperative code. I'm also particularly interested in it at this moment because I'm working on a refined auto-quote mechanism that could be hard to take advantage of if you're not programming this way. ;)

---

---

[1] http://github.com/awwx/ar now keeps tests in a separate file. Does that weaken the case for defining things before using them? Perhaps you could define your tests bottom-up but write your code top-down, or something.

I still want to try out a test harness that analyzes dependencies and runs tests bottom-up: http://arclanguage.org/item?id=12721. That way you could write your tests in any order and they'd execute in the most convenient order, with test failures at low levels not triggering noisy failures from higher-level code.

Not by design, as it happens. I wrote some new tests for code written in Arc, and stuck them into a separate file because I hadn't gotten around to implementing a mechanism to load Arc code without running the tests.

Though I do view writing dependencies-first as a form of scaffolding. You may need or want scaffolding for safety, or because you're working on a large project, or because you're in the midst of rebuilding.

Does that mean that you always need to use scaffolding when you work on a project? Of course not. If you're getting along fine without scaffolding, then you don't need to worry about it.

I have effectively the same resolution, but only 'cause of Not Invented Here syndrome. :-p Nah, I use plenty of libraries; they just happen to be the "libraries" that implement Arc. I use all kinds of those. :-p

---

http://github.com/awwx/ar now keeps tests in a separate file. Does that weaken the case for defining things before using them?

That file is loaded after the things it depends on, right?

---

...you could write your tests in any order and they'd execute in the most convenient order, with test failures at low levels not triggering noisy failures from higher-level code.

I'm not sure I understand. Do you mean if I define 'foo and then call 'foo in the process of defining 'bar (perhaps because 'foo is a macro), then the error message I get there will be less comprehensible than if I had run a test on 'foo before trying to define 'bar?

---

In any case, aw's post mostly struck me as a summary of something I'd already figured out but hadn't put into words: If a single program has lots of dependencies to manage, it helps to let the more independent parts of the program bubble together toward the top, and--aw didn't say this--things which bubble to the top are good candidates for skimming off into independent libraries. If you're quick enough to skim them off, the bubbling-to-the-top can happen mentally.

Lathe has been built up this way from the beginning, basically. It's just that the modules are automatically managed, and it acts as a dependency tree with more than one leaf at the "top," rather than something like yrc or Wart with a number on every file.

I'm interested in making a proper unit test system for Lathe, so we may looking for the same kinds of unit test dependency management, but I'm not sure yet about many things, like whether I want the tests to be inline or not.

If bar depends on foo (foo can be function or macro), and some tests for foo fail, then it's mostly pointless to run the tests for bar.

---

"That file is loaded _after_ the things it depends on, right?"

Yeah well, you gotta load code before you can run the tests for it :)

My understanding of aw's point was this: if you load your code bottom-up, then you can test things incrementally as you define them, and isolate breakage faster. Defining the tests after their code is irrelevant to the argument because it's hard to imagine an alternative.

Something I've been thinking about, though I haven't implemented anything yet, is that there's code, and then there's things related to that code such as prerequisites, documents, examples, tests, etc. The usual practice is to stick everything into the source code file: i.e., we start off with some require's or import's to list the prerequisites, doc strings inline with the function definition, and, in my case, tests following the definition because I wanted the tests to run immediately after the definition.

But perhaps it would be better to be able to have things in separate files. I could have a file of tests, and the tests for my definition of "foo" would be marked as tests for "foo".

  > (def maximin (x) (check x number (apply max (map minimax x))))
  #<procedure: maximin>
  > (def minimax (x) (check x number (apply min (map maximin x))))
  Warning: Oh no, what are you doing?
  > O_O

I still plan to do 'production' work with wart, so I'm going to try not to go off the deep end. One experiment at a time, try to falsify new changes as fast as possible, etc.

Well, that's both sides of the issue I'm talking about. I do want people to put pressure on each other's experiments, but I expect to promote that by reducing the difficulty involved in migrating from one experiment to another. Unfortunately, I expect that'll also make it easier for people not to put any more than a certain amount of pressure on any one experiment.

  > O_O

> They're not compatible with Arc programmers who want to get those errors. Not all errors signify places where extensions can roam free.

Thank you so much for saying that. XD All of it. I feel the same way, but I can't help but think I'm just out of my element and resistant to change. Managing a single character here or there isn't a trivial problem, but I personally don't care much either way as long as I can also manage my code well on a large scale. Good to know I'm not the only one.

---

...warn when assigning to an already-bound variable...

Maybe. :) These ideas aren't totally unheard of or untested. For example, PicoLisp has auto-quote for the number case, and it's been around since the 80s! XD

From http://software-lab.de/doc/faq.html#advantages:

Instead of '(1 2 3) you can just write (1 2 3). This is possible because a number never makes sense as a function name, and has to be checked at runtime anyway.

But all this is just gut instinct. I'll try anything once :)