I don't have much experience with Tcl, so I'm surprised to hear that it has macros. Can you point me to a description? I want to see what you mean by "macro system built in pure Tcl".
I've heard many criticisms about Tcl over the last 15 years, though, primarily having to do with everything being a string, from what I remember.
---
I certainly don't think Lisp is the end-all. For example, Forth has some cool capabilities much more elegantly provided (http://yosefk.com/blog/my-history-with-forth-stack-machines....). Factor basically has Lisp macros (http://arclanguage.org/item?id=19593). Haskell's pretty nice too. I particularly hanker after its automatically curried functions. Javascript has an eco-system we should all covet.
Past a point, languages don't matter. At some point you have to build stuff with the languages you know. So there's no point optimizing for the absolute best thing possible anywhere. Otherwise you risk becoming just a peanut gallery, bikeshedding because you cannot do.
But that's not an argument to not learn languages. Most people (including myself in the past) are led astray by some misplaced notion of efficiency, of thinking that the right language will make them faster programmers, better at their day jobs. Thinking this way leads to questions like, can I use this in production, will this replace my current language X, what is the value proposition, how many lines does it take to write this program. But learning languages is actually for resilience (or what Nassim Taleb calls anti-fragility). It's for having new ways of thinking about things as insurance against the day you run up against just the right sort of really difficult problem.
You can't prepare for an unknown future by working at it. That gets old quickly. Instead you treat it as play, you find ways to make it entertaining. In fact, that's why play evolved. Play is an evolutionary adaptation for anticipating unknown unknowns. Bear cubs play because it lets them practice skills that they'll never have a chance to fail at (because the one time they need it, they die if they don't have it).
So don't ask if a language is the be all end all. Languages are worth learning as a form of play, to gain breadth in the variety of rube goldberg machines (https://www.youtube.com/watch?v=GXrRC3pfLnE) it is possible to build using computers. Programming isn't about generating characters in the right syntax, following 'best practices'. Those trappings are not the essence. The essence of programming is being able to do things with many different (often unfamiliar) primitives, and about being able to bounce around between abstractions. Learning languages helps with both.
(My big beef with the world is that the way we do software today encourages people to live within rigid abstractions and so stifle their own potential: http://akkartik.name/post/libraries2)
If after learning a language it turns out to immediately be a good fit in production, in your day job, great! But that's just an unanticipated bonus, not the reason to do the learning.
I don't really build stuff in Lisp anymore :) What I'm interested in is teaching programming (http://akkartik.name/post/mu), that's why I continue to hang out here: for the chance to meet newcomers who are trying to learn something new, and to immerse myself in the world of someone unfamiliar with all this.
My Mu project is very far from a dynamic language. It's intended to replace C, so there's no garbage collection. There's not even any nesting expressions. Unlike Lisp where everything is an expression, in Mu everything is a statement. And yet, there are things in there that I picked up from Lisp, and which wouldn't exist if I didn't know about Lisp. Types in Mu are arbitrary s-expressions, because Lisp taught me that (map (address array character) (list number)) handles complexity more gracefully than map<string, list<number>>. That was an unanticipated benefit; I started learning Lisp long before Mu was a twinkle in my eye.
Those criticisms of tcl are true in the sense that yes, everything is a string, but unfounded in the sense that so what? Values are stored as strings so that they can be passed around as data and used immediately as code. But if you pass "5" to a command that needs to use it as an int, a conversion will take place, and the int value will be stored along with the string representation. The most recently used type of a variable is kept as the primary representation for efficiency (basically caching).
Here is a link to some docs about the tcl macro system, written by its author: http://wiki.tcl.tk/11155
From a practical perspective, I agree with what you say about "Past a point, languages don't matter. At some point you have to build stuff with the languages you know." Programming is about building stuff that works and is useful, and no matter how much theory you know, at some point you just have to sit down and type.
My motivation for evaluating and comparing the relative merits of languages is that programming, for me, is so much more fun when using certain languages. It's not about finding some holy grail of PL. It's about finding a language that is powerful and will pretty much let me do whatever I want.
I completely agree with your point about learning languages to keep learning languages. Even though I consider tcl to be more fun to program in than forth, my time writing forth code was extremely valuable to the way I think about programming.
I really like that analogy to bears.
Edit: The tclers wiki, to which I linked above, contains a massive amount of information on the language, programming in general, and many other topics, the likes of which I've never seen concentrated as well in another programming community.
"It's not about finding some holy grail of PL. It's about finding a language that is powerful and will pretty much let me do whatever I want."
I don't see the distinction. Isn't "let me do whatever I want" the holy grail of PL? :)
My point is that it's equally important to keep learning new things _to_ want. No single language will consistently do that.
"The tclers wiki contains a massive amount of information on the language, programming in general, and many other topics, the likes of which I've never seen concentrated as well in another programming community."
That is high praise! Feel free to submit your favorite links as new threads.
"..everything is a string, but.. so what? ..if you pass "5" to a command that needs to use it as an int, a conversion will take place.."
Here's another way to think about this. Types are useful but they're also speed-bumps. Dynamic languages push types to runtime so that you can run partially correct programs and gradually make them right. The drawback is that you can never be sure that your program doesn't have a type error; there can always be some code path that you haven't tested that causes everything to come crashing down. Passing just strings around makes this drawback exponentially worse: now you can't be sure your program is right even if you cover all possible code paths and your tests have perfect coverage, because type safety is a function of the lines executed and the data they rely on.
On a hunch I also took a look at Tcl's scope design (http://wiki.tcl.tk/12245) because that's something non-lisps (ahem, Python) mess up in subtle ways. And sure enough, Tcl is brain-damaged in this area. (I have sympathy for this mistake, because I too have spent time trying to "design hybrid approaches that strikes a great balance in maximizing the advantages of both lexical and dynamic scoping while minimizing the disadvantages of each": http://arclanguage.org/item?id=15137. Everyone should play with these things, but there's no known improvement on Lisp's dynamic scope and Scheme's lexical scope -- exactly as they're implemented there. Everyone should play with scope strategies so they can understand why.)
It looks to me like the confusion in that SO post is because Javascript does something sneaky in order to support several syntaxes that mean the same thing. Tcl has no syntax. Braces aren't used to delimit blocks and denote dictionaries. They're just used as an escaping mechanism, period. Once you know the language, it is always clear how the interpreter will treat your data.
About the holy grail quote... I should say it's not about __finding__ such a language, more like it's about __searching__ for such a language. Which involves constantly looking at different ones.
upvars is a terrible idea. You don't want capturing variables from the lexical environment to be something you have to explicitly ask for everytime. If you don't tend to use lexical scope all the time you end up with all sorts of action-at-a-distance bugs.
Newlisp is another language that uses dynamic scope by default. All it does is save a little bit of implementation effort. In every other way it's bad for users.
Interestingly, Mu plays this game as well. In Mu you have to explicitly specify what level variable you mean. But Mu explicitly isn't a high-level language. It's a language for building Lisp compilers in. And all Lisp compilers implement lexical scope under the hood using these tricks.
So what I'm saying is that Tcl is trying to avoid some work related to automatically managing lexical scope, and trying to pass this off as good for programmers.
But these comments of mine aren't as concrete as I usually strive for. I doubt you'll be persuaded by them. Just keep them in mind as you continue your search.
"..it's not about __finding__ such a language, more like it's about __searching__ for such a language. Which involves constantly looking at different ones."
