| (Apologies for making this a really rough draft. ^^; I'm finding the need to talk about Penknife, but I won't have time to actually introduce Penknife as comprehensively as I want to in the foreseeable future, so this is a hurried introduction to get things started. Feel free to ask for dozens of clarifications. :-p ) Penknife's a language I'm making, which is intended to be customizable. It has a very uniform syntax that embraces the sequence-of-characters nature of code, so that the majority of imaginable custom syntaxes will all naturally fall within its domain, thanks to being based on sequences of characters. This syntax... foo[text text text]
...compiles by compiling "foo" first to get a "parse fork", then forking it as an "op" using the body "text text text", which hasn't been parsed yet. The parse fork of foo will take care of the parsing itself.So, parse forks. Parse forks are intermediate values that represent all the different meanings of an expression in various contexts. The op context is one example, and others include the top-level command context, the standalone expression context, and the settable-place context (where Arc uses 'setforms). When a parse fork is forked in the op context, it's provided with the body as a sequence of characters, like "text text text" here. The parse fork parses that body manually according to its own behavior. Then the result is another parse fork, so that the operator form can be meaningful in various surrounding contexts. For instance, parsing foo[bar][baz] relies on the parse fork of foo[bar], which relies on the parse fork of foo... which is obtained by magic for our purposes. ^_^ This parsing method effectively allows several things all at once. It allows us to have operators that implement custom string syntaxes. (In fact, the string syntax I currently use in Penknife is q[This is a string.], and I'm not disappointed.) It also allows us to have macros. But more interestingly than that, it allows us to have metafns; if a custom parse fork is "op"-forked with one body, it can return another custom parse fork that consumes yet another body, and so on until it's quite bodiful enough for anybody. In particular, Penknife already copies Arc's 'compose metafn, which is the only one I really care about anyway. :-p == But... all text? == There's a caveat to foo[text text text] syntax. The body must have matched square brackets. Penknife automatically strips out comments before doing any parsing, so there's that too. I'm using semicolons now, like Arc, but it turns out I use semicolons when writing in English, which is important for the contents of strings, so I've been reconsidering that decision. == Brackets on the outside == The foo[bar][baz] syntax isn't that friendly when trying to indent big blocks of code, IMO. I'm used to Arc, where the parentheses are on the outside: ((foo bar) baz). So Penknife allows [[foo bar] baz], which is equivalent to foo[ bar][ baz]. Note the spaces in foo[ bar][ baz], so the other syntax is still necessary for the q[This is a string.] syntax and so forth. == Infix syntax == Also, Penknife supports a variant of ssyntax, which I've hinted at here and there on the forum. To implement that, there are two categories of identifiers in Penknife: Identifiers that include only letters, digits, and the characters in "+-/<=>", and identifiers that include only characters other than those, like . ! & : ' " , etc., with the exception of square brackets and whitespace. These are called "alpha" identifiers and "infix" identifiers respectively. The Penknife expression a!!!!b is parsed the same way as !!!![a][b]. In general, if the Penknife expression (not including parts inside brackets) contains an infix identifier it doesn't start with, the last such infix identifier is taken as the operator for the parts before and after it. The "doesn't start with" condition is important. It's just fine to have infix operators at the end: a! just becomes ![a][], taking the parts before and after like always. But if we try to have !a parse like ![][a], then that will parse like ![][[][a]], that will parse like ![][[][[][a]]], and we'll be caught in a loop. This automatically takes care of a commonly encountered shortcoming of Arc's ssyntax. You can finally say [foo a b c].d!e.[bar f] in a fully consistent way. Also, nothing else distinguishes infix identifiers from alpha identifiers. In Penknife, you can locally bind . and use it right away for a.b syntax. In conclusion, Penknife's syntax supports the equivalents of Arc's metafns and ssyntax in a totally consistent way, without hacking the language. It also requires no inherent literal syntaxes whatsoever, just operators that provide them. == Not very iconic == One criticism of Penknife's syntax is that it isn't homoiconic. A Penknife syntax operator accepts its body as a list of characters, which isn't very semantic. However, I believe with the right tools, dealing with lists of characters won't be too bad. The default function call syntax just breaks its characters up into words and has the words parsed individually, and I've already set up a macro system that invisibly uses that parser so that you don't have to worry about it as much. I'm even satisfied with quasiquoting. Here's my current definition of let: [mac* let [var val] body
qq.[[tf [\,var] \,body] \,val]]
(The tf operator is the most bare-bones kind of lambda.)In general, I think the conveniences homoiconicity provides can be regained through the accumulation of enough parsing utilities. | 
