You may be interested in my "autocall" patch, that I'm renaming to "defvar" and will be writing some documentation for soon. It extends the Arc compiler to allow you to provide your own implementation for a global variable, so that you can type "a" instead of "(a)".
Sounds vaguely similar to symbol macros, which are a more general facility in which you can make a given symbol expand to arbitrary code (rather than just the invocation of a given symbol). I'd be more interested in seeing a hack to implement those in arc; you can do some interesting things with them
In this case, I honestly don't find that writing the extra parens is much of a bother to me (and I have used dynvars quite a bit in one of my projects) - it actually helps remind me "oh hey, this is a dynamic variable". It's like an enforced naming convention; "(foo)" is just as succinct as 'foo, really.
Edit: dammit, that's supposed to be 'foo surrounded by stars - the CL dynamic variable naming convention. Unfortunately it's misinterpreted as italics. Is there some way around this?
pg's convention in the arc code is to just have one star after the name, instead of one on either side. Less typing, and it doesn't conflict with italics ;)
I guess putting the definition of + on hash tables into arc.arc makes sense. Either one of the algorithms is a more high-level procedure than the other behaviors of '+.
I like this 'redef macro. How come it's not in arc.arc already? I can't think of any reason to not have this be there, along with the definitions of other key macros like 'do, 'def, and 'mac.
Now, granted, this function actually contributes nothing to <insert large project here, including news.yc>. However, I think that this function strikes at the heart of the "hackable language" mentality, by making it much easier to quickly modify behavior on the fly. It also sends a message to somebody reading the source code: "One of the key principles in this language is that you can redefine behavior across the board with one macro call."
I think that for quick exploratory patches, it's a better solution than editing the original definition. If I want to modify this behavior, I just have one self-explanatory (redef + ...) form to edit, rather than having to sift through the original '+ definition. It is a clean solution, which adds hackability to the language.
'extend is great! However, I think it's more of a great library/patch, than something that should sit with the core functions. 'redef, on the other hand, is elegant, transparent ('extend is transparentially challenged...), and works well for quick hacks. 'extend seems to me to be more suited for setting in stone some behavior sketched out with 'redef (for example, the method-like behavior that you described in the original thread about 'extend http://arclanguage.org/item?id=8895)
The example you have of + on tables uses the algorithm that conses up a storm...
One thing I like about 'extend is that I can run it several times while developing an extension without my older, buggy versions remaining on the call chain. Try actually hacking with this version of 'redef, it turns out to be really quite a pain!
oops, this 'redef is now calling the previous 'redef, I'm stuck!
On the other hand I don't think the explicit separation in 'extend between the test of whether to apply the extension from the implementation of the extension has turned out to be all that useful, so maybe some combination of the two would be better.
I tried storing a list as each value in the table, and popping or pushing to/from the list in redef and undef, so that you could step backwards more than just one definition at a time. However, that made the entire thing much more unstable, because it turns out that 'push and 'pop rely on a bunch of functionality which breaks quite easily if you're messing with the internals.
So, this stripped down version is my little 'redef/'undef hack-pack.
I do think that this would be a useful facility to add to Arc. I think it has high leverage, in that it is a small addition to Arc that has a lot of power: with this change several features can be written with code that can be loaded instead of needing patches.
pg tends to go for short names for function/macro names that need to be typed a lot. I was concerned that something as short as "mz" was too valuable an identifier to be used for something that most people wouldn't be typing.
But if people want to name it "mz" or something else that refers to MzScheme, that's fine with me!
Maybe it's just that I've been writing a lot of libraries, but I use 'mz a lot ;)
Besides, I can't imagine needing that combination of letters for anything else. Especially after having it refer to mzscheme for so long now, it would be hard to switch it.
It doesn't matter to me what it gets named as. pg could add it to the Arc core as %%%internal-compile-to-mzscheme, and I could still use "mz" in the code I write using load-w/rename or some similar mechanism.
I think that FOR NOW, 'mz is fine. Even though each person has their own patched-up Arc, it seems as though the vast majority are based on pg's Arc on mzscheme. If somebody is using an Arc built on some other VM (eg Rainbow on the JVM), then they will be putting quite different code within the body of an 'mz, so I don't think it's an issue that they'd also have to change the name of the symbol.
I completely agree ac-tunnel (or mz) is a useful facility to add to arc, especially to use for once-off hacks, or to figure something out before committing to an xdef. I wrote rainbow just so I could use java libs while playing in arc (among other reasons).
As the author, you get to name your babies. I didn't agonise over java-new, java-implement, or (ugh) java-static-invoke - they're mostly buried under arc macros in any case.
In the worst case, if there's a conflict with mz (or ac-tunnel), I would recommend an excellent symbol renaming hack available on github at http://catdancer.github.com/load-rename.html
Oh. That's good then. Whoops. :) Now I feel like a bit of a jerk for ragging on about the name so much.
I am completely in agreement that this is a good feature. When anarki was still up-to-date, I used '$ quite a bit, and this is essentially just a better version of that - one that lets you access local variables and has less overhead to boot!
Speaking of Anarki, what needs to be done to get it back up to date?
I haven't heard much about that. Maybe we should make a new thread about updating it?
You're comment about local variables made me think of something:
What if we used ac-tunnel, like CatDancer suggested, and then make mz defined as:
(mac mz body
(list 'ac-tunnel (cons 'quasiquote body)))
That way mz is defined in terms of the tunnel, and adds the feature of being able to unquote variables to pass the value through to scheme. $ had that feature, but called seval instead, so it lost the local environment. This way, we get the local environment, and the quasiquoting.
Making the entire thing quasiquoted doesn't work. That would mean that the following:
(let x 2 (mz (+ 2 ,x)))
Would compile down to this:
((lambda (x) `(+ 2 ,x)) 2)
Which obviously is not the intended functionality - it evaluates to the list (+ 2 2), not to the value 4. A quasiquote is, after all, a quote. So you'd need to add an 'eval outside it, and then you just have '$. Anyway, you already get the local environment with 'mz, because arc compiles local variables down to the same-named variables in mzscheme. That is to say, with 'mz as it currently stands, the following:
(let x 2 (mz (+ 2 x)))
Compiles into:
((lambda (x) (+ 2 x)) 2)
So it just works! This does depend on the internals of the compiler, but then, that's the whole point of 'mz.
As for getting anarki back up to date, I discussed that a little in the arc3 release thread. Ultimately I think it's a wasted effort to try and port the entire thing. Instead I think the best idea is probably to port the best features from anarki (the help system, the ability to define how to call tagged objects, various utility functions and macros) separately to arc3, a la CatDancer's "Sharing arc hacks", and publish them as a sort of stripped-down new anarki. I've done some preliminary work here; in particular, I've ported the help subsystem to arc3. I have a repository at http://github.com/rntz/arc, though I haven't uploaded the help changes to it just yet and it's very badly organized. I think I'll probably make a more concerted effort once arc3.tar has been finalized.
You are given the full literal contents of what appears in (foo ...) in args. You can do anything you want to with it, including searching for (unquote x) forms and replacing them with something else. (The reader expands ,x into (unquote x), so that's what you'll see when you look at args).
Now, what would you like this macro expression to expand into? Something like this?
(with (x1 a x2 b)
(ac-scheme (+ x1 (* x2 2))))
(yeah, this morning I'm trying out the name "ac-scheme" for "ac-tunnel" :-)
Now, let's say that expansion is what you want. So your next step is to write a program that takes
(+ (unquote a) (* (unquote b) 2))
and turns it into
(with (x1 a x2 b) (ac-scheme (+ x1 (* x2 2))))
this part has nothing to do with macros; you're just creating a list from another list.
arc> (myprogram '(+ ,a (* ,b 2)))
(with (x1 a x2 b) (ac-scheme (+ x1 (* x2 2))))
note that's a regular quote ' character there, not a backquote `. We're just feeding a regular old list to myprogram, and it's giving us a regular old list back.
OK! Got that program written? Now the macro is easy:
I'm aware of the process for making macros, however my attempts at writing the transformation itself have been rather unsuccessful.
What I originally tried was transforming
(mz (+ 3 ,a))
into
(mz (+ 3 __a))
But I couldn't get it to work. The problem is that I seem to need two levels of transformation and evaluation, but macros only give you one.
What is the reason that 'eval is so looked down upon? Isn't that pretty much what macros do themselves? How hard would it be to get an eval that could use the current local scope?
It should apply f to each node of the tree, preserving the tree structure. Seems pretty useful to me, if you're going to be doing much transformation of arbitrary list structures.
In retrospect, it's not much good for transforming the structure, but great for many other things.
For everything else, the other tree functions like trav, treewise, ontree, etc. might be more useful.
How would you make a more general purpose version of maptree that could encompass your transform function? It needs to be able to selectively modify some branches of the tree, and avoid calling map on them. Mine can't do that because it restricts you to atoms. Maybe if it took in two functions, one that operates on atoms, and one on lists, which can then optionally continue the mapping on a subnode. But then what has been gained? You're pretty much writing the whole pattern over again anyway.
Hmm. There seems to be a pattern here, but I can't see how to abstract it out, beyond the somewhat useful but restricted maptree posted above. Ideas?
'eval is not particularly looked down on, it's just rather inflexible. Given the way arc works, an 'eval that can use the current local scope is impossible - because arc compiles down into scheme, and an 'eval in arc compiles down to an 'eval in scheme, and an 'eval in scheme cannot use the current local scope. In order to get such an eval, you'd need to rewrite arc as an interpreter.
Edit: Unless mzscheme itself has some special 'eval with that functionality. I didn't think of that. I don't think it does, though.
parser.arc (at least, the parser lib I wrote; there are at least two others in anarki) uses coerce to get corresponding atoms from the token in string form. 'eval would probably have worked too. Otherwise, parser.arc just returns the forms it reads in, only transforming bracket syntax and string interpolations.
The ac-global-name function will prefix a symbol with an underline for you to create the global symbol name.
As for getting at the lexical scope, with eval or something else, that's up to the Scheme implementation. You'd need to go looking in the MzScheme documentation, or ask on the MzScheme user mailing list, to find out if there's some way to do that. Though why would you need that for this?
So, anyway, when you say you couldn't get "it" to work, what is the "it" that you're trying to do? I thought you were saying you didn't know how to transform "(mz (+ 3 ,a))" into "(mz (+ 3 __a))", but you say you know how to do that, so what is the it that's not working?
Actually what's funny is that I had no idea that people were using it! I mean, it's not like a web application where I can see how many how any hits it's getting, right? I just throw this code out there and it drifts away into the Internet, never to be see again. Then I suggest naming it something else and people say "no! no! Don't take away my mz!" :-) Oh. I guess some people found it useful! ^__^
You'll get an "tcp error writing" when in your browser you navigate away from a page before the browser has finished downloading everything.
Usually Arc sends the entire response and closes the connection. When the browser doesn't wait, it closes the connection. Arc is then complaining that the connection has been closed on it.
If you're absolutely sure that everything has been loaded in the page before you click on anything, that would be something to look into further.
(def match-json-string ()
(match-char #\"
(liststr:accum a
(while (~match-char #\")
(atend-err "missing close quote")
(a (or (match-json-backslash)
(readc (stdin))))))))
I took shader's suggestion to use an input-port with peekc and readc. I figured that using a parameter to keep track of the parsing state was the way to go, and then realized that we already had a parameter (stdin) which did everything I needed it to. And taking Adlai's suggestion I made match-char a macro which avoids the extra fn's I had in the original.
Getting the parse position out of the function really broke the logjam. The old version was doing two things at once -- keeping track of the parse position and matching characters -- and that made it really hard to pull out bits of functionality into their own functions to make the main function shorter and clearer. Now it's much easier to tell what the function is doing! :)
Here's the rest of the code:
(def hexdigit (c)
(and (isa c 'char)
(or (<= #\a c #\f) (<= #\A c #\F) (<= #\0 c #\9))))
(def json-unicode-digits ()
(let u (n-of 4 (readc (stdin)))
(unless (all hexdigit u) (err "need 4 hexadecimal digits after \\u"))
(coerce (int (coerce u 'string) 16) 'char)))
; json-backslash-char is the same
(mac match-char (c . body)
`(when (is (peekc (stdin)) ,c)
(readc (stdin))
,@body))
(def atend-err (msg)
(unless (peekc (stdin))
(err msg)))
(def match-json-unicode-escape ()
(match-char #\u (json-unicode-digits)))
(def match-json-backslash ()
(match-char #\\
(atend-err "missing char after backslash")
(or (match-json-unicode-escape)
(json-backslash-char (readc (stdin))))))
(def liststr (l)
(coerce l 'string))
As you can see there's some more that could be done: a next-char function for (readc (stdin)) perhaps, and match-json-unicode-escape is so simple now that it could be inlined. That will be easy to work on :-)
According to arcfn.com and the code that it references, it looks like it's supposed to default to stdin, but it's written in such a way to require at least one argument. If you pass nil, it reads from stdin because of the default.
Which requires at least on argument. It should probably be changed so that that argument is actually optional, but I don't know how. I should read up on my mzscheme ;)
I'm not certain - I haven't tested it, but it looks right. Unless scheme and arc have different ideas of false, which could cause more problems. Maybe it should just be:
hmmm. It seems that mzscheme (at least the version that I'm running arc on, 360) doesn't support optional args. I guess I'll have to do it some other way.
It's fairly tedious to be doing this in Scheme, isn't it? We might let Scheme handle implementing the low level readc, and then in Arc redefine readc to be a more advanced function that can take an optional argument:
(mac redef (name parms . body)
" Redefine a function. The old function definition may be used within
`body' as the name `old'. "
`(do (tostring
(let old (varif ,name nilfn)
(= ,name (fn ,parms ,@body))))
,name))
It's the same as yours, except (a) it suppresses the warning on re-assigning an identifier, and (b) it calls the original function old.
btw, why is there no peekb? Or any other functions that work on bytes?
As part of the design process of finding the shortest Arc implementation, pg is careful not to include functions that he isn't actually using for news. This leads to some surprises (car, cdr, cadr, cddr, but no cdar?) but also removes cruft that builds up implementing things that people might need some day but turn out not to.
It turns out not to be a problem, since Arc is so concise it's really easy to extend, and so people quickly implement the things they need that pg happens not to be using for news.
I find I prefer the Arc approach, since libraries that try to provide everything I might need often have so much stuff that ironically they make it harder to implement what I actually need.
If you need the change right away you can patch ac.scm yourself, or, if you don't mind waiting, pg will eventually have a new arc3.tar containing the update.
Side note: compare the "profiles" of this version, and the earlier versions -- this one has a much more "functional" profile.
I'm a bit confused what you mean about atend:err. Do you basically mean that there would be function composition between a macro, and a call to, for example, (err "missing char after backslash")? Something like
Do you basically mean that there would be function composition between a macro, and a call
Yes, a composition, though not a function composition. Because the Arc compiler rewrites (a:b ...) as (a (b ...)) when a:b appears in the first position in an expression, it works for macros also.
Thus
(atend:err "missing close quote")
expands into
(atend (err "missing close quote"))
which macro expands into
(unless (peekc (stdin))
(err "missing close quote"))
Are you able to offer all the data your competition does? $300/month would be half off ^__^
Also, is there some way to include the user's name in the downloaded data? No doubt they could edit the data to remove their name before passing it on to their colleagues, but that might be more trouble than it's worth to them.
no doubt they could edit the data to remove their name before passing it on...
I could as they do - watermark on PDF, but my model is to make the data more accessible/usable and in doing so I am making it easier to freeload, hence why I am trying to make it cheap enough that users would just sign up and pay the monthly fee and get the benefits of some customization, rather than freeload.
Also the real money isn't in the data it's in other things, but I need the audience so I'm going to offer it really cheap, get the audience, get them famliar with the software then sell modules (the real pot of gold). I could jut get it out there for free, but I need to learn this pricing/billing stuff.
That all sounds good, my only thought is if you are charging $5 for something your market is used to paying $600 for, you may appear to lack credibility... imagine going up to a guy who gets $100 haircuts and offering him a haircut for $3. Would he take the risk of getting a $3 haircut, even if it looks like it would be just as good as his usual $100 one? But, of course, if they're getting other stuff than just the data for the $600, offering just the data for $5 plus expensive modules might be perfect ^_^
If b is a lexical variable, eval won't help, since it isn't run in the lexical scope in which it's called. An Arc lexical variable will get compiled to an MzScheme lexical variable, and I don't know of a way to refer to an MzScheme lexical variable by name determined at run time. If there isn't, I suspect your only hope might be to modify the Arc compiler to generate code like
(case v
((a) (set! a n))
((b) (set! b n))
((c) (set! c n))
((d) (set! d n))
...
for all the lexical variables available at that point.
There are two classes of people: people who want to pay you, and people who don't.
Consider focusing all your attention on people who want to pay you. How can you make it really easy for them to pay? What do they want, and how can you do more of that for them? How do you find more people who want to pay you, how do you let them know you exist? Is your app so great that the people who want to pay you are telling their friends who want to pay you?
People who don't want to pay you don't make any difference whether they're using your service or not (you're not getting any money from them either way), unless there are so many of them that it's costing you something in server load or your time, in which case you can do something about it based in the specific pattern of abuse that you're getting.
It think that trying to force people who don't want to pay you to pay you is probably wasted effort, they'll just go away and not pay someone else :-)
(mac def-jsniffer (name c next (o arg 'cdr.j))
`(def ,name (j)
So we have three answers to the question, what to do if you're passing some state through a lot of functions, to the extent that passing it around is taking up as much code as what you're actually doing?
1. Put all the functions inside a surrounding lexical scope, and keep the state in some lexical variables that all the functions can access.
2. Write some macros so that you're still passing the state through all your functions, but the variables don't appear in your code.
3. Use some kind of dynamic binding (like Scheme parameters) so the functions can access the state without it having to be passed in as part of the function's arguments.
I think the question is what to do with the common operations such as moving the forward the pointer to what we're going to parse next. One way to do it is to keep state in a some surrounding lexical scope (similar to what lib/parser.arc in Anarki does):
I think that char-char is supposed to be next char ;)
Other than that, I'm not good enough yet at writing parsers in arc to help you much. Your original was already pretty short, though there were a few spots that you could have used the . operator more (car, cdr, etc.)
I am also working on a parser too, though, so whatever you discover will certainly be helpful ;)
You know, if you used a input-port instead of a string, then next-char and bump-j would just be peekc and readc. I don't know if that would make you're life much easier, but it's an idea.
Right, a port encapsulates some state (where you're reading from, what your position is in the input), and then Arc uses a Scheme parameter (stdin) so that the input port doesn't need to be passed to every function that calls some function that calls some function that calls read or readc. So like you say I could use an input-port if that did what I needed, or I could implement my own thing if that turned out to be necessary.
