Yeah, that whole side is a bit clunky. It gives readers feedback when things are broken so they aren't misled. But the job of the writer is still hard. Things would be improved by my last idea, to run examples along with other tests.
Here's a couple more issues:
a) Expected results are currently unevaluated, so there's no way to represent tables, and so on.
arc> (help counts)
[fn] (counts seq)
Returns a table with counts of each unique element in 'seq'.
Examples:
arc> (counts '(b a n a n a))
(obj b 1 a 3 n 2) <-- ERROR
b) We can't provide examples for non-deterministic functions like rand-elt.
c) We can't show stdout and stderr.
---
Last but not least, I've been thinking about the freezing idea myself in other contexts. I think there's something big there.
As for the 'freezing', I've been thinking about it in other contexts as well. I noticed that one of the companies I worked at seemed to want to do some of their regression tests that way, so it makes sense to provide a way to automate it.
In particular, I would like a way to convert an interactive console session (my normal method of testing code) into a set of unit tests. Not sure exactly what the right way to do that would be though.
It's still not a very good form of unit test, as it won't cover cases as mentioned above with randomness or side-effects. Mutation testing or fuzz testing / generators are probably better ways to generate unit tests, but they don't make very good examples.
Yeah. B can probably work just fine with dynamically evaluated examples, but stderr/stdout would require user interaction, which may not be acceptable. Maybe some way to indicate if an example should be evaluated or not?
What do you mean by 'run examples along with other tests'? Does that mean that you would include other tests in the help output? That the examples would be included when running the tests? Or something else?
(examples do
(do (prn "line 1")
(prn "line 2")
(prn "line 3"))
_)
Here's how it looks:
arc> (help do)
[mac] (do . args)
Evaluates each expression in sequence and returns the result of the
last expression.
Examples:
arc> (do (prn "line 1")
(prn "line 2")
(prn "line 3"))
It makes sense to specify the example expr / expected pairs as sexps & strings: run repl-output on exprs, compare those strings to the expected strings.
Thanks for thinking through that! Yes, I'd had a similar though not as fully-formed idea, but was stuck on distinguishing stdout from return value in the examples. But yeah, that's overthinking it -- we have to mentally parse the two at the repl anyway.
Only other quibble: a string isn't very nice to read next to code, even if it looks nice at the repl. Hmm..
Maybe we should make an auto scraper/notifier? Though iirc, the arc server doesn't really appreciate frequent queries. On closer look, it should be fine; the default throttle window should be 10s. It could query once a minute, and update an rss feed for arc forum posts and comments. Or provide webhook + email support.
I actually built one back in the day: http://akkartik.name/post/2011-05-13-05-47-29-soc. Trouble was, it kept getting banned/blocked even when I scaled back to crawl HN every five minutes. Eventually I was crawling so infrequently that I was missing stories that dropped off /newest in the meantime, so I stopped crawling.
I actually had an arc version for a brief period[1], but it didn't seem like there was as much need for it here. But perhaps the email notifications would be useful. I'll see if I can bring it back.
[1] I was certain I'd showed it here, but no I only showed it over email to a couple of folks. I really should bring it back just so y'all can look at it. Or at least locate the (rails) sources.
Cool idea! I used to do something a long time ago: use a little-known option of ctags (-r) to match wikiWords, so that hitting C-] in vim or M-. in emacs with the cursor on say 'databaseSubsystem' would take you to '$<databaseSubsystem>' in the codebase.
The nice thing about it is that a name could potentially belong to multiple different tags/narratives. Multiple overlapping trails is a pretty decent way to narrate an intrinsically non-linear thing like a codebase. See also http://leoeditor.com.
You would have to make sure that all the variables used in arc are shared behind locks. That's a big enough problem that it hasn't been attempted, to my knowledge.
I think you're telling fibs. :-p I double-checked srv.arc (which defines 'defop), and the code there opens a thread for every request. This is true in Anarki, in official Arc 3.1, and even way back in Arc0.
Even without parallelism, this would come in handy to prevent I/O operations from pausing the whole server.
Arc does have threads, yes, but it also has a style of mutating in-memory globals willy-nilly. As a result, all its mutator primitives run in atomic sections (http://arclanguage.github.io/ref/atomic.html#atomic) at a deep level. The net effect is as if the HN server has only one thread, since most threads will be blocked most of the time.
I can't find links at the moment, but pg has repeatedly said that HN runs on a single extremely beefy server with lots of caching for performance.
Edit: Racket's docs say that "Threads run concurrently in the sense that one thread can preempt another without its cooperation, but threads do not run in parallel in the sense of using multiple hardware processors." (http://docs.racket-lang.org/guide/concurrency.html) So arc's use of atomic wouldn't matter in this case. It does prevent HN from using multiple load-balanced servers.
Looking back, I see that I did indeed inaccurately answer zck's question about "running single-threaded". I'd like to amend my answer to "No, it runs multi-threaded, but the threads use a single core." Rocketnia is right that arc has concurrency but not parallelism.
"The net effect is as if the HN server has only one thread, since most threads will be blocked most of the time."
Well, in JavaScript, I do concurrency by manually using continuation-passing style and building my own arbiter/trampoline... and using it for all my code. If I ever do something an easier way, I have to rewrite it eventually. Whenever I want to try out a different arbiter/trampoline technique, I have to rewrite all my code.
Arc's threading semantics are at least more automatic than that. Naive Arc code is pretty much always usable as a thread, and it just so happens it's especially useful if it doesn't use expensive 'atomic blocks (or the mutators that use them).
"Automatic" doesn't necessarily mean automatic in a good way for all applications. Even if I were working in Arc, I still might resort to building my own arbiters, trampolines, and such, because concurrency experiments are part of what I'm doing.
All in all, what I mean to say is, Arc's threads are sometimes helpful. :-p
Absolutely. I was speaking only in the context of making use of multiple cores.
I see now that I overstated how bad things are when I said it's as if there's only one thread. Since I/O can be done in parallel and accessing in-memory data is super fast, atomic isn't as bad as I thought for the past 5 years.
arc> (help do)
[mac] (do . args)
Evaluates each expression in sequence and returns the result of the
last expression.
See also [[do1]] [[after]]
This is awesome. Yes, worth updating. Let me think about how to sync it with the reference. (I'd also forgotten that the reference is generated from anarki. Is that still true, or is this copy redundant? Need to check..)
Sorry, I think I was mistaken. If the hypertext reference is generated from docstrings at all, I don't see how. Even if there were a tool, it would require some manual work afterward to place the definitions into appropriate categories.
I've updated the docs. I've also taken out the help/ dir entirely and inlined all the docstrings into arc.arc and elsewhere.
I'm still unsure how to organize the arcfn reference guide at http://arclanguage.github.io/ref so that we remember to update it when we make changes, and so it's convenient to update the website. Another complication is that the arclanguage account contains multiple dialects of arc with subtle differences, and the current organization of documentation is misleadingly monolithic. Any suggestions to fix this most appreciated. (We discussed this previously a year ago: http://www.arclanguage.org/item?id=17774)
Something like marginalia[1] might prove to be better than the arcfn docs. Not only because the docs would be fully integrated with the source code, but because it would also solve the multiple dialects problem. i.e. If some given code can be tagged with a dialect name then automation could also apply a dialect filter.
Of course this would probably be quite a bit of an undertaking.
The crux is colocating the rendered docs online with the repo. Would marginalia help us use github's hosting with github pages, managing branches, etc? If it does I think I'd be willing to go on a significant undertaking.
Marginalia is clojure specific so I expect it will not help other than to provide ideas.
To create an arc equivalent you probably need build an arc library which provides some code inspection/dissection capabilities and ideally also be able to attach metadata to any given function or macro. With such a library you then build a script to auto generate the docs.
As for GitHub syncing; well no, I'm guessing users would need to trigger the script and then check in the updated docs.
This is still better for a few reasons...1. developers can generate docs, locally, that are in sync with the code base they are actually using (checked out or branched). 2. Even if the online docs gets out of sync for a while you're still only a script trigger away for updating all outstanding changes.
The alternative is what you just went through; having someone remind you to do the work manually as an after-thought, which I've only seen happen once.
What happened here was that the type of oo was expanded to sym, and then sym was eval'd in the caller's scope to return a function (thanks to arc being a lisp-1).
The point is, your macro is trying to call a symbol as a function. Instead, this will do what you want:
(mac somemacro (someobj)
`(if (,someobj 'field) t nil))
How did you figure that out? I found I couldn't macex1 it, because it was already erroring. I guess it's related to when ssexpansion happens in relation to macroexpansion?
(mac somemacro (someobj)
`(if ,someobj!field t nil))
When we enter any code at the REPL, it causes three phases to occur: Reading, compilation, and execution.[1]
The read phase processes the ( ) ` , symbols and gets this data structure of cons lists and symbols:
(mac somemacro (someobj)
(quasiquote (if (unquote someobj!field) t nil)))
Note that someobj!field is a single symbol whose name contains an exclamation point character.
At this point you can probably see the problem already. What you may have expected was ((unquote someobj) (quote field)), but what we got was (unquote someobj!field). This is technically because Arc doesn't implement ssyntax at the reader level; instead it uses Racket's reader without modification, and then it processes the ssyntax in the next phase.
Even though the issue should already be clear, I'm going to go through the rest of the process to illustrate macroexpansion.
At this point the compilation phase starts. It expands the (mac ...) macro call, and then it re-processes whatever that macro expands to. As it goes along, at some point it also processes the (quasiquote ...) special form, and it expands the someobj!field syntax. The result of expanding someobj!field is (someobj 'field), and since this isn't a special form or macro, it's compiled as a function call.
The overall result is Racket code. In case it helps show what's going on, I just went to http://tryarc.org/ and ran the following command:
($ (ac '(macro somemacro (someobj) `(if ,someobj!field t nil)) (list)))
This produced a bunch of Racket code, which looks like this if I format it nicely:
Personally, I never think about the raw Racket code. Instead I pretend the result of compilation is Arc code again, just without any macro calls or ssyntax:
Either way, you can see the original (if ... t nil) is still in there somewhere. :)
Finally, this Racket code is executed. It modifies the global environment via Racket's namespace-set-variable-value! and puts a macro there. The macro is simply stored as a tagged value where the tag is 'mac and the content is a function. Then the result of execution is printed to the REPL as "#(tagged mac #<procedure: somemacro>)", and the REPL prompt appears again.
Later on, we execute the following command:
(somemacro oo)
The reader parses this to make this s-expression:
(somemacro oo)
Then the compilation phase starts. It starts to expand the (somemacro ...) macro call. To do this, it invokes the macro implementation we defined earlier. It passes in this list of s-expressions:
(oo)
The macro's implementation is the function that resulted from this Racket code:
Or alternately, in my imagination, it's the result of this macroless Arc code:
(fn (someobj)
(quasiquote
(if (unquote (someobj (quote field)))
t
nil)))
When this function is called, someobj's value is the symbol named "oo". When we try to call the symbol, we get an error.
The compilation phase terminates prematurely, and it displays the error on the console. The execution phase is skipped, since there's nothing to execute. Then the REPL prompt appears again.
I hope this gives people a good picture of the semantics of macroexpansion and ssyntax.
[1] Technically we might add printing and looping phases to get a full Read-Eval-Print-Loop. The eval step of the REPL does compilation followed by execution.
---
As I said above, the confusing point is probably that the reader doesn't give the result that you might expect when it sees ",someobj!field". On the one hand, this is a technical limitation of the fact that Arc uses Racket's reader which doesn't process ssyntax. On the other hand, I think it's debatable if this interpretation of the syntax is better or worse than the alternative.
The ssyntax is a red herring here, since the behavior is the same even if you don't use ssyntax like in my previous comment. I'd also bet that it's the same in common lisp. Just a consequence of macros operating in name space rather than value space. Though this is a kinda subtle consequence of that; I had to run that type example I mentioned before to fully grok what was going on.
The reader parses the comma to make (unquote someobj!field) before ssyntax processing even occurs. I just wrote a long comment to describe the whole process, but this is the main point.
Yeah I've seen that image too. It's older, and predates the public release of Arc by a few years. The current logo here came even later. Probably just an abortive attempt.
Wart isn't that different from arc, it still shows its roots.
for x 1 (x < 10) ++x
prn x
map prn '(1 2 3 4 5 6 7 8 9 10)
def (f n)
prn n
if (n < 10)
(f n+1)
f.1
Unlike smile it needs those parens around the if condition.
Edit 13 hours later: Ack, I was wrong. Wart would drop the parens just fine!
if n < 10
(f n+1)
I'm not sure I like it, but the design of infix fundamentally supports it by having higher precedence than function calls. If the infix operands grow complex it can get hard to read without parens.
if (function-call n x1 x2 x3) < 10
(f n+1)
# still works, but yuck..
