Arc Forum | Unit testing Arc

Arc Forum

5 points by dido 4649 days ago | 4 comments

Development on Arcueid is progressing nicely, and now one of the things I'm trying to develop is a test suite that can be used to validate any Arc implementation, not just my own, against the reference Arc implementation provided by Paul Graham.

But of course, before anything like this can be accomplished one naturally needs to have a unit testing framework of some sort. I have been able to find spec.arc by Will Fitzgerald [1]. While Will's code is fine, its licensing status is unclear. I asked for and explicitly got permission from him to distribute it along with Arcueid, but he seemed reluctant to comment on what license it was supposed to be released, so I have to assume it is essentially proprietary, and as such it makes me nervous to build something like this on top of it. Are there any other Arc unit testing frameworks available? Google doesn't get me much else.

I don't mind writing one myself if it has to come to that, but I'd rather not invent the wheel if it has already been invented.

[1] http://willwhim.wordpress.com/2008/02/02/more-better-spec-for-arc/

3 points by rocketnia 4649 days ago | link

I believe Rainbow has unit tests intended for "any Arc implementation." I also think there are extensive unit tests in ar, Nu, and Wart, but I don't know that they're very language-implementation-agnostic (especially with Wart). Nu's benchmarks are meant to be runnable on multiple Arc implementations for comparison purposes, so they could be a good start.

Personally, I consider something to be my kind of Arc once it's easy for me to adapt Lathe (my library of frameworks) to its quirks, even if Lathe doesn't work out of the box--and even if not all features of Lathe are available. That is to say, whatever set of assumptions I implicitly rely on in Lathe forms a common subset of several Arc-like languages, and I might as well call that common subset "Arc." I admit it's vague (what features of Lathe are optional and what aren't?) and that it isn't necessarily the same as what someone else would call Arc. Nevertheless, Lathe has a few unit tests to make this goal more discrete, in the examples/ directory.

I've had "try Lathe with Arcueid" on my mind for a while, and I keep applying my motivation to other things instead. Actually, the way I think of it is more like "try Lathe with Rainbow.js, Arcueid, the Nu family, and the most recent version of Anarki," so when I do work on it, I dedirect my attention toward the sub-task of getting Rainbow.js to load libraries. >.>

-----

4 points by conanite 4649 days ago | link

Implementation-neutral arc tests from rainbow are here:

https://github.com/conanite/rainbow/tree/master/src/arc/lib/...

It's safe to use these as a specification.If you fire up an arc3 repl within the rainbow src directory you can run the same tests to verify you get the same behaviour.

Rainbow-specific tests are in another directory.

-----

2 points by dido 4648 days ago | link

Hmm, core-evaluation-test.arc seems to hang Anarki, as well as Arc 3.1.

Well, I've tried to run the tests that do work fine with 3.1 and Anarki under Arcueid and find a lot of issues. For starters, I had no idea that Arc treats symbols with |'s specially. Looks like more accidental behavior inherited from MzScheme/Racket. Scheme48 says that '|abc| contains illegal characters. Guile creates a symbol |abc|. I don't see anything in R^6RS that mandates any of this behavior. Heh, looks like I've got a lot of work to do!

Apparently the bars in symbols is a sort of convention when it comes to case sensitivity of symbols in Scheme. It seems that Arc, in its current implementation anyway, is unintentionally inheriting a lot of onions from MzScheme...

-----

2 points by Pauan 4648 days ago | link

"I also think there are extensive unit tests in Nu"

Older versions of Nu had lots of unit tests, yeah, but I haven't ported them over to the latest version of Nu yet and probably won't do so anytime soon, as I've unfortunately lost the motivation to work on anything Arc related.

---

"Nu's benchmarks are meant to be runnable on multiple Arc implementations for comparison purposes, so they could be a good start."

Yeah, but for now all the Arc implementations need to be built on top of Racket, so it works for ar, Arc 3.1, Nu, etc. but not, say, Rainbow. Getting it to work with non-Racket processes is probably doable - albeit difficult - and it would come at the cost of accuracy in the tests.

The accuracy problem could be mitigated with some sort of namespace system such as the one Nu could have if I ever actually built the damn thing. But in that case, you might actually be better off building the benchmark tester program in C/D/Go/Racket/whatever and using FFI to talk to the different implementations...

-----