Arc Forumnew | comments | leaders | submitlogin
State of the arc
3 points by shader 4040 days ago | 12 comments
Hey all, I'm considering coming back to using arc for a project after a moderate hiatus, and wondering where the language is now. Is anarki still the most mature variant, or has someone hacked on their own fork enough that its cleaner, faster, has better libraries, etc.

I've been using python a lot recently, but sometimes the language just frustrates me with a missing feature that I would have in a lisp, and I enjoy writing arc the most. Only problem is, 3.1/anarki always felt a bit rickety and is lacking a bit in libraries. If that's still the best combination of language features and existing code, I'll deal with that and maybe help improve it some in the process, but if someone else thinks they have a good enough competitor with additional features like modules and fexprs, I'm open to trying something new.



2 points by Pauan 4040 days ago | link

I believe Anarki still has the largest number (and variety) of libraries, but if you want something cleaner, I recommend Arc/Nu:

https://github.com/Pauan/ar

Arc/Nu doesn't have modules, but I've been thinking about adding in hyper-static scope, which would give you modules.

As for fexprs... gosh that's hard to do because Arc is compiled to Racket. You'd really need Arc to be interpreted if you wanted fexprs.

-----

1 point by shader 4039 days ago | link

Interesting. An arc rebuild that's faster and cleaner while maintaining most compatibility certainly sounds attractive. I'll have to try it out at least a bit.

A slightly unrelated question: Have you ever tried compiling a standalone binary for Nu or anarki, at least for the compiler? I'm pretty sure racket can do that, but I don't think I've heard of anyone doing it.

As for fexprs, I think something can probably be done even in arc to some extent, using a copy of the original code and dynamic recompilation with memoization for speed. It would have the advantage of also enabling the self-documentation features that I added to anarki too, though it might be a bit slow.

-----

1 point by Pauan 4039 days ago | link

"Have you ever tried compiling a standalone binary for Nu or anarki, at least for the compiler? I'm pretty sure racket can do that, but I don't think I've heard of anyone doing it."

No, I haven't, though I suppose it could be done... I don't see much point to it, though. I mean, I guess if you're on Windows, but...

---

"As for fexprs, I think something can probably be done even in arc to some extent, using a copy of the original code and dynamic recompilation with memoization for speed. It would have the advantage of also enabling the self-documentation features that I added to anarki too, though it might be a bit slow."

I've experimented with using fexprs in a compiled environment like Racket, but you really really do want things to be interpreted. At the very least, using fexprs won't blend well with Arc code, because the entire Arc language has been designed around macros, to the point where adding in fexprs just makes things more clunky. If you want fexprs, I'd recommend a language that was designed from the start for fexprs, like Kernel, PicoLisp, or newLISP.

-----

2 points by shader 4039 days ago | link

Well, one of the reasons I was interested in compiling at least the standard arc library was for improving startup speed, say for use as a simple cgi script on a shared host, or a basic command line utility. It also might reduce distribution size, instead of making users install all of racket and the arc source, which can be somewhat bothersome.

I suppose its possible that trying to add fexprs to arc would just make things worse, but I was always attracted to the idea of making environments and scope easier to manipulate, and macros fit in better. As it currently is, macros are very separate from the rest of the language and stick out a little bit. I've rarely had a really good reason to need it, but it just bothers me sometimes that they aren't first class citizens.

-----

1 point by Pauan 4039 days ago | link

"Well, one of the reasons I was interested in compiling at least the standard arc library was for improving startup speed, say for use as a simple cgi script on a shared host, or a basic command line utility."

Oh! That's what you're talking about! Sure, I've experimented with that too. I think it's possible, but it would require mucking about with Racket's code loader... still, that's not a bad idea.

---

"I suppose its possible that trying to add fexprs to arc would just make things worse, but I was always attracted to the idea of making environments and scope easier to manipulate, and macros fit in better. As it currently is, macros are very separate from the rest of the language and stick out a little bit. I've rarely had a really good reason to need it, but it just bothers me sometimes that they aren't first class citizens."

Absolutely. I too like the idea of fexprs. I'm just saying that from a purely practical perspective, Arc was not designed around fexprs, so tacking them on after the fact won't work so well. It shouldn't be hard to make a variant of Arc that is built around fexprs. In fact, wart is very Arc-like but has fexprs (or something that's like fexprs anyways).

-----

1 point by akkartik 4040 days ago | link

Welcome back, shader!

Analogously to Pauan, I'd love to have you try wart. It's much slower and has fewer libraries than arc. But on the other hand it's cleaner, and has fexprs, python-like indent-sensitivity and python-like keyword args. At the very least, maybe worth a few minutes play? (And tell me what you think? :) Somebody using it may well be the impetus I need to go back to work on speeding it up.

http://github.com/akkartik/wart#readme

  $ git clone http://github.com/akkartik/wart.git
  $ cd wart
  $ ./wart
(Requires linux, gcc.)

Other than that, any contributions to anarki are also most welcome. Can you share what project you're planning?

-----

2 points by shader 4039 days ago | link

Hello akkartik, I'm glad to be back. I've been lurking on and off for the past few years, but it will be nice to get involved again.

I guess I should be fair and try yours out too while I'm at it, though I have to say that the whitespace sensitivity makes me somewhat cautious. For some reason, I've always liked the more traditional sexpr syntax for lisp. Does wart mind if I use parens for everything?

Even more disconcerting would probably be your infix support. Maybe it won't be as much of a problem as I'm expecting, but I like being able to use random symbols in my variable names.

Also, numbered git commits strikes me as a little odd, as does the numbered source files. In the latter case its little more than taste, and I see that Pauan is doing the same thing. Maybe you have a good reason for it and I'd start doing the same if I only understood.

Anyway, enough casting of doubt on someone else's pet project. I'm certainly interested in a clean language with the possibility of fexprs to try out and maybe even keyword args. I'm not sure what I would actually need them for, but more power is never something I will turn down.

I'm also somewhat interested in the wat/taf projects, but they seem to be a bit more experimental right now.

--

As for my project plans, I'm thinking of doing two or three web service projects on the side, as a long term investment counter-point to my current hourly contracting job.

The first one I'm thinking of focusing on is a sort of easy, data-driven unit-testing-as-a-service concept. If you've ever seen the fit or fitnesse testing frameworks, this idea was originally based off of those. Instead of writing unit test code, you would use a website to input test values and corresponding expected results in a table format. The first row of the table specifies the function or object being tested and its arguments or property names, and each row after would give the values for that test case, with the last column or set of columns specifying the return value. Fitnesse did that for c# and java, but it had a few major flaws. In the first case, it would only interact with classes that inherited from the fitnesse test classes, so you were forced to write test harness code anyway. Second, the user had to format the tables manually using a wiki format, so it required a bit too much manual formatting and there wasn't any way to provide additional metadata or add any more dimensions to the tables.

The first few incarnations would probably be something really simple that would only be useful for testing code locally, but eventually it would be expanded to a web service that supports multiple languages and has a way to point it at any vcs repo and run tests interactively in the cloud. This would hopefully be a cheapish testing and specification solution for startups or foss projects that would be easy enough to add to existing code that people would actually do it. That and an enterprise version that can be deployed internally, which would hopefully make it so that business analysts and the QA team can write, run, and review the tests without having the dev team write a separate test harness for them.

There's a bit more to the idea, but right now none of it has been written, so I probably shouldn't advertise features that I may never get to, or might not even be feasible in the first place. Of course, I like talking even more than I like coding, so I'm sure we can discuss it if you're interested. In fact, I would be very open to discussion, as I'm sure what other people actually want/need/would be willing to pay for in a test system won't match up exactly my own ideas.

-----

2 points by Pauan 4039 days ago | link

"In the latter case its little more than taste, and I see that Pauan is doing the same thing."

The reason I numbered them was just to make it easier to navigate the code. As a user, if you see "01 nu.rkt" you know it's loaded first, and that "02 arc.arc" is loaded second. And each one builds on the stuff defined previously, so you can read it in a linear order.

I only did that for the stuff that's automatically loaded when you start up Arc. You'll notice that the "lib" and "app" stuff is un-numbered. And I don't expect user-created code or libraries to use numbers! So I definitely don't take it as far as wart does.

-----

1 point by akkartik 4039 days ago | link

Ah, I was unaware of fitnesse! Thanks for the pointer, that's a really neat idea. Tests are a huge part of wart's goal of 'empowering outsiders'[1].

Sucks that fitnesse is stuck in the java eco-system. Just porting it to lisp/arc/wart would be awesome for starters..

Arguably much of the benefit of testing comes from the same person doing both programming and testing. Organizations which separate those two functions tend to gradually suck. If you accept that, inserting a web interface between oneself and one's tests seems like a bad idea. Perhaps fitnesse-like flows would be best utilized to involve the non-programmer in integration testing, testing the entire site as a whole rather than individual functions. Perhaps script interactions with a (non-ajax for starters!) app so that the CEO/QA engineer doesn't have to know about REST and PUT/GET? Hmm, that would be cool..

Don't mind me, I'm just thinking aloud :)

[1] https://github.com/akkartik/wart/blob/531622db6a/000preface. I spend a lot of time thinking about this, and at half an opening will fill your ears about all the ways in which tests, whilst awesome, aren't sufficient. Along with ideas for complementary mechanisms.

-----

3 points by shader 4039 days ago | link

Organizations which separate those two functions tend to gradually suck

Hmm... Well, that's certainly a valid opinion, and it may even be true in a lot of cases, however I think the issue is largely due to two other related issues: 1) The requirements aren't specified clearly enough, and are dissociated from the tests as well, and 2) they just don't have good enough tools.

Tests can serve many purposes. The most basic, given the fundamental concept of a test, is to tell you when something works, or when it doesn't. TDD focuses on the former, unit testing and regression testing focus more on the later. Tests can be used at different points in the development cycle, and if you use the wrong kind at the wrong point, it's not really all that helpful.

My concern is that its too difficult to write the correct kind of test, so most developers use the wrong kind or don't use any at all. There's nothing really wrong with that, I think it's just an unfortunate inefficiency, like trying to debug without stacktraces. >.> Hmm. Something to look forward to going back to arc I suppose. Anyway, my goal is to make testing easy enough to do, either for developers who just want a quick way to check if they broke something after making a 'minor' change, or for larger companies that want to know that all their requirements have actually been met.

So, to solve the first problem I'm hoping to utilize a lot of reflection and code inspection so that at least the outline of the test cases can be generated automatically, if not more. Then it should be really easy for the programmer to just add the missing details, either as specific test vectors or by using a more general definition of requirements using something like QuickCheck's generators.

In the long run the plan is for the tool to be able to support working the other direction, from requirements to tests. Hopefully with better tool support, and more intelligent interaction with the system under test, it should be possible for the architects to specify the requirements, and the tool should be able to verify that the code works.

Yes, divorcing tests from code could mean that different people do them. Doesn't have to be the case, but it becomes a possibility. And that means that they could will have a different perspective on the operation of the system, but not necessarily a worse one. If it's the architects or BAs writing the tests, then they might actually have more information about how the system should be working than the programmers, especially in the case that the programmers are outsourced. At which point allowing someone else to write the tests is an improvement. When developers write the tests, it doesn't help if they use the same incorrect understanding of the requirements for both the tests and the code.

Hopefully by making an easy enough tool that supports rapidly filling in tests based on code analysis (which would help anyone that doesn't know much about the actual code base match it up with the requirements they have) reducing boiler plate and barriers to testing, making it a much easier to use tool for developing. Maybe if it gets easy enough, developers would find that testing actually saves enough time testing to be worth the few seconds specifying test vectors for each method. And if it can do a good enough job at turning requirements into tests in a way that is clear enough to double as documentation, it should save the architects and BAs enough time, as well as make implementation easier for developers, that I might actually be able to sell licenses :P

-----

2 points by akkartik 4039 days ago | link

"If it's the architects or BAs writing the tests, then they might actually have _more_ information about how the system should be working than the programmers,"

Oh, absolutely. I didn't mean to sound anti-non-programmer.

I tend to distrust labels like 'architect' and 'programmer'. Really there's only two kinds of people who can work on something: those who are paid to work on it, and those who have some other (usually richer and more nuanced) motivation to do so. When I said, "Organizations which separate those two functions tend to gradually suck", I was implicitly assuming both sides were paid employees.

If non-employees (I'll call them, oh I don't know, owners) contribute towards building a program the result is always superior. Regardless of how they contribute. They're just more engaged, more aware of details, faster to react to changes in requirements (which always change). Your idea sounds awesome because it helps them be more engaged.

But when it's all paid employees and you separate them into testers and devs, then a peculiar phenomenon occurs. The engineers throw half-baked crap over to testers because, well, it's not their job to test. And test engineers throw releases back at the first sign of problems because, well, it's not their job to actually do anything constructive. A lot of shuffling back and forth happens, and both velocity and safety suffer, because nobody cares about the big picture of the product anymore.

(Agh, this is not very clear. I spend a lot of time thinking about large organizations. Another analogous example involves the patent office: http://akkartik.name/blog/2010-12-19-18-19-59-soc. Perhaps that'll help triangulate on where I'm coming from.)

(BTW, I've always wondered: what's that cryptic string in your profile?)

-----

2 points by akkartik 4039 days ago | link

"Does wart mind if I use parens for everything?"

Nope, that will always work as expected: https://github.com/akkartik/wart/blob/531622db6a/004optional.... I threw it out there since you mentioned python :) but syntax is the least important experiment in wart.

Re numbered files: they keep me from ever needing to refer to filenames in code: https://github.com/akkartik/wart/blob/531622db6a/001organiza...

Don't feel like you have to be fair :) The world isn't fair, and I understand about differences in taste. My goal with wart and some other stuff has been to figure out how to empower outsiders to bend a codebase to their will and taste with a minimum of effort. For example, one experiment I'd love to perform on you is to measure how long it takes you to fork wart to toss out infix and get back your beloved hyphens :) But no pressure.

-----