Arc Forumnew | comments | leaders | submit | dido's commentslogin
2 points by dido 2525 days ago | link | parent | on: Iso

Equality is actually a rather complicated thing. The way I understand it 'is' is an object equality predicate, roughly equivalent to Scheme's 'eqv?' for most data types. The 'iso' function on the other hand sounds like it was intended to be a structural equivalence primitive (which is what isomorphism means in mathematics), which should roughly have the semantics of Scheme's 'equal?'. Arc 3.1 defined iso only for lists, but it seems that just about every other third-party implementation including my own Arcueid extended it to arbitrary other objects such as tables.

-----


The assertion error is a stack overflow. :) At present Arcueid makes use of a fixed-size stack within its virtual machine, and since the compiler still can't properly optimise tail recursion, your code overflows the stack.

-----

1 point by akkartik 2544 days ago | link

That was what I thought :) Is there a place where I can increase the size of the fixed stack? I looked for the limit in the code, but couldn't find it.

-----

3 points by dido 2543 days ago | link

src/vmengine.h:236:

#define TSTKSIZE 65536

Increase it as high as you like. :)

-----

1 point by akkartik 2543 days ago | link

Thanks!

Does it make sense to resize it on the fly when we discover we've overflowed the stack? Or are there potentially continuation pointers into the stack that would be non-trivial to track down?

-----

1 point by akkartik 2543 days ago | link

Yeah, no luck: https://github.com/akkartik/arcueid/commit/ae31b82540

-----

1 point by akkartik 2533 days ago | link

Possibly relevant: http://wingolog.org/archives/2014/03/17/stack-overflow

-----


Well, thanks for the bug report. Well, I'll look into it when I have the time: the work which pays my bills has caught up with me once again and I don't have a lot of time to do hobby development. :)

-----

2 points by dido 2853 days ago | link | parent | on: Regular expressions in Arc

I ran into a similar issue with regex syntax when attempting to incorporate it into Arcueid's reader. There seems to be no easy way to parse a regular expression using Perl-like /.../ syntax, not if you also want symbols that use /'s for other things, e.g. the division function. Arcueid thus uses for now, r/.../ for regular expressions, and that syntax could be more easily distinguished from other legitimate uses of symbols with a minimum of fuss.

-----

3 points by dido 2855 days ago | link | parent | on: Regular expressions in Arc

Interesting, although I don't know if it can be done with efficiency in the worst case. Backtracking regexes can take in their worst-case exponential time for matching, as illustrated in the regex in my original post.

-----

3 points by dido 2855 days ago | link | parent | on: Arc DB

I wonder why Arc seems to have stayed away from the traditional SQL databases that have long been used for web apps. There are few problems for which relational databases are inappropriate, and not using one results in reinvention of the wheel which I think is hardly a good thing to be doing. A DBI work-alike by which one can connect to a MySQL, Postgres, or Sqlite database would be a lot simpler to my mind.

-----

3 points by Pauan 2855 days ago | link

Because it's the simplest solution for Arc. Arc does not have any kind of SQL library, and it doesn't even have an officially supported way to drop down to Racket libraries, nor does it have any kind of FFI whatsoever. So flat files are indeed the fastest way to get up and running.

Perhaps in the long run it might be better to have some sort of way to connect to an SQL DB, but keep in mind that Arc is still a work in progress, and unfortunately has not been updated in a long time. For a while, Arc didn't even have Unicode support, because pg was working on more important things.

If you think of Arc as being a prototype, then it all makes sense. pg probably intended to eventually flesh it out into a full language, but I hear he hasn't had the time.

-----

3 points by shader 2855 days ago | link

If arc had support for something like DBA or SQLAlchemy built in, I might just have used it with either postgres or sqlite. However, neither of those databases really fit the arc data model very well, imo, because arc is very hash table and list oriented. Objects have very little in the way of a set schema, and hash tables map pretty well to... hash tables.

Anyway, I mostly want to leave all the objects in memory and use direct references between them; my data relations aren't that complicated, and explicit relations where necessary are actually fairly efficient. In fact, that's what most orm's a la SQLAlchemy seem to do; whenever an object is loaded, you can specify desired relations that also get loaded in memory, so you don't have to explicitly query the database each time.

Memory is cheap these days, and I was hoping for something that allowed versioning and perhaps graph-db features.

-----

2 points by akkartik 2855 days ago | link

Hmm, do you care about threading and consistency at all? If not, you could probably do everything with just arc macros over the existing flat file approach..

-----

3 points by shader 2853 days ago | link

I think that some form of scalability would be valuable, but that could easily be achieved with some sort of single threaded worker for each db 'server', and then have multiple instances running to provide the scalability. In order to make the single threaded semantics work well even in a multi-threaded application, I already have a short library for erlang-style pattern matched message passing.

Given the data volumes I've been planning on working with, I mostly want to use the permanent storage for history and fault tolerance, as opposed to live access. That could probably be handled in-memory for the most part. So maybe some form of flat file system would work without causing too many problems.

I originally started using git to effectively achieve that design without having to manage the trees, history, and diff calculation myself, but I've discovered that storing thousands of tiny objects in the git index may not be very efficient. I still think something similar is a good idea, but I would want to separate 'local' version states for each object from the 'global' version, so that it doesn't take forever to save the state of a single object. Maybe storing each object in a git 'branch' with the guid of the object as the branch name would work, since only one object would be in each index. The overhead for saving each object would be slightly higher, but it should be constant, rather than linear with the total number of objects.

Any obvious flaws with that idea that I'm missing? Have any better ideas or foundations to build off of?

-----

1 point by akkartik 2853 days ago | link

Building atop git is an interesting idea, and you clearly have more experience with it. Do you have any pointers to code?

-----

3 points by shader 2852 days ago | link

Here's the code I had written before, using the shell git interface to interact with the repo: https://github.com/shader/metagame/blob/master/git-db.arc

That code is pretty rudimentary, but allows low level access to git commands from arc, plus storage and retrieval of arc objects. After my previous comment though, I'll probably change it so that each object gets a separate branch, with 'meta branches' listing which branches to load if necessary.

-----

1 point by akkartik 2855 days ago | link

Let's build this for the LISP contest! http://arclanguage.org/item?id=17640

-----

4 points by dido 2854 days ago | link

So I guess that makes a consistent foreign function interface something very important for Arcueid to start having then. I think I've built up a foreign function API (sorta) and am now working out the details for dynamic loading of shared libraries so you can do something like (load "mysql.so") and have it dynamically link into Arcueid, as well as a way to compile C sources into such an extension shared object.

-----

1 point by dido 2855 days ago | link | parent | on: Regular expressions in Arc

I've not yet come across a Lisp dialect that incorporates regexes the way Perl and Ruby do, so I wonder what's a clean way of using them that respects the idioms of Arc. Then there's the matter of taint mode... How would a taint mode in Arc be like?

-----

2 points by dido 2866 days ago | link | parent | on: Arcueid 0.1.1 released

Just a bugfix release, but this is the first version that can actually run news.arc in full. Performance is still rather dismal though, sorry about that, but I'd like to get it to work right first before getting it to work fast. :)

-----

1 point by lark 2863 days ago | link

Can you add in the repository a script for compiling Arcueid?

-----

2 points by dido 2862 days ago | link

I take it you're trying to compile Arcueid from sources cloning the repository on Github. There is no configure script in the source repository because that is automatically generated by autoconf. It is a general rule to never include anything in the repository that is automatically generated. Basically, you need to install the autotools (autoconf, automake, and libtool), the dependency headers (most importantly gmp and readline), and then run autoreconf -i in the main source directory. This creates the configure script and everything else it needs.

As I mention in more detail in Arcueid's README:

    If you are trying to build this by cloning the Git
    repository (git://github.com/dido/arcueid.git), you
    need the following prerequisites installed:

   - The autotools: autoconf, automake, and libtool.
   - GNU MP and any development headers (libgmp and
     libgmp-dev on Ubuntu/Debian)
   - GNU Check (http://check.sourceforge.net, available
      under the package name 'check' on Ubuntu)
   - GNU Readline and any development headers
     (libreadline-dev)
   - pkg-config (http://www.freedesktop.org/wiki/Software/pkg-config,
     pkg-config on Ubuntu/Debian)

   You need all of this since the process will create a
   complete configure script and all the development
   dependency headers are required for it to generate the
   rules for all the checks configure has to perform.  Once
   all these prerequisites are installed run 'autoreconf
   -i' in the main directory.  This should generate a
   configure script and Makefile.in's, and from there it
   should be possible to do ./configure ; make ; make
   install.
Well, there's some interesting stuff in the present git head, such as nested quasiquotes and some stability fixes (stomping out memory leakage and such), although be warned that it isn't always 100% stable.

-----

5 points by dido 2868 days ago | link | parent | on: Why did you write your programming language?

First of all I don't like the fact that the present reference Arc depends on MzScheme/Racket. This makes setting it up and using it a chore, and I think this is a not insignificant factor in what hobbled wider acceptance of the language. I wanted to be able to just do a ./configure ; make ; make install and start using it, the way I can presently do with, say, Ruby. As it is you have to install a rather large Scheme interpreter (Racket 5.1.1 has over 500,000 lines of code; it might have been a little different if it were a smaller Scheme like scheme48) just to get started.

I've also wanted to have a language like Tcl or Lua that one could easily embed as a scripting language in a larger project. This is one of the things that Arcueid is designed to do: just link a single library and have an interpreter available with a relatively simple API. Arcueid is also be designed to be easily extensible via a simple foreign function interface because I don't want to use it for just programming web applications, although that is something that I would doubtless use it for a lot as well.

Writing in C (or something that compiles to C) is also probably the only way to get an Arc faster than the implementations we have today. Of course my present implementation is still somewhat slower than Arc on Racket, but then again the goal for now is to build a foundation upon which further optimisation can later be done. As always, it's better to get it to work right first, and then make it run quickly later... Eventually Arcueid will have a JIT compiler of sorts but that's a little further down the roadmap.

As for Arc, it seems like it's much better designed than any other Lisp dialect out there. The parentheses are still there, but they are used more sparingly than Scheme, which seems out to irritate you by putting extra parentheses for something so basic as a let binding when you could easily do with less and have more comprehensible code. This fact was illustrated to me about a year ago when rocketnia gave me versions of the Hanson-Lamping algorithm for dynamic-bind in Scheme and in Arc here:

http://arclanguage.org/item?id=15536

The contrast was rather startling to me. I understood the algorithm immediately on seeing the Arc code.

Another thing about other Lisps that is really irritating to me are the overly long function names like multiple-value-bind and invoke-restart-interactively and stuff like that. They make a program look rather cluttered. Arc has for the most part avoided excessively long identifiers in the core and that has been good.

Macros are something that I've also been really interested in, and something that looks like is only really possible with a Lisp. Ruby is my programming language for most of my day to day work and the simple metaprogramming it makes possible makes me wish that it could do macros too, though how it could do that without also transforming into Lisp is another matter.

-----

1 point by Pauan 2868 days ago | link

"[...] though how it could do that without also transforming into Lisp is another matter."

My experiments with Nulan have convinced me that it's easier (and better) to start with a Lisp and then add Ruby syntax which desugars to S-expressions, rather than trying to turn Ruby into a Lisp.

-----

2 points by rocketnia 2868 days ago | link

I assume by "s-expressions" we mean nested lists whose first elements are often symbols which represent operators. Apparently, Ruby already has built-in support for that code representation: http://stackoverflow.com/questions/6894763/extract-the-ast-f...

-----

1 point by Pauan 2868 days ago | link

Well then, using that it'd be possible to make Ruby macros.

-----

2 points by rocketnia 2868 days ago | link

Hmm, even if the package for doing this comes with Ruby MRI, I'm not sure there's a corresponding out-of-the-box way to take these s-expressions and get running code. But here's a five-year-old article talking about Ruby2Ruby, a library that converts these s-expressions to strings: http://www.igvita.com/2008/12/11/ruby-ast-for-fun-and-profit...

I don't think a third-party tool should count as language support, unless (and inasmuch as) the people maintaining the language regularly go out of their way to avoid breaking that tool.[1][2] This is where I draw the line, because of how it affects backwards compatibility.

Everyday language users tend to think of language updates as backwards-compatible as long as their existing code has the same behavior. But tools like eval(), JavaScript minifiers, and Ruby2Ruby place the bar higher, since their input is arbitrary language code, and it might be written in a newer language version than the tool itself. Incidentally, even a program that calls these tools will continue to work after a language update, as long as the input of these calls is all generated programmatically, rather than directly taken from the program's input.

[1] Well, I guess the term "third-party" might not apply in that case!

[2] I have no idea how much I'm actually talking about Ruby2Ruby, and this Ruby s-expression format in general, when I make this comment.

-----

2 points by lark 2863 days ago | link

I too wish Arc could be embedded. The discussion on the size of a language core is suspiciously parallel to the discussion on the size of an operating system core.

-----

1 point by kinleyd 2868 days ago | link

Since I'm on a roll here, albeit at another level entirely... thanks dido, and more power to you too!

-----

2 points by dido 2876 days ago | link | parent | on: What is the arity of [uniq]?

Well, how did the change not break setforms in arc.arc? Was the original definition for make-br-fn silently overwritten somewhere? Last I checked there is still a (let argsyms (map [uniq](cdr expr) ...) inside of the definition of setforms in arc.arc even in the latest versions of Anarki. This will not work with the change to bracket functions that has been made.

Frankly, I think that bracket functions should always have at least one argument permitted.

-----

3 points by rocketnia 2876 days ago | link

"Well, how did the change not break setforms in arc.arc? Was the original definition for make-br-fn silently overwritten somewhere?"

Yes. The [...] syntax is redefined in make-br-fn.arc after arc.arc is loaded.

I described my opinion of this at https://github.com/nex3/arc/commit/957501d81bafecab070268418.... It did break my code, but I don't mind breaking changes these days (maybe because I'm hardly a user!), and I think a:b syntax is already more useful than either version of [...].

-----

3 points by rocketnia 2876 days ago | link

"Frankly, I think that bracket functions should always have at least one argument permitted."

Groovy does this. The Closure syntax { ... } is shorthand for { it = null -> ... }, which is a one-argument Closure whose argument "it" defaults to null. When I moved from Groovy to Arc, I missed this.

-----

2 points by akkartik 2876 days ago | link

How do you feel about tossing out arity checking altogether in this instance? I could just add a rest arg so extra args never throw errors.

-----

2 points by dido 2874 days ago | link

Well, that would be helpful. After all the [...] syntax is meant to be a short-cut.

-----

2 points by akkartik 2874 days ago | link

Done: https://github.com/nex3/arc/commit/80571a42a6.

After some readability-golfing:

  (mac make-br-fn (body)
    (let args (rem '__ (dedup:sort > -mbf-argsyms.body))
      (if args
        (= (cdr lastcons.args) '__)
        (= args '__))
      `(fn ,args ,body)))
(https://github.com/nex3/arc/commit/a8a4008fed)

And after still more readability-golfing:

  (mac make-br-fn (body)
    (let args '__
      (each arg (dedup:sort > -mbf-argsyms.body)
        (pushnew arg args))
      `(fn ,args ,body)))akkartik/wart
(https://github.com/nex3/arc/blob/d03eed03f5/lib/make-br-fn.a...)

In the process, though, I've made some more backwards-incompatible changes. pushnew now behaves like push on non-lists or dotted lists.

  > (= x 3)
  > (push 2 3)
  (2 . 3)

  > (= x 3)
  > (pushnew 3 x)
  3
  > (pushnew 2 x)
  (2 . 3)
What do people think of this? Basically I want all list operations to treat atoms as degenerate dotted lists. Does this have any adverse implications for other aggregates (tables)?

All my changes, if you're curious: https://github.com/nex3/arc/compare/a8a4008fed...d03eed03f5

Update: As I expected, taking out the round-trip to table in the first version above saves us some consing. All the changes to get to the second version don't seem to result in any extra consing over the first.

  $ git checkout 80571a42a6
  $ racket -f as.scm
  arc> (load "cons.arc")  ; fallintothis's http://arclanguage.org/item?id=11122
  arc> ([+ _1 _2] 2 3)
  5
  arc> cons-count*
  20

  $ git checkout a8a4008fed
  $ racket -f as.scm
  arc> (load "cons.arc")
  arc> ([+ _1 _2] 2 3)
  5
  arc> cons-count*
  16

  $ git checkout d03eed03f5
  $ racket -f as.scm
  arc> (load "cons.arc")
  arc> ([+ _1 _2] 2 3)
  5
  arc> cons-count*
  16

-----

2 points by rocketnia 2874 days ago | link

"What do people think of this? Basically I want all list operations to treat atoms as degenerate dotted lists. Does this have any adverse implications for other aggregates (tables)?"

I don't see a problem, but I may not be the right person to ask. If it came down to it, I'd be willing to use 'table-pushnew, 'alist-pushnew, 'maxheap-pushnew, and so on. :-p

What other list operations do you have in mind? While 'pushnew makes sense to use with degenerate dotted lists, it's a very special case: It only clobbers the list at a finite depth, and that depth is actually zero. (Or zero in the unmodified list, anyway.)

-----

1 point by akkartik 2874 days ago | link

"What other list operations do you have in mind?"

Well, I've already changed reclist and some so things like this work:

  > (find 3 3)
  3
  > (find 3 '(2 . 3))
  3
  > (find 4 '(2 . 3))
  nil
More: https://github.com/nex3/arc/blob/76d078bcd0/arc.arc.t. I expect I've still missed some cases, and I'll keep tweaking them as I run into them until someone objects.

-----

3 points by rocketnia 2874 days ago | link

Oh, whoops. I forgot 'pushnew actually does traverse the input list to find out whether the element is new. I was thinking of 'push. XD

I disagree with the notion of membership you're using for dotted lists. If I 'pushnew nil onto (1 2 3 . nil), I want to get (nil 1 2 3 . nil), and it won't work that way if nil is already considered to be a member.

-----

1 point by akkartik 2873 days ago | link

Hmm, it seems to work:

  arc> (= x '(1 2 3))
  (1 2 3)
  arc> (pushnew nil x)
  (nil 1 2 3)

-----

2 points by rocketnia 2872 days ago | link

I think the membership check you're using is like this:

  (The final cdr is an element iff it isn't nil.)
  If the list is...
    A cons cell:
      If the car is the value we're looking for, succeed. Otherwise,
      continue by searching the cdr.
    Nil:
      Fail.
    A non-cons, non-nil value:
      Continue by comparing the list to the value we're looking for.
I feel we could simplify this specification by rewriting the last two cases in one of these ways, ordered from my least to most favorite:

  (The final cdr is an element.)
  (This breaks (pushnew nil ...) in existing Arc code.)
  ...
    A non-cons value:
      Continue by comparing the list to the value we're looking for.
  
  (The final cdr is not an element.)
  ...
    A non-cons value:
      Fail.
  
  (The final cdr is always nil and is not an element.)
  (Arc 3.1 already uses this.)
  ...
    Nil:
      Fail.
    A non-cons, non-nil value:
      Raise an exception.
By using the "element iff it isn't nil" approach, you're able to use 'pushnew to traverse the simple argument lists you build as intermediate results of that 'make-br-fn implementation. But I don't know if it's worthwhile to complicate the notion of "list" just to accommodate a special case of argument lists.

-----

1 point by akkartik 2872 days ago | link

Yeah, that's a valid summary of what I've done.

"I don't know if it's worthwhile to complicate the notion of "list" just to accommodate a special case of argument lists."

Yeah I see your point. My aim was to extend the correspondence between the syntax for vararg params, rest params and lists of regular params. But those features merely match a (proper) list of args to some template. I'm entangling the template with the notion of a list. Hmm..

-----

More