Arc Forumnew | comments | leaders | submit | stefano's commentslogin

The source of this problem is not only in chaining itself: redefining standard functions will almost always lead to problems when distinct libraries redefine the same function. That's why other languages have module systems. Moreover, another problem arises from this chaining: performance. The more the chain is long, the more function calls it will need to get the original behavior. Were you thinking about a multimethod dispatch such that of CLOS?

-----

3 points by almkglor 6345 days ago | link

> The source of this problem is not only in chaining itself: redefining standard functions will almost always lead to problems when distinct libraries redefine the same function.

But this should be supported. For instance, in C++, you can redefine the "standard function" operator+ :

  Foo something(Foo a, Foo b){
      return a + b;
  }
Or in Ruby, you can redefine the "standard function" each:

  a = Foo.new()
  a.each{ |i|
    puts i
  }
C++ does it by strict static typing, while Ruby does it by attaching to the object. Neither way feels very Arcish; what is a better Arc solution?

> Were you thinking about a multimethod dispatch such that of CLOS?

Possibly, if we can hack this into Arc somehow.

-----

3 points by stefano 6345 days ago | link

The CLOS solution is pretty similar to the Ruby solution: it checks the type passed to the function and dispatch to the correct function. The big difference is that CLOS looks at all the arguments, while Ruby only at the first argument (the object). The CLOS approach seems to me the best to solve the problem, but I think that applying it to functions as basic as car would kill performance without a really good optimizer.

-----

3 points by almkglor 6344 days ago | link

> The CLOS solution is pretty similar to the Ruby solution: it checks the type passed to the function and dispatch to the correct function. The big difference is that CLOS looks at all the arguments, while Ruby only at the first argument (the object)

IMO the difference is big enough for CLOS Way != Ruby Way. ^^

Still, I wonder - how does CLOS implement this? How about for variadic functions?

> but I think that applying it to functions as basic as car would kill performance without a really good optimizer.

Hmm. I've been trying to grok through dynamic dispatching speedup techniques - the Sun Self papers are pretty good, Sun's index doesn't have the papers themselves but you can ask Google for them.

-----

2 points by stefano 6343 days ago | link

> Still, I wonder - how does CLOS implement this?

I think that for every method with the same name an hash table indexed on the types of the argument is created, and when the method is called, a lookup is made on the table.

> Hmm. I've been trying to grok through dynamic dispatching speedup techniques - the Sun Self papers are pretty good, Sun's index doesn't have the papers themselves but you can ask Google for them.

I've found those papers in the past (a few weeks ago), but I've never found the will to read them, they are written for people already in the compilers' world and I'm still learning the basics about compilers. BTW, a good type inferencer + a good function inliner could solve the problem, but I wouldn't know even where to start to implement them :(.

-----

2 points by almkglor 6343 days ago | link

The gist of the papers are mostly this:

Use a "Polymorphic Inline Cache" (PIC). Basically if a piece of code could call several different methods, we figure out which one it is, then we create a copy of the calling function which does the type checking at the top and specialize all method calls to that type:

  (defm method ((t n my-type))
    (my-type::method n))
  (defm method ((t n your-type))
    (your-type::method n))
  (defm other-meth ((t n my-type))
    (my-type::other-meth n))
  (defm other-meth ((t n your-type))
    (your-type::other-meth n))

  (def general-function (n)
    (+ (method n) (other-meth n)))

  (general-function (your-type-creator 42))
  ===>
    (do
      (def general-function (n)
        (if (isa n 'your-type)
          (+ (your-type::method n) (your-type::other-meth n))
          (+ (method n) (other-meth n))))
      (general-function (your-type-creator 42)))
Everything else in the papers that go beyond the PIC is mostly about debugging and making the PIC lazy.

Edit:

Okay, I've been thinking. Basically the call* table is about specializing on 'apply, and we might think of 'defcall as:

  (defcall type (val . params)
    (code))
  ==>
  (defm apply ((t val type) . params)
    (code))
Could we possibly generalize this at the language level and make a PIC, say in arc2c/SNAP?

-----

1 point by stefano 6342 days ago | link

To do the optimization when we see

  (general-function (your-type-creator 42))
we need a type inferencer to discover the return type of your-type-creator. The cache should also be able to change. For example I could write:

(general-function (your-type-creator 42))

(set your-type-creator another-type-creator)

(general-function (your-type-creator 42))

Now the optimization doesn't work if the cache doesn't change. This seems a rare case, though, and it's useless to optimize rare cases.

-----

1 point by almkglor 6342 days ago | link

Actually we don't: general-function is actually defined this way:

  (with (PIC (table) ;init empty table
         num-calls 0
         orig-fn
         (fn (n)
           (+ (method n) (other-meth n))))
    (def general-function (n)
      ((aif
         ; don't infer type: just look at the runtime type
         (PIC:type n)
            it
         (is optimization-trigger-level** (++ num-calls))
            (do (= num-calls 0)
                (= (PIC:type n)
                   (optimize* orig-fn (type n))))
            orig-fn)
        n)))
Basically s/type inference/just look at it directly/

-----

1 point by eds 6343 days ago | link

> Still, I wonder - how does CLOS implement this? How about for variadic functions?

I don't think CLOS lets you check types on &rest, &optional, or &key parameters. So you couldn't use CLOS for the current behavior of '+.

Also note that CLOS only works on methods with "congruent lambda lists", that is, methods with the same number of required, optional, and keyword arguments. So you can't have

  (defmethod foo ((a type-a)) ...)
  (defmethod foo ((b type-b) &optional (c ...)) ...)

-----

1 point by almkglor 6343 days ago | link

ah, I see. So I suppose this greatly simplifies things then.

Hmm. This certainly seems easier to hack into arc. We could have each method start with a generic function whose lambda list we have to match.

As an aside, currently the base of Arc lambda lists are simply &rest parameters (i.e. optional parameters are converted into rest parameters with destructuring). Should we match on the plain rest parameters or should we properly support optionals?

-----

1 point by stefano 6354 days ago | link | parent | on: restartable: modifier

The ultimate goto! :)

-----


If someone wants to implement an IDE for Arc, or any other desktop (as opposed to web-based) application a GUI library is essential. I had already started the job some time ago by writing a simple GTK binding (very few function at the moment) on Anarki. We could use it as a back-end for a more Arc-ish GUI library.

-----

1 point by conanite 6362 days ago | link

an IDE is coming! if you don't mind it being in java, that is ...

-----

1 point by almkglor 6362 days ago | link

How about in Rainbow? Is it implementable there?

-----

4 points by conanite 6361 days ago | link

i'm working on it. hope to post news this weekend. it's also my daughter's birthday, she's 2 tomorrow :)

-----

1 point by almkglor 6361 days ago | link

LOL. Hope it goes well ^^

-----

1 point by oconnor0 6362 days ago | link

Rainbow?

-----

1 point by oconnor0 6362 days ago | link

http://arclanguage.org/item?id=6002

Ah, that Rainbow...

-----


This is really necessary, especially if we want to keep using short names and not something like:

  (gui-window-set-title w "my title")

-----

1 point by stefano 6365 days ago | link | parent | on: Rethinking macros

The real point in favor of unhygienic macros is that they are less constraining and, personally, I find them easier to write and read than hygienic macros. I don't find a bad idea to have both hygienic macros and unhygienic macros.

-----

1 point by rkts 6364 days ago | link

Sigh...

Obviously I like unhygienic macros when they are necessary. The problem is, I keep writing higher-order functions and getting tired of typing the "fn" over and over, and then I have to convert the function to a macro, making it twice as long and hard to read. Hygienic macros help, but they still are not as easy to write as plain functions.

I know I'm not the only person who has this problem, but maybe I'm the only one who realizes I have this problem. I see people on the Internet raving about the AMAZING POWER of macros, and most of their examples are just higher-order functions with some small cosmetic changes. Most of the macros in Arc are of the same kind.

I'm not denying that macros are powerful. I just think there is a gross inefficiency in using them where you shouldn't have to, just because of minor syntactic concerns.

I wanted to solve this with a short, clean syntax for function literals, but I haven't been able to come up with one and neither, apparently, has anyone else. So instead I decided to try something that would generate macros out of HOFs.

I thought this would be evident from my post, but apparently it wasn't.

-----

1 point by shader 6364 days ago | link

Maybe we could use { args | body } ? I don't think the braces are taken.

Now, maybe that's a bad idea; instead, we could redefine the brackets, so that a pipe divides args and body, and if it doesn't have a pipe, it defaults to binding _ ? I don't know how hard that would be, or some variation on the concept, but it would be a bit shorter than (fn (args) (body)), if you don't want to type that all the time.

And how exactly does 'w/hof work, as defined so far? And if it's "standard" now, why not just implement it?

-----

3 points by almkglor 6364 days ago | link

Since Anarki defines [ ... ] as (make-br-fn ...), it's actually possible to have the syntax:

  [params || body]
Which would be:

  (make-br-fn (params || body))
(The double bar is needed because a single | is reserved for weird symbols)

By simply redefining make-br-fn, you can redefine the syntax of [ ... ] to an extent

-----

1 point by shader 6364 days ago | link

I wondered if redefining [...] might be possible. It seems to me that the new double bar syntax is practically a super-set of the old one: if it doesn't have the double bar, just treat it like the old bracket function and define _ to be the argument; otherwise use the argument list provided. Including an empty list, I would hope.

Any word on how hard that would be?

-----

1 point by almkglor 6364 days ago | link

  (let old (rep make-br-fn)
    (= make-br-fn
       (annotate 'mac
         (fn (l)
           (if (some '|| l)
               (do
                 your-stuff)
               (old l))))))
Be careful of supersets: someone's code might unexpectedly break ^^

-----

1 point by shader 6364 days ago | link

Isn't that to be expected in an evolving open source language :)

Do you think || is the best choice, or something else?

-----

2 points by rkts 6364 days ago | link

I think it's a bad choice, personally. I'm not crazy about the single pipe either, but || is awful.

Tangent: this may be a dumb question, but do we really need the pipe character for symbols? I know I've never used it. Why not disallow spaces (and the like) in symbols, and free the pipe for new syntax?

-----

2 points by shader 6364 days ago | link

If you don't like the pipe, then recommend something :)

Other possibilities, in no particular order:

  [ # ]
  [ - ]
  [ = ]
  [ -> ]
  [ : ]
  [ => ]
  [ > ]
  [ ~ ]
  [ % ]
  [ ! ]
  [ $ ]
  [ ^ ]
  [ & ]
  [ * ]
  [ @ ]
  [ + ]
  [ | ]
  [ || ]
  [ ? ]
Most of those are either bad looking or already taken. Anything stand out as a good / ok / not bad choice?

-----

2 points by almkglor 6364 days ago | link

:, =>, and -> don't look bad.

# can't be redefined

Let's try some mockups:

  [ a b c :
    (/ (+ (- b) (sqrt (+ (* b b) (* 4 a c))))
       (* 2 a))]

  [: (thunk this)]

  [ a b c ->
    (/ (+ (- b) (sqrt (+ (* b b) (* 4 a c))))
       (* 2 a))]

  [-> (thunk this)]

  [ a b c =>
    (/ (+ (- b) (sqrt (+ (* b b) (* 4 a c))))
       (* 2 a))]

  [=> (thunk this)]

-----

1 point by shader 6350 days ago | link

So, did we ever make a decision about this? Does someone who knows more than I do about this want to implement it?

Also, is there a way to compose or nest these lambda shortcuts? Or would that make this almost impossible to implement?

-----

1 point by almkglor 6349 days ago | link

Nesting doesn't seem impossible: the reader, I think, will handle nesting as:

  [foo [bar]]

  (make-br-fn (foo (make-br-fn (bar))))
As for implementation, it's easy:

  (given ; this gives us access to the old
         ; implementation of [] syntax; it
         ; is used when we don't find the
         ; separator
         old (rep make-br-fn)
         ; use a variable to easily change
         ; the separator
         separator ': ;for experimentation
    (= make-br-fn
       ; a macro is just a function that has
       ; been tagged (or annotated) with the
       ; symbol 'mac
       (annotate 'mac
         ; the reader reads [...] as
         ; (make-br-fn (...))
         (fn (rest)
               ; find the separator
           (if (some separator rest)
               ; note the use of the s-variant givens
               ; the "s" at the end of the name of givens
               ; means that the variables are specifically
               ; bound in order, and that succeeding variables
               ; may refer to earlier ones
               (givens ; scans through the list, returning
                       ; an index for use with split
                       ; (no built-in function does this)
                       scan
                       (fn (l)
                         ((afn (l i)
                            (if (caris l separator)
                                i
                                (self (cdr l) (+ i 1))))
                          l 0))
                       ; now do the scan
                       i (scan rest)
                       ; this part destructures a two-element
                       ; list
                       (params body)
                         ; used to get around a bug in split
                         (if (isnt i 0)
                             (split rest i)
                             (list nil rest))
                 ; it just becomes an ordinary function
                              ; body includes the separator,
                              ; so we also cut it out
                 `(fn ,params ,@(cut body 1)))
               ; pass it to the old version of make-br-fn
               ; if a separator was not found
               (old rest))))))
Edit: tested. Also reveals a bug in split: (split valid_list 0) == (split valid_list 1)

  (= foo [ i :
           [ : i]])

  ((foo 42))
edit2: p.s. probably not really easy much after all^^. As a suggestion, (help "stuff") is good at finding stuff.

edit3: added comments

-----

1 point by shader 6349 days ago | link

Hmm. It doesn't seem to work with the older version. If I try ([+ _ 10] 3) it complains: "reference to undefined identifier: ___"

It used to complain "#<procedure>: expects 1 argument, given 3: + _ 10", but something seems to have changed between updates :)

-----

1 point by almkglor 6349 days ago | link

Have you tried restarting Arc and then repasting the code?

Probably some dirt left from older versions ^^

-----

1 point by shader 6363 days ago | link

I agree, those aren't bad.

I think that out of those, : makes the most sense. They all make logical sense with arg lists, but : looks the best without any.

-----

1 point by almkglor 6364 days ago | link

> Tangent: this may be a dumb question, but do we really need the pipe character for symbols? I know I've never used it. Why not disallow spaces (and the like) in symbols, and free the pipe for new syntax?

Wanna start implementing a reader for Arc?

-----

1 point by rkts 6363 days ago | link

Looks like this is configurable in MzScheme. Do

  (read-accept-bar-quote #f)

-----

1 point by rkts 6364 days ago | link

How would you write a function with no arguments?

-----

1 point by shader 6364 days ago | link

leave the part before the pipe empty, I suppose

  {| body}
it might need a space between the brace and the pipe, but I don't know

-----

4 points by stefano 6365 days ago | link | parent | on: Poll: Priorities

What kind of problems exactly does get-http.arc give you? It's a translation of a piece of Common Lisp code I had written to be able to fetch RSS feeds. The CL version worked well for the task.

-----

1 point by antiismist 6364 days ago | link

Here is what happens:

  arc> (get-request (str->url "http://yahoo.com/"))
  Error: "procedure ...narki/arc/ac.scm:1231:11: expects 2 arguments, given 3: 
  #hash((\"CONTENT-TYPE\" . \"text/... \"LOCATION\" #hash((\"CONTENT-TYPE\" . \"text/..."

(I think the problem lies with me, in that I don't know how to use the library...)

-----

3 points by stefano 6364 days ago | link

I've tried your example and gives me (obviously) the same error, but trying

  (get-request (str->url "http://www.yahoo.com/"))
works. I will investigate further.

BTW, get-request returns a list containing the header of the response and a string containing the page (when it doesn't raise and error, of course).

Edit: bug solved. New version on github!

-----

1 point by antiismist 6363 days ago | link

Thanks you are awesome. So what do you use for RSS parsing?

-----

2 points by stefano 6363 days ago | link

In my CL project I thought to parse it manually, but I've abandoned the project after finishing the HTTP part. I've never tried using it, but you could have a look at treeparse in Anarki, if you haven't done already.

-----

1 point by stefano 6366 days ago | link | parent | on: Poll: Priorities

There is a wiki on github. For the moment I think putting all libraries on Anarki should suffice. When the libraries begin to proliferate (if this will ever happen) we could start thinking about something like cpan.

-----


To do async IO there are 2 ways: use OS threads or use green threads and OS-specific asynchronous I/O calls. Either way, the VM should, obviously, abstract away the OS-dependant features. Using OS threads seems easier to me.

-----

1 point by almkglor 6367 days ago | link

Both ways seem quite OS-specific to me ^^

Hmm. This will also have to be done in arc2c too.

-----

1 point by stefano 6367 days ago | link

I don't think there is some way to avoid using OS specific functions for async IO. Abstracting this shouldn't be very hard, though.

-----

1 point by shader 6366 days ago | link

Well, there is a Boost library for cross-platform threading. We could just use that and "simulated" async using threads to offload the synchronous io. Now, maybe that's a bad idea, and we need to use some abstracted os specific async libraries, but it would work.

Boost also seems to have plenty of libraries that would be useful for SNAP. How is that going anyway, almkglor? I should probably ask questions pertaining to that subjects in it's own thread.

Pretty quiet around here. Hopefully that means everyone's busy ;)

-----

1 point by almkglor 6366 days ago | link

> Well, there is a Boost library for cross-platform threading. We could just use that and "simulated" async using threads to offload the synchronous io.

Boost also has asio, but it appears to be concentrated more on socket I/O than I/O in general.

What I would like is to have a general-purpose asynchronous I/O library, which would handle all details of asynchronicity.

Basically, I intend to allow SNAP to be compiled in two modes:

1. Single worker thread, multiple process. For cases where the OS doesn't have decent threads (dorky embedded systems? LOL, quite a few of the embedded systems I've seen recently actually have decent threads)

2. Multiple worker OS-level threads, single runqueue.

async I/O is needed for 1, and would be nice for 2: in case 2, if a worker thread blocks on synchronous I/O, the others will still fetch processes to run from the runqueue, although the blocked worker thread is one less thread that can work.

However in case 1, there's just one thread, so it can't block.

-----


The positive thing of Arc being implemented in top of MzScheme is that this way you can easily modify the core language and try new features without messing with assembly or virtual machines, but in the long run an independent implementation seems to me inevitable. Arc2c is moving this way and I think it will eventually abstract over C, maybe by targeting a general bytecode that can be translated to C, to assembly or interpreted by a VM (this is I think what SNAP is supposed to do).

-----

1 point by mr_luc 6347 days ago | link

Absolutely.

I constantly see comments bemoaning the lack of progress in defining new language ideas in Arc. I don't say I agree 100% with this complaint, although probably the closest that the community has come to this is with hcase.

And of course pg gives library design the same status as language design -- or Arc does, by being so damned flexible. It's not as though Arc is standing still.

But really, the thing that's going to advance the language will be when we use Arc to make things! And that takes some time, and the things we want to make may not be all that important but they'll be interesting to us.

And so while Arc's built-on-PLT status is an obvious hindrance to winning language pissing matches, it means that little is preventing us from doing very interesting things. In fact, that's why FFI's have been such an exciting area of discussion; many of us were limping along with fun little pet projects in another language that wasn't ideal but that had a convenient interface to a couple of C libraries.

In my case, ruby has a good interface to Gosu (for smooth, fast, trouble-free operations on images, even big ones, abstracting away a good chunk of opengl use cases while still letting you use opengl where you want to) and chipmunk (for relatively pain-free 2D physics), and so I made a little screen-saver that has my photographs, mixed with random pg whodehouse quotes from drones.com, tumbling down the screen waterfall-style, and then set about cooking my own code. I got it down to under 100 lines with room for improvement.

(I was living in a fishing village in pacific south america for the last 6 months, with nothing but a $300 windows vista craptop a friend loaned me, so my projects were necessarily of the silly, one-off, only-until-the-surf-gets-good-again variety. I just got back a week ago.)

At the moment, I'm mulling over redoing the same project in Arc, and so my first step is figuring out how to use Chipmunk and Gosu in PLT Scheme. I've never used a C lib from scheme before, so this could be educational.

But I can at least consider doing this project in Arc, in the short-term, because it's implemented on top of PLT. Bad for winning a pissing contest, good for hacking.

-----


I use Arc because:

1) It's different and I like to try different languages

2) Very compact syntax

3) I think it has a lot of space for evolution.

-----

5 points by shader 6368 days ago | link

Ditto. I especially like the fact that, because there are so few people working on it, I actually have a chance at making a difference. Not a very big chance, since as yet I'm still a noob, but it's still something to hope for ;)

-----

7 points by lojic 6368 days ago | link

I thought it would be cool to be part of new Lisp community also, and it may eventually develop into that, but it's increasingly seeming like just a pet project of pg's that may or may not develop into something more. That's not a criticism - I think it was generous of pg to open source Arc and invite folks to participate, and he's been straightforward with his intentions. It's just a slightly different project than I expected initially.

I'm also a Lisp newbie. If I was already experienced in Common Lisp or Scheme, I might be more interested in investing more time with Arc to learn something new, but at this point, I'm simply looking for the best Lisp to learn. Given that I primarily develop web based software, you'd think Arc might be the one; however the ease of creating simple web apps is offset by the lack of other stuff I need and the question of long term viability.

So, learning PLT Scheme seems like a reasonable course of action because I expect I can bring most of that knowledge back to Arc if it gains more traction. Who knows, if developing web apps with PLT Scheme becomes too painful, maybe I'll just switch back to Arc and give it a few months of learning.

-----

1 point by projectileboy 6363 days ago | link

I agree with your comments, and those above, and I actually think it's a huge strength of this project and why I remain interested. I think way, way too much stuff in the software community is focused on DO LOTS OF STUFF RIGHT AWAY GET IT DONE YESTERDAY WAAAAHHHHHHHH FASTER FASTER FASTER!!! That may be the right way to start a company, but it's a really crappy way to build a "one-hundred year language".

I like the fact that this project is moving slowly, and that the community (from newbies like me to pros like Tilton) has time to try things and reflect, and not in internet time.

I'm 36. I've already learned and thrown away about 3 different major development ecosystems in my career. I accept that I'll continue to do that in order to pay my mortgage, but in my own time, when I'm actually programming for fun, I want to hitch my wagon to something that will actually have some lasting value. And Arc (or at least something very similar) is the best-looking option to me at the moment.

-----

1 point by schtog 6360 days ago | link

True, most language-developers don't have the luxury of skipping backwards compatability.

Obviously this complicates things for library-developers but Arc will be very interesting when it is done.

Even if it might not be something new it will hopefully together with the Arc-community things what´s is wrong with all current LISPs.

-----

More