Arc Forumnew | comments | leaders | submitlogin
1 point by Pauan 4758 days ago | link | parent

Alright, that's better. I feel like we're finally on (more or less) the same page, so we can actually discuss the idea, rather than arguing about what the idea even is.

---

"Maybe we just have a difference in meaning here, or maybe the "scalable" part was crucial to what you meant, or maybe you've changed your mind now. Is it any of those things?"

I think we just have a different view of extensibility.

---

"In fact, in the examples you've given, it would make it a harder problem to solve, thanks to encapsulation."

Um... extending is just as encapsulated as message passing is... I'm not sure what you were trying to say.

---

"Thus I continue to advocate 'extend, which solves the issue of creating new data types and solves even more problems I have (or expect to have)."

Please give a single example where extending is better than message passing, with regards to creating or modifying data types.

I'm not arguing that we get rid of extend. Extend is great, it gets to stay. I'm arguing that implementing the low-level data types using message passing would be less verbose and easier than using extend.

Even if we adopted message passing, extend would still have a large role to play. In fact, I envision extend being used to add/modify/remove messages:

  (def bad-function (m)
    (case m
      'get ...          ; bad-function has a bug in it!
      'set ...))
      
  (extend bad-function (m) (is m 'get) ; fix bad-function, hurray!
    ...)
This could play well with prototypes as well (assuming we used them)... for instance, imagine there was a table-proto* function that served as the base for all tables, and you wanted to add a new message:

  (extend table-proto* (m) (is m 'my-message)
    ...)
Voila, now all tables support your new message.

---

"I believe there is actually a solution to this problem that lets me keep using predicate dispatch, but your style of message-passing doesn't inspire me."

I'd be interested in hearing that.

---

"Your example isn't fair to 'extend. You assume 'iso, 'fill-table, and 'vals would be derived functions in PyArc, [...]"

I'll note that with message passing, it can use a type of 'table, and thus work seamlessly with code that uses isa, whereas in wart you need to create a new type, and then extend code (like iso and filltable) to recognize your new type.

But... let's assume that there was a built-in called `table?` that tested for "tableness", and that the other built-ins used that rather than isa:

  (def clone(x)                                   ;  (def clone(p)
    (annotate 'heir (obj val (table) parent x)))  ;    (let t (table)
                                                  ;      (annotate 'table (fn (m k v)
  (defcall heir h                                 ;                         (case m
    [or rep.h!val._                               ;                           'keys (join (keys p) (keys t))
        rep.h!parent._])                          ;                           'get (or (t k v) (p k v))
                                                  ;                           'set (sref t k v))))))
  (defset heir(h key value)                       ;
    (= rep.h!val.key value))                      ;
                                                  ;
  (def table?(x) :case (isa x 'heir) t)           ;
                                                  ;
  (def keys(x) :case (isa x 'heir)                ;
    (join (keys rep.x!val)                        ;
          (keys rep.x!parent)))                   ;
Not as huge as before, I admit, but still a 2x difference. Does it matter? Perhaps. Perhaps not. But with message passing, you get correct behavior with modules out of the box... you already mentioned that extend will cause problems with modules unless you either A) design the module system so it's no longer doing it's job right, in which case why have the module system at all? or B) jump through hoops.

I'll also note that I dislike how this requires you to create a new type every time you want to extend something... thus even if we moved to a `table?` approach, we'd still be keeping isa and type around, solely to support extensions... So if they're in the language, why not co-opt them for a more useful purpose (like message passing)? Or figure out a better way to create data types that doesn't even use isa?

---

"You use the same (m k v) argument list for the 'keys, 'get, and 'set functions, when really different methods will tend to have different argument lists (at least in my mind)."

You're right: that's done for simplicity, but that is easily solved. For instance, consider a macro like this:

  (object table
    keys () ...
    get (k v) ...
    set (k v) ...)

  (object cons
    call args ...
    car () ...
    cdr () ...)
The system doesn't care how the functions are implemented, as long as they follow the interface. You could use nested functions, currying, inheritance, whatever floats your boat.

---

"Meanwhile, you overlook that the 'extend approach is extensible enough to support a message-passing system like yours inside it (which is what should really matter in the core, IMO)."

Uh huh. And how are you going to get that to play nicely with the built in table type? Or the built in cons type? It's much easier to design things so the built-in types already use message passing.

---

"[...] here's an illustration, extrapolating from my earlier example: [...]"

Now do that with tables. Or lists. Or numbers. Harder, ain't it?

---

"However, I currently don't have enough motivation for encapsulation to consider it a positive aspect of my designs;"

Once again... how is message passing "more encapsulated"?

---

"and at the beginning of your post you claimed that your message-passing only tries to tackle one half of that problem: extending the system with new kinds of value, not extending it with new operations."

Yes, because extend is already pretty darn good at covering the other half. So by combining the two, you can get the best of both worlds.

---

My point is not that "extend can't do that" because extend is definitely powerful enough to do it. OOP is also powerful enough to describe any program. That doesn't mean every program should be written in OOP style.

My point is that message passing takes the creation and modification of types, and makes it easier/shorter. That's all. If you prefer to use the more-verbose extend, then go ahead.



1 point by rocketnia 4757 days ago | link

"Um... extending is just as encapsulated as message passing is..."

If we define a whole bundle of behaviors in one closure, or in a nearby set of closures, then the value can naturally encapsulate variables in the lexical scope. Essentially, the value itself is a closure but with a different interface.

If instead we extend the behaviors at the top level, outside the construction context, then we have to design the value's representation in such a way that those extensions can inspect it. This encourages a lesser degree of encapsulation, which in turn gives future programmers more freedom in using the code. On the other hand, sometimes there are just no other operations worth adding, and this just makes the code more scattered and harder to reason about.

Oh, I overlooked your post at http://arclanguage.org/item?id=14307. The encapsulation aspect is exactly the part that allows for certain optimizations.

---

"Please give a single example where extending is better than message passing, with regards to creating or modifying data types."

Well, I gave an example before the part you replied to, and I gave a more in-depth version in the same post as you replied to. I dunno, I think they're "better," but keep in mind that we're solving different problems: in my case, the expression problem, and in your case, convenient construction of custom data types. Based on the motivations you've expressed, I believe my problem has a strictly broader scope than yours, but by the same token, I do believe a narrower scope can allow for a more elegant special-case solution.

---

"I'm arguing that implementing the low-level data types using message passing would be less verbose and easier than using extend."

That actually sounds like a pretty nifty way to go, not because of verbosity and ease of implementation ('cause I have a pretty high tolerance of complexity in the core) but because it may eliminate some primitive types.

There'd be trouble eliminating all of them though: If you pass 'car a message, then when that message is parsed from the argument list, there may be an infinite loop.

---

"Voila, now all tables support your new message."

I think I specifically said it would be hard to add behaviors low on the prototype hierarchy. :) Specifically, it would be hard to add behaviors to things whose behaviors are determined by their constructors, rather than their prototypes; I'll get to this topic at ( * ).

---

"I'd be interested in hearing [how you plan to reconcile predicate dispatch and modules]."

I think I'd rather start a new thread for that. I'll try to get around to it soon.

---

"I'll note that with message passing, it can use a type of 'table, and thus work seamlessly with code that uses isa, whereas in wart you need to create a new type, and then extend code (like iso and filltable) to recognize your new type."

I haven't heard you say what happens when something needs to act seamlessly both as a 'table and as an 'input (my example at http://arclanguage.org/item?id=14258). You did have an example of alists being tables and cons cells, but I don't see how it would work, since you said (type my-alist) was 'cons. (http://arclanguage.org/item?id=14261)

---

"But with message passing, you get correct behavior with modules out of the box..."

( * ) Actually, I don't see the connection between message-passing and modules at all. That's probably your point, but if I did end up wanting to add a new operation on heirs (a desire you don't claim to be prepared for), the most straightforward way would be to redefine 'clone completely, in a way that intentionally defies module separation so that all the program's heirs use the new definition.

---

"[...]even if we moved to a `table?` approach, we'd still be keeping isa and type around, solely to support extensions[...]"

In http://arclanguage.org/item?id=14265 I mentioned that we could get rid of 'type and 'rep completely by using a (deftype wrap-foo unwrap-foo isa-foo) form, but that I'd prefer to let people have access to 'type and 'rep too, if only for emergencies.

---

"For instance, consider a macro like this"

How about failcall and a macro like this:

  ; This makes a new type, extends 'keys, 'ref, and 'sref with rules
  ; that check that their first argument is that type and delegate to
  ; its inner behaviors, and defines a function 'fn-make-foo and a macro
  ; 'make-foo for constructing values of the new type.
  ;
  (def-extension-type fn-make-foo make-foo
    keys ref sref)
  
  (make-foo
    ()                         ...
    (key (o default))          ...
    (new-val key (o default))  ...)
With failcall, there'd be no need to extend things like 'supports-keys too, and 'isa would be mostly redundant.

Note the fact that you have to say 'def-extension-type this way, but that each construction of the type is slightly shorter since it doesn't have to reiterate which "methods" it implements.

Anyway, I expect one could actually duplicate this exact syntax in a message-passing system that has failcall. So you probably still have a potential brevity advantage, thanks to the fact that your problem has a narrower scope than mine, but it's pretty close.

---

"Uh huh . And how are you going to get that to play nicely with the built in table type?"

You're right, if message-passing is merely a library, there's a good chance it won't be seamless with existing things in the language.

---

"Now do that with tables. Or lists. Or numbers. Harder, ain't it?"

I address tables and lists here: http://arclanguage.org/item?id=14337

Numbers, on the other hand, are more interesting. ^_^ It's super-tempting to make them efficient with the help of algorithms that manipulate the binary representation of the number. IMO, that means numbers should ideally be represented as binary strings.

In practice though, who makes custom types of numbers? Methinks there are two kinds of custom number type: small bundles of abstract properties, which 'extend is good for, and large bundles of optimized algorithms, which neither of these approaches is especially good for. :-p

-----

1 point by Pauan 4757 days ago | link

"If instead we extend the behaviors at the top level, outside the construction context, then we have to design the value's representation in such a way that those extensions can inspect it."

Yeah, that's what message passing does... gives a way to get at the internal details of the function. The alternative is to use a closure + extend. Both systems use closures to hide internal data, they just have different interfaces for "getting at" the internal data.

---

"Based on the motivations you've expressed, I believe my problem has a strictly broader scope than yours, but by the same token, I do believe a narrower scope can allow for a more elegant special-case solution."

Yeah, you're right. A good solution to the expression problem would be nice, but in the meantime, how about sprinkling a little sugar on the current difficult areas?

---

"There'd be trouble eliminating all of them though: If you pass 'car a message, then when that message is parsed from the argument list, there may be an infinite loop."

How so? If cons is defined as follows:

  (def cons (x y)
    (object cons
      car () x
      cdr () y))
And then `car` and `cdr` are defined as follows:

  (def car (x)
    (attr (coerce x 'cons) 'car))

  (def cdr (x)
    (attr (coerce x 'cons) 'cdr))
Then where's the problem?

---

"I think I specifically said it would be hard to add behaviors low on the prototype hierarchy. :) Specifically, it would be hard to add behaviors to things whose behaviors are determined by their constructors, rather than their prototypes;"

But isn't a prototype a constructor? The whole point of prototypes are that they remove the distinction between class/instance, so in a sense, every prototype is both an instance and a class at the same time.

---

"I haven't heard you say what happens when something needs to act seamlessly both as a 'table and as an 'input"

I was actually mulling that over last night... I see two (compatible) ways to do that: duck typing and allowing values to have multiple types.

For instance, let's say you wanted to make a table that also happens to be an iterator... tables have a 'keys, 'call, and 'set message. Iterators have a 'next message. So you would just combine them together:

  (object table
    keys ()    ...
    call (k d) ...
    set  (k v) ...
    next ()    ...)
If code just uses (attr foo 'next) then the above would work, but it fails if the code expects something of type 'iterator. So then comes in the other idea... allowing values to have multiple types at once. If we think of types as being interfaces, this is equivalent to saying, "I support this interface, oh, and that interface too."

  (object (iterator table)
    keys ()    ...
    call (k d) ...
    set  (k v) ...
    next ()    ...)
Neat. Now code that expects a table works, and code that expects an iterator works too. This is nice because it allows for data to use the same message name, without causing conflicts. Imagine if you created a new data type that uses the 'next message, but it has different behavior from an iterator. If you pass that data to a function that expects an iterator, it would probably fail silently.

But by using types-as-interfaces, this lets you give context to the messages. So an iterator that has a 'next message is different from something else that has a 'next message. If you really don't care about the type, you can just use `attr`.

---

"Actually, I don't see the connection between message-passing and modules at all. That's probably your point, but if I did end up wanting to add a new operation on heirs (a desire you don't claim to be prepared for), the most straightforward way would be to redefine 'clone completely, in a way that intentionally defies module separation so that all the program's heirs use the new definition."

The point is that you want one module to be able to create a new data type, and then let other modules use it. Because isa uses a symbol, and symbols are interned across modules, (isa foo 'table) would work just fine. This is not true with things like `table?` unless you jump through some hoops or have every module extend __built-ins*

As for adding a new operation on 'heir... it's possible with message passing, but I don't think it would be particularly superior to extend. In fact, it may just be easier to define that new behavior using extend, since message passing is more concerned with data and interfaces than behavior.

---

"Anyway, I expect one could actually duplicate this exact syntax in a message-passing system that has failcall. So you probably still have a potential brevity advantage, thanks to the fact that your problem has a narrower scope than mine, but it's pretty close."

Or to put it another way, rulebooks could be implemented as a library on top of message passing. Also, another concern... efficiency. When you extend a function, it has some overhead, yes? Message passing shouldn't have that overhead, so if you're creating many rules, that could make a difference.

-----

1 point by rocketnia 4756 days ago | link

"Yeah, that's what message passing does... gives a way to get at the internal details of the function."

By the time someone adds all the messages needed to extract the private data out of the type (which they can't do without modifying the library), it'll

---

"A good solution to the expression problem would be nice, but in the meantime, how about sprinkling a little sugar on the current difficult areas?"

Not to belittle, but it doesn't strike me as difficult. I'd rather sprinkle sugar on 'extend if I need to, using something like my 'def-extension-type example, but I haven't needed to yet.

---

"How so? If cons is defined as follows[...]"

If 'car is implemented as a function and you call it and 'car tries to get its argument using 'car, you're in trouble. There's a similar loop if you're sending the message 'car rather than calling a function 'car, but removing the built-in implementation of 'car for built-in cons cells is the problem here, not message-passing.

---

"But isn't a prototype a constructor?"

In your example at http://arclanguage.org/item?id=14331, the (annotate 'table (fn ...)) expression is an example of constructing an instance, and you specify instance-specific behavior in that expression, and they get to refer to some encapsulated data. It's hard, if not impossible, for someone using 'clone as a library to add behaviors to that instance which have access to the same data, whether we used prototypes or not.

---

"So then comes in the other idea... allowing values to have multiple types at once."

Would 'type return a list of types? I see no problem with the way this is going, but it feels more complicated now.

---

"This is nice because it allows for data to use the same message name, without causing conflicts."

Actually, I bet you'd get a conflict when a value needed to implement two interfaces that had completely different meanings for 'next. Whether you consider that important is up to you; it's a common gotcha in OO languages with inheritance. I consider it to be a namespacing issue, and that encourages me to use the global namespace somehow. I'd put 'next in the global namespace as an extensible function, but it could easily be a global variable holding a gensym message name instead.

---

"The point is that you want one module to be able to create a new data type, and then let other modules use it."

Yep, I understand that's what you want, and I understand that's all you expect your technique to tackle.

To phrase the other half of the expression problem that in those terms: I want one module to be able to create a new data type, and then let other modules and application code interpret that value according to interfaces of their own devising.

"Because isa uses a symbol, and symbols are interned across modules, (isa foo 'table) would work just fine. This is not true with things like `table?` unless you jump through some hoops[...]"

If two modules use different notions of table, then a single value like 'table can't uniquely identify them both. There's a name clash.

If they use the same notion, then they should both depend on whatever module introduces that concept anyway (probably the core), and that module can provide them access (and extension access) to functions like 'table? and 'keys.

---

"Or to put it another way, rulebooks could be implemented as a library on top of message passing."

Well, the kind of message-passing system you're talking about makes encapsulation very easy. The kind of 'extend system I'm talking about only thrives on unencapsulated types. They have an adversarious relationship right now.

That said, if you make your message receivers less encapsulated, rulebooks might indeed work as a library. ^_^

Failcall not so much, though. I managed to get failcall mostly working as an Arc library (in Lathe's arc/failcall.arc), but any preexisting code that uses 'apply is a potential abstraction leak. Failcall pretty much needs to be in the core to be seamless.

---

"When you extend a function, it has some overhead, yes?"

Actually, it can, but I expect almost all programs to have a small constant number of rules in each rulebook, and I expect (and encourage) constant-time failure conditions so one rule doesn't encumber the others. So I expect most rulebooks to have constant-time overhead, and that's the same complexity I expect of message-passing.

-----

1 point by Pauan 4756 days ago | link

"If 'car is implemented as a function and you call it and 'car tries to get its argument using 'car, you're in trouble. There's a similar loop if you're sending the message 'car rather than calling a function 'car, but removing the built-in implementation of 'car for built-in cons cells is the problem here, not message-passing."

It's not using the function car to get the message... it's using the symbol 'car. An infinite loop should only happen in the following situation (or similar):

  (= foo (object cons
           car () (car foo)))
In other words... a recursive loop. But that's the fault of the designer of the code, no different than any other recursive loop. This should work fine:

  (= foo (cons 1 2))

  (= bar (object cons
           car () (car foo)))

  (car bar) -> 1
But... let's assume you were using `car` as an example of a situation where something could potentially go wrong... in which case, I'm still not sure what the problem is. You get the internal details by using `attr`, which is a very simple function that shouldn't cause infinite loops:

  (def attr (x . args)
    (apply (coerce x 'fn) args))

  (attr foo 'car)
  (attr foo 'cdr)
---

"In your example at http://arclanguage.org/item?id=14331, the (annotate 'table (fn ...)) expression is an example of constructing an instance"

Hm... solvable, but a lot trickier. That's a good point.

---

"Would 'type return a list of types? I see no problem with the way this is going, but it feels more complicated now."

That would be the simple way to do it, yeah, which is what I was planning on doing with py-arc. There are other ways, but I doubt they'd be any simpler. Then isa would be defined like so:

  (def isa (x t)
    (in (type x) t))
---

"Actually, I bet you'd get a conflict when a value needed to implement two interfaces that had completely different meanings for 'next."

That's where coerce comes into play. Let's say we had two interfaces, foo and bar. They both have the 'something message, but the two interfaces give it different meanings. You could then use coercion to choose which meaning you want:

  (attr (coerce x 'foo) 'something) ; use foo's meaning
  (attr (coerce x 'bar) 'something) ; use bar's meaning
As for defining the actual coerce rules... that's clunky, so I'll need to think about that.

---

"If two modules use different notions of table, then a single value like 'table can't uniquely identify them both. There's a name clash."

Right, that's why you would use a different type... each type would map to a single interface. Two interfaces == two types.

---

"Well, the kind of message-passing system you're talking about makes encapsulation very easy. The kind of 'extend system I'm talking about only thrives on unencapsulated types. If you make your message receivers less encapsulated, rulebooks might indeed work as a library. ^_^"

I'm really not understanding you when you say "encapsulation", which seems to be very different from the way I think of "encapsulation". Please explain how extend is less encapsulated, and how message passing is too encapsulated.

---

"Actually, it can, but I expect almost all programs to have a small constant number of rules in each rulebook, and I expect (and encourage) constant-time failure conditions so one rule doesn't encumber the others. So I expect most rulebooks to have constant-time overhead, and that's the same complexity I expect of message-passing."

If you wish to create a new type that behaves like tables... you need to extend at least keys, sref, and ref, correct? Now if you wish to have 5 different table types, then that means that those functions need to be extended 5 times. Message passing should always be O(1) but extend is O(n) where n is the number of times the function has been extended.

Probably not a big deal in ordinary programs, but it might matter in a large program that uses libraries and has many rules. Obviously you shouldn't choose message passing solely because it's more efficient, but it is a (minor) benefit.

-----

2 points by rocketnia 4756 days ago | link

"It's not using the function car to get the message... it's using the symbol 'car."

Did you stop reading halfway through what you quoted? "There's a similar loop if you're sending the message 'car rather than calling a function 'car," and here's the loop:

1. The cons pair (car . nil) receives the message 'car with no arguments.

2. The message-passing function implementing the cons pair (car . nil) receives the argument list '(car).

3. The message-passing function implementing foo needs to determine the first element of its argument list to see what message it should handle. It sends that argument list the message 'car with no arguments.

4. Go to 1.

---

That's where coerce comes into play.

I don't see it. I don't think something really counts as two types unless you can use that value as either type right away. The way you're using 'coerce, every method pass will need to have an accompanying 'coerce just to be sure, and a few method-passing functions will need to check the current type as well as the method name. At that point you might as well just combine the type name and the method name into one symbol and go back to square one.

---

"Right, that's why you would use a different type..."

Two modules can be developed by two people who don't realize they're using the "same" type. You might as well tackle the issue of a module system by saying "that's why you would use a different global variable."

---

"I'm really not understanding you when you say "encapsulation", which seems to be very different from the way I think of "encapsulation". Please explain how extend is less encapsulated, and how message passing is too encapsulated."

This again.... Okay, here's an explanation by example. ^_^

  (def counter ()
    (let count 0
      (fn ()
        ++.count)))
If we call (counter), we get a value, and we can tell very little by inspecting this value.

  arc> (= foo (coun­ter))
  #<procedure: counter>
  arc> type.foo
  fn
  arc> rep.foo
  #<procedure: counter>
  arc> (is foo rep.f­oo)
  t
Because we constructed it, we know it will act as a counter, but we can't pass it to other parts of the program and expect them to figure that out, since they only have access to 'type and 'rep (unless they're super-sneaky with 'write or Racket pointer operations).

This value is encapsulated. Specifically, we can think of its implementation as encapsulating the mutable variable 'count by storing it in a lexical closure hidden within the 'fn value. This hypothetical implementation doesn't matter to what we can do with the value, but it does indicate how to reimplement it with less encapsulation:

  (def counter ()
    (annotate 'counter list.0))
  
  (defcall counter (self)
    (++ rep.self.0))
Now we can tell a lot more about the value dynamically:

  arc> (= foo (coun­ter))
  #(tagged counter (0 . nil))
  arc> type.foo
  counter
  arc> rep.foo
  (0)
Future code can observe and change its internal state in ways we didn't imagine. For instance, it might consider things of type 'counter to be iterators, and it might define a utility 'peek that acts on certain iterators, including counters.

  (extend peek (str) (isa str 'counter)
    (inc rep.str.0))
This is future code, such as an application that uses our library. Its developers know what they need better than we do, and they (should) know the representations of our types might change in future versions of the library, so this is a deliberate choice. They could also do the "right thing" and patch the library or contact the library author, but this way they don't have to stop their work, and they can even provide the author with working demos of why such implementation details should be exposed.

So, back to message-passing. Message-passing constructions like (annotate 'table (fn ...)) capture their lexical surroundings just like regular closures do, making it natural to use them for implementations similar to the first version of 'counter.

---

"Now if you wish to have 5 different table types, then that means that those functions need to be extended 5 times."

If you implement those five types using (annotate 'table (fn ...)), then you have to implement three methods per type, for the same overall result. I believe this is what akkartik meant by "moving parens around."

If there's actually duplicated code in those fifteen cases, we both have strategies for that: You can remove some of it using prototype inheritance, and I can remove some of it by way of rulebook "inheritance" using intermediate rulebooks (as explained in http://arclanguage.org/item?id=14330). Of course, we can both use plain old functions and macros too.

-----

1 point by Pauan 4756 days ago | link

"Did you stop reading halfway through what you quoted?"

Nope! Just not understanding why it's a problem, since I don't see how that would cause an infinite loop.

---

"1. The cons pair (car . nil) receives the message 'car with no arguments.

2. The message-passing function implementing the cons pair (car . nil) receives the argument list '(car).

3. The message-passing function implementing foo needs to determine the first element of its argument list to see what message it should handle. It sends that argument list the message 'car with no arguments."

You mean like this?

  ; argument list '(car)

  (fn (m)
    (case m
      'car 'car
      'cdr nil))


  ; cons pair (car . nil)

  (fn args
    (case (attr args 'car)
      'car car
      'cdr nil))
---

"At that point you might as well just combine the type name and the method name into one symbol and go back to square one."

Yeah, I admit it's not that great at handling the situation where a value wants to implement two different interfaces, but the interfaces have the same message name.

---

"This again.... Okay, here's an explanation by example. ^_^"

Alright, I'm starting to get it. Rather than representing types as a function that maps between keys/behaviors, you're representing it as a simple flat list. But that's quite possible to do with message passing as well:

  (def counter ()
    (let methods (obj call  (fn () (++ methods.'value))
                      value 0)
                      
      (annotate 'counter methods)))


  (= foo (coun­ter))
  (type foo) -> counter
  (rep foo)  -> #hash((call ...) (value 0))
I admit that's a tiny bit more verbose than your simple example, but it's nice because rather than exposing the type as an ordered sequence... we expose it as a table. Which means that rather than saying, "the 0th element is the value" we instead say, "the key 'value is the value", which makes it more robust if say... we decide to change it so the value is the 2nd element.

It also serves as self-documenting code. It also allows the built-in functions like `keys` to dispatch on the 'keys method... which doesn't work so well with a list. It also means we can easily write a macro to do the wrapping for us:

  (def counter ()
    (object counter
      call () (++ self.'value)
      value 0))
And if types are represented as tables... then you can use rep + sref to actually modify the internal representation... without needing to use extend at all. Unfortunately, if we went that route, then my simple examples like...

  (fn (m)
    (case m
      'keys ...
      'set  ...))
...wouldn't really work. But that's okay. Implementing types as tables seems like it'd be more flexible and overall better. The only downside is that now they're carrying around an internal table... but how is that worse than carrying around an internal list?

So... would you consider the above (using a table to implement message passing) to be "more encapsulated" than your idea?

-----

1 point by rocketnia 4756 days ago | link

"You mean like this?"

I don't know what you're getting at, but the cons cell (car . nil) and the argument list '(car) are supposed to be 'iso in my example; otherwise, why would I say it's a loop? I'm just referring to them in different ways to emphasize their different roles in the story.

I'm sure you're confused about something else in my example, but I don't know what it is. Here's a quick take at summarizing the issue anyway: Inspecting a list by sending it messages is incompatible with decoding a message by inspecting an argument list.

---

"The only downside is that now they're carrying around an internal table... but how is that worse than carrying around an internal list?"

Here's an old post of mine where I go over the pros and cons: http://arclanguage.org/item?id=12076. (Since then, I've learned to stop worrying about 'rep and 'annotate and love predicate dispatch. I put inheritance-aware multimethods in Lathe after that too, but I found they were less useful than I expected for what I was doing.)

With a predicate dispatch approach, it hardly matters whether the internal representation is a table or a list or even a function which dispatches to a complete getter/setter protocol, since I usually only interact with it in a single clump of extensions. Nevertheless, I do find this feature of list-based types pretty helpful:

  (let (foo bar baz) rep.x
    ...)
---

"So... would you consider the above (using a table to implement message passing) to be "more encapsulated" than your idea?"

Actually, I consider that example to be less encapsulated. Future code can come in and replace the 'call method and add new fields. That would be just fine with me.

...Do note that tables would have to be built into the core that way though. ;) It's to avoid loops again: Getting a table key by sending a message is incompatible with sending a message by getting a table key.

-----

1 point by Pauan 4756 days ago | link

"Here's a quick take at summarizing the issue anyway: Inspecting a list by sending it messages is incompatible with decoding a message by inspecting an argument list."

Okay... let me see if I understand you. What you're saying is, that if a function is given an argument list (like with rest args), then it can't extract the data out of that list, by sending messages? Why not? A cons is represented as the following form:

  (obj type 'cons
       rep ...
       car ...
       cdr ...)
You retrieve those by using attr [1]:

  (attr foo 'car)
  (attr foo 'cdr)
  ...
Now, the argument list is represented as a series of conses, and each individual cons is a table. Incidentally, this is similar to how I'm implementing conses in Python, except I'm using classes rather than tables. So suppose you have the following:

  (= args '(car))
Which is the same thing as this:

  (= args (obj type 'cons
               car 'car
               cdr nil))
               
  (= args.'rep args)
You can extract the car with (attr args 'car), and the cdr with (attr args 'cdr). All conses would be represented this way, including built-in types. So in the following function...

  (fn args ...)
...the variable args would also be represented that way, so calling (attr args 'car) inside the function should work fine. You did mention a recursive loop earlier:

"If 'car is implemented as a function and you call it and 'car tries to get its argument using 'car, you're in trouble."

Which would be represented as follows...

  (def car args
    (car args)
    ...)
...which is obviously a naive infinite loop. But that is solved by using attr:

  (def car args
    (attr args 'car)
    ...)
So I fail to see what that (infinite loop) has to do with message passing. What have I misunderstood?

---

"Nevertheless, I do find this feature of list-based types pretty helpful:"

Hm... that does seem pretty useful/neat/nifty! Not sure if it offsets the advantages of a table, though.

---

"Here's an old post of mine where I go over the pros and cons: http://arclanguage.org/item?id=12076. "

Neat! Great summary. For the record, I'm leaning toward the second form, with type being an attribute on the table. Then `type` can just be a convenience function (or defined for backwards compatibility).

"no way for different types' x fields to be accessed using the same code without doing something like standardizing the field order"

Yeah, that's kinda the killer (for me) for list-based types. Using lists seems so... short and light-weight, though.

Also, I found this amusing: "Hmm, that gives me an idea. Maybe what I miss most of all is the ability to tag a new datatype so that an existing utility can understand it"

Which is precisely what message passing does. But as you say, you seem to have moved onto other... different directions.

---

"...Do note that tables would have to be built into the core that way though. ;) It's to avoid loops again: Getting a table key by sending a message is incompatible with sending a message by getting a table key."

No worse than having fns in the core. :P Especially if I figure out a way to define fn using message passing. Then... tables would be primitives... and fn would be layered on top...?

---

* [1]: Why `attr`? Why not just use (foo 'car)? Because when a table has a 'call key, that will be used when calling it in functional position. In this case, it didn't have a 'call key, but in actual Arc code, conses would have a 'call key, which means that calling (foo 'car) wouldn't actually retrieve the 'car key... it would instead be equivalent to ((foo 'call) 'car). Thus, `attr` is necessary to distinguish between calling a table, and retrieving a key of the table.

-----

1 point by rocketnia 4755 days ago | link

"What you're saying is, that if a function is given an argument list (like with rest args), then it can't extract the data out of that list, by sending messages? Why not?"

Because that just sends arguments to another function, and that function tries to extract them, so it sends arguments to another function, and so on. At no point does any function succeed in determining what its first argument is.

"Why `attr`?"

FWIW, I believe I understand what 'attr is for. ^_^ It passes direct messages to a value without passing a "call" message. It just doesn't solve the paradox.

"But that is solved by using attr"

But how does the 'attr function extract its first argument?

---

"Not sure if it offsets the advantages of a table, though."

Me neither. ^_^ To even the field a bit:

Clojure has its own kinda wordy form of table destructuring. An Arc macro can do better than that when it's only trying to do non-recursive destructuring.

  (w/keys a b c rep.x  ; gets the 'a, 'b, and 'c keys
    ...)
I'm not sure if something this simple would help all the time, since I use alternate names for things pretty often (e.g. when dealing with two things of the same type), but it covers the more common cases.

Personally, I'd probably go for 'def-extension-type and blissfully ignore the internal representation altogether (except when I'm reading things at the REPL, hm...).

---

"['the ability to tag a new datatype so that an existing utility can understand it'] is precisely what message passing does."

It's what 'extend does too. The tagging is just done in a meta way. The way I see things now, the meaning of a value is only the op-understands-value relationships it supports (duck typing?), so extending an op is a valid way to change (tag) the meaning of a value.

---

"Especially if I figure out a way to define fn using message passing. Then... tables would be primitives... and fn would be layered on top...?"

I think you gotta encode behavior in some type or other. I see two ways to avoid functions: You could have tables hold s-expressions and interpret them, or you could have tables encode the behavior themselves. The latter would make tables objects, and they'd be indistinguishable (on a high level) from message-passing closures.

-----

1 point by Pauan 4755 days ago | link

"Because that just sends arguments to another function, and that function tries to extract them,"

Hm... well, I still don't get what you're saying, but I'll try implementing it and see if it's a problem or not.

---

"But how does the 'attr function extract its first argument?"

In a worst case scenario, it can be a built-in and use Python magic to avoid the paradox. Having one built-in (attr) is still waaaay better than having a ton of other built-ins (car, cons, cdr, maptable, etc.)

---

"Clojure has its own kinda wordy form of table destructuring. An Arc macro can do better than that when it's only trying to do non-recursive destructuring."

JavaScript also has a form of object destructuring [1]:

  {a, b} = {a: "foo", b: "bar"}

  // or was it like this...?

  [a, b] = {a: "foo", b: "bar"}
I think something that simple would help 99% of the time, so that's okay with me.

---

"Personally, I'd probably go for 'def-extension-type and blissfully ignore the internal representation altogether (except when I'm reading things at the REPL, hm...)."

That's all well and good for high-level coding, but somebody's going to have to determine what the low-level data type will be. I'm going with a table, in py-arc, thanks to our discussions.

---

"It's what 'extend does too. The tagging is just done in a meta way. The way I see things now, the meaning of a value is only the op-understands-value relationships it supports (duck typing?), so extending an op is a valid way to change (tag) the meaning of a value."

Ah, but the key difference is: where is the operator stored? Right now we're only using global functions as operators, but we could also store them as functions in a table. This is better in some situations... and extend is better in other situations.

So yeah, I'm leaning toward duck typing as well, and actually getting rid of the `type` function completely. My idea is that the "type" of something is based solely on it's prototype: `table` would be both a prototype and a constructor of tables (at the same time). You can then create new prototypes that inherit from `table`, and voila, they're tables.

This is sorta-kinda like how JavaScript does it, except JavaScript actually screwed up prototypes, so we can do a lot better in Arc. For starters, in JavaScript you can access the prototype, but you can't change it. That's definitely gonna change: in Arc, you should be able to dynamically change the prototype.

Something's a table, but want it to be a cons? Just change it's 'prototype attribute and bam, it now inherits from cons. Anyways, then the `isa` function would test for inheritance, rather than type. So rather than saying (isa foo 'table) you'd say (isa foo table), which would mean, "does foo inherit from table?" [2].

As far as I can see, the only downside is that isa would now be O(n) rather than O(1), but the prototype chain should be small enough that it won't make a big difference in ordinary programs.

---

"I think you gotta encode behavior in some type or other. I see two ways to avoid functions: You could have tables hold s-expressions and interpret them, or you could have tables encode the behavior themselves. The latter would make tables objects, and they'd be indistinguishable (on a high level) from message-passing closures."

I'm kinda leaning toward the second approach. Even in a worst case scenario, having two built-in primitives that can't be created in Arc (tables and fns) is still better than the current situation.

---

* [1]: https://developer.mozilla.org/en/New_in_JavaScript_1.7#Destr...

* [2]: I'll note that `isa` would probably be used rarely... just as I rarely use `instanceof` in JavaScript. It's only there on the off chance that somebody really does care what "type" something is.

-----

3 points by Pauan 4755 days ago | link

"JavaScript also has a form of object destructuring [1]:"

Okay, I just realized something... what if we changed destructuring so it worked on tables too? Then this would work:

  (let '(a b c) (obj a 1 b 2 c 3)
    ...)
This would also allow us to get a clunky form of keyword arguments:

  (def foo ('(a b c))
    (prn a b c))

  (foo (obj c 1 b 2)) -> nil 2 1
Which could be made nicer with a little sugar...

  (foo :c 1 :b 2) -> nil 2 1
  (foo {c 1 b 2}) -> nil 2 1
One advantage of this approach is that you can pass either lists or tables to the function:

  (foo '(1 2 3)) -> 1 2 3

-----

1 point by Pauan 4756 days ago | link

By the way, I just had an idea. We could even write the `annotate`, `type`, and `rep` functions in Arc:

  (def annotate (type rep)
    (fn (m . args)
      (case m
        'type type
        'rep  rep
              (apply attr rep m args))))
    
  (def type (x)
    (attr x 'type))
    
  (def rep (x)
    (attr x 'rep))
So calling `annotate` on a value would create a wrapper around it. But this also allows us to do something else... modify the type of an existing value, rather than wrap it.

-----

1 point by Pauan 4756 days ago | link

"If you implement those five types using (annotate 'table (fn ...)), then you have to implement three methods per type, for the same overall result. I believe this is what akkartik meant by "moving parens around.""

...but if creating a single table with extend is twice as verbose as message passing, then creating 5 tables with extend will still be twice as verbose as message passing. That seems to be more than just "moving parens around", though perhaps you could argue that the 2x difference doesn't matter.

Or perhaps you could argue that the 2x verbosity could be curbed with macros... or a construct like def-extension-type... in which case I reply that we can use the same techniques for message passing. But with O(1) efficiency rather than the O(n) efficiency of extend, and with the ability to easily define core functions in Arc itself.

-----

1 point by rocketnia 4756 days ago | link

Yes, that was exactly what I was saying with 'def-extension-type, and I even mentioned that you could use 'def-extension-type verbatim too (not that the name would make sense).

I've just written my reply to "the O(n) efficiency of 'extend" over here: arclanguage.org/item?id=14374

-----

1 point by rocketnia 4756 days ago | link

I missed this the first time around:

"Message passing should always be O(1) but extend is O(n) where n is the number of times the function has been extended."

As I said, the number of extensions of a function is almost always a constant in any given program (and a small one at that). I do believe your constant factor would be smaller than mine, but we can't decide that without runnable code. ^_^

-----

1 point by Pauan 4756 days ago | link

Right, which means you shouldn't be choosing one or the other based solely on efficiency. I mention it for completeness, since we're dissecting the pros and cons of the two approaches. And who knows, somebody might actually find the O(1) approach better for their needs.

However... if message passing can do everything just as good as extend (at least pertaining to data types), then that small efficiency advantage could be enough to tip things over.

-----

1 point by akkartik 4756 days ago | link

"It's not using the function car to get the message... it's using the symbol 'car."

Whoa. Should methods/messages be first-class like functions? Can I pass them around, store them in symbols, generate objects with gensym'd methods? How does 'call by name' interact with all that?

We can provide feedback about these things until we're blue in the face and it won't count for a thing because the first attempt at implementation will run into some blazingly obvious issue that we all missed. Why expend so much collective attention so inefficiently when you can just build it and see what happens?

-----

1 point by Pauan 4756 days ago | link

"Whoa. Should methods/messages be first-class like functions?"

Sure, why not? I actually envision them being implemented as functions, but other ways are possible. In JavaScript, methods are ordinary functions.

As for messages being first-class... they sorta already are, in the sense that the messages are symbols. I don't see much reason to give messages a special type; they can just be symbols.

---

"How does 'call by name' interact with all that?"

Hm... not sure.

---

"Why expend so much collective attention so inefficiently when you can just build it and see what happens?"

Because py-arc is in a pretty sad state right now. :P Once it's spruced up, implementing message passing should be trivial.

-----

1 point by akkartik 4756 days ago | link

"Because py-arc is in a pretty sad state right now. :P Once it's spruced up, implementing message passing should be trivial."

:) I think you're underestimating the difficulty of this. You need isa to somehow simulate a set of messages to an object before deciding that yes, it fits that interface. I have no idea how to do that without causing undesirable side effects. Performance is another concern.

I think interfaces work in Go because it's statically typed.

-----

1 point by Pauan 4756 days ago | link

It doesn't need to simulate anything... if something has a type of 'table, then it uses the table interface. If something has a type of 'stream then it uses the stream interface. Thus, isa could work exactly the same way it does in pgArc[1].

* [1]: Of course, when I try to implement multi-types, then isa would need to change, but that's not strictly speaking necessary right now.

-----

1 point by akkartik 4756 days ago | link

Hmm, so each value in arc would carry around a table mapping messages to functions?

-----

1 point by Pauan 4756 days ago | link

Yes, basically. That means if you want to create a custom table type, you only need to create an annotated function that maps messages (symbols) to behavior (functions), and voila, it all works seamlessly.

---

I find it amusing that your explanation pretty much sums up thousands and thousands of words that I've said. I really am too verbose. ^^;

-----

2 points by akkartik 4756 days ago | link

"I find it amusing that your explanation pretty much sums up thousands and thousands of words that I've said. I really am too verbose. ^^;"

You were probably more intelligible to everyone else; I seem to have been uncommonly dense :) What you're describing is indeed message passing. I think I was misled by your code snippets.

You're basically describing an s-expression syntax for smalltalk.

-----

1 point by Pauan 4756 days ago | link

"You were probably more intelligible to everyone else; I seem to have been uncommonly dense :) What you're describing is indeed message passing. I think I was misled by your code snippets."

I was implementing message passing by using functions + closures, so it's definitely a weird form of it... most languages implement message passing with objects + methods. So, your confusion is understandable.

-----

1 point by akkartik 4756 days ago | link

Once I make the connection to smalltalk, wrapping these objects in the (annotate ..) layer seems redundant. Just get rid of types entirely.

We're probably going to rediscover why a s-expression syntax for smalltalk is a bad idea, but it should be a fun process.

I'm reminded of this thread: http://people.csail.mit.edu/gregs/ll1-discuss-archive-html/m...

-----

1 point by Pauan 4756 days ago | link

"Once I make the connection to smalltalk, wrapping these objects in the (annotate ..) layer seems redundant. Just get rid of types entirely."

So... what... just use duck typing for everything? :P Duck typing is easier, but has a greater possibility for strange bugs and conflicts. But Arc is a LFSP, so that might be a good fit for Arc.

---

Neat! In the link you mentioned, I found this: http://okmij.org/ftp/Scheme/oop-in-fp.txt

Which, in fact, describes something almost completely identical to my message passing idea, except that it's more verbose in Scheme than in Arc. It even demonstrates prototypical inheritance.

The only difference is that they're using a separate function to represent each method... whereas in my examples, I've basically written the methods "inline" so there's only one function. However, I did demonstrate one way to give each method a separate function: http://arclanguage.org/item?id=14368

-----

1 point by akkartik 4756 days ago | link

Are you planning to have all say string values contain a pointer to the same table instance, or give each string value its own copy of the table?

-----

1 point by Pauan 4756 days ago | link

Not sure. Either way could work. That seems to be more of an optimization, though. The interface should work the same either way.

-----

1 point by akkartik 4756 days ago | link

It probably bears thinking about. It'll affect, for example, how you say, "I want all tables to have this new operation."

-----