"It's not using the function car to get the message... it's using the symbol 'car."
Did you stop reading halfway through what you quoted? "There's a similar loop if you're sending the message 'car rather than calling a function 'car," and here's the loop:
1. The cons pair (car . nil) receives the message 'car with no arguments.
2. The message-passing function implementing the cons pair (car . nil) receives the argument list '(car).
3. The message-passing function implementing foo needs to determine the first element of its argument list to see what message it should handle. It sends that argument list the message 'car with no arguments.
4. Go to 1.
---
That's where coerce comes into play.
I don't see it. I don't think something really counts as two types unless you can use that value as either type right away. The way you're using 'coerce, every method pass will need to have an accompanying 'coerce just to be sure, and a few method-passing functions will need to check the current type as well as the method name. At that point you might as well just combine the type name and the method name into one symbol and go back to square one.
---
"Right, that's why you would use a different type..."
Two modules can be developed by two people who don't realize they're using the "same" type. You might as well tackle the issue of a module system by saying "that's why you would use a different global variable."
---
"I'm really not understanding you when you say "encapsulation", which seems to be very different from the way I think of "encapsulation". Please explain how extend is less encapsulated, and how message passing is too encapsulated."
This again.... Okay, here's an explanation by example. ^_^
(def counter ()
(let count 0
(fn ()
++.count)))
If we call (counter), we get a value, and we can tell very little by inspecting this value.
Because we constructed it, we know it will act as a counter, but we can't pass it to other parts of the program and expect them to figure that out, since they only have access to 'type and 'rep (unless they're super-sneaky with 'write or Racket pointer operations).
This value is encapsulated. Specifically, we can think of its implementation as encapsulating the mutable variable 'count by storing it in a lexical closure hidden within the 'fn value. This hypothetical implementation doesn't matter to what we can do with the value, but it does indicate how to reimplement it with less encapsulation:
Future code can observe and change its internal state in ways we didn't imagine. For instance, it might consider things of type 'counter to be iterators, and it might define a utility 'peek that acts on certain iterators, including counters.
This is future code, such as an application that uses our library. Its developers know what they need better than we do, and they (should) know the representations of our types might change in future versions of the library, so this is a deliberate choice. They could also do the "right thing" and patch the library or contact the library author, but this way they don't have to stop their work, and they can even provide the author with working demos of why such implementation details should be exposed.
So, back to message-passing. Message-passing constructions like (annotate 'table (fn ...)) capture their lexical surroundings just like regular closures do, making it natural to use them for implementations similar to the first version of 'counter.
---
"Now if you wish to have 5 different table types, then that means that those functions need to be extended 5 times."
If you implement those five types using (annotate 'table (fn ...)), then you have to implement three methods per type, for the same overall result. I believe this is what akkartik meant by "moving parens around."
If there's actually duplicated code in those fifteen cases, we both have strategies for that: You can remove some of it using prototype inheritance, and I can remove some of it by way of rulebook "inheritance" using intermediate rulebooks (as explained in http://arclanguage.org/item?id=14330). Of course, we can both use plain old functions and macros too.
"Did you stop reading halfway through what you quoted?"
Nope! Just not understanding why it's a problem, since I don't see how that would cause an infinite loop.
---
"1. The cons pair (car . nil) receives the message 'car with no arguments.
2. The message-passing function implementing the cons pair (car . nil) receives the argument list '(car).
3. The message-passing function implementing foo needs to determine the first element of its argument list to see what message it should handle. It sends that argument list the message 'car with no arguments."
You mean like this?
; argument list '(car)
(fn (m)
(case m
'car 'car
'cdr nil))
; cons pair (car . nil)
(fn args
(case (attr args 'car)
'car car
'cdr nil))
---
"At that point you might as well just combine the type name and the method name into one symbol and go back to square one."
Yeah, I admit it's not that great at handling the situation where a value wants to implement two different interfaces, but the interfaces have the same message name.
---
"This again.... Okay, here's an explanation by example. ^_^"
Alright, I'm starting to get it. Rather than representing types as a function that maps between keys/behaviors, you're representing it as a simple flat list. But that's quite possible to do with message passing as well:
I admit that's a tiny bit more verbose than your simple example, but it's nice because rather than exposing the type as an ordered sequence... we expose it as a table. Which means that rather than saying, "the 0th element is the value" we instead say, "the key 'value is the value", which makes it more robust if say... we decide to change it so the value is the 2nd element.
It also serves as self-documenting code. It also allows the built-in functions like `keys` to dispatch on the 'keys method... which doesn't work so well with a list. It also means we can easily write a macro to do the wrapping for us:
And if types are represented as tables... then you can use rep + sref to actually modify the internal representation... without needing to use extend at all. Unfortunately, if we went that route, then my simple examples like...
(fn (m)
(case m
'keys ...
'set ...))
...wouldn't really work. But that's okay. Implementing types as tables seems like it'd be more flexible and overall better. The only downside is that now they're carrying around an internal table... but how is that worse than carrying around an internal list?
So... would you consider the above (using a table to implement message passing) to be "more encapsulated" than your idea?
I don't know what you're getting at, but the cons cell (car . nil) and the argument list '(car) are supposed to be 'iso in my example; otherwise, why would I say it's a loop? I'm just referring to them in different ways to emphasize their different roles in the story.
I'm sure you're confused about something else in my example, but I don't know what it is. Here's a quick take at summarizing the issue anyway: Inspecting a list by sending it messages is incompatible with decoding a message by inspecting an argument list.
---
"The only downside is that now they're carrying around an internal table... but how is that worse than carrying around an internal list?"
Here's an old post of mine where I go over the pros and cons: http://arclanguage.org/item?id=12076. (Since then, I've learned to stop worrying about 'rep and 'annotate and love predicate dispatch. I put inheritance-aware multimethods in Lathe after that too, but I found they were less useful than I expected for what I was doing.)
With a predicate dispatch approach, it hardly matters whether the internal representation is a table or a list or even a function which dispatches to a complete getter/setter protocol, since I usually only interact with it in a single clump of extensions. Nevertheless, I do find this feature of list-based types pretty helpful:
(let (foo bar baz) rep.x
...)
---
"So... would you consider the above (using a table to implement
message passing) to be "more encapsulated" than your idea?"
Actually, I consider that example to be less encapsulated. Future code can come in and replace the 'call method and add new fields. That would be just fine with me.
...Do note that tables would have to be built into the core that way though. ;) It's to avoid loops again: Getting a table key by sending a message is incompatible with sending a message by getting a table key.
"Here's a quick take at summarizing the issue anyway: Inspecting a list by sending it messages is incompatible with decoding a message by inspecting an argument list."
Okay... let me see if I understand you. What you're saying is, that if a function is given an argument list (like with rest args), then it can't extract the data out of that list, by sending messages? Why not? A cons is represented as the following form:
(obj type 'cons
rep ...
car ...
cdr ...)
You retrieve those by using attr [1]:
(attr foo 'car)
(attr foo 'cdr)
...
Now, the argument list is represented as a series of conses, and each individual cons is a table. Incidentally, this is similar to how I'm implementing conses in Python, except I'm using classes rather than tables. So suppose you have the following:
(= args '(car))
Which is the same thing as this:
(= args (obj type 'cons
car 'car
cdr nil))
(= args.'rep args)
You can extract the car with (attr args 'car), and the cdr with (attr args 'cdr). All conses would be represented this way, including built-in types. So in the following function...
(fn args ...)
...the variable args would also be represented that way, so calling (attr args 'car) inside the function should work fine. You did mention a recursive loop earlier:
"If 'car is implemented as a function and you call it and 'car tries to get its argument using 'car, you're in trouble."
Which would be represented as follows...
(def car args
(car args)
...)
...which is obviously a naive infinite loop. But that is solved by using attr:
(def car args
(attr args 'car)
...)
So I fail to see what that (infinite loop) has to do with message passing. What have I misunderstood?
---
"Nevertheless, I do find this feature of list-based types pretty helpful:"
Hm... that does seem pretty useful/neat/nifty! Not sure if it offsets the advantages of a table, though.
Neat! Great summary. For the record, I'm leaning toward the second form, with type being an attribute on the table. Then `type` can just be a convenience function (or defined for backwards compatibility).
"no way for different types' x fields to be accessed using the same code without doing something like standardizing the field order"
Yeah, that's kinda the killer (for me) for list-based types. Using lists seems so... short and light-weight, though.
Also, I found this amusing: "Hmm, that gives me an idea. Maybe what I miss most of all is the ability to tag a new datatype so that an existing utility can understand it"
Which is precisely what message passing does. But as you say, you seem to have moved onto other... different directions.
---
"...Do note that tables would have to be built into the core that way though. ;) It's to avoid loops again: Getting a table key by sending a message is incompatible with sending a message by getting a table key."
No worse than having fns in the core. :P Especially if I figure out a way to define fn using message passing. Then... tables would be primitives... and fn would be layered on top...?
---
* [1]: Why `attr`? Why not just use (foo 'car)? Because when a table has a 'call key, that will be used when calling it in functional position. In this case, it didn't have a 'call key, but in actual Arc code, conses would have a 'call key, which means that calling (foo 'car) wouldn't actually retrieve the 'car key... it would instead be equivalent to ((foo 'call) 'car). Thus, `attr` is necessary to distinguish between calling a table, and retrieving a key of the table.
"What you're saying is, that if a function is given an argument list (like with rest args), then it can't extract the data out of that list, by sending messages? Why not?"
Because that just sends arguments to another function, and that function tries to extract them, so it sends arguments to another function, and so on. At no point does any function succeed in determining what its first argument is.
"Why `attr`?"
FWIW, I believe I understand what 'attr is for. ^_^ It passes direct messages to a value without passing a "call" message. It just doesn't solve the paradox.
"But that is solved by using attr"
But how does the 'attr function extract its first argument?
---
"Not sure if it offsets the advantages of a table, though."
Me neither. ^_^ To even the field a bit:
Clojure has its own kinda wordy form of table destructuring. An Arc macro can do better than that when it's only trying to do non-recursive destructuring.
(w/keys a b c rep.x ; gets the 'a, 'b, and 'c keys
...)
I'm not sure if something this simple would help all the time, since I use alternate names for things pretty often (e.g. when dealing with two things of the same type), but it covers the more common cases.
Personally, I'd probably go for 'def-extension-type and blissfully ignore the internal representation altogether (except when I'm reading things at the REPL, hm...).
---
"['the ability to tag a new datatype so that an existing utility can understand it'] is precisely what message passing does."
It's what 'extend does too. The tagging is just done in a meta way. The way I see things now, the meaning of a value is only the op-understands-value relationships it supports (duck typing?), so extending an op is a valid way to change (tag) the meaning of a value.
---
"Especially if I figure out a way to define fn using message passing. Then... tables would be primitives... and fn would be layered on top...?"
I think you gotta encode behavior in some type or other. I see two ways to avoid functions: You could have tables hold s-expressions and interpret them, or you could have tables encode the behavior themselves. The latter would make tables objects, and they'd be indistinguishable (on a high level) from message-passing closures.
"Because that just sends arguments to another function, and that function tries to extract them,"
Hm... well, I still don't get what you're saying, but I'll try implementing it and see if it's a problem or not.
---
"But how does the 'attr function extract its first argument?"
In a worst case scenario, it can be a built-in and use Python magic to avoid the paradox. Having one built-in (attr) is still waaaay better than having a ton of other built-ins (car, cons, cdr, maptable, etc.)
---
"Clojure has its own kinda wordy form of table destructuring. An Arc macro can do better than that when it's only trying to do non-recursive destructuring."
JavaScript also has a form of object destructuring [1]:
{a, b} = {a: "foo", b: "bar"}
// or was it like this...?
[a, b] = {a: "foo", b: "bar"}
I think something that simple would help 99% of the time, so that's okay with me.
---
"Personally, I'd probably go for 'def-extension-type and blissfully ignore the internal representation altogether (except when I'm reading things at the REPL, hm...)."
That's all well and good for high-level coding, but somebody's going to have to determine what the low-level data type will be. I'm going with a table, in py-arc, thanks to our discussions.
---
"It's what 'extend does too. The tagging is just done in a meta way. The way I see things now, the meaning of a value is only the op-understands-value relationships it supports (duck typing?), so extending an op is a valid way to change (tag) the meaning of a value."
Ah, but the key difference is: where is the operator stored? Right now we're only using global functions as operators, but we could also store them as functions in a table. This is better in some situations... and extend is better in other situations.
So yeah, I'm leaning toward duck typing as well, and actually getting rid of the `type` function completely. My idea is that the "type" of something is based solely on it's prototype: `table` would be both a prototype and a constructor of tables (at the same time). You can then create new prototypes that inherit from `table`, and voila, they're tables.
This is sorta-kinda like how JavaScript does it, except JavaScript actually screwed up prototypes, so we can do a lot better in Arc. For starters, in JavaScript you can access the prototype, but you can't change it. That's definitely gonna change: in Arc, you should be able to dynamically change the prototype.
Something's a table, but want it to be a cons? Just change it's 'prototype attribute and bam, it now inherits from cons. Anyways, then the `isa` function would test for inheritance, rather than type. So rather than saying (isa foo 'table) you'd say (isa foo table), which would mean, "does foo inherit from table?" [2].
As far as I can see, the only downside is that isa would now be O(n) rather than O(1), but the prototype chain should be small enough that it won't make a big difference in ordinary programs.
---
"I think you gotta encode behavior in some type or other. I see two ways to avoid functions: You could have tables hold s-expressions and interpret them, or you could have tables encode the behavior themselves. The latter would make tables objects, and they'd be indistinguishable (on a high level) from message-passing closures."
I'm kinda leaning toward the second approach. Even in a worst case scenario, having two built-in primitives that can't be created in Arc (tables and fns) is still better than the current situation.
* [2]: I'll note that `isa` would probably be used rarely... just as I rarely use `instanceof` in JavaScript. It's only there on the off chance that somebody really does care what "type" something is.
By the way, I just had an idea. We could even write the `annotate`, `type`, and `rep` functions in Arc:
(def annotate (type rep)
(fn (m . args)
(case m
'type type
'rep rep
(apply attr rep m args))))
(def type (x)
(attr x 'type))
(def rep (x)
(attr x 'rep))
So calling `annotate` on a value would create a wrapper around it. But this also allows us to do something else... modify the type of an existing value, rather than wrap it.
"If you implement those five types using (annotate 'table (fn ...)), then you have to implement three methods per type, for the same overall result. I believe this is what akkartik meant by "moving parens around.""
...but if creating a single table with extend is twice as verbose as message passing, then creating 5 tables with extend will still be twice as verbose as message passing. That seems to be more than just "moving parens around", though perhaps you could argue that the 2x difference doesn't matter.
Or perhaps you could argue that the 2x verbosity could be curbed with macros... or a construct like def-extension-type... in which case I reply that we can use the same techniques for message passing. But with O(1) efficiency rather than the O(n) efficiency of extend, and with the ability to easily define core functions in Arc itself.
Yes, that was exactly what I was saying with 'def-extension-type, and I even mentioned that you could use 'def-extension-type verbatim too (not that the name would make sense).
I've just written my reply to "the O(n) efficiency of 'extend" over here: arclanguage.org/item?id=14374
