Arc Forumnew | comments | leaders | submit | kinnard's commentslogin

This is off-topic but pg is shockingly Eurocentric, he seems from that excerpt to be completely oblivious to or completely willing to elide the generations of advances made in almost every field by the luminaries of the muslim empires including things like . . . ALGEBRA!

reply

3 points by kinnard 14 hours ago | link | parent | on: Ask AF: Ordered Tables?

It's for an application called pipeline: https://github.com/Kinnardian/pipeline

Meant to operate as a generalized pipeline for items to move through.

Could function as a basis for a kanban or as a sales management system.

In which cases the "stacks" as I call the stages in a pipeline would be:

"todo" "fortnight" "week" "today" "done"

"attention" "interest" "decision" "action" "love" "refer"

Respectively.

I was working on converting the whole thing from lists (https://github.com/Kinnardian/pipeline/commits/templatize) when I realized that tables are not order preserving (I can't believe this never came up working with json).

Order definitely matters, and I'm envisioning functionality like repeating tasks that once moved to the "done" stack/marked done wrap around to `pipe.0`(Imagine something that must be done once a month or once a week). I suppose I could tag items as monthly or weekly in which case they'd get appended to pipe!month or pipe!week as soon as they're done.

Another example is the `(createItem)` function which should take an item and an optional "stack" parameter. If no stack parameter items should be appended to the zeroeth stack `pipe.0` identified by index because it will be named by different keys in different pipelines.

Another example is a `(push)` function which pushes an item from its current stage to the "next" stage. With order this is simple.

EDIT: Order even matter matters within a stack. In the current kanban system I have a bunch of things I need to do today, the top priorities are actually at the top. I wouldn't want them getting all disheveled.

. . .(please excuse my disheveled codebase)

reply

3 points by i4cu 13 hours ago | link

I agree that order matters. It matters in pretty much every application that uses data. Even HN has data ordered by newest, rank (best) etc. And Arc has a tonne of functionality to support sorting and comparing data to let you do that very thing.

But I don't see the need to order data in your application equating to the need for tables that support constant insertion order.

I 'm not going to say what you're doing is wrong because it's not, but I am going to say it's non-standard and I think there other ways to manage your data that doesn't require ordered tables (which have downsides with data growth). It's just my opinion, but there's not much you can't do with the standard storing of table records and maintaining of indexes.

That said if your data load is always low with little to no growth it could very well be a good fit.

reply

1 point by kinnard 4 hours ago | link

This may have no bearing on how tables are implemented in arc but a more efficient implementation happened to be ordered:

https://morepypy.blogspot.com/2015/01/faster-more-memory-eff...

"One surprising part is that the new design, besides being more memory efficient, is ordered by design: it preserves the insertion order."

reply

1 point by kinnard 12 hours ago | link

I do think the data load for this would always be low.

I feel like implementing tables with indices would basically be like implementing ordered dictionaries (new nomenclature I've arrived at per: http://arclanguage.org/item?id=21037 ), no?

reply

2 points by kinnard 15 hours ago | link | parent | on: Ask AF: Ordered Tables?

Another example and probably better name is Ordered Dict:

https://pymotw.com/2/collections/ordereddict.html

https://docs.microsoft.com/en-us/dotnet/api/system.collectio...

reply

1 point by akkartik 5 hours ago | link

How about AutobiographicalDict? ^_^

More serious suggestion: Journaling Dict. Maybe the Arc type could be `jtable`?

'order' is pretty ambiguous here. The fact that some places use the word to mean what we mean doesn't seem like sufficient reason to follow suit.

reply

1 point by kinnard 4 hours ago | link

I mean 'insertion order'

reply

1 point by akkartik 2 hours ago | link

But do you see that other interpretations are possible?

reply

1 point by kinnard 2 hours ago | link

Yes like a variety of consistent sort orders

reply

1 point by akkartik 2 hours ago | link

Not quite. The distinction I'm drawing is between ordering the elements based on their intrinsic properties, and ordering the elements based merely on the order they're inserted in.

reply

1 point by kinnard 5 hours ago | link

Dicts are ordered as a feature in Python 3.6^:

https://stackoverflow.com/questions/39980323/are-dictionarie...

reply

1 point by kinnard 4 hours ago | link

Sadly: https://stackoverflow.com/a/44687752/1236793

reply

2 points by kinnard 1 day ago | link | parent | on: Ask AF: Ordered Tables?

Damn. I thought I might've gotten that mixed up. Should've checked.

I think clojure's array-maps do it but with some limitations:

https://clojure.org/reference/data_structures#ArrayMaps

http://arclanguage.org/item?id=21014

reply


I overlooked the existence of `sym` which does make my request altogether superfluous. Higher-order coercion combinators will take some digestion on my part!

reply

2 points by kinnard 1 day ago | link | parent | on: Ask AF: Advantages of alists?

I completely overlooked the "care about order" part.

I realized belatedly that ordering matters for my application. . .

Even looked at Assocative Containers[1] before I thought, "alist!"

[1] https://en.wikipedia.org/wiki/Associative_containers

reply

3 points by i4cu 1 day ago | link

Yeah, in clojure we have the standard hash-maps, but we also have array-maps (which maintain the insertion order) and sorted-maps (which are sorted by keys).

In the last ten years I've only needed ordered maps once or twice. In one case it was for a custom query language I wrote that generated queries from data alone.

eg. (query db :users (array-map :gender "female" :name "xena"))

so in this example adding the :gender clause first would restrict the query and improve performance.

I also remember, in my early clojure days, I made the mistake of relying on standard hash-maps:

eg. (query db :users {:gender "female" :name "xena"})

Until I discovered a query where the performance died and found out hash-maps are actually array-maps (under the hood) until 9 entries. It then auto converts to real hash-maps and loses its order (Clojure does this because maintaining order on bigger data sets is costly from an efficiency/memory perspective).

In arc I would have to use alists for this.

reply

2 points by kinnard 1 day ago | link

This sucks because notation, and lookup are so hairy with alists and they get even worse with nesting.

reply

3 points by rocketnia 1 day ago | link

Can you give an example of where the notation is better with tables than with alists? Maybe there's something you can do about it, e.g. writing a macro, using `defcall` or `defset`, or extending `sref`.

reply

2 points by kinnard 1 day ago | link

I'm running into trouble with both actually.

  (= tpipe {todo 
              '({id 1 cont "get eggs" done '(nil)}
                {id 23 cont "fix toilet" done '(nil)})
            week '(nil)
            today '({id 23 cont "Build something that works in arc" done '(nil)})
            done '({id 23 cont "Research Ordered Associative Arrays" done '(2019 1 21)})})

  (= apipe '((todo ({id 1 cont "get eggs" done '(nil)} 
                    {id 23 cont "fix toilet" done '(nil)})) 
             (week (nil)) 
             (today ({id 23 cont "Build something that works in arc" done '(nil)}))
             (done ({id 23 cont "Research Ordered Associative Arrays" done (2019 1 21)}))))

  arc> ((alref apipe 'todo) 0)
  '(%braces id 1 cont "get eggs" done '(nil))
vs

  arc> tpipe!todo.1
  '(%braces id 23 cont "fix toilet" done '(nil))
But neither works like so

  arc> tpipe!todo.1!id

  arc> ((alref apipe 'todo) 0) 'id)
One must employ (list) or quasiquotation

    (= npipe {todo 
                (list {id 1 cont "get eggs" done '(nil)}
                      {id 23 cont "fix toilet" done '(nil)})
              week '(nil)
              today (list {id 23 cont "Build something that works in arc" done '(nil)})
              done (list {id 23 cont "Research Ordered Associative Arrays" done '(2019 1 21)})})

    (= unqpipe {todo 
                `(,{id 1 cont "get eggs" done '(nil)}
                  ,{id 23 cont "fix toilet" done '(nil)})
                week '(nil)
                today `(,{id 23 cont "Build something that works in arc" done '(nil)})
                done `(,{id 23 cont "Research Ordered Associative Arrays" done '(2019 1 21)})})

This is less of a problem with tpipe since in practice items would get pushed into a "pipe".

Ideal world with key and index access:

  arc> pipe!todo.1!id
  23

  arc> pipe.0.1!id
  23
EDIT: I think tables might be better overall and that the order-necessitating functionality may not be worth the trouble of alists.

reply

2 points by rocketnia 22 hours ago | link

I recommend not expecting `quote` or `quasiquote` to be very useful outside the context of metaprogramming.

Quotation is helpful for metaprogramming in a tangible way, because we can easily refactor code in ways that move parts of it from being quoted to being unquoted or vice versa.

And quotation is limited to metaprogramming in a tangible way, because we can only quote data that's reasonably maintainable in the same format we're maintaining our other code in. For instance, an Arc `quote` or `quasiquote` operation is itself written inside an Arc source code file, which is plain text, so it isn't very useful for quoting graphics or audio data.

We can of course use other functions or macros to construct those kinds of data. That's essentially Arc's relationship with tables. When we've constructed tables, we've just used (obj ...) and (listtab ...) and such.

Adding tables to the language syntax is doable, but it could have some quirks.

  ; Should this cause an error, or should it result in the same thing as
  ; '(let i 0 `{,++.i "foo"}) or '(let i 0 `{,++.i "foo"})? Each option
  ; is a little surprising, since any slight edit to the code, like
  ; replacing one ++.i with (++ i 1), would give us valid code to
  ; construct a two-entry table, and this code very well might have
  ; arisen from a slight edit in the opposite direction.
  '(let i 0
    `{,++.i "foo" ,++.i "bar"})
  
  ; Should this always result in 2, or should it result in 1 if "foo"
  ; comes last in the table's iteration order?
  (let x 0
    `{"foo" ,(= x 1) "bar" ,(= x 2)}
    x)
Personally, I tend to go the other way: I prefer to have as few kinds of data as possible in a language's quotable syntax.

A macroexpander needs to extract two things from the code at any given time: The name of the next macro to expand, and the region of syntax the macro should operate on. Symbols help encode macro names. Lists and symbols together help encode regions of plain text. I think it's for these reasons that symbols and lists are so essential to Arc's syntax.

Arc has other kinds of data that commonly occur in its quotable syntax, like strings and numbers, but it doesn't need them nearly as much as symbols and lists. Arc could expand symbols like |"Hello, world!"| and |123| as ssyntax, or it could simply let everyone put up with writing things like (string '|Hello, world!|) and (int:string '|123|) each time.

Tables would fit particularly well into a language's quotable syntax if they somehow helped encode regions of syntax. For instance, if a macro body consisted of all the files in a directory, then a table could be an appropriate represention of that file collection.

reply

1 point by akkartik 10 hours ago | link

I'm having a lot of trouble parsing this comment.

> I recommend not expecting `quote` or `quasiquote` to be very useful outside the context of metaprogramming.

My immediate reaction is to disagree. A lot of the reason Lisp is so great is that quasiquotation is orthogonal to macros/metaprogramming.

    > ; Should this cause an error, or should it result in the same thing as
    > ; '(let i 0 `{,++.i "foo"}) or '(let i 0 `{,++.i "foo"})?
Those two fragments are the same?

In general it feels unnecessarily confusing to include long doc comments in code fragments here. We're already using prose to describe the code before and after.

Code comments make sense when sharing a utility that you expect readers to copy/paste directly into a file to keep around on their disks. But I don't think that's what you intend here?

Finally, both your examples seem to be more about side effects in literals? That is a bad idea whether it's a table literal or not, and whether it uses quasiquoting or not. Do you have a different example to show the issue without relying on side-effects?

reply

3 points by i4cu 1 day ago | link

> EDIT: I think tables might be better overall and that the order-necessitating functionality may not be worth the trouble of alists.

I would agree. At least your examples seem to point to that.

I would suggest you flatten your data and add indexes:

  (= data (obj 1 (obj cont "get eggs")
               2 (obj cont "fix toilet")
               3 (obj cont "Build something that works in arc")  
               4 (obj cont "Research Ordered Associative Arrays")))
Then:

  (= todo  '(1 2))
  (= today '(3))
  (= done  '(4))
And use the indexes to lookup the records. You'll notice by doing it this way you're able to control the order and don't need alists to do so.

Trying to go down the road of deeply nested tables or alists will only lead you to pain and suffering (at least in arc).

Edit: wow, I think arc needs 'sets' too :)

reply

1 point by kinnard 1 day ago | link

The main issue with alists is that the special syntax doesn't work and the notation is so verbose . . . I don't know if the efficiency issues would even come to bear for me.

EDIT: nesting is not behaving as I expect but that may be a product of my own misunderstanding.

reply

2 points by i4cu 22 hours ago | link

I would be careful not to structure data/logic to accommodate a special syntax.

i.e while using:

  pipe!todo.1!id 
is certainly fancy, writing a function is just as effective and most likely more performant since it doesn't require read de-structuring:

(fetch pipe todo first id)

So I'm suggesting you shape your syntax usage around your data, not your data around your syntax. You can always write a macro to obtain your desired level of brevity.

reply

2 points by kinnard 15 hours ago | link

Shaping syntax around data rather than data around syntax sounds like the move, I'm probably just not used to having that option.

reply

3 points by shawn 1 day ago | link

Thanks for pointing this out. I’ve pushed a fix. Can you confirm?

reply

2 points by kinnard 1 day ago | link

Nice. I'm getting this error:

  $ arc
  ac.rkt:347:43: tablist: unbound identifier
  in: tablist
  location...:
   ac.rkt:347:43
  context...:
   raise-unbound-syntax-error
   for-loop
   [repeats 2 more times]
   finish-bodys
   for-loop
   finish-bodys
   lambda-clause-expander
   loop
   [repeats 66 more times]
   module-begin-k
   expand-module16
   expand-capturing-lifts
   expand-single
   temp74_0
   compile16
   temp68_2
   ...

reply

3 points by shawn 1 day ago | link

Hmm. I know why. My mistake.

Un momento; fix incoming.

The general idea behind the fix is that quoted literals need to be treated as data. Arc now has two new functions for this purpose: quoted and unquoted.

The fact that (quote {a 1}) now becomes a hash table is a little strange. I’m not entirely sure that’s correct behavior. It depends whether (car '({a 1})) should yield a hash table. It seems like it should, which is reified in the code now.

EDIT: Ok, I've force-pushed the fixed commit. (Sorry for the force-push.)

If you `git reset --hard HEAD~1 && git pull` it should work now.

reply

3 points by kinnard 1 day ago | link

Works great! The only step further that comes to mind to me is:

  arc> '(pipe "water")
  '(pipe "water")

  arc> "she"
  "she
  
  arc> 23
  23

  arc> {pipe "water"}
  {pipe "water"}
rather than

  arc> {pipe "water"}
  '#hash((pipe . "water"))

reply

3 points by shawn 18 hours ago | link

I tried improving anarki's repl experience at https://github.com/arclanguage/anarki/pull/145

It sort of went well, but mostly not. :)

Personally, I found I prefer racket's pretty-printing with the horrible hash tables compared to something like {pipe "water" a 1 b 2 c ...} because if you try to evaluate `items` or `profs` you won't have a clue what the data is without pretty-printing.

And it turns out I suck at writing pretty-printers. Someone else do it!

reply

3 points by shawn 1 day ago | link

Was already working on that. It's clear it's time.

Few minutes. Maybe an hour.

reply

3 points by shawn 1 day ago | link

Close: https://gist.github.com/shawwn/03b936d37e4cd83ca6652bb03c527...

not bad for precisely 59 minutes.

Brb, transferring to starbucks.

reply

2 points by kinnard 1 day ago | link | parent | on: Ask: When to use lists vs tables?

I realized belatedly that ordering matters for my application. . .

Even looked at Assocative Containers[1] before I thought, "alist!"

[1] https://en.wikipedia.org/wiki/Associative_containers

reply


Touché.

reply


Wow. Freaky fast. Thanks! I was thinking of going even further and finding out if arc could output tables and read in tables in that structure?

  {todo:({id 1 name "get eggs" done nil} {id 8 name "fix computer" done t})} 
looks so much better than the #hash() equivalent and this gets extreme with nested tables. It's also much easier to think through a table structure writing it out.

reply

3 points by shawn 4 days ago | link

I switched `(write ...)` to `(pretty-print ...)` for repl values. https://github.com/arclanguage/anarki/pull/142

Let me know if that seems sufficient for now.

You're right that Arc still can't read tables written via `write`. That is definitely worth supporting. Here is an example of how it could work: https://github.com/sctb/lumen/blob/55b14ca8aafeaf6b0ca1b636d...

It would be important to ensure that circular structures don't cause an infinite loop, and I'd be nervous about straying too far from Racket's `write` facility. For better or worse, it's a limitation of racket that you can't `read` a table you've written. But it could be worth doing.

reply

2 points by rocketnia 3 days ago | link

"it's a limitation of racket that you can't `read` a table you've written"

Eh? You definitely can, in Racket. :) The only problem I know of is that when you write a mutable table, you read back an immutable one.

---

"You're right that Arc still can't read tables written via `write`."

Arc's supported this for as long as I can remember. The only problem is that they come back immutable.

Here's Arc 3.2 on Racket 7.0:

  arc> (= foo (fromstring (tostring:write:obj a 1 b 2) (read)))
  #hash((a . 1) (b . 2))
  arc> (= foo!a 3)
  Error: "hash-set!: contract violation\n  expected: (and/c hash? (not/c immutable?))\n  given: '#hash((a . 1) (b . 2))\n  argument position: 1st\n  other arguments...:\n   'a\n   3"
And here's the latest Anarki:

  arc> (= foo (fromstring (tostring:write:obj a 1 b 2) (read)))
  '#hash((a . 1) (b . 2))
  
  arc> (= foo!a 3)
  3
  
  arc> foo
  '#hash((a . 3) (b . 2))
Oh, I guess it works!

It looks like that's thanks to a March 8, 2012 commit by akkartik (https://github.com/arclanguage/anarki/commit/547d8966de76320...)... which, lol... Everything I was saying in a couple of recent threads about replacing the Arc reader to read mutable tables... I guess that's already in place. :)

reply

3 points by kinnard 2 days ago | link

It'd be cool if this worked with curly brace syntax. You could read in (and write) a file that looked like this:

  {'id 3 
   'c {
      'name "james c clarke" 
      'age 23 
      'addr "1724 Cox Ave. NY, NY 90210"
     }
  }
I think it'd make reading and writing a much better experience.[1]

[1] http://arclanguage.org/item?id=20803

EDIT: I guess this would be called "table literals"?

reply

1 point by kinnard 1 day ago | link

Does the necessity of quasiquote + unquote feel natural?

  (= tpipe {todo 
              '({id 1 cont "get eggs" done '(nil)}
                {id 23 cont "fix toilet" done '(nil)})
            week '(nil)
            today '({id 83 cont "Build something that works in arc" done '(nil)})
This won't work:

  arc> tpipe!todo.1!id
One must employ (list) or quasiquotation

    (= npipe {todo 
                (list {id 1 cont "get eggs" done '(nil)}
                      {id 23 cont "fix toilet" done '(nil)})
              week '(nil)
              today (list {id 83 cont "Build something that works in arc" done '(nil)})
              done (list {id 44 cont "Research Ordered Associative Arrays" done '(2019 1 21)})})

    (= unqpipe {todo 
                `(,{id 1 cont "get eggs" done '(nil)}
                  ,{id 23 cont "fix toilet" done '(nil)})
                week '(nil)
                today `(,{id 83 cont "Build something that works in arc" done '(nil)})
                done `(,{id 44 cont "Research Ordered Associative Arrays" done '(2019 1 21)})})

reply


Wow.

reply

More