Arc Forumnew | comments | leaders | submit | fallintothis's commentslogin
3 points by fallintothis 5437 days ago | link | parent | on: Tips on implementing arc

Is this how most languages get tail call elimination?

Well, more languages do TCO than use CPS -- never mind a CPS interpreter. Though CPS is functionally equivalent to the more popular SSA (Static Single Assignment) form, it remains relatively unused; see http://lambda-the-ultimate.org/node/3467. I imagine the comment about using an interpreter is rooted in http://billhails.net/Book/v10.html, a great chapter from a book that goes through writing a Scheme-like interpreter in Perl, including a CPS transform and TCO using a trampoline. (For quick info about what those mean, there's of course http://en.wikipedia.org/wiki/Continuation-passing_style and http://en.wikipedia.org/wiki/Tail_call)

In fact, CPS benefits from TCO. By converting a program into explicit CPS, every function call becomes a tail call (to the current continuation); without TCO, stack space would run out quickly. Certain TCO implementations can also use CPS; see http://en.wikipedia.org/wiki/Tail_call#Implementation_method....

A great discussion about TCO implementation is http://lambda-the-ultimate.org/classic/message1532.html. To get really into it, the master's thesis at http://users.cecs.anu.edu.au/~baueran/thesis/ talks about tail calls in GCC (I've not read it, but it looks interesting).

-----

1 point by evanrmurphy 5436 days ago | link

Wish I could upvote you more than once. Great reply.

-----

2 points by fallintothis 5445 days ago | link | parent | on: Command line arguments

Well, blame (https://github.com/nex3/arc/blame/5ac5d567bce08004c0dce6fc4c...) tells us that the commit (https://github.com/nex3/arc/commit/5ac5d567bce08004c0dce6fc4...) wasn't really belabored. My guess is that there's no particular rationale. It does make more sense to me to pass in argv parameters rather than just aloading everything.

-----

2 points by fallintothis 5446 days ago | link | parent | on: Command line arguments

I'm not aware of any option parsers written in Arc. There are plenty of good libraries to mimic (e.g., http://docs.python.org/library/argparse.html), but in the meantime, you could monkey-patch with Racket (http://docs.racket-lang.org/reference/Command-Line_Parsing.h...) or with getopt (http://man.cx/getopt) in arc.sh (not that that's portable).

-----

2 points by fallintothis 5451 days ago | link | parent | on: Ask: what does annotate do?

You're most of the way there. annotate is non-destructive. Hence,

  arc> (= x 10)
  10
  arc> (type x)
  int
  arc> (annotate 'foo x)
  #(tagged foo 10)
  arc> x
  10
  arc> (type (annotate 'foo x))
  foo
but

  arc> (= x (annotate 'foo 10))
  #(tagged foo 10)
  arc> (type x)
  foo
  arc> (annotate 'foo x) ; annotating with the same type does nothing
  #(tagged foo 10)
  arc> (annotate 'bar x)
  #(tagged bar #(tagged foo 10))
  arc> x
  #(tagged foo 10)
  arc> (type (annotate 'bar x))
  bar
See http://www.paulgraham.com/ilc03.html (the name has since changed from tag to annotate).

-----

2 points by hasenj 5451 days ago | link

So, what can you do with it?

Can you make it behave a certain way with pr and + and string, for example?

If yes, then how?

If no, then what's the point? I mean, if you can't overload existing operators to work with the new type, what's the use?

-----

2 points by fallintothis 5451 days ago | link

Not to be snarky, but I linked to http://www.paulgraham.com/ilc03.html for a reason. One salient point (that's independent from annotate):

  arc> (let orig +
         (def + args
           (if (all acons args)
               (do (prn "Blah blah, special, blah blah")
                   (apply orig args))
               (apply orig args))))
  *** redefining +
  #<procedure: +>
  arc> (+ 1 2 3)
  6
  arc> (+ '(a b c) '(d e f))
  Blah blah, special, blah blah
  (a b c d e f)
  arc> (+ '(a b c) 1 2 3)
  Error: "car: expects argument of type <pair>; given 1"
The only things you might have trouble extending this way are macros (since any definition up to the point where you redefine it will have been expanded)

  arc> (mac m (x) `(do (prn "orig") ',x))
  #(tagged mac #<procedure: m>)
  arc> (def f () (m 5))
  #<procedure: f>
  arc> (let orig (rep m)
         (mac m (x)
           (if (number x)
               `(do (prn "redef") ',x)
               (orig x))))
  *** redefining m
  #(tagged mac #<procedure: m>)
  arc> (m "a string")
  orig
  "a string"
  arc> (m 5)
  redef
  5
  arc> (f)
  orig
  5
and names treated specially by the compiler (if, fn, quote, quasiquote, unquote, unquote-splicing, assign, nil, t).

Also relevant: http://awwx.ws/extend0.arc, http://awwx.posterous.com/defrule.

-----

1 point by akkartik 5451 days ago | link

Are you suggesting we don't need annotations for anything that extend works with? That isn't how I read that essay at all.

-----

1 point by fallintothis 5450 days ago | link

I meant that

  (let orig f
    (def f args
      ; you can put ANY code you want in here
      ))
Hence, extend is independent from annotate: just as you can annotate without using extend, you can extend without using annotate. extend is just a byproduct of closures and first-class functions. Of course, combining annotate and extend lets you dispatch based on custom types.

-----

1 point by rocketnia 5451 days ago | link

In official Arc, there's not much of a point. As akkartik says, tagged types are used to designate macros, but that's about the only thing that cares about them.

The slight benefit is that 'annotate is the single most obvious way to do type labels in Arc, meaning people don't usually have to invent their own representation for dynamically typed values. As we explore future directions of the Arc language, we can compare type ideas 'annotate, and in many cases we can implement them using 'annotate too, in order to try them out right away, in a way that's often compatible with other people's code.

In other words, 'annotate only has the value we Arc users give it, but it's there, so we occasionally do add value to it.

-----

2 points by akkartik 5451 days ago | link

Yes types are underused in baseline arc. I think they're only used to distinguish macros from other functions.

I made queues tagged types in anarki, and overloaded len to work with queues.

pr is a good suggestion. Let me know if you have others.

I've thought about overloading +, but couldn't find a clean way since + is implemented at the scheme level. There also seems to be a consensus that overloading + for concatenation is a bad idea, so it doesn't seem worth changing.

-----

3 points by rocketnia 5451 days ago | link

There also seems to be a consensus that overloading + for concatenation is a bad idea

I thought there was a majority for concatenation. XD Well, either way, I'm for it. Maybe you've heard this all before, but I'll summarize:

I've convinced myself that '+ is a sensible standard name for monoid operations, including concatenation, and that '+ and '* together should describe a field whenever that makes sense (http://arclanguage.org/item?id=12850). To appease people who are concerned with efficiency, we can always leave in the math-only version of '+ as a separate variable, or just have the '$ drop-to-Racket operator so they can get it themselves.

As far as implementation goes, the technique I recommend is making a separate two-argument version of the varargs function, so that it's easier to extend consistently (preserving the left-associativity of the varargs version). Then a double dispatch technique comes in handy, but it only has to be a simple one. I've posted some rough examples before to illustrate this strategy: http://arclanguage.org/item?id=12561 http://arclanguage.org/item?id=12858

-----

1 point by akkartik 5450 days ago | link

"I thought there was a majority for concatenation."

I was thinking of http://arclanguage.org/item?id=12347, where it seemed like PG and everyone else didn't like '+ for concatenation :) The thread on matrices and vectors was a separate issue. I think we're all ok overloading + for adding different kinds of mathematical entities, and so on. I'm not sure what sort of monoid allows + for list concatenation.

-----

2 points by rocketnia 5450 days ago | link

I was thinking of http://arclanguage.org/item?id=12347, where it seemed like PG and everyone else didn't like '+ for concatenation :)

I was thinking of the same thread. ^_^ We must have kept score in different ways, depending on which arguments convinced us and so on.

The thread on matrices and vectors was a separate issue.

I was using 'v+ as an example of an extensible, left-associative varargs operator, just to give an idea of how to implement '+ and '* so they're extensible.

I think we're all ok overloading + for adding different kinds of mathematical entities, and so on. I'm not sure what sort of monoid allows + for list concatenation.

Well, lists are mathematical entities. If that isn't intuitive, consider how we reason about basic properties of rational numbers. It's usually easiest to break them apart into numerators and denominators and form conclusions about those as integers. But if we can reason about something in terms of two pieces it has, it makes sense for our reasoning system to support arbitrary pairs rather than just the special case of rational numbers. Those pairs can be built up into linked lists.

If your concern is more about monoids, here's the Wikipedia version: "In abstract algebra, a branch of mathematics, a monoid is an algebraic structure with a single associative binary operation and an identity element."

Concatenation is associative (a . b . c is the same regardless of whether you do a . b or b . c first) and has an identity element (a . nil and nil . a are both just a), so the set of lists (or strings, etc.) together with the concatenation operator makes a monoid. Also, real numbers are a monoid under addition, since it too is associative and has an identity element (zero).

A few accumulation and repetition utilities here and there, like '++, 'mappend, and 'summing, can be generalized to all monoids... but really, the main benefit in my book is having a consistent excuse to use '+ for concatenation. :-p

-----

2 points by rocketnia 5449 days ago | link

Oh, I keep forgetting: I'm not sure I like having '+ be implemented using just a two-argument version. Naively concatenating two things at a time is embarrassingly inefficient (http://en.wikipedia.org/wiki/Schlemiel_the_Painter%27s_algor...). So, I'm sorry I suggested it!

Instead of that approach, I've been thinking about having one function that wraps the first argument in some kind of output stream, one function that submits a new argument to that stream, and one function that gets the value accumulated in that stream. So here's yet another untested, throwaway code example illustrating the functionality I'm talking about (but not the extensibility, unless you use 'extend):

  (= orig-inside inside orig-+ +)
  
  (def +streamer (x)
    (case type.x string  (w/outstring str (disp x str) str)
                 int     x
                 num     x
      (if alist.x
        (annotate 'basic-streamer
          (obj test alist func [apply join _] args list.x))
        (err "unrecognized case"))))
  
  (def fn-add-to (segment streamer)
    (if (and (isa streamer 'output) (isa segment 'string))
      (do (disp segment streamer)
          streamer)
        (and (in type.streamer 'int 'num) (in type.segment 'int 'num))
      (orig-+ streamer segment)
        (and (isa streamer 'basic-streamer) rep.streamer!test.segment)
      (do (push segment rep.streamer!args)
          streamer)
      (err "unrecognized case")))
  
  (mac add-to (segment place)
    (w/uniq g-streamer
      `(zap (fn (,g-streamer)
              (fn-add-to ,segment ,g-streamer))
            ,place)))
  
  (def inside (x)
    (case type.x output          orig-inside.x
                 basic-streamer  (rep.x!func rep.x!args)
                 int             x
                 num             x
      (err "unrecognized case")))
  
  (def + args
    (iflet (first . rest) args
      (let s +streamer.first
        (each arg rest
          (add-to arg s))
        inside.s)
      0))
Inefficient concatenation is probably what's bogging down Penknife's parser, so I'll put this technique to the test sooner or later to see how much it helps.

-----

1 point by rocketnia 5449 days ago | link

In order to get an apostrophe in that URL (http://en.wikipedia.org/wiki/Schlemiel_the_Painter%27s_algor...), I had to escape it as %27. Even when I went to edit my post, the quote had been stripped out of the recreated input. Bug?

-----

1 point by akkartik 5449 days ago | link

You can probably do a quick performance comparison and count conses like waterhouse does in that + thread. I'd be curious to see your results.

-----

1 point by rocketnia 5449 days ago | link

I haven't found my way into cons-counting.... I know it's probably sloppy, but I allocate as though it takes constant time. Given that assumption, I'm less concerned with an overall constant-time slowdown and more concerned with allowing a '+ of N sequences of length L to take O(NL) time rather than O(N^2 L) time.

This is also a mostly[1] less limited design from an interface point of view. It's straightforward to port an extension from my other design to this one: The '+streamer and 'inside functions will be extended with (fn (x) x), and 'fn-add-to will be extended with the desired two-argument behavior. (I'd personally make a macro for this, and I'd use it to (re)implement the number-and-number case.)

[1] The exception is if you want to use '+ on streamer types themselves, since a '+streamer behavior of (fn (x) x) will cause that type to be confused with whatever other type wraps itself up as that streamer type.

-----

2 points by akkartik 5450 days ago | link

"I was thinking of the same thread. ^_^ We must have kept score in different ways, depending on which arguments convinced us and so on."

Ack, you're right. So it's just me and waterhouse against +?

I tried replacing + on anarki and got immediate pushback. join does seem a long name, especially inside prn's to generate html, but what other one-character name can we use besides +? I'm leaning back towards + again, perhaps mirroring the experience of people who've tried this before.

-----

1 point by evanrmurphy 5450 days ago | link

> So it's just me and waterhouse against +?

IIRC, pg found himself against it too, but rtm was for it.

> what other one-character name can we use besides +?

One feature I like in PHP is the use of the dot (.) for concatenation. We've already loaded up that character quite a bit here in Arc, with its use in conses, rest parameters and for the ssyntax `a.b` => `(a b)`. But concatentation is at least vaguely isomorphic to consing. I wonder...

Probably not. `+` is your best bet, IMHO.

-----

1 point by akkartik 5450 days ago | link

I should mention, just for completeness, that haskell uses ++.

-----

1 point by evanrmurphy 5450 days ago | link

Didn't know that. Could you give a quick example?

-----

2 points by akkartik 5449 days ago | link

In haskell you can designate any sequence of characters as an infix operator. Here's the definition of ++ from the prelude (http://www.haskell.org/onlinereport/standard-prelude.html):

  (++) :: [a] -> [a] -> [a]
  []     ++ ys = ys
  (x:xs) ++ ys = x : (xs ++ ys)
so [1, 2, 3] ++ [4, 5] = [1, 2, 3, 4, 5]

-----

1 point by akkartik 5450 days ago | link

:) I'm convinced.

"if we can reason about something in terms of two pieces it has, it makes sense for our reasoning system to support arbitrary pairs rather than just the special case of rational numbers. Those pairs can be built up into linked lists."

Perhaps lists are continued fractions? :)

-----

4 points by fallintothis 5456 days ago | link | parent | on: Noobing with news.arc

BIG WARNING: I've never really run, read, or modified news.arc thoroughly. This is as much as I can gather from just looking.

There are several ways you might login in news.arc, and correspondingly, there are specific bits of code for each of those cases: when you click the top-right login link (see topright in news.arc), when you try to vote (see newsop vote), when you try to submit (see submit-login-warning), when you try to comment (see comment-login-warning), when you try to reply (see newsop reply), etc.

What do these have in common? The login-page function, defined in app.arc:

  (def login-page (switch (o msg nil) (o afterward hello-page))
    (whitepage
      (pagemessage msg)
      (when (in switch 'login 'both)
        (login-form "Login" switch login-handler afterward)
        (hook 'login-form afterward)
        (br2))
      (when (in switch 'register 'both)
        (login-form "Create Account" switch create-handler afterward))))
In each context, different things happen with the afterward parameter passed to login-page. topright logs that the user signed in via the topright link, newsop vote registers the user's vote after they log in, submit-login-warning redirects the user to the submit page, etc. Since login-page is the single point of entry, you might change it. However, there's this comment in news.arc:

  ; Need this because can create users on the server (for other apps)
  ; without setting up places to store their state as news users.
  ; See the admin op in app.arc.  So all calls to login-page from the 
  ; news app need to call this in the after-login fn.

  (def ensure-news-user (u)
    (if (profile u) u (init-user u)))
Aha. This whittles down a single-point of entry specifically for news.arc logins, since they all need to call ensure-news-user in the afterwards parameter of login-page. So, it makes more sense to change this.

You still need to make sure ensure-news-user behaves the same way (i.e., returns the proper user) so that the functions that use ensure-news-user don't break. Shouldn't be too bad to add a ++ in there, then just return the user.

I'll interpret "it is a new day" as "it's a new day between now and when they last logged in" (which isn't necessarily 24 hours, just a new date). This requires knowing when they last logged in. Probably the easiest way to get this would be adding some new info to the profile template, and recording it on each login (an action we're doing now anyways, so not a big deal). Look for deftem profile and add a line to it:

  (deftem profile
    id         nil
    name       nil
    created    (seconds)
    ...
    last-login (date) ; default last-login = whenever the new profile was made
    ...
    delay      0)
Now all you need to do is record their last login in ensure-news-user and do some date arithmetic to see if it's a "new day".

  ; These two functions take in the today argument (i.e., (date)), since
  ; otherwise you might do the equivalent of
  ;
  ;   (it-is-a-new-day u (date))
  ;   ...
  ;   (update-last-login u (date))
  ;
  ; But (date) might change between the times when you make the two function
  ; calls, so it should be bound outside, like
  ;
  ;   (let today (date)
  ;     (it-is-a-new-day u today)
  ;     ...
  ;     (update-last-login u today))

  (def it-is-a-new-day (u today)
    (with ((new-y new-m new-d) today
           (old-y old-m old-d) (uvar u last-login))
      (or (> new-d old-d)                   ; e.g., Apr 15, 2010 -> Apr 16, 2010
          (> new-m old-m)                   ; e.g., Apr 30, 2010 -> May 01, 2010
          (> new-y old-y))))                ; e.g., Dec 31, 2010 -> Jan 01, 2011

  (def update-last-login (u today)
    (= (uvar u last-login) today))

  (let old ensure-news-user
    (def ensure-news-user (u)
      (with (u (old u) today (date))
        (when (it-is-a-new-day u today)
          (++ (karma u)))
        (update-last-login u today))
        u)) ; return the result of old-ensure-news-user
Per my warning, this is untested. Someone else is probably more qualified to gauge the quality of this response. Alternatively, you could increase everyone's karma as a batch job that runs every midnight, but that removes the login part of your requirement. Good luck!

-----

3 points by shader 5456 days ago | link

Just one question: Is their karma supposed to be increased every time the log in and it's a new day, or every time they are logged in and it's a new day? I ask because I almost never have to log in to the arc forum; my session apparently lasts a long time, possibly lasting until I log in from a different computer or longer. As such, I never actually go through the official log in process, but I'm still online.

Does ensure-news-user get run on page load as part of the code that checks whether their logged in or not? From a cursory look at the code, it didn't appear to work that way.

How would you write it differently if you wanted to increase their karma for every day they logged on, as opposed to for day they logged in?

-----

2 points by akkartik 5456 days ago | link

I was wondering about that as well. Perhaps it makes sense to insert a hook (http://files.arcfn.com/doc/variables.html#defhook) into get-user (top of app.arc) rather than ensure-news-user.

You'd be adding some overhead to every single request. Not a big deal to begin with, but something to keep in mind.

-----

3 points by thaddeus 5456 days ago | link

Great response - as always!

the date comparison could be easier:

  (deftem profile
      ...
      last-login (datestring))

   (let old ensure-news-user
      (def ensure-news-user (u)
        (with (u (old u) today (datestring))
          (when (> today (uvar u last-login))
            (++ (karma u)))
          (update-last-login u today))
          u))

-----

1 point by akkartik 5456 days ago | link

Since time always increases, you can also just:

  (unless (is (date) (uvar u last-login))
    (++ karma.u)
    (= (uvar u last-login) (date)))
update: Hmm, I just realized there's a few places I can use this macro:

  (mac updating(place expr . body)
    (w/uniq rhs
      `(let ,rhs ,expr
         (unless (is ,place ,rhs)
           (= ,place ,rhs)
           ,@body))))

  ..
  (updating (uvar u last-login) (date)
    (++ karma.u))
Perhaps we need to make the is parameterizable as well in the general case. Hmm, this could be a generalization of my firsttime (http://arclanguage.org/item?id=12889)

  (mac updating(place ? expr t iff 'is . body)
    (w/uniq rhs
      `(let ,rhs ,expr
         (unless (,iff ,place ,rhs)
           (= ,place ,rhs)
           ,@body))))

  (mac firsttime(place . body)
    `(updating ,place
        :body
          ,@body))
(requires my keyword args and new optional syntax: http://github.com/akkartik/arc)

Now you can run several kinds of test-and-update:

  (firsttime user!loggedin
     (prn *welcome-message*))

  (updating (uvar u last-login) (date)
    :body
      (++ karma.u))

  ; track largest value seen so far
  (ret max 0
    (each elem '(1 3 2 6 5)
      (updating max :iff > elem
        (prn "max now: " max))))

  max now: 1
  max now: 3
  max now: 6
  6
Hmm, I might get rid of firsttime altogether. Is updating the right name for this macro?

-----

2 points by rocketnia 5456 days ago | link

For this:

  (mac updating(place ? expr t iff 'is . body)
    (w/uniq rhs
      `(let ,rhs ,expr
         (unless (,iff ,place ,rhs)
           (= ,place ,rhs)
           ,@body))))
You're evaluating the subexpressions of 'place twice, which isn't necessary thanks to 'setforms. Also, I like to have macros macro-expand and evaluate their parameters from left to right if I can, just for similarity with 'do. (I still try to evaluate only as much as I need to though.) To accomplish these things, I'd go with this (untested) implementation instead:

  (def fn-updating (current-val setter-getter new-val comparator body)
    (unless (do.comparator current-val new-val)
      (.new-val:do.setter-getter)
      (do.body)))
  
  (mac updating (place ? expr t iff 'is . body)
    (let (binds val setter) setforms.place
      `(withs ,binds
         (fn-updating ,val (fn () ,setter) ,expr ,iff (fn () ,@body)))))
In the meantime, for the purposes of dates, I'd at least use 'iso instead of 'is. You're probably not going to get the same (date) list as one you have from earlier today. Even if (date) itself made that guarantee, you've potentially persisted and un-persisted the user data today.

In fact, since you've already made 'iso extensible, I'd use it as the default instead of 'is.

-----

1 point by akkartik 5456 days ago | link

All great points, thanks! I have in fact stopped using is everywhere in my code, don't know what I was thinking.

-----

1 point by akkartik 5455 days ago | link

Why the do's?

And what does the .new-val do?

-----

3 points by rocketnia 5455 days ago | link

Saying (do.foo ...) instead of (foo ...) just makes it so that 'foo isn't in function position. That means the local variable 'foo will always be used instead of some macro named 'foo that happened to be defined in another library.

The ssyntaxes .foo and !foo are short for get.foo and get!foo. Also, a:b ssyntax is handled before a.b ssyntax, giving the : lower precedence.

That means (.new-val:do.setter-getter) compiles as though it's ((get new-val) ((do setter-getter))), which makes it a roundabout way to accomplish ((do.setter-getter) new-val) with one less pair of parentheses, one less character, or (by my count) one less token.

It's really sort of silly in this case, since you save on so little while rearranging so much. In its favor, he (.a:b ...) idiom makes a bit more sense when it's a field/element access like (!width:get-rect ...) or (!2:setforms ...). It's especially helpful when it's part of a big chain of composed forms like (.a:b:.c:.d:e ...).

In my own code, including Lathe, I define this:

  (def call (x . args)
    (apply x args))
Then the code is just call.setter-getter.new-val, giving a savings of two characters, two pairs of parentheses, or (by my count) four tokens, all without causing the expressions to be rearranged.

Seems like a lot of brevity boilerplate, huh? :-p

-----

2 points by aw 5455 days ago | link

Saying (do.foo ...) instead of (foo ...) just makes it so that 'foo isn't in function position. That means the local variable 'foo will always be used instead of some macro named 'foo that happened to be defined in another library.

I remembered talking about this once and found the comment: http://arclanguage.org/item?id=11697

It looks like it would be an easy change to Arc to have local variables take precedence, though I haven't tried the patch myself.

-----

1 point by akkartik 5455 days ago | link

Oh nice. So is there a way using this trick to do

  a."b".c.d.g
without parens?

-----

3 points by rocketnia 5455 days ago | link

Not in Arc. The best you get is (.g:.d:.c:a "b"), I think. You can shove "b" into ssyntax with string!b, but you need to use it as an argument somehow, and any ssyntax containing .string!b will pass the 'string function itself as an argument instead of calling it first.

Of course, if "b" were a symbol instead of a string, it would just be "a!b.c.d.g".

In Penknife, the answer to your question is actually yes: It's "q.b'a.c.d.g", where q is the string-quoting operator and ' is an operator that acts like a reverse a.b. It's easy to stump Penknife too, but I'm hoping to make a full thread about Penknife's take on infix operators in a couple of days or so.

-----

2 points by thaddeus 5452 days ago | link

Optionally, in ac.scm:

  (define (has-ssyntax-char? string i)
    (and (>= i 0)
         (or (let ((c (string-ref string i)))
               (or (eqv? c #\:) (eqv? c #\~) 
                   (eqv? c #\&)                
                   (eqv? c #\%)                
                   ;(eqv? c #\_) 
                   (eqv? c #\.)(eqv? c #\!)))
             (has-ssyntax-char? string (- i 1)))))

  (define (expand-ssyntax sym)
    ((cond ((or (insym? #\: sym) (insym? #\~ sym)) expand-compose)
           ((or (insym? #\. sym) (insym? #\! sym)(insym? #\% sym)) expand-sexpr)
           ((insym? #\& sym) expand-and)
       ;   ((insym? #\_ sym) expand-curry)
           (#t (error "Unknown ssyntax" sym)))
   sym))

  (define (build-sexpr toks orig)
    (cond ((null? toks)
           'get)	    
          ((null? (cdr toks))
           (chars->value (car toks)))
          (#t
           (list (build-sexpr (cddr toks) orig)
                 (cond ((eqv? (cadr toks) #\!)
                        (list 'quote (chars->value (car toks))))
                       ((eqv? (cadr toks) #\%)
                         (list 'string (list 'quote (chars->value (car toks)))))
                       ((or (eqv? (car toks) #\.)(eqv? (car toks) #\!)(eqv? (car toks) #\$)(eqv? (car toks) #\%))
                         (err "Bad ssyntax" orig))
                       (#t 
                       (chars->value (car toks))))))))
Then:

  arc> (= test (obj "a" "my a val" "b" "my b val"))
  #hash(("a" . "my a val") ("b" . "my b val"))

  arc> test%a
  "my a val"
[edit: actually, it would be nicer to have the percent symbol represent the spaces in the string and have some other symbol signify string handling, but I never got around to it + my scheme foo is lacking :)]

-----

1 point by akkartik 5455 days ago | link

Is fn-updating generally useful?

-----

2 points by markkat 5456 days ago | link

Wow. Thanks for such a thorough response! Funny, I had 'last-log' in the deftem profile, and was tweaking ensure-news-user. I couldn't get it working, but's nice to know I was tweaking in the right places. BTW, your interpretation of what I meant by 'new day' was spot on.

I am going to spend some time trying this out. Thank you. I'll let you know how it goes.

-----

1 point by markkat 5456 days ago | link

Thanks again. Based on the login issue shader pointed out, I decided to change things up a bit, and ++ karma if it is a new day, and the user comments or replies. This is what I did, and it seems to work. I started with process-comment:

  (deftem profile
  ...
  karmadate nil
  ...
  )

  (def process-comment (user parent text ip whence)
  ...
         (day-karma user)
         whence)))

  (def day-karma (u)
    (when (~iso (date) (uvar u karmadate))
      (++ (karma u))
    (update-karmadate u)))

  (def update-karmadate (u)
    (= (uvar u karmadate) (date))
    (save-prof u))
But now looking at akkartik's

  (unless (is (date) (uvar u last-login))
    (++ karma.u)
    (= (uvar u last-login) (date)))
Maybe I'll go for that. BTW, karma.u what's up with the . ?

-----

3 points by shader 5456 days ago | link

Arc has a feature called "ssyntax", short for symbol syntax, that allows us to use certain characters in a symbol as shorthand for a commonly used longer form.

In this case, "karma.u" expands to (karma u), which is a macro that looks up the users karma. Functions applied to single arguments are extremely common in arc, so it is quite useful to have an abbreviation.

Other ssyntax include:

  a!b -> (a 'b)
  ~a -> (no a)
  a~b -> (compose a (no b))
  a:b -> (compose a b) ;(a:b c) is equivalent to (a (b c))

-----

1 point by markkat 5455 days ago | link

Ah... thanks. Great way to throw noobs like me off though. :) I was wondering about the colon too, I couldn't find an explanation anywhere.

-----

1 point by akkartik 5456 days ago | link

Don't forget to replace is with iso like rocketnia pointed out.

-----

1 point by markkat 5455 days ago | link

Will do. Thanks for that.

-----

3 points by fallintothis 5457 days ago | link | parent | on: QuickCheck for Arc

Okay! I've finally gotten around to writing this. Sorry it took so long.

One thing that might be nice to show in your tutorial is how to use it more like a traditional unit test syste[m].

Well, they serve different purposes. QuickCheck properties are universally quantified: every x should satisfy (p x). There are obviously practical problems with testing these, so we use a flood of random values for x. Unit tests are existentially quantified: for this particular x, (p x) should be satisfied.

In practice, people unwittingly reinvent one in terms of the other. In a unit-testing library, they'd change the existential property into a "universal" one, in the sense that we can test random values (which is as universal as we usually get).

  (unit-test test-foo
    "foo holds for strings"
    (let x (rand-string (rand max-int*)) ; whatever max-int* is
      (foo x)))
This is an anti-pattern that is solved more cleanly with QuickCheck by abstracting away the random generation.

  (prop prop-foo (x 'string)
    "foo holds for strings"
    (foo x))
Conversely, we can change a universal statement into an existential one easily. Just don't use any parameters to prop.

  (prop prop-foo ()
    "foo doesn't break on an edge case"
    (let x edge-case
      (foo x)))
This, too, is an anti-pattern. E.g., a unit-testing library would probably have better facilities for shared data set-up/tear-down, grouping together tests into suites, and asserting several things per test. (The last one's not necessarily true if we try to keep "functional-programming-y" about it like QuickCheck does.) As it stands, this is a square peg in a round hole for quick-check.arc. It'll run (prop-foo) 100 times, testing a property that's verifiable after only 1 run.

In a way, QuickCheck properties seem more powerful. You test a range of random data instead of the same ol' '(8 6 7 5 3 0 9), "foobarbaz" test inputs. QuickCheck lets you discover edge cases. In the presence of such a tool, unit-tests become less of a default ("Agile Methodology dictates we write unit tests first!!") and more of a backup plan in the fight against regressions: discover edge cases with QuickCheck, then make sure those edge cases never break your code again with unit tests.

Even so, you'll still want to write unit tests for other reasons. If you know a priori that a certain input should produce a certain output, it's easy to put that in a unit test. Often, to test it as a universal property basically relies on the very program you're writing. Silly example, for illustration: converting Roman numerals to integers. You could write

  (generate roman-int (int-between 1 4000))

  (prop test-roman (x 'roman-int)
    (is x (roman->int (int->roman x))))
which is a useful property, but relies on both roman->int and int->roman having been written correctly. As long as int->roman and roman->int are inverses, they'll pass the property. They could even be something stupid like

  (def roman->int (x) (+ x 5))

  (def int->roman (x) (- x 5))
So you need to test just one of them. How do you do that?

  (is (int->roman x) ???)
x is random, so you won't know what the other side of this equality should be. In this case, you'd want unit tests against known values -- 1 and "I", 4 and "IV", etc.

This problem with inverses actually came up when I was testing sscontract (http://arclanguage.org/item?id=11179), hence test/canonical.arc: http://bitbucket.org/fallintothis/contract/src/d1b4ff38afaf/....

Now! You probably knew that stuff already. But it gives us a framework for my understanding, so we can discuss the rest of your post.

i.e. specify ranges of values to test. I don't mean write lots of test cases, but rather an example generator that produces all of the important edge cases, and then as many as possible inside the desired range.

I'm not exactly sure what you mean. Per the above, it doesn't sound like a "traditional unit test system". Specifying ranges of values sounds like what QuickCheck is supposed to do. But there's no real way of automatically knowing all of the important edge cases, so I assume you mean that edge-cases are provided manually. That seems to contradict your "don't write lots of test cases" line, though. Finally, "as many as possible inside the desired range" makes me think you're talking about disjoint input sets to a single property: first test all of the edge cases, then start randomizing.

Using quick-check.arc to do the random testing it's meant to do, that's

  (sized edge-case n
    (rand-choice      ; or frequency, if you want to bias the choices
      edge-case-1
      edge-case-2
      edge-case-3
      (arbitrary default-range n)))

  (prop foo (x 'edge-case)
    (bar x))
But that won't enumerate all of the edge cases. It will with fewer than 100 of them, on average, but it's all still random. You could un-randomize the generation a little bit with something like

  (let index 0
    (sized edge-case n
      (if (< index (len list-of-edge-cases))
          (do1 (list-of-edge-cases index)
               (++ index))
          (arbitrary default-case n))))
Then each call goes through list-of-edge-cases in sequence. Once it gets to the end, it starts pumping out arbitrary values.

  arc> (= list-of-edge-cases '(foo bar baz))
  (foo bar baz)
  arc> (= default-case 'int)
  int
  arc> (arbitrary 'edge-case 10)
  foo
  arc> (arbitrary 'edge-case 10)
  bar
  arc> (arbitrary 'edge-case 10)
  baz
  arc> (arbitrary 'edge-case 10)
  -10
  arc> (arbitrary 'edge-case 10)
  -5
If you have more edge-cases than the number of times quick-check runs (presumably 100, though sometimes more with whenever clauses and such), you still won't go through all of them. So you could manually go through each edge case separately from the quick-check call, then run quick-check, and...

Square peg, meet round hole.

However, this thread's gotten me interested in writing a proper unit-testing library for Arc. The main considerations are API design and the output's readability. After figuring those out, the programming part's easy. And there are lots of example libraries to base it off of. We'll see if I get around to it any time soon. (Maybe this weekend? No promises, though; I've got a bunch of work to do.)

What becomes important and interesting, then, is to automate the synergy between QuickCheck and unit tests. Instead of "run QuickCheck, get a failure, copy/paste into a unit test, fix code, run QuickCheck again, pass, run unit tests, pass", there should be a way to spill over QuickCheck failures into unit tests transparently, so we don't even need to think about it. Again, easy enough to do, once I settle on the way it should be done. Is that closer to what you meant?

-----

2 points by akkartik 5457 days ago | link

(Request: please post an email on your profile.)

"this thread's gotten me interested in writing a proper unit-testing library for Arc.. automate the synergy between QuickCheck and unit tests. Instead of "run QuickCheck, get a failure, copy/paste into a unit test, fix code, run QuickCheck again, pass, run unit tests, pass", there should be a way to spill over QuickCheck failures into unit tests.."

I'd love to help. I've been testing a lot with arc. Readwarp has more tests than code.

One recent idea to improve testing is to run tests in dependency tree order: http://arclanguage.org/item?id=12721

-----

2 points by fallintothis 5457 days ago | link | parent | on: Access Lexical Variables

[Insert caveats about how crazy the idea is, yadda yadda, stuff rntz already pointed out, blah blah, who cares?]

There are too many edge-cases for the idea to work well once implemented (as I said in the giant rant above :P), but you can get the basic behavior down, which fixes the issues you were finding early on in the thread. I gather that's what you were asking. So, for the sake of fun:

  (mac lexbound (x)
    `(if ((locals) ,x)
         t
         (bound ,x)))

  ; ac won't care about shadowing a special-form, but we don't want to treat it
  ; like an arg to the br-fn below

  (def keyword (name)
    (or (nonop name)
        (in name 'nil 't 'if 'fn 'assign 'locals)))

  (mac make-br-fn (body)
    (w/uniq args
      `(fn ,args
         (with ,(mappend
                  (fn (v) `(,v (if (lexbound ',v) ,v
                                   (no ,args)     (err "Too few args to br-fn")
                                                  (pop ,args))))
                  (sort < (keep (fn (_) (and (isa _ 'sym) (~keyword _)))
                                (dedup (flat body)))))
           (when ,args
             (err "Too many args to br-fn"))
           ,body))))

  arc> ((make-br-fn (+ x y z)) 1 2 3)
  6
  arc> ((make-br-fn (+ x y z)) 1 2 3 4)
  Error: "Too many args to br-fn"
  arc> ((make-br-fn (+ x y z)) 1 2)
  Error: "Too few args to br-fn"
  arc> (let x 3 ((make-br-fn (+ x y z)) 1 2))
  6
  arc> (let + (fn (x y z) (prf "x = #x, y = #y, z = #z") (prn))
         ((make-br-fn (+ x y z)) 1 2 3))
  x = 1, y = 2, z = 3
  nil
  arc> ((make-br-fn (do (prs x z y) (prn))) 1 2 3) ; bound in alphabetic order
  1 3 2
  nil
You have to do the work within the expansion itself (at run-time), since that's where you need to check whether variables are lexically bound. I.e., if you do it in the macro, you'll just get the locals from macroexpansion time. This essentially works the same way the following do.

  arc> (let x (if (lexbound 'x) x 5)
         (+ x 5))
  10
since the call to locals winds up happening outside the outside of the fn that let generates:

  arc> (macsteps '(let x (if (lexbound 'x) x 5) (+ x 5)))
  Expression:

    (let x (if (lexbound 'x) x 5)
      (+ x 5))

  Macro Expansion:

      (let x (if (lexbound 'x) x 5)
        (+ x 5))

  ==> (with (x (if (lexbound 'x) x 5))
        (+ x 5))

  Expression:

    (with (x (if (lexbound 'x) x 5))
      (+ x 5))

  Macro Expansion:

      (with (x (if (lexbound 'x) x 5))
        (+ x 5))

  ==> ((fn (x) (+ x 5)) (if (lexbound 'x) x 5))

  Expression:

    ((fn (x) (+ x 5)) (if (lexbound 'x) x 5))

  Macro Expansion:

      (lexbound 'x)

  ==> (if ((locals) 'x) t (bound 'x))

  Expression:

    ((fn (x) (+ x 5)) (if (if ((locals) 'x) t (bound 'x)) x 5))

  nil
Whereas

  arc> (let x 10
         (let x (if (lexbound 'x) x 5)
           (+ x 5)))
  15
since the call to locals winds up happening inside the fn that the outermost let generates:

  arc> (macsteps '(let x 10 (let x (if (lexbound 'x) x 5) (+ x 5))))
  Expression:

    (let x 10
      (let x (if (lexbound 'x) x 5)
        (+ x 5)))

  Macro Expansion:

      (let x 10
        (let x (if (lexbound 'x) x 5)
          (+ x 5)))

  ==> (with (x 10)
        (let x (if (lexbound 'x) x 5)
          (+ x 5)))

  Expression:

    (with (x 10)
      (let x (if (lexbound 'x) x 5)
        (+ x 5)))

  Macro Expansion:

      (with (x 10)
        (let x (if (lexbound 'x) x 5)
          (+ x 5)))

  ==> ((fn (x)
         (let x (if (lexbound 'x) x 5)
           (+ x 5)))
       10)

  Expression:

    ((fn (x)
       (let x (if (lexbound 'x) x 5)
         (+ x 5)))
     10)

  Macro Expansion:

      (let x (if (lexbound 'x) x 5)
        (+ x 5))

  ==> (with (x (if (lexbound 'x) x 5))
        (+ x 5))

  Expression:

    ((fn (x)
       (with (x (if (lexbound 'x) x 5))
         (+ x 5)))
     10)

  Macro Expansion:

      (with (x (if (lexbound 'x) x 5))
        (+ x 5))

  ==> ((fn (x) (+ x 5)) (if (lexbound 'x) x 5))

  Expression:

    ((fn (x)
       ((fn (x) (+ x 5)) (if (lexbound 'x) x 5)))
     10)

  Macro Expansion:

      (lexbound 'x)

  ==> (if ((locals) 'x) t (bound 'x))

  Expression:

    ((fn (x)
       ((fn (x) (+ x 5)) (if (if ((locals) 'x) t (bound 'x)) x 5)))
     10)

  nil
That said, there are plenty of bugs.

- The most obvious is that you'd need a full-blown walker of the body argument to see where you really need to make new variables. E.g.,

  (make-br-fn (let x 5
                (+ x 10)))
macroexpands to

  (fn gs2296
    (with (+   (if (lexbound '+) +
                   (no gs2296)   (err "Too few args to br-fn")
                                 (pop gs2296))
           let (if (lexbound 'let) let
                 (no gs2296)       (err "Too few args to br-fn")
                                   (pop gs2296))
           x   (if (lexbound 'x) x
                   (no gs2296)   (err "Too few args to br-fn")
                                 (pop gs2296)))
      (when gs2296
        (err "Too many args to br-fn"))
      (let x 5
        (+ x 10))))
which winds up expecting an x argument, because it wasn't bound outside of the br-fn, even though it's just a local binding introduced in the body by the let.

  arc> ((make-br-fn (let x 5 (+ x 10))) 'ignored-x)
  15
- make-br-fn should probably start with (zap macex-all body), since macros may introduce arbitrary unbound variables. E.g.,

  arc> (macex1 '(prf "x = #x"))        ; introduces an unbound reference to x
  (let gs396 (list) (pr "x = " x))
  arc> ((make-br-fn (prf "x = #x")) 1) ; prf not expanded
  Error: "Too many args to br-fn"
Couple this with the above bug, and you have even more problems, since in the above expansion of prf, you can't tell that gs396 is a bound local in body without doing a code-walk.

- You'll still have problems from vanilla Arc's lexically-shadowed-macros bug in some cases, but they're the ones you'd expect from knowing about the bug to begin with. That is, the following works because do is not in a function position.

  arc> (let do 10 ((make-br-fn do)))
  10
But, instead of generating an error (applying 10 to 5), the following macroexpands do.

  arc> (let do 10 ((make-br-fn (do 5))))
  5
This thread's made local variables in Arc interesting in ways I hadn't thought of before. Macros introduce odd contours to lexical environments that don't exist in, say, Python (which still has a locals() function). Basically, any use of (locals) should come with a big, fat warning: "magic happens here".

-----


Not much to do with content, but some sort of syntax highlighting (even just bold-facing keywords or something) would make longer code samples on such a site much more readable -- both for Arc and Scheme snippets. At least to me. There's still the sect of people who are too manly for syntax highlighting or whatever, so you could have a link at the bottom of codeblocks to toggle it.

This also applies to inline fixed-width fonts, which you should have as well. It's annoying as hell to have a billion conventions for inline code here: some people 'quote, others "quote", you could italicize, some don't do any formatting, and sometimes I just get lazy and put it

  in its own block.
A sane syntax for this would be good. I'm never sure if arc-globals* interact well with the italicizing syntax (asterisks) in different contexts: (= global 5)*?

-----

2 points by fallintothis 5461 days ago | link | parent | on: Share your useful functions/macros

And then, exploding forth with a dramatic surplus of conceit and utter lack of humility, came ... ME!

cons counting

http://arclanguage.org/item?id=11135

tracing

http://arclanguage.org/item?id=10372

visualizing macroexpansion

http://arclanguage.org/item?id=11806

the list goes on

http://arclanguage.org/submitted?id=fallintothis

(Okay, that last one was excessive, but I thought it was funny. I'm just glad anyone likes things I write. :P)

P.S.: Gah! I hadn't realized rand-elt would break on empty sequences before. Good catch. Should probably use (unless (empty ...) ...), in case you want a random string character.

-----

2 points by fallintothis 5462 days ago | link | parent | on: Share your useful functions/macros

Ah, that's a fun one. Could use counts to get a table in vanilla Arc, except that it hard-codes recursion on cdrs, so it won't work for strings. From arc.arc:

  (def counts (seq (o c (table)))
    (if (no seq)
        c
        (do (++ (c (car seq) 0))
            (counts (cdr seq) c))))
It's also easy to not notice the built-in sortable from srv.arc:

  (def sortable (ht (o f >))
    (let res nil
      (maptable (fn kv
                  (insort (compare f cadr) kv res))
                ht)
      res))
So,

  (= count-up (compose sortable counts [coerce _ 'cons]))
until counts gets fixed to use each, at which point it's just

  (= count-up sortable:counts)

-----

1 point by akkartik 5461 days ago | link

Ah, getting rid of the optional arg lets me define counts using defgeneric!

I've updated my repo with your suggestions: https://github.com/akkartik/arc/commit/a5b3c6d6bf616f51bc4ab...

-----

More