I recommend using publisher.hackname.release, where publisher is your Arc forum username.
For example, Linus Torvalds writes hardly a line of kernel code these days, he largely integrates the work of others. So there's a role for authors and a role for integrators.
Under my hack naming scheme, all the hacks you publish would be rntz.hackname.release. The hack may have one author or multiple authors, you may not be the author or may not be one of the authors.
"catdancer.exit-on-eof.1" is an exception, because it already has a name, and you're really just importing the hack into git, rather than doing your own integration. (If, however, you changed it to get it to work with Anarki, then you'd use "rntz.exit-on-eof.0").
The reason for using the publisher in the hack name is to make it globally unique. For example, you and I may both happen to port "arcsh" to arc3. For people who aren't using git, it's useful to be able to refer to hacks and download them by a unique name. By using "rntz.arcsh.0" and "catdancer.arcsh.0" there's no conflict. It should be understood that the presence of the publisher's username "catdancer" or "rntz" in the name does not imply authorship.
Should there be a periodical (i.e. once/twice a month) effort to merge together various publishings of the same hack? I think it's important to do that, so that we don't have five differently-named but almost identical versions of a hack floating around after a few people add features. A list of contributers to a hack/library can be kept as commented "meta-data" within the file, and the hack can then be renamed to anarki.hackname.release.
However, I will point out that the name of the project/community is "Anarki", and so it is very likely to be distributed and each person will probably want their own version anyway.
Just like Linus Torvalds decided he wanted to be the Linux integrator, someone will emerge who wants to put together the best version of various variations of various hacks. If he or she does a good job, he or she will become respected and people will look first to him or her to find a version of a hack they want to use.
Information about a hack, like "this hack is the best of the bunch", is really best not put into the hack name, because later if the information changes (maybe you find out the hack has a bug), now you have a name that's giving bad information.
While my attempt to use git as a way of sharing libraries and managing library dependencies (as opposed to, or in addition to, using git in the normal way as a version control system) didn't work out, I learned a number of useful things from it. So while I don't have all the details, I do have some important design principles:
Every release of a library should have a globally unique name: the publisher (we can use our Arc forum username), the name of the library, and the release number. An example of the full name of my toerr library would be "catdancer.toerr.3"
Every release of a library has to be immutable. Once I publish "catdancer.toerr.3", I must never change it. The reason for this is meta-data about a libraries, conflicts, and dependencies; I might say Fred's foo library release 19 doesn't work with my toerr release 3, but there's no way to know if this meta-data is valid if Fred or I are changing our released versions.
I can unpublish a release if I need to; if "catdancer.toerr.3" has a horrible bug I can remove it from my server and publish a fixed version "catdancer.toerr.4". But I must never reuse "catdancer.toerr.3" for a changed version.
The release number is used internally by the tools. A new release number doesn't mean that the version is better. Compared to "catdancer.toerr.2", "catdancer.toerr.3" may be an alpha version or a release candidate or broken in some way.
If you want to have a meaningful version number for your library, make it part of the library name: "foo0", "foo1", "foo2". That way you can release a fixed version of "foo2" if you need to.
We should also be able to publish meta-data about my libraries. For example, I should be able to publish that "catdancer.toerr.3" is, in my opinion, the best available version of the catdancer.toerr library. Unlike a particular library release, which can be deleted but must never change, meta-data can be updated. I can later say that "catdancer.toerr.4" is now better.
The tools should naturally let me abbreviate library names. If I say (load "toerr.arc") or the equivalent, and there's only a "catdancer" toerr library available (or maybe I have "catdancer" first in my search list), and my meta-data says that "4" is the best one, then it should load "catdancer.toerr.4" for me. But if I want to explicitly load "fred.toerr.19", I should be able to do that.
Every library should be available at an http location by username. For example, if you're using git and github, you can publish your repository using the github Project Page feature (http://pages.github.com/).
Now we'll have a list associating usernames with http locations:
this list needs to be kept at some central location, but nothing else does.
Now the tool can download any library it needs to. Given "catdancer.toerr.2", it can lookup "catdancer" in the list of locations, and download "http://hacks.catdancer.ws/catdancer.toerr.2.arc
I agree with several of your pronouncements, such as the fact that libraries need to be immutable, etc.
I think that the naming system should use "user/libname/version" so that it can be reflected in the directory structure of the lib folder of arc where they will be downloaded, and can be easily turned into a path. I'd prefer to make it a 'sym instead of a 'string for less typing ;)
Thus the library loading code would be effectively:
(use libname)
where libname is just a symbol representing a path to which "$arc_home/lib/" and ".arc" are attached. This makes it easy to load files which are already in your lib folder that you wrote, or downloaded. It works with downloaded libraries, and can hopefully detect if you've already loaded it or not, and avoid reloading it. In theory since it is a library it won't be changing and thus only needs to be loaded once.
We can also have other functions that fetch the libraries, or make 'use fetch them automatically. In that case it will need to know how to find them. To that end I propose we add a file to the lib folder called something like "server-list" or something. The file is merely a list of http locations, such as hacks.catdancer.ws, and fred.github.com/arcstuff. Each location will be a folder that contains a file named "libs.arc" or similar, which contains all of the metadata necessary to find the libraries belonging to that server.
Possible meta-data: Name of library, file name and location, date updated, author, dependencies etc.
In theory the package system can determine by reading these files which version of the library is newest, (by version number and date) or if given more particular criteria, find an older version.
I'm basing my ideas off of the debian package system, which I have found very useful. The nice thing about it is that anyone can host a package server, and a package can be located on any server. They aren't tied to a particular server by their name, for instance. It also automatically downloads prereqs, which can be very handy.
Summary:
1) Library is named by symbol which can be easily converted to a path: catdancer/erp/0
2) Servers which have meta-data on finding packages, a la debian packages/yum/gems, etc. for ease in hosting and finding packages.
3) The 'use function, whatever it's called should be able to load local libraries as well as online ones. It should probably also be capable of taking in a direct web address like CatDancer's lib function.
Ok. Why not support both? That way someone can create a large and structured library with multiple sub-folders if they need to.
I was also hoping of making a version of your 'lib hack to manage the libs already in the lib folder, instead of just those on the web.
(lib binary)
seems like much less typing than
(load "lib/binary.arc")
and also has the advantage of not reloading it if it's already been loaded.
On a side note, how hard would it be to selectively reload individual functions?
(reload example-fn)
Since the anarki help system keeps track of what file the function was declared in, it could presumably be used to automatically read in the file and eval the proper form. I'm just somewhat tired of working in a large library file and having to reload all of the functions, even if I don't need to. (I'm using a very impotent linux box for development, so it can get rather slow)
a large and structured library with multiple sub-folders
I don't think we need to design this system to do everything that somebody might someday need. There's already plenty of solutions for distributing large collections of files such as zip or tar files; we don't need to invent something to solve that problem.
EDIT: I didn't explicitly say this here, but I will now: I think that the version numbers should make a definite statement that "catdancer.toerr.3" is an inferior predecessor to "catdancer.toerr.4". I think that in the interest of avoiding "dependency hell", people should be expected to use the latest stable version of a library. However, to ease backwards compatibility, if a library developer sees that the next stable version of their library will break compatibility with other libraries which depend upon their library, they should inform the developers of the other libraries about this, so that once the new library is released, other libraries can be quickly updated to work with the new version.
I think that a good system for version numbers (and who doesn't love copying good systems?) is to have two "latest" versions always available -- a stable version, and an alpha/beta version. The stable versions are the odd version numbers, and the alpha/beta are the evens. Obviously that could be switched, but everybody should use the same convention, and I think this one makes sense because a project starts at version 0, and then the first stable version would be 1.
If this system were used, than only the odd-numbered versions would be required to remain constant. The even-numbered versions could vary as the dev(s) fixed bugs or added features. Odd-numbered versions which depended on other libraries would have to depend on odd versions of those libraries. An even version could depend on any library.
I like the idea of using forum nicknames, because they're unique. URLs are (not entirely, thanks to Internet Explorer...) case-insensitive, so maybe pg should change the forum so that two usernames can't be case-insensitively equal? (If that's the case already, scratch what I just said...)
Meta-data can come in a separate file, named the same as the library. It should probably be some form of alist:
This last part could work because libraries would be uniquely identified by a string, as CatDancer explained above.
Also, some form of standard directory structure could be good. Each person would be able to customize where their lib/ directory would be, and what it would be called (in the example, my directory is arc/keystones/). However, within that directory, I think there should be some convention of how libraries would be organized. I think one that makes sense is that each library would have a directory, within which each version would have a separate directory. If this is nested too deeply, it could instead be a wide nesting -- arc/keystones/foo.xyzlib.3/
Within the library directory, the file which gets loaded should have some standard name too -- the most obvious one would be the name of the library. The directory could contain other files containing more code, and those files would be loaded (or required) by foo.xyzlib/3/xyzlib.arc. Meta-data would be in the file meta.arc.
Customizability of the lib folder is probably a good idea, but it will probably done via hacking the code for the lib functions ;) It shouldn't be that hard to do anyway. With my naming scheme, you'd just change the string that was prepended to the library name.
I'm not sure that odd/even version numbers is such a good idea. It could be very confusing that way. I think that CatDancer's requirement of libraries to be static is a much more reliable concept. Otherwise like he said you'd need to check periodically for updates.
Libraries should also be able to depend on whatever they want. That's the author's decision. If they need the beta version, but they've tested it and know that what they have written is stable, then they should be allowed to publish it that way. They can always make a new version if they need a bug fix.
Also, since the version is just part of the lib name, you can have as many layers of minor version that you want i.e. 1.5.200906015.
Libraries should also be able to depend on whatever they want
Dependencies should actually be managed outside of the libraries themselves. For example, I have a library foo that depends on bar. Later a new version of Arc comes out that implements what bar did. Now foo doesn't need bar any more. But foo itself hasn't changed, so I shouldn't have to release a new version of foo just to say that it doesn't need bar with this new version of Arc.
Instead we publish dependency information about libraries. For example, I can say that foo needs bar 0 and arc 3, or just arc 4... something like:
The odd/even numbering doesn't have to be exactly that way. It could also be something like foo/xyzlib/1b for the beta, and foo/xyzlib/1 for the "stable" version.
only the odd-numbered versions would be required to remain constant
the version numbers should make a definite statement that "catdancer.toerr.3" is an inferior predecessor to "catdancer.toerr.4"
There's a difference between a release number and a version number. A version number, as you say, can be used to indicate that a later release is better, or indicate the stable vs. alpha/beta status of a release, etc. The release number merely identifies releases.
For example, pg had several releases of arc3. Under my naming system, they would have been named "pg.arc3.0", "pg.arc3.1", "pg.arc3.2", etc.
Regardless of the alpha/beta/stable status of a release, two releases of a library should never be released with the same name and release number for several reasons:
- If I'm telling you about a bug in your library, then I can tell you which release if saw the bug in. If you change your library without giving it a new release number, then we won't know if I'm talking about your old release or your new release.
- It's clear when a tool such as my "lib" library which downloads a library from a URL needs to download a new release. If there's a new release number, and I want that new release, then "lib" knows it needs to download the new release. If the release can change at the same URL, then "lib" has to periodically check to see if the file at the URL has changed.
- Just because I think that a release of mine is a "alpha/beta" version doesn't mean that you might not want to keep using it.
I see what you mean -- I was a bit confused about version vs release.
However, I still think that the name of the library should be "arc". Maybe the releases would be named "pg.arc.3.0", "pg.arc.3.1", etc. I just think that the library name should be distinct from the version and release numbers.
So if you want your library foo.arc to land in Anarki in the lib directory, create a patch to Anarki that adds foo.arc to the lib directory in a single commit.
One way to do this is to have one branch that tracks Anarki, and then a separate branch for your own development. You can merge new Anarki commits into your private development branch, but avoid ever merging from your development branch back into your Anarki tracking branch directly. Then in your private development branch you can have a messy development history (create foo, change bar, change foo, do something else, then change foo again), but only have add clean commits to your Anarki tracking branch that you push to Anarki ("create foo" or "update foo").
the hard part is making it work, and making it work without leaving the old definitions behind when the lambda no longer exists
I don't know about the first part, but for the second, what you want is a weak hash table (http://download.plt-scheme.org/doc/372/html/mzscheme/mzschem...) with the lambda's stored as the keys. The weak hash table holds the keys "weakly", which means it doesn't prevent the keys (the lambda's) from being garbage collected, and when they are garbage collected, the related value is dropped from the table.
If all the following lines go together as a group, and the arguments on the first line aren't part of that group, then I'll indent the following lines differently to emphasize the grouping:
(each (k v) mytable
(prn "my key is " k)
(prn "and my value is " v))
It doesn't matter to me whether it's a macro or a function:
(copy (obj a 1)
'b 2
'c 3)
I use the two space indentation a lot more with macros than I do with functions because many macros use the pattern of some arguments followed by a body and not many functions do, but I don't change my indentation depending on whether it's a function or a macro.
If the lines after the first line don't fall into their own group, then I'll line up the arguments:
(+ '(a b c d)
'(e f g h)
'(i jk))
(map (fn (x) ...)
(generate-my-list ...))
For me the important consideration is using indentation to show which arguments go together.
rntz notes that this isn't as powerful as symbol macros, and that's true: this lets you specify the behavior of a global variable at run time, while symbol macros would let you manipulate the effect of a symbol during macro expansion (compile time).
I don't get symbol macros. Is there an example of a symbol macro in use? On another thread someone mentioned that a symbol macro can expand into arbitrary code. Is this the same as an ordinary macro with no arguments (except for a pair of parens) ?
On a possibly related note, when compiling (assign ...) forms, ac.scm calls macex on the 1st arg:
(assign (foo bar) 12)
'(foo bar) gets macexed. Later, ac-set requires that its first argument be a symbol. So '(foo bar) must expand into a symbol. I didn't see this obviously being used anywhere in arc ... do I need to look harder? Is its usage buried under layers of macros (deep inside '= for example), or what's the story? Is this somehow related to the concept of symbol macros?
I'm not sure about your related note, but for an example of symbol macros in use:
This example is from Common Lisp's object system, called CLOS. When you want to do some code while accessing specific slots of an object, you can use a 'with-slots form, which lets you name symbols that you'll use to reference specific slots of the object. The 'with-slots form then generates local symbol-macros for each symbol you mentioned, which expand into code to access those slots. An example, taken from PCL [1]:
(with-slots ((bal balance)) account ; Here 'balance is the name of the slot, and 'account is the specific object
(when (< bal *minimum-balance*) ; *symbol* is a CL convention for global variables
(decf bal (* bal .01))))) ; decf is like --
[1] Practical Common Lisp, by Peter Seibel, available online at www.gigamonkeys.com/book/
Another use for symbol macros is in writing package/module systems, where you want to determine which package a symbol goes into based on the compilation context.
You can do this to a certain extent already, using package!symbol notation, but this still doesn't allow using symbols without prefixes.
This uses rntz's dynvar name to declare a dynamic variable, and I'm trying out the "ac-scheme" name for the syntax to tunnel a Scheme expression through the Arc compiler.
I was using github's cool Github Page feature, where you can push a git repository to them, and they'll automatically create a web site out of it. The thing is, after you do the push, it goes into a queue, and some random time later depending on how busy github is the push will be processed to actually publish the changes. I found it was breaking my flow; the first thing I want to do when I publish something or write some instructions "go here and get this" is to try it and make sure that it's actually working. Nothing against github, it just wasn't a good fit for my particular work pattern.
I'm also slowly automating the hack documentation, so while naturally I do want to keep the source under version control it's just an extra step to have the generated HTML also in a git repository.
And I have aspirations of someday making this into a web app where anyone can publish their hacks, so it couldn't stay on github forever anyway.
I plan to still push the hacks to github, that way if my personal server goes down people won't be stuck waiting for me to fix it.
To implement my hacks I've already needed to pull a few pieces of the Arc compiler up into Arc, such as ac-denil and ac-global-name. So the direction this is heading in is to make parts of the Arc compiler available in Arc: ac-denil, ac-global-name, ac-scheme.
This allows us to do various powerful things, of which one is getting at Scheme functionality easily.
You could then write a plain "mz" macro that expanded into "ac-scheme" directly, or, if you wanted, a more sophisticated one that did things like giving easy access to global variables in the expansion.
The "scheme" part of the name is a visible reminder that the code inside is literal Scheme code instead of Arc code.
