This isn't actually related to the work you are doing, but I was curious why you chose to use notabug.org over, say, gitlab.com. I hadn't actually seen the service before you linked to it on this forum.
Under knarks "Why the fork?" section hjek links to the 'ethical repository'. Since notabug.com is only for open source projects then the repo's will likely be considered 'free software' which I believe makes it grade A in 'ethical repository' terms.
Personally I find the term 'ethical repository' offensive. It insinuates that non-free software is unethical when the majority of non-free software has no nefarious intent or code. Not exactly the greatest sales pitch in my book.
I think free (or "open source") and ethical mean the same in most cases.
Exceptions might include something like Facebook, which is technically somehow usable w/o non-free JS when using their basic mobile web page, but where the company is still engaging in other unethical activities, like selling user data to sway elections.
Or something like Amazon, where you might possibly be able to buy something w/o non-free JS (haven't checked), but where the treatment of their employees is unacceptable.
But, I think, when we're talking git hosting sites, there's no difference?
But FSF considers Gitlab ethical enough for hosting GNU packages.
As I understand it - the 'Open Source' movement concerns itself with improving the software by making the code openly accessible, where as the 'Free Software' movement concerns itself with a fighting for users rights (i.e. having the freedom to access, modify and distribute the code in a manner that empowers the user).
And so, an 'Open Source' repository holds code that is openly accessible for the purpose of improving the software. Where as an 'Ethical Repository' holds code that is graded by its' ability to guarantee users rights according to a specific set of morals (established by free software foundation). It so happens that open source repos tend to align well the ethics associated with free-software, but they should not be mistaken for each other. As an example to illustrate: If a repo SaaS were built for open source code, but restricted users from a certain country it wouldn't rank high in ethical repository grading. This is because while having the code openly accessible leans towards a Grade A rating (excellent), the restricting some users part puts it at a Grade F rating (unacceptable).
"Despite initially accepting it, Richard Stallman of the FSF now flatly opposes the term "Open Source" being applied to what they refer to as "free software". Although he agrees that the two terms describe "almost the same category of software", Stallman considers equating the terms incorrect and misleading. Stallman also opposes the professed pragmatism of the Open Source Initiative, as he fears that the free software ideals of freedom and community are threatened by compromising on the FSF's idealistic standards for software freedom. The FSF considers free software to be a subset of open-source software, and Richard Stallman explained that DRM software, for example, can be developed as open source, despite that it does not give its users freedom (it restricts them), and thus doesn't qualify as free software."
> They are categorically demonizing innocent people.
I'm sorry; that was not my intention.
Perhaps I can make a comparison to clarify? As an example, some people think that guns are unethical because they may be seen as an unjust instrument of violence. Even if a particular gun hasn't killed anyone (yet), or even if most guns happened not to be used to kill, then surely it can still be legitimate for people to object to the passive presence of guns, because it gives gun owners the power to kill, and that power may be considered unjust by principle.
Similarly, some people think that non-free software is unethical because it gives programmers the power to do bad stuff, regardless of whether some particular non-free program is actually malware (yet).
(Sorry in advance if I've derailed this discussion into a more controversial subject.)
(continuing the discussion for clarity... no emotional connotation is intended)
I realize this is a comparison for clarification, but isn't it still just 'categorically demonizing innocent people'?
You picked a more controversial topic where more people are likely to agree with the demonization I suppose, but your assertion that the power to kill "may be considered unjust by principle" is not well supported by vague assertions that "some think" guns "may be seen as" unjust instruments of violence. I fully support everyone's right to object to something they see as dangerous; opinion does not constitute principle, however.
To me, something is just or unjust based on whether or not it aligns with or infringes anyone's rights. So, I suppose I might actually agree that a power could be "unjust in principle" if it could be shown that the power could not be used justly - that is, without infringing on anyone else's rights. For some powers, mostly political ones, this is the case. In this case I think guns may be a poor comparison, because they actually can be used in ways which are just (defense, etc.), even if you believe that those cases are unlikely and so desire strict gun control, etc.
In contrast, it may be that producing nonfree software is always 'wrong' (in that it infringes on the supposed rights of the users to understand and modify the program they are running) and therefore having or providing the power to do so would be 'unjust in principle'. If the concern is merely that some may produce malware, and there are actually legitimate reasons for producing nonfree software, then it is not unjust in principle to do so, or to provide someone with said power.
I hope I've understood all that correctly, and restated it well. I'm not sure that I agree with the idea that nonfree software is always bad, but I am open to it. Perhaps what I'm missing is a clear understanding of the specific rights that nonfree software violates.
No worries, I know you (as well as the authors) are simply trying to apply implicit safety measures to counter bad actors. And I'm certainly not offended by you adopting the program. It's my feeling, however, that their approach is horribly wrong and bordering on corruption. I simply don't believe they will have any success when trampling over the good actors in their process of trying to better the world. IMO; If they really wanted to make a dent, they should push for a regulation requiring that browsers provide functionality that enforces a free-software configuration OPTION. Then allow society to decide for themselves (this is a free world after all). I'd even be ok if the default setting was on. But as it sits right now they will get nowhere really fast.
edit: oh and as for the gun analogy... I'm from Canada and fully support gun control (we have it), but I'm not going around and implying that every gun owner is unethical in the process of asking for gun control. That would be shooting myself in the foot!
However I am opposed to that call for action given it's an all-or-none implementation. I feel it's the role of each country to regulate, which is why I expressly suggested it as a configuration option (ideally it could be enforced at the browser level country by country and if not then user by user).
I think if we fork the community site to run on anarki. which I think is more likely than being given control over the Arc Forum, we should consider ways to archive and bring forward all of the stuff on the existing arc forum. It shouldn't be too hard to crawl the forum, though I think there might be some DoS prevention that would slow it down.
For anarchic service management, what do you think about trying to set it up so that the service is automatically updated if the latest commit passes some form of CI testing? The CI tests themselves could be updated if the new version passes the latest working version of the site.
It would still be possible for someone to break things in two passes of "delete the tests" then "delete the server" but regular automated backups should mitigate the risks somewhat. And we haven't had many problems with spammers anyway...
Given the open nature of the anarki repo, it's likely that news will break. And when it does we wouldn't be able to discuss it.
So unless these tests could prove that the forum would work (which is highly unlikely) then my vote would be not to do this. It's akin to putting the services issue logging/tracking system under the same service . It's a bad idea IMO.
Or maybe the immediate and painful result of breaking the forum would motivate us to be more careful and fix the issues more quickly. It probably won't happen that often anyway.
I was about to say that an outage might risk killing the community, which would be bad, but 1) we still have this forum and GitHub (as krapp points out), and 2) if the community is really so weak that it can't revert a commit in order to get the forum running again, it's probably not worth hosting a separate site anyway.
The idea is growing on me, just because of how audacious it is. (^^)
I do agree with some of your point though; it would be good to have some separate logging and bootstrap systems in place so that we can detect and repair faults more easily, without the intervention of a specific admin. For one thing, the software that pulls the changes will probably not be arc-based, so it should still be running even if the forum goes down. Secondly, we could try to set it up so that it always pulls hotfix patches immediately when the logging / monitor system indicates failure.
Also, marking a particular branch (probably not master) as a more 'stable' version might be good.
Yeah, that's part of why I've never seriously considered it before.
The only reasons for thinking about it now are that 1) we want to add some features to the forum, and there's no way to test them here, and 2) it isn't actually a bad idea to have a community site for Anarki. The risk of weakening the community has deterred me from the idea of forking the arc forum, but if we still treat this as the 'official' arc community, and make a separate site more focused around anarki, I don't think that would be too bad.
It might actually help some, since separating more could allow us to really focus on and develop our unique points of experimental language hacking.
Don't let my comments stop you. Your thinking is quite valid. I'm just trying to contribute my opinion in hopes of helping you shape whatever you decide.
> It might actually help some, since separating more could allow us to really focus on and develop our unique points of experimental language hacking.
As far as I can tell, pretty much everyone has moved over to anarki, so I don't understand your comment. How does creating a separate forum for anarki help to "focus on and develop our unique points of experimental language hacking"?
Thank you for your opinions; it really does help me clarify my own thoughts. That and I rather dislike talking to myself for more than a few minutes at a time...
Currently, this is an Arc forum. People are drawn here from pg's posts about Arc, and their attraction to the simplicity and beauty of the language. Also probably dreams of silver bullets... That won't stop just because we make a separate community, and I really don't think there's anything wrong with it.
However, I think we're also torn a bit between maintaining basically a bugfix version of arc, or going on to develop it further.
The question then is what "further" means - which direction would it go? One answer is that it can't really be predicted; creating such a community would be a way to find out.
On the other hand I can speculate, based on what I've seen of this community so far.
I suspect this is partly because arc is so small, and does not have many standard libraries or a package system. You can read it all in a fairly short amount of time. So the fact that most of the code is aimed at developing arc means that the easiest thing to develop with it is... arc.
This trend of exploration and extension will undoubtedly continue, and I think it will be much more free to develop into something significant if we simply look at anarki with the slightly different perspective that having a separate community site we could actually upgrade might offer. Instead of being weighed down by arc as the 'community bugfix edition', it could become a 'language based on arc' with a solid foundation, but room to grow.
The core paradigms and strengths of arc appear to be "exploratory programming", and "language hacking". I think it would be cool if we could develop the former beyond the latter, but who knows how it will turn out?
Of course, I may be entirely off-base here myself...
Hmm, you know, a new anarki forum could have the benefit of adding a tags feature for posts. A few good tags could be 'lang-design', 'arc', 'anarki', 'help', etc., etc. Not only would this allow our subtypes of members to zero in on their content of interest, but would make searching for meaningful info much easier. Now that would probably make me jump over.
Yep, improving the forum would help a lot, in a lot of ways.
I've often thought that the structure of the news.arc forum is rather unhelpful for the arc community, especially now that we're so small and lethargic. Conversations can go extended periods of time without comment, so they get locked. Or they fall off the front page, and hard to find again. Neither is conducive to long-term development and improvement; we probably lose a lot of valuable work and ideas that way.
One thing I'm considering as part of developing a new community site is collecting and archiving all of the arclanguage.org content, so we can actually access it. And preserve it, if the site goes down.
But I end up wasting all my time on discussion, instead of actually making progress on that...
Looks as though it ranks how bad you are and always keeps the baddest of the bad-asses in cache, while never deleting any from disk. In a low volume site like this I doubt you'll get out of it without contacting them.
Sure, but from what I can see they soon quickly discover they need to use anarki and move over.
And impacting newbies does not appear to be considered in "focus on and develop our unique points of experimental language hacking". So...
edit: maybe I'm wrong, but it seems to me what he really wants is a language design group. And I'm fine with that, but I think it's wrong to conflate anarki users with language designers. They are not one in the same.
I was expecting arclanguage.org to stay mostly as it is. Support for newcomers to arc would be included in that.
You may not be entirely wrong, but I probably shouldn't have tried to compress a description of a language used and developed by a loosely federated group of unique individuals into a single phrase.
I can try to unpack it a bit...
"Experimental (language) hacking" -> Exploratory programming is supposedly a primary paradigm of arc
"Experimental language hacking" -> Arc isn't exactly production ready; it's a very experimental language, and that makes it fun (and sometimes frustrating) to use, and easier to explore new directions and possibilities. You're less likely to reverently assume that the way it is is the way it must be.
"Experimental language hacking" -> And yes, we hack on arc itself. So I am thinking a bit of a language design community, I guess. In an anarchic language community though, the lines between 'users' and 'designers' become rather vague...
If that's an argument against us using Anarki then it also seems like an argument against anyone using Anarki, or at least against anyone blindly committing it to production. Breakage can happen with any open source project, but given the generally slow nature of the community, even if news is likely to break, it isn't likely to break often.
Also, we already have Github to check and discuss it, and I think there is a more appropriate venue than here for those issues.
The idea behind a 'hook' is that it is a place where code can be inserted without having to modify the original code. They're pretty common in emacs, for instance, where lots of packages provide hooks for easy extension in your .emacs file.
So, if there's a function bound to that name in the hooks table, it is run with the arguments. You only call 'defhook when you want to bind to that point in the code.
This is a rather interesting implementation to me, because I'm used to the emacs-lisp concept, where a hook is actually a list of functions that all get called on the arguments, so it can be added to multiple times. The corresponding names in elisp are 'run-hooks and 'add-hook. https://www.gnu.org/software/emacs/manual/html_node/elisp/Ho...
True, Guile and Picolisp do not make self-contained binaries, but they're pretty lightweight and fast, and seem like they still might be half-decent choices. I guess I should probably remove Guile from the list of candidates, since it's really for plugin extensibility in other programs; I may even have been slightly overlapping it with Chicken in my mind.
Arc would have to be "polished" a bit to make it work, but that's what we're here for, isn't it?
Yeah, I probably wouldn't use it for that either, at least as much because of the relative instability and hackishness of arc. It was fun for a few minutes to imagine modifying it to be used that way though.
No, I was forgetting that ac is considered a compiler :) But it compiles the Arc codebase every single time Arc starts up. Maybe we should start memoizing its outputs to disk somehow, see if that makes it noticeably faster. My suspicion is egregious runtime processing like ar-denil-last and ar-apply-args will cause it to not make a difference.
Given the pervasiveness with which Arc has to make such transforms at runtime, I've gotten into the habit of thinking of it effectively as an interpreter.
I have this constant problem of dissatisfaction with existing tools and paradigms, to the point that I often work my way down to the bottom of the stack considering building a new OS...
My most recent descent has started with a dissatisfaction of container orchestration tools, then discovering that it's not really easy to manage containers with unprivileged code under Linux, and now wondering about building a microkernel OS / language for distributed actor-model development from the ground up, and thinking it would be nice to have an "OS development framework" so it is easier to experiment with novel architectures without having to deal with a lot of driver development grunt work... I originally asked this question somewhere near the beginning of all that.
Speaking of building a new OS, check out https://gitlab.com/giomasce/asmc which uses some of the bootstrappable infrastructure to build an OS. It uses a stack-based (Forth-like) intermediate language called G to implement C.
Interesting that you mention the sandboxing problem, since that's something I'm currently thinking about...
My current thought for solving the sandbox problem is to have a thoroughly first-class and recursive system for creating subsets of resources and providing them to children. Fortunately, I don't think we ever have to worry about running _unexpected_ code; just code that does unexpected things. So if we have the tools to take part of the resources available for the current "process" and pass them on to any child interpreters (tabs, etc.), then it wouldn't matter how many layers you decide to add to the system.
That said, a lot of complexity with sandboxing comes from the desire to deliberately break isolation and provide access to shared resources... I think that temptation should be avoided as much as possible, and solved with easy to use communication primitives. While that does impose an overhead, we're already quite used to isolation overheads, and it should be constant. And if the abstractions around resources are good enough, the difference shouldn't be that noticeable.
And now you know why I'm thinking about an actor-model OS, where everything (including files, etc.) are represented as isolated actors that communicate via messaging. Then it doesn't make a difference whether you "own" the 'filehandle' or a different actor does - it's just an address that you send authorized messages to anyway. Fits pretty well with microkernel and distributed OS design too.
"My current thought for solving the sandbox problem is to have a thoroughly first-class and recursive system for creating subsets of resources and providing them to children."
I hadn't been thinking about sandboxing, but my interest in testable interfaces lately has me asking how I can design an interface for memory allocation in a testable way. And that requires precisely such a recursive way for one allocator to be able to do book-keeping on memory allocated by a parent allocator. This would be the start of the capability to run the tests of an OS while running on itself.
The only way to have an address tagged as used in one allocator and free in a child allocator: metadata must be somewhere else. I'm imagining something like this, where 'the 'allocator' blocks are book-keeping metadata on used vs free areas:
Yeah; I guess I didn't state it that clearly, but the sandboxing reference was from the reddit thread you linked
"Sandboxing is challenging partly because our boundaries for what is trusted shift imperceptibly over time. Browsers came out in a world where the desktop was considered the crown jewels, while browsers were considered untrusted. Over time local disk has disappeared (e.g. Chromebooks) while our lives are ruled by the browser. This has made the original design of browser security (shielding the local file system; cookies; single origin) if not obsolete then at least dangerously incomplete. So any sandboxing solution has to think about how to fight this gradual creep. No matter how you organize your sandboxes using Qubes or whatnot, something inside a sandbox is liable to become more important than other stuff inside that sandbox. And even outside that sandbox."
The idea of running tests against a live application, particularly an OS, is intriguing. It shouldn't be too hard though, since the code and its state don't have to be in the same place. So you could run the live code against separate testing data in a different sandbox or something...
Your specific example of memory management makes me think that vague "recursive subsets of resources" probably won't cut it for a lot of things. After all, if you make memory access too indirect, it would be very very slow; but if you try to use the actual CPU management interfaces, it may not allow the kind of control that we want...
How much do you really need the bootstrappable stack? There shouldn't be anything stopping you from using something that's already been bootstrapped, instead of raw hex code. E.g. existing C tools, etc. There's also the picolisp-based PilOS project; basically picolisp (which is mostly bootstrapped from an assembler they wrote anyway) running directly on the metal as a kernel. (https://picolisp.com/wiki/?PilOS)
Oh sorry, I did understand your sandboxing reference. Just meant I wasn't concerned with it when thinking about the memory allocator. Should have said, "I haven't been thinking about sandboxing.."
I'm less concerned about overhead because I'm primarily concerned about tests, which by definition don't run in production.
But I think in the case of memory allocation you can do several levels of sandboxing before you start hitting performance bottlenecks. You'd have no extra indirections when using memory, because its just a pointer and you just have an address. You'd only hit extra levels of indirection when you try to allocate and the allocator is out of memory. Then you may need to request more from your parent, who needs more from its parent, and so on, all the way down to the OS.
"How much do you really need the bootstrappable stack?"
My basic idea is that code is easier to quickly understand if you can run it and see it in operation in a variety of tiny slices. I think it would be easy to quickly modify codebases if you had access to a curriculum of logs of the subsystems you care about, click on any line to jump to the code emitting it, and so on. The curriculum would be created automatically while running tests (because tests are selected manually to capture interesting scenarios).
This idea requires being able to drill down arbitrarily deep into the traces, because we can't anticipate what a reader is trying to change. I may be trying to investigate why a certain sequence of operations causes my laptop to lock up. That may require drilling down inside a save operation to see the data structures inside the OS that perform buffering of disk writes. And so on.
Compilers are part of this infrastructure I want to drill down into. If I want to understand how the if keyword works, it does me no good if it terminates in some opaque binary that wasn't built on my computer. It's also only mildly nicer to find out that some code in this compiler I'm reading requires understanding the whole compiler. Metacircularity is a cute trick, but it's hell on comprehension. Any sort of coiling is bad. What noobs need is to see things laid out linearly.
I've looked at PicoLisp before. I even modeled Wart's memory allocator on PicoLisp. So it's great in many ways. But like all software today it's not really intended for end-users to look at the source code. Its code is intended for insiders who have spent months and years building up an intimate understanding of its architecture. That assumption affects how it's written and managed.
That could be true; I'm not really familiar with CPU facilities for memory isolation, but this is probably one of the most solved of the isolation challenges, since it does require CPU support.
"You'd only hit extra levels of indirection when you try to allocate..."
Good point. I wonder if there's any way to improve that, or if the worst case is rare enough that it's acceptable?
"This idea requires being able to drill down arbitrarily deep into the traces..."
That's a cool idea, and fits very well with the GNU objective of fully open and transparent source code. I'm not sure that bootstrapping is what is required to achieve that goal, however. What you really need is transparent source all the way down, which can be satisfied with self-hosted code, even if it's not bootstrapped. In fact, I'd argue that multiple layers of bootstrapping would make the drill-down process very challenging, because you'd have to cross very sharp API boundaries — not just between functions or libraries, but between languages and runtime environments. Making a single debug tool that handles all of that would be impressive, let alone expecting your users to understand it.
"Metacircularity is a cute trick, but it's hell on comprehension"
Metacircularity applies to interpreters built on interpreters, not compilers. I can agree that it makes things opaque though, because it reuses the existing platform rather than fully implementing it. A self-hosted compiler, even if written in its own language, could be fully comprehensible, however (depending on the quality of the code...). It's just a program that takes syntax and produces binary.
I do think it would be really cool and possibly also useful if every compiler was self-hosted, and could be drilled into as part of the debugging process. It would mean that you could read the code, since it would be written in the language you are currently using, and the same debugger should be able to handle it.
"But like all software today it's not really intended for end-users to look at the source code"
I haven't actually looked at picolisp's source much myself, but what would it take to satisfy your desires for "end-user" readable code?
Very interesting. I'm going to read that paper as well. And think about what you said about the difference between interpreting and compiling.
> what would it take to satisfy your desires for "end-user" readable code?
See code running. My big rant is that while it's possible to read static code on a screen and imagine it running, it's a very painful process to build up that muscle. And you have to build it from scratch for every new program and problem domain. And even after you build this muscle you screw up every once in a while and fail to imagine some scenario, thereby causing regressions. So to help people understand its inner workings, code should be easy to see running. In increasing order of difficulty:
a) It should be utterly trivial to build from source. Dependencies kill you here. If you try to install library version x and the system has library version y -- boom! If you want to use a library, really the only way to absolutely guarantee it won't cause problems is to bundle it with your program, suitably isolated and namespaced so it doesn't collide with a system installation of the same library.
b) It should be obvious how to run it. Provide example commands.
c) There should be lots and lots of example runs for subsets of the codebase. Automated tests are great for this. My approach to Literate Programming also helps, by making it easy to run subsets of a program without advanced features: http://akkartik.name/post/wart-layers. https://travis-ci.org/akkartik/mu runs not just a single binary, but dozens of binaries, gradually adding features from nothing in a pseudo-autobiographical fashion, recapitulating some approximation of its git history. In this way I guarantee that my readers (both of them) can run any part of the program by building just the parts it needs. That then lets them experiment with it more rigorously without including unnecessary variables and moving parts.
I think the problem is probably a combination of lack of motivation and barriers to contribution.
On the one hand, most projects are not very well documented, advertised, supported by tests, or have communities ready to help people get started in development. So the difficulty of getting started helping is probably high.
On the other hand, most projects are the unique vision of a single developer. Everyone else probably has different ideas, and other projects they'd rather work on. It takes a certain kind of charisma, influence, productivity, etc. to get past the point where people feel like contributing to _your_ project is worth _their_ time. It would be interesting to figure out exactly what it takes to make that happen, so I could use that power for my own nefarious ends...
To perhaps start a discussion instead of just leaving a comment: where should we go next with the self-documenting / exploratory code ideas? What's missing or frustrating about the current help system?
I've recently discovered the Caddy server (https://caddyserver.com/), which makes SSL and application proxy deployments super easy. Like, 2 lines of config:
domain.com #uses the domain to automatically set up SSL with Let's Encrypt
proxy / localhost:8080 #redirect everything to Arc on 8080
I will say that Arc runs a bit resource intensive, and the slightly slow boot times mean you don't want it to have to re-launch because of infrequent requests. I don't know how well it would work on Heroku.
Also, some VPS services like vultr.com offer $5/mo nodes that have more resources than what you get from Heroku at $7/mo anyway.
That would be nice I suppose, but we can't actually modify the arclanguage forum, so it would be of limited benefit. I guess a lot of people come to arc because they want to run an HN clone though, so it might be worth pursuing.
> File upload
Yeah, I think enhancing some of the fundamental web-service functionality in arc would be good. Beyond file upload, better support for OAuth, ssl, etc. would be good too.
Good idea. Not sure how to make that better myself, but demystifying and enabling easy ecommerce would be cool.
I think that also has the same motivation for the previous category; we need better support for web application development, because that's what most people want to do.
It's just insane that Arc Forum doesn't encrypt HTTP but, well, as you write, we can't change Arc Forum. Maybe they'll update it some day, or someone will set up an unofficial one that's more up to date?
It looks quite humble but it has both a repl and an interface for saving/running various web apps.
I've set it to be enabled by default in Anarki. And it's only available for admins, because it can run system commands. I wonder if it would be possible to allow any user to develop apps securely, e.g. by disabling unsafe commands or sandboxing it somehow? Perhaps there is a simple way of doing that? It could open Arc web app development up to any user of an Anarki driven page.
I think those features mentioned above still need better documentation, too.
Other HN features that Anarki News is lacking:
* hide item
* past front pages
* show all stories from same site by clicking on domain name