~2017.3.1 Update

~sorreg-namtyv + ~ravmel-ropdyl

Curtis (~sorreg-namtyv) has reached code-complete (compiling but not tested) on the arvo.hoon rewrite. The super-impatient can have a look at this file.

The entire boot sequence from Nock through the Arvo kernel has now been rewritten, and actually makes sense. The old Arvo did work, it's true, under most conditions. It didn't work because it made sense, but mainly through pure persistence and luck. This is not viable for the deep core of a stable platform. It has, however, managed to stay alive on the network for the past 7 months.

This all-new Arvo kernel (at a very fluffy 1065 lines) joins the all-new %ames and %jael vanes (kernel modules), and the %zuse library, in the new family of reference-quality Hoon code. (%ames is almost there -- the code is good, but not documented.)

Roughly speaking, Urbit including all application, library, and kernel code, is about 40,000 lines of Hoon, in three grades, categorized by how clean and easy to read they are.

C-grade Hoon is found in %eyre (Web server) and %dill (console); a bit of it has accumulated in :talk; the markdown parser; some of the examples; some of hoon.hoon; etc.

C-grade Hoon is write-only code. To even look at it, even if you know Hoon, is reverse engineering. It follows no conventions, not even its own. If it has comments, they mean nothing, except (possibly) to the author. (One bad property of Hoon is that, like C, it does nothing to stop you from writing obfuscated code.) Most C-grade Hoon dates to 2014 or early '15.

B-grade Hoon can be followed with some effort. It is structurally simple, but not always conceptually simple. It makes no particular effort to explain itself, either through documentation or otherwise. %behn, %clay (B+ for decent comments), %ford, %gall, most of hoon.hoon and most userspace code is B-grade. Most B-grade code is from late '15 or early '16.

The A-grade Hoon is everything produced since our crowdsale: %ames, %zuse, %jael, and %parv. Hoon of this quality, you just spread on a cracker and eat. It's completely smooth. You can almost see what it's doing even if you don't know Hoon. The only places where A-grade Hoon is hard to understand are places where the problem is actually tricky.

Urbit succeeds exclusively through software quality. All A-grade code should be permanently maintainable at consistent quality. All B-grade and C-grade code will deteriorate over time and eventually needs to be replaced.

But cc-release is an "infrastructure MVP" in which the critical parts are A-grade: the boot sequence and OS, networking, and secrets. Lower-grade code remains suitable for light-duty kernel and userspace work. The good news is that the longer we wait to rewrite it, the better the replacement will be.

These new A-grade subsystems remain untested (except %jael and %zuse) and have not been integrated. This will involve further tinkering with other vanes and userspace code. However, Arvo should be the last major module rewrite before cc-release, so we're on our way.

Anton (~pittyp-pittyp) and Galen (~ravmel-ropdyl) are working from the opposite end. They're designing the new, easy-to-use, command-completing Urbit UI from the top down, using ES6 and React.

This new framework isn't even connected to Urbit. Its first use is actually a demo video. We published a few screenshots from our mockups in the last blog post, and want to give a more complete impression of where we're headed to the public. Making this interface real will, of course, require nontrivial console engineering on the Urbit side. We think it's going to be great.

Like our new server-side code, the new Urbit client-side framework is designed to be high-quality code that other human beings can read and extend.

Raymond (~maldeb-hapben) has finished breaking the new vere system into client, manager and worker processes. Curtis earlier rewrote the event loop and the logging system, with correct use of concurrency and fsync() in an actual two-phase commit.

There's still a large number of small tasks to perform, but we're essentially turning vere into the C equivalent of A-grade Hoon: clean code that just obviously works. Also, the new vere works like an actual, industrial-grade server, and should be reasonably adapted to the task of hosting nontrivial numbers of urbits.

The absolutely crucial requirement for cc-release is ships that don't sink. We have not observed any problems with the new event log so far, but we still need to stress-test it with powercycles. Yes, Urbit really is an ACID database!

Keaton (~livweb-havweb), our intern, has written a bunch of examples and cleaned up a bunch of doc.

Elliot (~ponnys-podfer), an outside contributor, sent an amazing doccords patch that adds documentation strings to Hoon. This sounds like a superficial change, but it actually involved deep work in the heart of the type system. He's also added a bunch of this doc himself.

You can check out the pull requests here and here. The original fora discussion is here.

As usual, comments are on fora. Check out this thread.