This starts with the Beatles, but it’s not about them. It’s about us.

So, a friend tweeted earlier today the question “what is the final Beatles song“, and as usual, these topics lead to fun conversations. You see, the “end” of the Beatles is a fuzzy matter, because they recorded their last two albums in 1969 and 1970, but released them in reverse order — Abbey Road was recorded last and released first, and Let it Be was the other way around. And then, do the Anthology sessions from 1994 with Paul, George and Ringo playing over an old recording of John count? (I feel they do!)

As the conversation went on, she shared with me this little story (on everything2! they’re still around!!), which only drives its point home really deep if you’re a hardcore Beatles fan, but one element in it is that it imagines a Beatles album made from songs that the individual members released immediately post-breakup. That on itself is not very remarkable: building those imaginary what-if-they-stayed-together album setlists is a favorite pastime for Beatles fans. Invariably, the semi-obvious conclusion is that if you took the best tracks from each of the four solo albums from a given year, you’d make an album that’s better than the individual works. And as you can imagine, those playlists are widely available. Still, those always sound like compilations to my ears (and, without giving away too much, the story linked above addresses that beautifully).

To me, the fact that such fan-fictional albums sound like compilations and not like true albums has to do with the reason why I think collective works — be it in music or not — tend to be better in general than solo efforts. Because work done in a collective yields results of a different nature.

“People change like colors bleed,
As I sense a shade of you in me”
— “Color Bleed pt. 2”, from the album Color Bleed (2011)

Work done by two or more people always reflects that multitude, it’s always unlike any single person’s work. When I’m in a collective environment — and by that I mean any setting where my work is presented to and discussed by others as it is developed — even when I’m doing work completely on my own, even before I’ve had my first piece of feedback, I feel a sort of mind game playing in my head where I “play the part” of my peers and imagine what their feedback would be, be it consciously or subconsciously. I’m doing the work not only for myself, but for others too, whose opinions I care about. That affects in subtle and not-so-subtle ways the results of what I do. Even the work I do alone, when in such environment, is different and better than the work I do alone-alone.

When I recorded Color Bleed, I invited my friends who had played in a Pink Floyd cover band with me to join in. Even though I wrote all the songs — some written years prior, some written as the project took shape — all of them ended up influenced by the fact that I was playing the songs with them, and even though I tried to venture away from the cover stuff we were doing, there are clearly some floydian moments here and there, which maybe don’t sound so much like Pink Floyd to my ears, but certainly alluded to our favorite moments on stage together (and let me tell you, playing keyboards along with another keyboardist is really cool!) — by the point we were recording the original songs, the point was never to sound like the cover band, but for it to sound like us. So even if that wasn’t truly a “band effort” in the idealized sense that we imagine “four people in a garage”, it was never a lonely project: from the very beginning it was me and Coutinho, who was the other keyboard player in our band, who also owned the studio and took the role of producer/engineer, and then the other folks joined in contributing parts as we went. The end result was much better than anything I could achieve on my own, and most importantly, even the things that I did were better than if I had done them on my own.

That feeling of being “in a band” is not exclusive to music. I definitely felt it as a software developer as well. At my last job, I made the interesting observation that it was the third time in my career that I was part of a team called “Core Team”. The first one was back in college, and it was the most special one — maybe because it was the first, maybe because it was the experience that shaped the rest of my career: the Core Team for GoboLinux, my first successful open source project.

Looking back, it’s funny how it started very much like Color Bleed. Back in 2002, I was at the university and I had this idea for a crazy Linux distribution which would require recompiling the entire system from scratch. A friend joined in and we did it together over the course of a weekend. One by one, more friends joined in, switched over their systems, created bootable CDs, made kernel patches adding cool features, we were just having lots of fun tinkering with the OS. Then we were slashdotted (think “HN frontpage x10”), then we were on a magazine cover, photo shoots and all, then we were invited for internships in Silicon Valley, all the way from Brazil. I never took music seriously, but at least in tech I had the “indie band having its one-hit-wonder moment” experience. And yes, it was as cool as it sounds.

Cover story! And a cool band pic!


There we are, next to KDE and Gentoo! We’re “for real”!

The second Core Team was at a local startup, where I worked with Guilherme, who was also in GoboLinux. Since we already had the chemistry from that project, this really felt more like a side project than a new band — think Petrucci and Portnoy doing LTE away from Dream Theater (yes, I just blatantly made that comparison haha).

In between the first and second Core Teams I got my Masters degree, and between the second and third, I got my PhD. I loved it at PUC-Rio (or else I wouldn’t have gone there twice!), and made great friends each time, but it never felt like a band. In both occasions we had a research group, but each person was running their own project, with little or no overlap. Opportunities for collaboration were limited, everyone was on a different schedule, and while we created a great environment which I’ve called home for many years, it just wasn’t “a band”.

The third Core Team was at my last gig, at Kong. Again, that felt like a band — a scattered group of hackers from all over the world — China, Finland, Spain, Brazil, Peru, Canada, US — brought together because of their Lua knowledge to maintain this open source project. Each one with very different skills and backgrounds, and it was complimentary: it felt like each one of us played a different instrument. And doing creative work as part of that group felt like doing it in a band context: even when I did stuff on my own, I had it in my mind that it is was being done for that particular team to review and maintain (even if each of us would still put our personal flavor to the code). I had a great 3½ years with that team, where I learned a lot and played different instruments—I mean, roles, and then I put in practice something that I learned from the many bands I played since I was a teenager: leave on a high note. Looking back, the only regret I have is that… apparently we never took a picture of our team? (To be honest, I’m not sure we’ve ever got my last lineup of the team together in the same room — it was supposed to have happened in 2020, but then the pandemic hit. Maybe a pic of some previous lineup at least?)

Not all of my coding projects were “bands”. Even though I had tons of pull requests with contributions over the years, the process of developing htop was always a solo endeavor. I liked it this way, for a good while it was my chill-on-my-own thing away from everything and everyone. But then I drifted away from it, and free/open source projects (FOSS) projects need maintenance. From a distance, I feel like that the new team who picked it up works like a band. I’m happy for them!

Maybe it’s better if FOSS projects work more like bands than solo projects — bands often outlast their members, after all. But then sometimes you just want to pick up an acoustic guitar and do stuff on your own. There’s got to be a place for that too. Now that I think of it, I’ve never been part of a really huge FOSS project — I have a tendency of starting projects rather than joining established ones! — and I don’t know if this “band mentality” of mine has prevented any of my projects from growing (whenever I read about the structure of the Rust project, even before the Foundation, it seemed super sprawling!) but I know that a team can only feel like a team when it is about the size of a band, and I know that a team feels best when it feels like a band.

Not all teams feel like a band, and to be honest, not even all bands. But when it happens, it’s somewhat magical. It’s something that build memories that you take with you forever, and which change you in some way or another. Whenever I listen to the solo works from my favorite artists who left my favorite bands, I can always tell that the influence of their old bandmates is always obviously there, whether they want it or not, whether they’re Paul McCartney or John Lennon, David Gilmour or Roger Waters. I’m sure the influences from all the great people from my past history are there whenever I play, and whenever I code.

André Garzia made a nice blog post called “Lua, a misunderstood language” recently, and unfortunately (but perhaps unsurprisingly) a bulk of HN comments on it was about the age-old 0-based vs. 1-based indexing debate. You see, Lua uses 1-based indexing, and lots of programmers claimed this is unnatural because “every other language out there” uses 0-based indexing.

I’ll brush aside quickly the fact that this is not true — 1-based indexing has a long history, all the way from Fortran, COBOL, Pascal, Ada, Smalltalk, etc. — and I’ll grant that the vast majority of popular languages in the industry nowadays are 0-based. So, let’s avoid the popularity contest and address the claim that 0-based indexing is “inherently better”, or worse, “more natural”.

It really shows how conditioned an entire community can be when they find the statement “given a list x, the first item in x is x[1], the second item in x is x[2]” to be unnatural. :) And in fact this is a somewhat scary thought about groupthink outside of programming even!

I guess I shouldn’t be surprised by groupthink coming from HN, but it was also alarming how a bunch of the HN comments gave nearly identical responses, all linking to the same writing by Dijkstra defending 0-based indexing as inherently better, as an implicit Appeal to Authority. (Well, I did read Dijkstra’s note years ago and wasn’t particularly convinced by it — not the first time I disagree with Dijkstra, by the way — but if we’re giving it extra weight for coming from one of our field’s legends, then the list of 1-based languages above gives me a much longer list of legends who disagree — not to mention standard mathematical notation which is rooted on a much greater history.)

I think that a better thought, instead of trying to defend 1-based indexing, is to try to answer the question “why is 0-based indexing even a thing in programming languages?” — of course, nowadays the number one reason is tradition and familiarity given other popular languages, and I think even proponents of 0-based indexing would agree, in spite of the fact that most of them wouldn’t even notice that they don’t call it a number zero reason. But if the main reason for something is tradition, then it’s important to know how did the tradition start. It wasn’t with Dijkstra.

C is pointed as the popularizer of this style. C’s well-known history points to BCPL by Martin Richards as its predecessor, a language designed to be simple to write a compiler for. One of the simplifications carried over to C: array indexing and pointer offsets were mashed together.

It’s telling how, whenever people go into non-Appeal-to-Authority arguments to defend 0-based indexes (including Dijkstra himself), people start talking about offsets. That’s because offsets are naturally 0-based, being a relative measurement: here + 0 = here; here + 1 meter = 1 meter away from here, and so on. Just like numeric indexes are identifiers for elements of an ordered object, and thus use the 1-based ordinal numbers: the first card in the deck, the second in the deck, etc.

BCPL, back in 1967, made a shortcut and made it so that p[i] (an index) was equal to p + i an offset. C inherited that. And nowadays, all arguments that say that indexes should be 0-based are actually arguments that offsets are 0-based, indexes are offsets, therefore indexes should be 0-based. That’s a circular argument. Even Dijkstra’s argument also starts with the calculation of differences, i.e., doing “pointer arithmetic” (offsets), not indexing.

Nowadays, people just repeat these arguments over and over, because “C won”, and now that tiny compiler-writing shortcut from the 1960s appears in Java, C#, Python, Perl, PHP, JavaScript and so on, even though none of these languages even have pointer arithmetic.

What’s funny to think about is that if instead C had not done that and used 1-based indexing, people today would certainly be claiming how C is superior for providing both 1-based indexing with p[i] and 0-based pointer offsets with p + i. I can easily visualize how people would argue that was the best design because there are always scenarios where one leads to more natural expressions than the other (similar to having both x++ and ++x), and how newcomers getting them mixed up were clearly not suited for the subtleties of low-level programming in C, and should be instead using simpler languages with garbage collection and without 0-based pointer arithmetic.

What the principles that underlie the software you make?

This short story is not really about htop, or about the feature request that I’ll use as an illustration, but about what are the core principles that drive the development of a bit of software, down to every last detail. And by “core principles” I really mean it.

When we develop software, we have to make a million decisions. We’re often driven by some unspoken general principles, ranging from our personal aesthetics on the external visuals, to our sense of what makes a good UX in the product behavior, to things such as “where does bloat cross the line” in the engineering internals. In FOSS, we’re often wearing all these hats at the same time.

We don’t always have it clear in our mind what drives those principles, we often “just know”. There’s no better opportunity to assess those principles than when user feedback asks for a change in the behavior. And there’s no better way to explain to yourself why the change “feels wrong” than to put those principles in writing.

Today was one such opportunity.

I was peeking at the htop issue tracker as an end-user, which is a refreshing experience, having recently retired from this FOSS project I started. I spotted a feature request, asking for a change to make it hide threads by default.

The rationale was sensible:

People casually using htop usually have no idea what userland threads are for.
People who actually need to see them can easily enable them via SHIFT+H.

With them currently enabled by default, it is very inconvenient to go through the list and see what is running, taking up RAM, CPU usage and whatnot, therefore I think it’d be more user-friendly to not show them by default.

He proceeded to show two screenshots: one with the default behavior, full of threads (in green) mixed with the processes, and another with threads disabled.

When one of the current developers said that it’s easier for the user to figure out how to hide things than for them to discover that something hidden can be shown, the counter-argument was also sensible:

Htop can also show Disk IO, which can be arguably very useful, but is hidden by default.

At that point, I decided to put my “original author” hat on to explain what was the intent behind the existing behavior. Here’s what I wrote:

Hi @C0rn3j, I thought I’d drop by and give a bit of historical background as to what was my motivation for showing threads by default.

People casually using htop usually have no idea what userland threads are for.

Yes! I fully sympathize with this sentiment. And the choice for enabling threads and painting them green was deliberate.

You have no idea how many times I was asked “hey, why are some processes green?” over these 15+ years — and no, it wasn’t annoying: each of these times it was an opportunity to teach someone about threads!

(credit: xkcd)

htop was designed to provide a view to what’s going on in the system. If a process is spawning threads like crazy, or if all your cores are overwhelmed because multiple threads of a process are doing work, I think it’s fair to show it to the user, even if they don’t know a thing about threads — or I would say, especially if they don’t know a thing about threads, otherwise these things would be happening and they wouldn’t even know where to look.

Htop can also show Disk IO, which can be arguably very useful, but is hidden by default.

One of my last projects when I was still active in htop development was to add tabs to the interface, so that the user would have a more discoverable path for navigating through these additional columns:

This code is in the next branch of the old repo https://github.com/hishamhm/htop/ — I think the code for tabs is actually finished (though not super tested); it’s pretty nice, you can click on them or cycle with the Tab key, but the Perf Counters support which I was working on, never really got stable to a point of my liking, and that’s when I got busy with other things and drifted away from development — turns out I enjoy writing interface code a lot more than the systems monitoring part!

Anyway, I think that also illustrates the pattern that went into the design: I implemented the entire tabs feature because I wanted to make IO and Perf Counters more discoverable. I wanted to put that information in front of users faces, especially for users who don’t know about these things! I considered the fact that I had implemented the entire functionality of iotop inside htop but people didn’t know about it to be a personal failure, and a practical learning experience: adding systems functionality is useless if the UI design doesn’t get it to users’ hands. (And even I didn’t use the IO features because there was no convenient way of using them.)

I never wanted to implement a tool for “super advanced Linux power users who know what they doing”, otherwise I would have never spent a full line at the bottom showing F-keys, and I would have spent my time implementing Vim bindings (ugh ;) ) instead of mouse support. I’ve always made a point that every setting can be settable via the Setup screen, which you can access via F2-Setup (shown at the bottom line!) and which you can control with the keyboard or mouse, no “edit this config file to use this advanced feature for the initiated only” or even “read the man page” (in fact it only has a man page because Debian developers contributed it!).

I wanted to make a tool that has the power but doesn’t hide it from users, and instead invites users into becoming more powerful. A tool that reaches out its hand and guides them along the way, helping users to educate themselves about what’s happening on the internals of their system, and in this way have more control of it. This is how you hand power to users, not by erecting barriers of initiation or by giving them the bare minimum they need. I reject this dicothomy between “complicated tools for power users” and “stripped-down tools for mere mortals”, the latter being a design paradigm made popular by certain companies and unfortunately copied by so many OSS GUI projects, without realizing what the goals of that paradigm really were, but that’s another rant.

And that’s why threads are enabled by default, and colored in green!

(PS: and to put the point to the proof, I must say that the tabs feature was a much bigger code change than Perf Counters themselves; it included some quite big internal refactors (there’s no “toolkit”, everything is drawn “by hand” by htop) with unfortunately might make it difficult to ressurect that code (or not! who knows?), and of course tabs are user-definable and user-editable!)

The user who proposed the change in the defaults thanked me for the history tidbits and closed the feature request. But in some sense writing this down was probably more enlightening to me than it was for them.

When I started writing htop, I did not set out to create “an instrument for user empowerment” or any such lofty goal. (All I wanted was to have a top that scrolled and was more forgiving with mistypes than circa-2005 top!) But as I proceeded to write the software, every small decision had to come from somewhere, even if done without much deliberate thought, even if at the time it was just “it felt right”. That’s how your principles start to percolate into the project.

Over time, the picture I described in that reply above became clear to me. It helped me build practical guidelines, such as “every setting must be UI-accessible”, it helped me prioritize or even reject feature requests based on how much they aligned to those principles, and it helped me build a sense of purpose for the software I was working on.

If you don’t have it clear to yourself what are the principles that are foundational to the software you’re building, I recommend you to give this exercise a try — try to explain why the things in the software are the way they are. There are always principles, even if they are not conscious to you at the moment.

(PS: It would be awesome if some enterprising soul would dig down the tab support code and ressurrect it for htop 3! I don’t plan to do so myself any time soon, but all the necessary bits and pieces are there!)

I was interviewed at the Changelog podcast about the surprising story of the maintainership transition in htop:

The Changelog 413: How open source saved htop – Listen on Changelog.com

I am not going to talk directly about the US protests. Instead, I will briefly note the role of the State in them, both as cause—promoting institutional racism—and as a continuing instigator. But aren’t the protests for justice, which supposedly needs to come from the State? But the government is clearly not interested in justice. So we have on one side the people, on the other side the State. But it didn’t have to be this way by definition: it’s _this_ particular state that’s the problem. And this makes me think of Peter Thiel.

Peter Thiel has said, in no uncertain terms, that he does not believe in democracy and ultimately wants the destruction of any form of State. That’s what he went up on stage in the elections for. That’s what the alt-right stands for. This is not a crisis, this is an ongoing plan. Historians will look at this as a multi-decade process, with the early 1980s under Reagan and Thatcher as the first inflection point, and the last few years with the alt-right, Trump and Brexit as the second one.

The first stage, neoliberalism, was about crippling the state in order to declare it inefficient and privatize it. In the 90s in particular this was sped up in practice and played down in discourse, but early on this was the stated goal.

Now at the second stage, it is no longer about crippling government institutions. It is about crippling the concept of government itself: get the incompetent to power, so that the supposed flaws of the democratic model become evident. Then offer something else.

Of course, the trickery there is that invariably the incompetent were led to power in various places by apparently legal but effectively criminal means: mass propaganda done in breach of campaign funding laws, voter supression, buying congress.

So now we’re at the stage where there’s a useful minority of radicalized fascists ensuring we get the worst possible government, and a mass of average people whose heroes are billionaires, ranging from your monopolist-turned-philantropist to your tweeting-techbro bigot. It may seem contradictory that the forces that are ultimately destroying the notion of nation-state ostensibly employ the discourse of nationalism. But if you pay attention, they are priming their base on adapting to the upcoming state of things.

This pandemic is the first time in history when we see a national crisis being addressed by the government by giving a press release and giving names of _companies_ who are going to be doing this and that to deal with the issue. This was very, very startling to see.

The notion that is up for companies and not the state to organize society is being normalized. America is at the forefront of this process, and has been for a long time. Other places still have things like functioning health and educational systems, but the pressure exists. In that sense, the landmark space launch — once a matter of national pride, now privatized into another triumph of business over country — led by Thiel’s associate Musk, is not at all disconnected from the ongoing events. They are deeply, closely connected.

Thiel’s company Palantir — named after the all-seeing eye in LOTR, another techbro favorite — provides machinery for mass surveillance in the the US. The surveillance people worldwide are under is already managed by a private company.

When you think of Russia and its oligarchs, or the growing number of billionaires in China’s parliament — why lobby the middleman, just buy a Congress seat for yourself — we see that the increasingly direct control of superpowers by businessmen is not an American-only phenomenon. It might be that the façade of a state will stick around for long, as a useful device as people cling to their flags and anthems, but I feel the foundations of the modern nation-state slowly crumble into dust under my feet, and I do not like what’s taking its place.

Zygmunt Bauman saw it back in the 1990s: in a world of companies, we’re no longer citizens, only consumers. And the rippling effects of this are much worse than they initially sound, but this is a conversation for another time.