🔗 There are two very different things called “package managers”

Earlier today, Ludovic Courtès asked these questions on Mastodon:

Are npm/crates.io just a different approach to distributing software, compared to Linux distros?

Or is there something more fundamental about their approach, how it affects user/developer relations, and how if affects user autonomy?

My take on these questions is that to me there are two fundamental differences:

1) npm/crates.io/LuaRocks (language package managers) are package managers for programmers. They ship primarily modules/libraries; apt/rpm/etc (system package managers) are package managers for end-users. I don’t expect a non-programmer to ever use cargo but I expect a Ubuntu user to use apt (even if via a GUI).

2) language package managers cut the “middleman” and allow module/library devs to make them available to other devs right away, without someone like a Debian developer having to deem them relevant/ready enough for packaging.

The fact that there is no curation is a feature for language package managers, just like the fact that there is curation is a feature for system package managers.

The reason why system package managers are curated is for end-user protection. The reason why language package managers are not curated is to provide developer autonomy. There is a wide gradient between these two things.

If as a developer you put too many hurdles between me and the library I want to use in the name of “protecting” me, I’m gonna skip your package manager and just fetch my dependencies from the upstream source.

For end-users just trying to keep their browser up-to-date, I understand the story is completely different.

There are other practical reasons why developers use language-specific package managers, of course. One of them is dealing with dependency versioning. By having per-project trees (think of node_modules in npm), each project can define their own dependency versions, regardless of what is available system-wide (if it is available at all). This is something that happens in practice, but that is not necessarily a fundamental design difference between language and system manager. In our 2019 paper we discussed alternatives, but the approaches we suggested there are not the general state of the world today.

An aside: when talking about projects adopting or not certain package managers, one needs to keep an eye for the motivations. I’ve seen questionable arguments from companies keeping packages away from community repos (for example, to retain control over download metrics). But in my experience, I’ve only ever seen this happening to system package managers, but never to language package managers, which just seem to reiterate how different these two universes are.

In an ideal world, system package managers and language package managers would be able to cross-reference each other to avoid the duplication of packaging work we see everywhere. This is something we talked about in the paper linked above. In fact, I’ve been trying to preach the gospel of cross-language links across ecosystems since at least… *checks git, or rather, cvs history*… 2003.

I’m as sympathetic to the ideal as they come, but the reality is that we need to deal with these two universes for now (…for a significant definition of “now”).

Amidst the lively debate in Mastodon, my friend Julio brought up the elephant in the room:

(puts Diogenes beard):
“Docker.”

…to which I can only nod my head in agreement and say: the success of Docker is the developer world’s tacit acknowledgement that package management has failed.

(For anyone else reading this tongue-in-cheek comment without further context: I have authored two package managers, both in the language side and distro side of the game. If that remark feels excessive, just s/has failed/is hard/, but my point still stands.)

Don’t get me wrong, package management has evolved greatly over the years, but as a community we’re still understanding it. There are two very different things that are both called package managers, and they serve different purposes. Another world is possible, but it will take a lot of gradual change, and “if only everyone would adopt my package manager where everything works” is not a solution (that applies to people living inside the bubble of their own language ecosystem as well!). Age has turned me from a revolutionary into a reformist.