Independence and Backends

Something that has been on my mind lately is the challenge of self-hosting. AWS, Google Cloud, Azure, all of them force you into a set of proprietary tools that lock you in. There is also Supabase or Firebase that promise you "Platform-as-a-service". They provide a tightly-integrated, hard-to-extend set of tools that lock you into their ecosystem.

What fascinates me is that these PAAS solutions already existed some twent-yish years ago, but had far less lock-in and relied on open source tooling. Some still exist today, for example there is Hetzner and Namecheap web hosting. They provide a managed Postgresql database, the ability to use any programming language you want via CGI, cron jobs and much more. They're all managed via open source tooling, so you can move away more easily if you want to.

But there is a trend that is quite clear which is that we are moving away from externally extensible systems towards locked-down "complete" systems, exemplified by the rising popularity of containers, SAAS and unikernels. Systems that are static, immutable, self-contained. I think it's sad, because IPC is a superpower, and there is some real power that comes with a malleable system that makes it easy for programs to interface. If you learn Unix, there are so many ways of stitching together programs quickly to do what you need. It gives you a lot of flexibility, and allows you to set up a workflow that more closely matches what your organization needs, all without needing to buy expensive subscriptions or learn vendor-specific tools. It is as if you learn a vocabulary, a programming language, rather than rigid "complete" systems you cannot control.

Yet, I think unfortunately the Unix ecosystem has not entirely kept up. Partly because it is out of fashion; partly because people think they will out-scale a single server; partly I think there is a fear that self-hosting is insecure. Thus, there are a few pieces in the ecosystem that are missing if you want to build a decent web application. These are instead provided by expensive SAAS solutions, but they could just be a systemd service.

So, as a result, if you want a programming language agnostic solution to secret management, or authnz, or experimentation, then you're bound to expensive heavy SAAS solutions or heavy, hard-to-host, open source solutions. These solutions just tend to do too much, and as a result, tend to assume you use a combination of docker, react, jwts, github, and more; they also only provide integration paths for those tools. It means that adopting them tends to have cascading effects on all other decisions you do, and so in one way or another, you tend to be forced into using a software vendor or technology that you didn't want to use. You wanted the banana but got the gorilla in the jungle holding the banana.

Instead, what I'd like to see is a set of very primitives that do exactly the hard part and are easy to integrate with from any language. I have been very inspired by Plan9, which provides a uniform interface via files. These primitives do not need to scale to support large businesses, but instead be there for individuals who are skilled and want to quickly build an app that they can self-host, without needing any SAAS or PAAS or whatever.

My aspiration for the next year is to build support tools that make it easier for you to host your own webapp. For example, an authnz module for a modern web server like Caddy, or an experimentation client configured via Plan9 or Unix IPC style. They should use a uniform interface and be administered via files or sockets. Many open source tools today come with docker containers, their own set of auth, user management, admin UI and more. The primitives I want to build would instead try to be good Unix-citizens.

As a starter, I built a generic job queue named File d'attente. It is entirely managed using files. I guess it is philosophical, but I hate the idea of spending my time learning how to use third-party vendor specific tools and services. I want people to be able to learn transferable skills instead, a vocabulary.

- Marc