Filesystems are Composable Interfaces

As a programmer, I have had numerous epiphanies that seem so obvious in hindsight. One such realisation came as I was exploring user-space file systems. Any system that exposes simple file formats allows for a large variety of preexisting programs to interface with it. If you expose your data as CSVs then Excel, Google Sheets, Numbers, and Libreoffice are able to open and make all kinds of modifications, calculations, etc.

Arriving at this basic premise doesn't require too much effort, but what is less explored and, perhaps, less obvious is that it is possible expose different "views" of data so that different programs can understand and interact with the data. I've been investigating this with my new program caldavfs (still WIP). I started writing caldavfs because I wanted a way to sync my custom notes between different devices and be able to view and open these notes easily on my phone. Since iCalendar supports VJOURNAL, I thought it'd be cool to expose files that I can write in my regular text editor, and have those be saved afterward as VJOURNAL files that I sync with my caldav server.

Typically an iCalendar file has the following format:

  BEGIN:VJOURNAL
  UID:19970901T130000Z-123405@host.com
  DTSTAMP:19970901T1300Z
  DTSTART;VALUE=DATE:19970317
  SUMMARY:Staff meeting minutes
  DESCRIPTION:1. Staff meeting: Participants include Joe\, Lisa
    and Bob. Aurora project plans were reviewed. There is currently
    no budget reserves for this project. Lisa will escalate to
    management. Next meeting on Tuesday.\n
    2. Telephone Conference: ABC Corp. sales representative called
    to discuss new printer. Promised to get us a demo by Friday.\n
    3. Henry Miller (Handsoff Insurance): Car was totaled by tree.
    Is looking into a loaner car. 654-2323 (tel).
  END:VJOURNAL

And so what caldavfs essentially does is take those files and build a view of them that is easier to edit and write, allowing you to use your tool of choice when engaging with the files.

What I realize, then, is that files are essentially a protocol, almost like REST, that expose an interface that contains methods, such as: read, write, stat, readdir, etc. Applications that work with files call those methods to perform operations. This is very similar to a message-passing system with late-binding similar to Smalltalk. While drafting out this entry, I did a quick search and found that, naturally, I am not the first one to have made this observation.

The paper also highlights another challenge that I perceived: when you try to fit a file-like interface everywhere, cases arise where the abstraction breaks and become confusing. For example, what does it mean to cp -r a process tree? Does it create a snapshot? What operations are supported?

It's a challenge I faced with caldavfs. It works great as I am able to create notes using any editor I want, and I can use rm, cp etc., to make modifications. At the same time, files do not necessarily map cleanly onto iCalendar format. For example: directories don't exist in ical, summaries are not unique but file names are. Then, there are a lot of cool features that are tougher to represent with files: note linking, categories, event types, etc. How should those be represented? It is not obvious.

This seems to be the Achilles' heel of composable systems, these abstractions become loose and open to interpretation. Files are great, but maybe not the best foundation. Maybe abstractions from Functional programming would be better? Maybe lenses should be the universal interface?

- Marc