File Systems: The Original Hypermedia

44 points
1/21/1970
2 days ago
by pilgrim0

Comments


recursivedoubts

> if we always had hypermedia with directories and files, why hasn't the web evolved into a mesh of interconnected file systems?

It kind of has. URLs have the notion of paths, which are obviously strongly associated with the notion of file system hierarchies. People sometimes (sometimes accidentally) put their file systems directly on the web, see DirectoryIndex in apache for example.

> Why isn't a website just a remote directory on someone's computer that we can explore via a file browser?

Well now we are getting into the meat of it. To be a hypermedia requires the presence of hypermedia controls in a media. Hypermedia controls can be as simple as links, but the web introduced more sophisticated controls such as forms, and allowed HTML authors to specify more significant interactions beyond click-to-link.

IMO the uniform interface is the most interesting aspect of hypermedia, and that really emerged post Web. I like the authors concept of a file explorer enhanced with hypermedia ideas though and would be interested to see more details on it.

5 hours ago

moritz

> You would create and manage content directly from the file explorer application, in the most natural way possible. This version of the web wouldn’t require users to learn advanced computer skills in order to participate.

My students at university (Gen Z) have no concept of the “file system”.

6 hours ago

pilgrim0

Inline, ordered multimedia is the backbone of all consumer information systems. So your students have internalized the the archetypal equivalent of file systems through a different vocabulary, such as tweet (for files) and threads (for directories)

4 hours ago

dialup_sounds

That's not even a generational thing. People have been (e.g.) saving everything to the desktop for as long as there have been desktops. "Managing files" has always been a subsidiary task to the things people wanted to use computers for.

3 hours ago

esafak

Do they not save files on their personal computers (phones, laptops)?

3 hours ago

TheNewsIsHere

A lot of folks in that demographic are what you might call “cloud natives”. Their hard drives are used for storing the software that connects them to Google Drive or OneDrive or what have you.

We grew up in a time when understanding file systems in terms of “a system for organizing your files” was not optional. Gen Z has grown up in a time when their data was a Google Drive search away from their fingers.

an hour ago

andyferris

I found it took a while to get to the point.

In the end, I actually agree with this! I have also been thinking about filesystems, which are trees of of dictionaries (directories) and blobs (files), and that there are many other examples of tree-like data. Data structures in our programs are tree shaped, perhaps with references/pointers to other parts of the tree. JSON is a tree of dictionaries (objects), arrays, and data (the primitive string/number/null). The arrays are ordered, much like the content inside a HTML/XML block.

I agree that adding an "array" style of directory to our file systems would be really cool. I've been toying with the idea of writing a FUSE driver that holds some structured data (possibly including arrays) and just converts the (integer) index into a string. The idea is that you could e.g. view and edit some JSON tree with the file explorer. And not just JSON - basically any piece of structured data that we have in our programs can be "viewed" as a FS this way (e.g. just convert structs into directories of fields). It could even be a pretty cool and universal debugger - a breakpoint could make the program pause and serve a FUSE driver and later continue when it is unmounted :)

And yes, exactly what follows from this is some program could be written to open some "directory" and render a "document" based on the contents. The filesystem supports links, so we have the "web" like experience.

The "document" angle does require adding a kind of directory with ordered array semantics rather than dictionary/map semantics. It's the first missing ingredient listed in the article. Though some filesystems use sorted dictionaries (b-trees or whatever) for directory maps so you could maybe hack this ordered semantics in that way.

The second missing ingredient listed in the article is the hypermedia part. I mean my computer is actually OK at inferring if a file is a movie or photo or text document, so we kind of have a way of dealing with that, too. The "blob of bytes is a narrow waste" thing is quite powerful. That said, sum types could be useful to demark different kinds of "stuff", and there's no reason an implementation of this idea couldn't support sum types as part of its data model.

2 hours ago

pilgrim0

> I found it took a while to get to the point.

You're just a great understandeour

> I agree that adding an "array" style of directory to our file systems would be really cool.

I think this is sort of a low-hanging fruit people have slept on. We've proved list-based systems are extremely versatile for data structures (s-expr) and programming (lisps). What about for media in general? Everywhere I look I just see lists, with very minor stylistic distinctions between them. Of course there're abysmal infrastructural differences between chats, feeds and what not, but it does not invalidate a universal list-based frontend, similar to what you developed in your comment.

2 hours ago

andyferris

True.

I didn't really think about (linked) lists much, only (flat) arrays, but maybe that's possible already with files laid out like:

/head /next/head /next/next/head ... etc

or whatever.

I'm not saying its _ergonomic_, mind you. I'd like my file viewer to lay out the list flat, for example. There might be technical limitains with the length of the file path. Etc.

2 hours ago

syberant

I'd argue that this concept for media is most commonly known as "playlists" and unfortunately only used within data silos, e.g. a playlist of YouTube videos, a Spotify playlist, a TV season of episodes, a series of episodes, a trilogy, etc. (Yes, I'd argue that your music library of mp3 files is also kind of a silo, although portable) Heck, even a slideshow is arguably a playlist.

I agree that putting a playlist-like concept into, say, the filesystem would be an extremely interesting idea but I think a big danger is running into the same problem as hardlinks and symlinks. This problem is that if a file is "present" in multiple places (or playlists) deleting/modifying/moving it can have unforeseen consequences and it's hard to reason about (and if you copy the file now you get to invent a way to track different versions too!). I think this is also holding tagging filesystems back.

I'm currently writing a non-hierarchical FUSE filesystem and have been thinking about this list-directory concept but I'm still not completely sure how it would work, especially since I need to remain backwards compatible with the POSIX interfaces. Will probably have to just try it out (xattrs to the rescue?) and see what sticks I suppose...

A linkdump of interesting somewhat related stuff:

- https://newsletter.squishy.computer/p/knowledge-structures

- https://newsletter.squishy.computer/p/all-you-need-is-links

- https://thesephist.com/posts/search-vs-nav/

- https://karl-voit.at/2017/02/10/evolution-of-systems/ Especially "Information-Centric Systems"

- https://www.theatlantic.com/magazine/archive/1945/07/as-we-m... A classic 1945 article cited as inspiration by Ted Nelson, Doug Engelbart and Tim Berners-Lee.

- https://en.wikipedia.org/wiki/Content-addressable_storage

- https://www.nayuki.io/page/designing-better-file-organizatio...

21 minutes ago

andyferris

I should probably have mentioned that there already exists FUSE-based JSON FS mounters, for example this one is 13 years old:

https://github.com/calebcase/jsonfs

2 hours ago

xnx

A usable equivalent of the file system is sorely missing from the web. Every email address should come with a place to publically share files. It could be as easy as https://user@emaildomain.com/

5 hours ago

Borg3

O really? It seems people forgot about userdir.path (or similar). So user can expose whatever he wants via: https://emaildomain.com/~user/

4 hours ago

xnx

Exactly. This convention is virtually non existent on the web in 2025.

3 hours ago

o11c

Same-origin security problems kind of killed `https://host.tld/~user` in favor of `https://user.host.tld`.

an hour ago

mongol

That is an http basic auth URL.

5 hours ago

p_ing

I struggle to parse this with every paragraph surrounded by a border. It feels unnatural and extremely distracting.

To the author's take of using a file system as an interconnected 'web', we have networked file systems today, typically clustered though.

We've also had the _concept_ of a media-rich file system, like WinFS [as an overlay to NTFS], which was dead before it was alive due to the WWW.

Networking file systems is _complex_. All vendors would need to agree to a common export model on top of their preferred file systems. Or users would need a specialized partition/overlay developed just for this purpose.

FSes are great, but they're not fit for WWW. Without a control plane, they lack any tooling that makes the WWW better -- redirection, access control, programmatic execution of content (ASP.NET, PHP, CGI ...), etc.

Ultimately this would be a complex solution. Just like many don't simply "open up" their web server to any and all traffic to any and all content, a file system would need to be carefully partitioned the same.

The time for file systems as the vehicle for WWW content is long since past. We have better ways to do things, better caching mechanisms, better performance [through CDNs], better security mechanisms, and so on.

...not to mention, I certainly don't want to open my personal computer's file system up to the Internet.

There would have to be a big leap in evolution of file systems across all major operating systems for the author's dream to come true. I would certainly be excited to see it, but we're talking about allocating _talented_ developers to create a new file system and certainly an open source file system. Like many file systems, it would take years to become a trusted file system to host any content of value.

In the mean time, the author can always investigate WebDAV. Slower than dog shit, but it's available with every major web server.

2 days ago

pilgrim0

Sorry for the poor experience with the current design, still experimenting.

I cannot disagree with you, you’re on point on everything when considering the file system as an OS component.

But if we entertain the thought of file system as a document model, or as a transactional data structure, it should come naturally that we can piggyback on the modern infrastructure, at the application level, to achieve the desired qualities.

This very website is an experiment on how this could be done. The main takeaway with my research is that we have much to gain if we leave presentation and layout concerns out of hypermedia documents, letting the client software decide on it, like we do with our editors and IDEs, choosing the theme and font we like, the information is the same no matter. To abandon the fetishism inherited from print media and to transact pure data is to make the web democratic. That’s precisely the recipe used by all social networks: standardized, systemic presentation of schematic payloads following a given ontological model. We need only to copy them with an open model

4 hours ago

p_ing

> Sorry for the poor experience with the current design, still experimenting.

Never stop experimenting.

> file system as a document model, or as a transactional data structure, it should come naturally that we can piggyback on the modern infrastructure, at the application level, to achieve the desired qualities.

I'm having a little difficulty groking what you're trying to say, here. Can you explain it like I'm stupid? My mind immediately bounced to SQL storing binary data or a document DB of some sort.

> if we leave presentation and layout concerns out of hypermedia documents, letting the client software decide on it

Every browser engine does this today. Every engine developer has their own idea of how a standard should be implemented. Granted, this isn't end-user choice, which is probably what you're after. Well, sorta, you have some customization of fonts and [link] colors.

> To abandon the fetishism inherited from print media and to transact pure data is to make the web democratic.

It sounds like you would like some simple scheme that provided you the text and binary data (images) of the website which allowed you to manipulate them into the "newspaper" layout of your choice. Am I getting that correct?

4 hours ago

pilgrim0

You cannot fully leave presentation concerns out of browsers. Layout and decoration is an integral part of authoring, it’s supported at the markup level. HTML and friends are practically rendering instructions at this point, as if a PostScript for screens. Worse, this has a double edge, for instance, not always what should be a single paragraph of text comes as a single <p> tag, because the mere availability of styling leads people to author ad-hoc structures, since they know it could always be visually reconciled.

What I’m proposing with the research is a data structure transmitted as text. This text payload is the equivalent of a file system volume or tree. Each line in this payload is the equivalent of a file. It’s only a semiotic analogy to file systems.

Given this is such a trivial structure, it’s easy to develop multiple rendering approaches that maintain an isomorphism between the text representation and graphical representation. It does not transmit layout information, the syntax itself is the higher-order layout.

You might wanna checkout the source of this article by appending index.fifo.txt to its URL.

3 hours ago

pwg

> Sorry for the poor experience with the current design, still experimenting.

It also is simply a blank page when browsed with UBlockOrigin blocking all the java script from executing.

3 hours ago

pilgrim0

I was expecting this. Sorry, it can’t be helped with the current architecture. This site is a proof of concept using client side rendering. But it’s only a handmade, ~20kb vanilla parser and renderer for the experimental document format. Sorry for the inconvenience

3 hours ago

mickael-kerjean

> WebDAV. Slower than dog shit, but it's available with every major web server.

You lost me there. WebDAV is nothing more than HTTP calls with some XML data with a slightly different syntax than the S3 API. There is no fundamental stuff in the spec that command the protocol to be "slower than dog shit" as a file transfer protocol. Please prove me wrong with another argument than: "the particular server implementation I tried was dog shit"

3 hours ago

groby_b

> This version of the web wouldn’t require users to learn advanced computer skills in order to participate.

The web doesn't require "advanced computer skills". (Unless you use non-flexbox CSS alignments ;) It is fairly trivial to create basic HTML files. SSG + MD have removed a lot of the remaining obstacles. Most web sites are structured files, just with a "compiler" and possibly a database to store the files.

But what they still do require is the ability to reason about structured data and its best configuration. And that is the truly hard problem, ever since Ted Nelson first talked about it.

It also requires us to reason about how to best make that data consumable for humans. It doesn't just magically "arise from the structure", as much as I wish it did. The web site is a clear example - the lack of understanding how humans consume info, and what helps/hinders, leads to odd boxes around each paragraph.

I still agree with the fundamental idea. The more structure we can encode in an easily graspable way, the easier it becomes to impose structure.

But even then, the fundamental advantage of the web over hierarchical file systems is the non-linearity. And yes, correct, hierarchies matter, but the fundamental point the article misses is that there isn't just _one_ hierarchy. Wikipedia is a great example here - it fundamentally cannot be expressed in a meaningful way as a tree, even though it has many hierarchies.

And hierarchies alone are insufficient. We've now learned, thoroughly I think, that hierarchical taxonomies always break down. If we're given to snark, Linnaeus took a good stab, he failed. In more practical terms, the emergence of "tags" has shown that we need a way to have non-hierarchical cross-cutting data.

I think for a discussion of the subject, there's value in separating a few topics:

* Presentation. The author is right, HTML made a grave mistake including that

* Local representation. Again, agreement here, giving a file system structure that allows to infer meaning for later presentation is super helpful. (See point about SSG/MD)

* Organization/Navigation: Any sufficiently complex set of data requires several separate overlaid structures to help humans navigate.

* Human psychology: We're bad thinking about relation schemes beyond trees & grids. That means our organization schemes need to mirror them at least partially so we don't break our head. Corollary is that any sufficiently complex set of data needs searchability.

There's probably more. It's a topic that's been brewing in my head for a while, you're getting a very rough first draft, sorry :)

3 hours ago

pilgrim0

> It is fairly trivial to create basic HTML files. SSG + MD have removed a lot of the remaining obstacles

These are advanced computer skills IMO

> It doesn't just magically "arise from the structure", as much as I wish it did. The web site is a clear example - the lack of understanding how humans consume info, and what helps/hinders, leads to odd boxes around each paragraph

I appreciate the criticism. It's impossible to please everyone in terms of design, and I think your antipathy towards this particular style agrees with the general premises.

Regarding if disposition arising from structure is desirable or not, I think it's a matter of culture and habit. The time and complexity savings for authoring and publishing afforded by this model, for me, satisfactorily offsets whatever could be said that it misses in the aesthetic or funcional department, which can always be patched and improved. The positive feedback I had from interested users, all of them tech-illiterate, is what gave me the confidence to pursue investing in the research, and also made me realize that my insecurities towards its acceptability, which stemmed from sentiments quite similar to what you put forth as criticism, were mere whims. As far as experience and perceptions can be trusted, I believe serial multimedia has been proved as a viable format.

> And hierarchies alone are insufficient.

Not disputing that. The fact the document model is hierarchical does not mean the document system has to be. In fact it was never planned to be. There are many mechanisms in place affording hypernavigation, down to the design of the in-memory representation. Just haven't been implemented for lack of resources.

> you're getting a very rough first draft, sorry :)

I'd love to hear more. Feel free to ping me anytime if this is a subject you find exciting to discuss!

2 hours ago

groby_b

> These are advanced computer skills IMO

To set up? Yes. Absolutely. But that's equivalent to asking users to install a filesystem before using the machine :) Past installation, it's "write text. It shall appear"

Nobody needs to hand-edit HTML any more. It has become, for better or worse, the assembly language of information design ;)

> The time and complexity savings for authoring and publishing afforded by this model, for me, satisfactorily offsets whatever could be said that it misses in the aesthetic or funcional department

I'm curious. Are you saying the boxes are helping you author/publish (in which case, please say more!), or am I wildly misunderstanding?

> As far as experience and perceptions can be trusted, I believe serial multimedia has been proved as a viable format.

Absolutely. But it doesn't create meaning by itself. It's merely a well-understood and simple way of organizing information. (I should've added "linear" to "tree" and "grid")

And just having a pre-defined structure doesn't give meaning in general. You'll need to conform to it, and you need to deal with the parts that just won't conform. (soft/hard links exist to satisfy a need, if we want to go back to file systems)

> I'd love to hear more. Feel free to ping me anytime if this is a subject you find exciting to discuss!

I just might take you up on that ;) The topic's near and dear to my heart. (Alas, it is not my main occupation, so... feel free to ping as well. I might fall off the face of the earth from time to time ;)

2 hours ago

pilgrim0

> Nobody needs to hand-edit HTML any more

Doesn't that prove it absolutely failed? I mean, you have to abstract it so much that, like you said, it's a compiler target at this point! That's precisely my reasoning when I said about needing a second system in the article. And, mind you, not only a second system, most of the time also a third or more! Might as well write a game engine with the budget needed for an HTML WYSIWYG editor.

> I'm curious. Are you saying the boxes are helping you author/publish (in which case, please say more!), or am I wildly misunderstanding?

Not the boxes, per se. I'm refering to the general approach of having independent elements that can be composed with no rigour or planning. This UI came out this way cause I ported the code for the data structure editor, which, for accessibility reasons, had a full-blown cursor around each element, that's why it retained the borders. It's obious that there should be a read mode with less clutter. But honestly, It does not worry me too much, for a proof of concept it was not worth to sweat every detail.

> And just having a pre-defined structure doesn't give meaning in general. You'll need to conform to it

Beautifully said. It's for precisely this fact (if you accept Peirce semiotics) that it becomes a cultural factor. You are compelled to think the way the medium demands you to think. And this is completely dystopian but also absolutely real, and serious! That's why design decisions in authoring interfaces are undeniably political. The abysm indeed looks back. One of the main reasons I started this research was to find a way to express myself in a more forgiving way, that better reflected my thought structure: loose. I can't bear the typewriter model, it drains all my energy and creativity, it smells of formalism and grandeur. Now, when I chat with people, my mind is free, I type at the speed of thought. So that influenced the whole design. I trust you can see it.

an hour ago