minecraft-fs is a filesystem for Minecraft; that is, a virtual FUSE filesystem that allows querying and interaction with the game through reading and writing files. It was inspired by TabFS that does similar for interacting with a browser.

It started as a silly but fascinating distraction from other projects, but became one that I realised I could actually get to a somewhat finished state in a reasonable time frame.

The filesystem

The full filesystem structure can be found in the project README, but for this post we'll deal with a cut-down version for simplicity:

├── player
│   ├── entity -> world/entities/by-id/135
│   └── world -> ../worlds/overworld
└── worlds
    ├── overworld
    │   ├── entities
    │   │   └── by-id
    │   │       ├── 135
    │   │       │   ├── position
    │   │       │   └── type
    │   │       ├── 140
    │   │       │   ├── position
    │   │       │   └── type
    │   │       ...
    │   ├── blocks
    │   │   └── 100,64,200
    │   │       ├── type

There are three types of entry - directories, symlinks and files.

Some of these entries are hardcoded and unchanging such as the player directory, whereas others depend on the state of the game; the directory for entity 140 only exists if an entity with that ID is currently alive. We'll look into the differences between these shortly.

Every entry in the filesystem has a unique 64-bit identifier known as an inode. Since this isn't a real filesystem that needs to be efficient with its bookkeeping, they can just be allocated willy-nilly by incrementing a counter as needed¹.

Every instance of an entry is unique, even if they behave identically. For example, the position file underneath the different entities have inodes 5005 and 5011 respectively.

$ ls -ilR mnt/worlds/overworld/entities/by-id # -i to show inodes
5003 drwxr-xr-x - dom 28 Feb 13:17 135
5009 drwxr-xr-x - dom 28 Feb 13:17 140

./135:
5005 .rwxr-xr-x 256 dom 28 Feb 13:17 position
5007 .rwxr-xr-x 256 dom 28 Feb 13:17 type

./140:
5011 .rwxr-xr-x 256 dom 28 Feb 13:17 position
5013 .rwxr-xr-x 256 dom 28 Feb 13:17 type

Static entries

Static inodes are the simplest type of entry that exist unconditionally², and generally form the filesystem structure. Their inodes are allocated incrementally starting from 1 (the root) up to the magically chosen limit of 5000.

$ find . -maxdepth 2 -type d -ls
#   inode          name
        1   <snip> .
        2   <snip> ./player
       10   <snip> ./player/control
       14   <snip> ./worlds
       15   <snip> ./worlds/overworld
       22   <snip> ./worlds/nether
       29   <snip> ./worlds/end

Dynamic entries

As the Minecraft world evolves, so must our filesystem. When a dynamic directory such as $world/entities/by-id is requested, the game is queried for a list of entities through a state request (covered below) and the corresponding directory structure is generated on the fly.

$ cd mnt/worlds/overworld/entities
$ find by-id -ls
       17    by-id
     5000    by-id/135
     5002    by-id/135/position
     5018    by-id/80
     5020    by-id/80/position
     5023    by-id/13
     5025    by-id/13/position

# wait a bit...

$ find by-id -ls
       17    by-id
     5000    by-id/135
     5002    by-id/135/position
     5028    by-id/160
     5030    by-id/160/position

# 135 still exists, 80 and 13 died, 160 is new

A dynamic directory tree is technically ephemeral, because the game state and therefore list of entities is constantly changing. Unfortunately this doesn't map well to a user's intuition when it comes to a filesystem; it should be possible to cd into an entity's directory and continuously poll its files.

If every FUSE operation resulted in totally new inodes each time, we end up with a very confused user:

$ cd mnt/worlds/overworld/entities
$ ls -lRi
5561 drwxr-xr-x - dom 28 Feb 14:15 135

./135:
5563 .rwxr-xr-x 256 dom 28 Feb 14:15 position
5565 .rwxr-xr-x 256 dom 28 Feb 14:15 type

$ cd 135
$ ls -li
".": No such file or directory (os error 2)
# what? we just saw the files in here. check the parent again

$ ls -lRi ..
5596 drwxr-xr-x - dom 28 Feb 14:16 135

../135:
5598 .rwxr-xr-x 256 dom 28 Feb 14:16 position
5600 .rwxr-xr-x 256 dom 28 Feb 14:16 type

# ???

Note the new inode numbers on the 135 directory. The initial ls shows it to be 5561, but after cding it's changed to something else, and the second ls on the parent shows it is now 5596. Chaos!

What we need is a way to preserve inodes for identical entries between game states, which is done with a check against the current game state when building a dynamic hierarchy. If there is an identical child with the same name under the same parent, then we can reuse the inode.

But care has to be taken for directories; it is feasible that the player entity 135 dies and is reused by Minecraft for an arrow entity. The existence of the living file within an entity directory represents that the entity is animate, which was true for the player but not for the arrow. If we blindly reuse the same inode then this stale living file will keep hanging around, falsely claiming the arrow is alive.

The solution is to keep track of which directory inodes were reused, then cleanse them for stale entries.

Associated entry data

Entries can have a tag that describes which game data they're associated with, which dictates the target of commands sent to the game. When a file or directory is queried we recurse up through its ancestors checking for these tags, and accumulate their "interests" in the request.

For example, when reading the file below, the request is associated with the specific entity 100 in the overworld.

cat worlds/overworld/entities/by-id/100/position
               |                     |     |
               |                     |     +-----> Command = EntityPosition
               |                     |
               |                     +-----------> TargetEntity = 100
               |
               +---------------------------------> TargetWorld = Overworld

Phantom entries

While entities within a world are countable and can be listed in the entities/by-id directory, the same can't be said about block coordinates. There are technically infinite blocks in a Minecraft world, so attempting to generate a full directory listing of every single block is infeasible.

Instead, directories for blocks are generated on-demand when they are explicitly browsed to via $world/blocks/$x,$y,$z, hence the description of phantom entries. This is inspired by Bash's /dev/tcp pseudo-device, where a socket can be opened to a host by writing to a non-existent file with path /dev/tcp/$host/$port.

This also has the advantage that we can be more flexible with block coordinate parsing, and accept floating point positions as well, such as an entity's position.

$ cd mnt/worlds/overworld/blocks
$ ls
README

$ cat README
Path format is ./x,y,z or ./x\ y\ z
e.g. 0,64,100 or "0.5 22.3 41.5555"

# directory is created on demand, which contains the `type` file
$ cat 100,64,200/type
minecraft:dirt

# access through player's floating point position
$ cat "$(cat ~/mnt/player/position)/type"
minecraft:air  

Architecture

 ┌──────────────────────────────┐
 │     Shell/file explorer      │
 ├──────────────────────────────┤
 │ - ls mnt                     │
 │ - cat mnt/player/gamemode    │
 │ - echo 0 > mnt/player/health │
 └──────────────────────────────┘
       ▲
       │  via lib FUSE kernel driver and userspace library
───────┼──────────────────────────────────────────────────────────────────
       │
       ▼
 ┌─────────────┐               ┌──────────────────────────────────────────┐
 │   FUSE fs   │◄─────────────►│     Filesystem structure definition      │
 ├─────────────┤               ├──────────────────────────────────────────┤
 │ - lookup()  │               │ - Allocates inodes                       │
 │ - readdir() │               │ - Maintains child/parent relationships   │
 │ - read()    │               │ - Defines read/write behaviour for files │
 │ - write()   │               └──────────────────────────────────────────┘
 └─────────────┘
       ▲ via FlatBuffers
       │ over Unix domain socket
───────┼──────────────────────────────────────────────────────────────────
       │
       ▼
 ┌───────────────────────────┐
 │  Modded Minecraft client  │
 └───────────────────────────┘

Communication between the FUSE filesystem and the Minecraft mod is a pretty simple synchronous reply/response protocol using FlatBuffers (defined here) over the Unix domain socket /tmp/minecraft-fuse-$USER.

There are 2 primary message types in the FlatBuffers protocol - state and command. State requests are issued implicitly during the browsing of the filesystem, and fetch relevant game data to build the directory structure.

Let's take an ls in the $world/entities/by-id directory as an example. The FUSE function readdir is invoked on the by-id directory inode, which has registered an interest in the list of entities in the world via associated entry data (as mentioned above). Before we can list any child directories we need to know which entity IDs to list, so a synchronous state request is fired off to the game, and its response is used to populate the directory.

The other message type is a command, which is issued on a file read/write to query/update the game world. The file entries in the filesystem are aware of the data type they expect (float, integer, 3D vector, etc) and can validate the input locally before sending it to the game.

A use case?

To be honest, legitimate use cases did not feature heavily, if at all, when planning this project. The main reason is that it seemed a pretty cool way to learn about implementing virtual filesystems with FUSE.

But during development, I discovered an actual useful purpose - a language-agnostic modding API! Any language with I/O can use the filesystem to mod Minecraft without needing to deal with the horrors of gradle. I made a start on a very basic Python API, and it really is very satisfying use a REPL to mess around with the game.