intro⌗

ft is a remote file browser, its function is to give any user(smartphone, raspberry pi, normal computer) the ability to transfer files to a number remote of storage units such as flash drives, phone and hard disks.

If that was the only intention with creating ft then it wouldn’t be of much use really. ft should support different file systems, such as network filesystems, filesystems for phones, cameras and other types aswell.

ft’s server language, Go, provides a package in the standard library that defines clearly what a file system is. So, one must consider Go’s io/fs as a good option in using it in this project, but there are several reasons why you shouldn’t.

go fs packages⌗

io/fs simply isn’t flexible. The definition of a filesystem in io/fs consists of a data structure with an Open method.

Open(name string) (File, error)

Do note that the method essentially returns a read-only file, as file only implements io.Reader, io.Closer and Stat() (FileInfo, error). That’s literaly all.

That essentially means you cannot:

  • Create Directories
  • Remove files
  • Create new files
  • Create files with specific flags
  • Modify already existing files
  • Modify permissions and groups

Simply put, io/fs is too abstract for a filesystem interface. There’s a slightly order and better package called afero by user, spf13 who also made hugo. afero, in contrast to io/fs, is a lot better.

It has convenient abstractions that are implemented by the default os package and pretty good data structures that allow for easier testing, see afero.NewMemMapFs().

But, while researching filesystem abstractions I thought: “Why go through all this trouble if this is not extensible”.

extensibility: an overview⌗

Extensibility is the capacity of being able to be extended. Extensibility can help a program stay secure and consistent while providing supreme usuability and power to the user.

One good case of this is the coreutils defined by the POSIX standard. For those who do not know, the coreutils are a collection of programs that are meant to do thing and doing it well.

A good example of using the coreutils together is by wanting to search through a file. Normally, programs such as text-editors provide a feature like searching through a hot key such as CTRL+F, where you could not only serach for word but also replace them.

In the coreutils, it is much simple. First, you read all the contents of a file via a program named cat. Second, you search for the word using the program grep.

Via a process called piping, one program can read another program’s output. The practical application of this is as follows:

cat file.txt | grep "my word"

In essence, extensibility is a good trait to have. However, requiring the user to write a Go package to support their favorite file system was too much to gain extensibility.

Add to that the fact that these file system implementations some times need updates or have bugs in them, and you have a pretty high risk of making your software assiociated with unreliability and buggy behaivor.

In ft’s situation, extensibility must come from outside and to that we can look forward to no abstraction but FUSE.

fuse: filesystem in userspace⌗

Usually, filesystems must be implemented through a driver, similarly to how Microsoft Windows does in-order to support phones and cameras. But, this approach is rather inefficient as it required elevated privileges and bugs can crash a system.

fuse is linux’s solution to mounting dynamic or otherwise ‘outsider’ filesystem. fuse is a kernel module that defines an abstraction to mount file system as a normal user.

The pros of using fuse for ft are:

  • Any programmer can implement, in their own language, a filesystem
  • ft can later use that filesystem from an outside language such as C++
  • Performance, stability and reliability via using tested implementations.

conculsion⌗

While afero provides a very good file system abstraction, it cannot compete with fuse, not at the slightest. fuse allows the end-user to customize how many directories are mounted at its root directory, allowing for pluggable phones, cameras, flash drives, and other sort of devices to co-exist in one space, making ft extremely useful.

One thing I’d like to note is how useful afero is in Go packages. To test out internal packages, you do not have to depend on OS access but instead rely on afero’s abstractions and using mock implementations. Definitely use afero over io/fs any time of the day since it is more developed and stable.