Project Banbury

Project Banbury aims to improve the experience of Linux users and administrators with hotpluggable storage.

Use cases

I unplug a drive from my Linux system. I realise I've unplugged the wrong drive and plug it back in.
- Linux tears down the block device immediately. Block layer returns errors to filesystems. Filesystems return errors to userspace. Writes are lost, applications crash. Need to fsck, maybe restore from backups.
- I/Os are paused. When the hardware is reattached, the I/Os restart.
I unplug a USB stick before the last write has finished. I get a notification on my desktop that the device was still busy and should be reinserted. The notification has three options -- discard writes silently, wait patiently and send errors.
The firewire cable is a bit wobbly and we get a rapid series of connect/disconnect events making I/O impossible. I receive a desktop notification to check the cable. Once it is fixed, everything continues.
Someone inadvertently powers off the fibrechannel switch between me and my storage, taking away all paths. After powering it back on, everything continues.

How can we make this work in Linux?

When a device is inserted, a userspace component (udev?) looks through its database of devices and decides whether it has seen this device since the last reboot. If it has, it tells the kernel that this new device is actually that old device. The kernel then reconnects the new physical device to the old block device and I/Os which were outstanding at the time of unplug are retried.

Work plan

Implement a mechanism for userspace to remove a logical block device.
Implement an option to not delete block devices after physical hardware is removed. This includes a new state for the block device, maybe called "removed" which indicates the hardware is not currently present.
Implement a mechanism for userspace to tell the kernel that this logical block device is actually that logical block device (implicitly deleting the temporary logical block device we were using and reattaching it to the old one).
Recruit someone who understands the userspace side of this to do all the hard work of determining whether two devices are the same, interacting with the desktop notifications, etc.

Refinements

Userspace can check the "last mount time" on filesystems which have it, and offer you the chance to discard the dirty data, overwrite any conflicting changes from the other system or mount another instance of the new filesystem while keeping the frozen filesystem around for you to do data recovery on.
The block layer can send userspace a message when it gets an I/O for a removed device (including the direction of the I/O). The desktop environment can pop up an alert to tell the user they need to reinsert the device.
Filesystems can check the state of the block device and change behaviour for devices that have gone (perhaps keep clean pages around more aggressively).
The userspace component can keep track of how many events it's seen recently for a given device and alert the user that something is flaky.
The userspace component should also notice when dozens of devices have disappeared in quick succession and send a single dbus notification noting that a lot of storage has disappeared.
We probably need a way to send I/O which won't block; anything marked with REQ_FAILFAST, for example. Maybe O_DIRECT? Lots of possibilities here.
How should we handle writers which are dirtying pages while a block device is unavailable? If they fill the page cache with unevictable pages, we're a little bit stuffed. It'd be nice to be able to pause particular userspace processes.

Why Banbury?

It's just a place name. You may have heard of it from the song.