kAFS and AF_RXRPC

The primary goal of the kAFS project is to provide a filesystem within the Linux kernel that can communicate with an AFS server to the same extent that OpenAFS can and, as such, can be a drop-in replacement for OpenAFS on the Linux platform. With the sources being in the upstream Linux kernel, kAFS can take advantage of GPL-only kernel APIs and can be fixed up by people wanting to make wholesale VFS interface changes.

kAFS comprises four in-kernel components:

• kAFS itself. This is a network filesystem that's used in much the same way as any other network filesystem provided by the Linux kernel, such as NFS.
• AF_RXRPC. This is a network protocol that provides the network interface for kAFS to use and also offers userspace access via the socket interface.
• Kernel keyrings. This facility's primary purpose is to retain and manage the authentication tokens used by kAFS and AF_RXRPC, though it has been kept generic enough that it has been adapted to a variety of roles within the kernel.
• FS-Cache. This is an interface layer acts as a broker between any network filesystem and local storage, allowing retrieved data to be cached. It can be used with NFS, CIFS, Plan9 and Ceph in addition to kAFS.

and three userspace components:

• kafs-client. This will be the configuration and systemd integration for automatically mounting the kAFS filesystem and tools for managing authentication.
• kafs-utils. This is a suite of AFS management tools, written in Python3, that use AF_RXRPC from userspace as a transport to communicate with a server.
• keyutils. This is a set of utilities for manipulating the keyrings facility from userspace.

A TODO list can be found here.

The kAFS filesystem is a Linux filesystem for accessing AFS servers over the network. It is mounted using something like the following:

	    insmod /tmp/kafs.ko
	    echo grand.central.org >/proc/fs/afs/rootcell
	    mount -t afs none /afs -o dyn

	    mount -t afs "#grand.central.org:root.cell." /mnt

Implemented features:

• Reading and writing.
• Advisory file locking.
• Encryption and authentication.
• Automounting of mountpoints.
• Failover.
• AFSDB and SRV DNS record lookup.
• Handle volumes being moved or being busy.
• Path substitution (@sys and @cell).
• IPv6 support, thus allowing access to current AuriStor servers.
• Kernel tracepoints.

• Network namespacing.
• Notifications (inotify and friends).
• Userspace tool interface (substitute for pioctl).

Features that could be added:

• RXAFSCB RPCs for debugging. Have to be careful of this is it can be a security problem, but it could require encryption to access.

The kAFS filesystem sources are integrated into the upstream Linux kernel here

The AF_RXRPC network protocol is the network transport for RX on behalf of the kAFS program and the Python tools. It opens UDP sockets internally and exchanges network packets with peers over it.

An open AF_RXRPC socket can support multiple calls to different peers, both service calls and client calls. The sendmsg() and recvmsg() system calls are used with the ancillary data being used to manage the multiplexing and to pass information.

The kernel handles all the RX protocol metadata internally so that the user only sees the content data. This allows virtual connections to be shared transparently between userspace processes.

Implemented features:

• Usable from userspace via socket(AF_RXRPC, ...).
• Supports client and server calls over the same socket.
• Kauth, Kerberos 4 and Kerberos 5 security; plain, partial and full encryption.
• IPv6 support.
• Kernel tracepoints.
• Slow start congestion management.
• Service upgrade.

Features that need to be added:

• RXGK security.
• YFS-RXGK security.

Features that could be added:

• Debug and stat packet support. These are a potential security hole as there's no encryption on them. It might be possible to limit access to these using netfilter - or just provide the information through /proc in userspace and let anyone who wishes to read it do so, provided they can log in to the system.

The AF_RXRPC sources are integrated into the upstream Linux kernel here.

An example client built using AF_RXRPC can be found here.

The kernel keyrings facility was created to carry tokens for kAFS and AF_RXRPC to identify and authenticate a user with the various servers and to provide cryptographic keys for the network transport. PAGs are not available, rather the session keyring to which a process subscribes is the container for these keys.

Keys and keyrings can be manipulated by users to a certain extent, including adding and deleting keys, though security features exist to control a user's access.

The PAM package and the keyutils package are the main keyring manipulators.

Looking in /proc/keys something like the following would appear:

       396198eb I--Q---     1   1d 3b010000     0     0 rxrpc     afs@your.cell.com

The current process's session keyring can be listed to see the kAFS/AF_RXRPC keys applicable to processes run under a shell:

       [root@andromeda ~]# keyctl show @s
       Keyring
        340802050 --alswrv      0     0  keyring: _ses
        566823513 --alswrv      0 65534   \_ keyring: _uid.0
        962697451 --als-rv      0     0   \_ rxrpc: afs@your-file-system.com

kAFS and AF_RXRPC clients will pick up the appropriate rxrpc-type key automatically by name and use it if it is available.

The kernel keyrings sources are integrated into the upstream Linux kernel primarily here and here.

The keyutils utility sources can be found here

The FS-Cache kernel module is a thin layer between any network filesystem and a cache. There is one caching back-end available (CacheFiles) that uses one disk file per network filesystem file to store data.

Supported features:

• Persistent across reboots.
• Usable by multiple filesystems: AFS, NFS, CIFS, Plan9, Ceph.
• Transparent.

Features that need to be added:

• Kernel tracepoints.
• More tuning knobs.
• Backends to make use of non-rotating cache media.

Features that would be nice:

• Disconnected operation support (requires netfs integration).

The FS-Cache sources are integrated into the upstream Linux kernel here

The kafs-client package will be the tools to configure and mount the kAFS filesystem and provide systemdintegration. It will also provide authentications support through aklog and PAM integration.

The kafs-utils suite is a replacement for bos, vos, pts and co. written in v3 Python, utilising AF_RXRPC to provide the transport.

Implemented features:

• Parameter, input and output equivalence with OpenAFS for use with existing scripts.
• Bash completion.
• Two layers, the command suite applications and wrappers for the raw RPC calls.
• Some command/subcommands are supported, including most of bos and pts and some of vos.

Features that need to be added:

• Kernel interface (substitute for pioctl).
• The rest of the commands and subcommands.
• The ability to use other transports than AF_RXRPC.
• Two more layers within the library to abstract some of the RPCs.
• User scripting support so that users and admins can use this to write scripts directly in Python.
• Better error processing.

Features that would be nice to have.

• Python 2 support.
• The ability to talk to other AFS kernel modules, such as OpenAFS and Auristor.

The kafs-utils sources can be found here