retrage.github.io

9pfsPkg: Network Boot from Bell Labs

I developed a Plan 9 file system (9P) client for UEFI to enable network booting from a commodity 9P server. By leveraging the simplicity and flexibility of 9P, the UEFI can do network boot from cloud storage without any effort. This blog post gives you a brief overview of 9pfsPkg.

The source code, introduction slides, and presentation video are available at:

What is Network Boot?

Network boot is a boot method which loading boot images over the network. To make this possible, the BIOS has its network stack. There are two methods for network booting: PXE Boot and HTTP Boot.

PXE (Pre-eXecution Environment) Boot is the most widely used method as it exists from the Legacy BIOS era. It is standardized and implemented as not only proprietary but also open source. PXE boot uses TFTP to transfer files. This protocol is not popular, so it requires a dedicated TFTP server.

HTTP Boot uses HTTP for transferring images. It has been standardized from UEFI 2.5 in 2015[0]. It supports modern features like DNS and TLS. Since it uses HTTP, we can use commodity HTTP servers (e.g. Apache HTTP Server, Nginx).

Below is the interface of the HTTP protocol.

EFI HTTP Protocol

Configure() sets the configuration, Request() sends a request, and Response() receives a response. By using these functions, we can implement HTTP Boot bootloader like this:

EFI_STATUS HttpBootLoader()
{
  // Send request
  Status = Http->Request (Http, TxToken);
  // Recieve response
  Status = Http->Response (Http, RxToken);
  // Start loaded image
  Status = gBS->StartImage (ImageHandle, NULL, NULL);
  return Status;
}

This example shows how it is easy to boot with HTTP on UEFI.

UEFI is Extensible

UEFI is an abbreviation of the Unified Extensible Firmware Interface. As it includes the word “Extensible,” it has a modular design. The modules are called “Protocol” and UEFI has features that load external protocols in its core. By calling EFI_BOOT_SERVICES.InstallProtocolInterface() with passing a loaded protocol to Interface argument, it installs an external protocol.

EFI_BOOT_SERVICES.InstallProtocolInterface()

As an example of a UEFI protocol, I introduce the Simple File System Protocol. This protocol provides a file system independent file operation interfaces. Here is a figure of the interfaces.

EFI Simple File System Protocol

OpenVolume() in Simple File System Protocol opens a volume and returns File Protocol Root that represents the root directory. File Protocol provides file operation functions like Open(), Read(), Write().

However, even it has abstract interfaces, UEFI supports the FAT file system only by default. There are some third-party non-FAT file system drivers. Here is an example of the use of such a file system driver: UEFI Rootkits. A rootkit is malware that targets kernels or firmware. Once infected, it installs other rootkits and/or agents. Hacking Team’s rkloader[2] and LoJax[3] are such UEFI rootkits. They have NTFS UEFI drivers to embed kernel rootkits to the target Windows system. This driver is a port of NTFS-3G, an open-source NTFS implementation, and has Simple File System Protocol as an interface. The following snippet from the rkloader shows how Simple File System Protocol makes embedding an agent easy.

EFI_STATUS
EFIAPI
InstallAgent(
  IN EFI_FILE_HANDLE CurDir,
  IN CHAR16	*  FileNameUser
  )
{
  // Open FileNameScout as FileHandle
  Status = CurDir->Open (CurDir, &FileHandle, FileNameScout, EFI_FILE_MODE_READ|EFI_FILE_MODE_WRITE|EFI_FILE_MODE_CREATE, 0);
  // Write pSectiondata to FileHandle
  Status = FileHandle->Write(FileHandle,&VirtualSize,(UINT8*)(pSectiondata));
  // Close FileHandle
  Status = FileHandle->Close(FileHandle);
  return EFI_SUCCESS;
}

First of all, open a file by Open(), deploy the agent by Write(), and close the file with Close(). As you can see, it is simple and does not have any trick.

Plan 9 File Protocol

Now, let’s think about network boot. I pointed out that the current network boots are network-aware and less flexibility. Thus, we want a network transparent file system and protocol for network boot while maintaining the feasibility of the file system.

Here is the answer: Plan 9 File Protocol (9P)[8]

Plan 9 from Bell Labs (Plan 9)[7] is a Unix successor OS developed by the original Unix developer in Bell Labs. “Everything is a file.”, which is a well known Unix philosophy, is a core design decision on Plan 9. P9 is a protocol developed by Plan 9 developers to deal with remote files in the same manner as local files. Below is a flow of loading a file with 9P.

9P Flow

The client negotiates by version and connects with attach to get a file descriptor of the root directory. walk searches to the target file, open it, and read it. As you can see, 9P is a protocol that file operations and messages correspond to one-by-one.

9P is popular in the fields of not related to Plan 9 due to clarity and simplicity. For instance, the Linux kernel has a 9P client file system called v9fs[4]. VirtIO has a virtio-9p 9P server to share the host file system with the guests[5].

Recently, Microsoft has released Windows 10 update, and Windows Subsystem for Linux 2 (WSL2) is now officially supported. It runs a guest Linux on the Hyper-V VM in contrast to WSL1. Because VM disk image is a monolithic file, it is hard to access inside files with the same manner of host file access. To solve this issue, WSL2 uses 9P to access guest files from the host. The host Windows has 9P client to access the guest Linux files. The guest has a 9P server to process requests from the host to share the files[6].

9pfsPkg

As I described in the previous section, 9P is still widely used nowadays. I implemented a 9P client file system for UEFI: 9pfsPkg.

9pfsPkg is a 9P client file system UEFI driver with a Simple File System Protocol interface. Because 9P is a network transparent file system, we can use existing non-network-aware tools (e.g. UEFI Shell) for file operations via networks without any modification by using 9pfsPkg. Another advantage of the file system is that it does not require dedicated servers (like TFTP in PXE Boot).

9P Boot

Let’s take a look at a boot by 9P (9P Boot). The below shows the overview.

9P Boot Overview

First of all, the 9P service runs on the server with an exported directory (e.g. /tmp/9). Next, the client loads the 9pfsPkg UEFI driver to create a new volume. The driver processes operations to the file system volume and communicates with the server via the UEFI network stack to handle the file operations.

The below video clip shows what it looks:

When the UEFI startups, we can see the local file system FS0: only.

Mapping table
      FS0: Alias(s):HD0a65535a1:;BLK1:
          PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x0,0xFFFF,0x0)/HD(1,MBR,0xBE1AFDFA,0x3F,0xFBFC1)
     BLK0: Alias(s):
          PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x0,0xFFFF,0x0)
     BLK2: Alias(s):
          PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x2,0xFFFF,0x0)

Load 9pfsPkg UEFI driver by load 9pfs.efi.

FS0:\> load 9pfs.efi
Image 'FS0:\9pfs.efi' loaded at 7E2E7000 - Success

The new file system FS1: has appeared.

Mapping table
      FS0: Alias(s):HD0a65535a1:;BLK1:                                          
          PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x0,0xFFFF,0x0)/HD(1,MBR,0xBE1AFDFA,0x3F,0xFBFC1)
      FS1: Alias(s):F1:
          PciRoot(0x0)/Pci(0x2,0x0)/MAC(525400123456,0x1)
     BLK0: Alias(s):
          PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x0,0xFFFF,0x0)
     BLK2: Alias(s):
          PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x2,0xFFFF,0x0)

In contrast to local FS0:, the device path of remote FS1: is PciRoot(0x0)/Pci(0x2,0x0)/MAC(525400123456,0x1). It represents that the volume is on the remote server.

By executing fs1: and grubx64.efi, it boots the bootloader GRUB.

FS0:\> fs1:
FS1:\> grubx64.efi
                             GNU GRUB  version 2.02

   Minimal BASH-like line editing is supported. For the first word, TAB
   lists possible command completions. Anywhere else TAB lists possible
   device or file completions.


grub>

At this point, UEFI Shell and GRUB deal with the remote files in the same manner as local files. There is no network boot specific process.

Proxy Boot

9P Boot enables non-network-aware network boot. To take more advantages of the 9P, I propose Proxy Boot as an application of 9P Boot. It can boot from other servers via the direct server as a proxy. By using Proxy Boot, UEFI can boot from cloud storage without any effort. Following is the overview of Proxy Boot.

Proxy Boot Overview

I used Google Cloud Storage (GCS) for the network boot. The storage bucket has boot images. The server mounts the bucket as a file system using gcsfuse[9]. The 9P server uses the gcsfuse’s mount point (e.g. /mnt/gcs) as an exported directory. The client mounts the volume in the same manner as 9P Boot. The client UEFI can treat the cloud storage files as if local files.

The below is the demo:

Create a GCS bucket and upload boot images. I used BitVisor (thin-hypervisor) as a practical boot image. loadvmm.efi is the loader, and bitvisor.elf is the actual BitVisor image. They are default build and no modification for the network boot.

GCS Bucket

Next, mount the bucket on the server using gcsfuse at /mnt/gcs mount point.

$ sudo -E gcsfuse proxy-boot /mnt/gcs
 Using mount point: /mnt/gcs
Opening GCS connection...
Opening bucket...
Mounting file system...
File system has been successfully mounted.

On the client, load the driver with load 9pfs.efi, move to fs1:, and call loadvmm.efi to boot BitVisor.

Shell> fs0:
FS0:\> load 9pfs.efi
FS0:\> map -u
FS0:\> fs1:
FS1:\> loadvmm.efi
Starting BitVisor...
Copyright (c) 2007, 2008 University of Tsukuba
All rights reserved.

Once again, the UEFI Shell and loadvmm.efi operates the cloud storage files as if local files, and there is no cloud-specific process.

Conclusion

In this blog post, I pointed out that the existing network boots are network-aware and less flexible. The 9P client file system for UEFI (9pfsPkg) enables network transparent network boot (9P Boot). As an application of the 9pfsPkg, I proposed a network boot from cloud storage via the server (Proxy Boot) without any effort.

References