FreeBSD for Research: CHERI/Morello

Long before breakthrough innovations hit VC pitch decks, let alone your laptop, microservice, or smartphone, they typically start as a hypothesis in a University or industrial research lab. 

Take Docker, for example. This building block of microservices and agile methodologies stands on the shoulders of the virtualization and system partitioning innovation in FreeBSD called Jails. And Jails, you may have guessed, emerged from research undertaken by Robert Watson and Poul-Henning Kamp. 

After his work on Jails, Professor Watson joined the Department of Computer Science at the University of Cambridge in the UK, where he continues to research and develop breakthrough ways to improve computer system security. His work earned him the EuroSys Jochen Liedtke Young Researcher Award, granted annually to European early researchers (i.e., those no more than 10 years on from receiving their PhD) who have demonstrated exceptional creativity and innovation in systems research. 

In selecting Professor Watson, EuroSys noted his long history of contributions in systems research, including co-developing the Jail security model that is an intellectual foundation for contemporary OS containerization; leading development of the kernel access-control framework used for application sandboxing in systems from macOS and Junos to FreeBSD; developing the Capsicum OS security model, and most recently leading developing of the CHERI computer architecture. CHERI is now the key technology being explored in the UK Industrial Strategy Challenge Fund (ISCF) £187M (roughly $225M) Digital Security by Design (DSbD) program. 

CHERI (Capability Hardware Enhanced RISC instructions) is an architectural feature that enables fine-grained C/C++ memory safety and scalable software compartmentalisation, and was developed by SRI International and the University of Cambridge starting in 2010.

In order to make CHERI available to write and run programs on, it requires a development board. This is where Morello comes in. Arm’s Morello is a processor design, System-on-Chip (SoC), and board that integrates support for the CHERI protection model. The Morello development board, compiler, and toolchain allow researchers and industry practitioners to write and run programs leveraging CHERI. AutoCHERI, for example, is testing CHERI and Morello as a hyper-secure architecture for in-vehicle Telematics Control Units.

Professor Watson was kind enough to sit with the FreeBSD Foundation to discuss his research. The discussion illuminates the role of academic research and the process by which breakthroughs in the lab make their way into our digital society. A key tool in Professor Watson’s quiver is FreeBSD. Like many other OSes, it is open source so supports the necessary customization. In contrast with Linux, the permissive BSD license is essential to supporting the public – private partnership his research relies on.   

To understand why open source in general, and FreeBSD in particular, is so central to Professor Watson’s work, and to academic computer system research more broadly, it helps to first understand the research process. Professor Watson explains “in systems research, one builds prototypes of ideas. You begin with a pure hypothesis about the way the system should work, or should not work, or perhaps how some tooling that you have come up with or a change to the design will work, something that is an intellectual contribution. You then apply it to an artifact, like an operating system or a compiler. You then evaluate it, and then you iterate it to correct for learnings about your hypotheses, undesirable outcomes, or there was a problem we did not understand, and so on. In all the work mentioned in the EuroSys award, FreeBSD has been that starting artifact.”

Groundbreaking academic research benefits enormously from a production system against which to test hypotheses and into which the innovations can be later transitioned. “Open source is essential to this research approach in the software space, and increasingly in the hardware space,” Dr. Watson explains. “You could build the reference baseline artifact yourself, but what you really want is something that is the current state of the art, that has the complexity that comes from large scale use, and so on. FreeBSD meets these criteria.”

To those familiar with the history of FreeBSD, it may come as no surprise that Professor Watson uses it so heavily at Cambridge. FreeBSD has a rich history in the academic community. It was derived from the 4.4-Lite version of the Berkeley Software Distribution developed by the Computer Systems Research Group at the University of California at Berkeley between the mid 1970s and 1990s. Over the last 30 years, the FreeBSD operating system has continued to provide those in academia a stable base on which to undertake research, and a path to industrial adoption via widely used products and services that run on, are based on, or incorporate, FreeBSD ¹

Professor Watson’s Mandatory Access Control (MAC) framework research continued along the lines from Jails, arguing that your OS should have pluggable components to allow it to work with many different pieces of hardware, and that access control and security should also be pluggable. 

Many users customize FreeBSD in their products—adding things like device drivers and file systems—this is often called localizing. Professor Watson’s MAC research argued that you could localize the operating systems and the security requirements of your product. This provides extensibility so that companies building a router or mobile phone operating system or some other appliance with FreeBSD can tune the OS to the environment they need in terms of security, in addition to hardware, storage, and network tuning. 

In terms of how this research transitioned to popular use, Professor Watson feels it was quite a successful piece of research (we’d call that an understatement). The research targeted FreeBSD and was incorporated into other systems, most notably iOS. 

CHERI, which stands for Capability Hardware Enhanced RISC Instructions, extends conventional processor Instruction-Set Architectures (ISAs) with architectural capabilities to enable fine-grained memory protection and highly scalable software compartmentalization. The Capsicum work took some ideas from the 1970s called capability systems, and argued that they can be made current by atoms-smashing with a current piece of software through BSD. The CHERI work takes this concept much further and applies it to processors. “With CHERI, we changed the hardware and we changed the software. And the software we chose to change to demonstrate these ideas, but also understand them, is FreeBSD,” remarked Professor Watson.

CHERI looks at hardware/software co-design, which requires coordinated changes to the software and the hardware. The interface between the two is the instruction set architecture, or ISA. A new ISA enables processor advancements to be expressed by software. So hardware/software co-design requires changes to both the hardware and the software.² 

At BSDCan 2023, Brooks Davis from SRI International explained that “CHERI mitigates vulnerabilities in C/C++ Trusted Computing Bases, such as hypervisors, operating systems, language runtimes, browsers, ….” CHERI is proving to be extremely effective. The Microsoft Security Response Center (MSRC) found that CHERI mitigates over two thirds of critical memory-safety security vulnerabilities in C/C++-language software.³

FreeBSD is essential to this kind of research thanks in part to the permissive BSD license that supports the modification of the entire hardware-software stack, and because of the comprehensive integration of LLVM to build a full kernel and user space. When the project began in 2010, FreeBSD was aggressively pursuing LLVM integration, which continues to be a challenge for Linux today. Also important were clean kernel support for multiple ABIs, integration of the Capsicum security model, and strong early support for the RISC-V architecture. 

The CHERI team has contributed back to FreeBSD extensively during this work, improving FreeBSD’s 64-bit MIPS and Armv8-A support, contributing a RISC-V port, adding numerous device drivers to better support Xilinx and Intel FPGA boards, as well as Arm IP, and developing QMEU userlevel and cross-build support for the base system and ports/packages.

Interested users can download a full FreeBSD-derived OS called CheriBSD. CheriBSD is a research operating system that makes extensive use of CHERI to improve software security – and demonstrates an incremental software adoption path for CHERI. Its features include a memory-safe kernel and userspace, support for two software compartmentalisation models, and roughly ten thousand memory-safe third-party packages including server applications such as nginx, a KDE Plasma desktop environment. Later this year, a memory-safe Chromium web browser, sponsored by Google and InnovateUK, will be added to the CheriBSD package collection. It can be installed on Arm Morello systems using a downloadable USB stick image. CheriBSD has an active user community of over 70 companies and universities that have Morello-based research projects, with almost all running CheriBSD, and its most active contributors are SRI, Cambridge, and Microsoft.

There is a build system that cross builds CheriBSD from Linux, Mac, or FreeBSD. Professor Watson and the others working on CHERI have actively contributed Linux and macOS cross-build support back to FreeBSD. This is important, Professor Watson argues, “because we think to make FreeBSD and our work really accessible, we can’t tell you what to run on your desktop. You have to run your OS of choice, and we help you get FreeBSD running on whatever device you want it to run on, which might be the desktop, but, it might also be an embedded device, or it might be a server somewhere, and we don’t want any obstacles standing in the way of that.”

As mentioned, the Morello processor design, System-on-Chip (SoC), and board integrate support for the CHERI protection model. Morello is a research prototype jointly funded by Arm and UK Research and Innovation (UKRI), and created in collaboration between Arm and Cambridge to evaluate CHERI as a technology for mainstream adoption. Over 600 boards have been shipped since mid-2022, primarily to universities, government labs, and industrial research laboratories. The FreeBSD Foundation will receive its first Morello board later in 2023, making this development environment available to the global FreeBSD community.

To get hands on with CHERI, go to the CHERI software distribution site and grab the latest release. Docker images are available for Linux and other OSes. The images include environments that allow users to run a QEMU-based VM with CheriBSD for the CHERI-RISC-V or Arm Morello architecture, and cross-compile software for it using the included SDK. For Arm Morello, use an image from: https://hub.docker.com/r/ctsrd/cheribsd-sdk-qemu-morello-purecap and once a Docker container is up, a single command starts CheriBSD⁴. How academic innovations transition from the lab to production

Transition for research can take many forms. In some cases, people and companies use the software. Sometimes, transition means that the ideas influence other people’s work, and then eventually that ends up somewhere. 

“For something like the MAC framework, or Jails, it’s quite easy to look out there and see all the use cases, it’s used in all kinds of things. For something like CHERI, we’re not yet at the point where we could really speak to what the ultimate transition will be, but there is a big transition project going on. The UK Government and various companies are spending about a quarter of a billion dollars on a five-year transition project currently, which is quite exciting.”

The CHERI work has a number of transition narratives. CHERI was originally done relative to the MIPS architecture when it began in about 2010. Now, all of the work is on Arm and RISC-V. RISC-V is the open source research platform, Arm is the current transition target of choice, although Professor Watson quickly adds that he and the others would be happy to transition to other hardware also. “Network routers and switches, firewalls, phones, tablets, were all the ambition, really. I think it’s quite possible to imagine well beyond that, servers and desktops are all in scope too, but that was what we were doing in early demonstrations.”

On behalf of all the computer users who have benefitted from his and his partners’ security research, the FreeBSD Foundation expresses its gratitude to Professor Watson. Far from ivory tower pursuits, these innovations undergird much of contemporary society’s digital infrastructure. As a completely open source project, we hope you will consider getting involved with FreeBSD and helping write the next chapter in computing. 

— Contributed by Greg Wallace