July 18, 2024

The FreeBSD Foundation kindly sponsored by trip to Ottawa for the BSDCan 2024 conference and FreeBSD developer summit.  The event spanned four days, with the first two for the developer summit and the second two for the conference proper. As usual, all of this took place at the University of Ottawa, a rather familiar setting at this point: I’ve been attending BSDCan for about 10 years now. Travel was also straightforward for me since I live in Toronto, a train ride away.

The first day of the developer summit opened with a presentation by the outgoing FreeBSD core team. Four of the nine core team members were present and John Baldwin did most of the speaking.  They presented results from the annual FreeBSD user survey, presented reflections on the past two years of the team, presented some plans to introduce a new srcmgr@ team, and discussed some infrastructural initiatives by some core team members.  For the latter, the use of Matrix for developer communication and Galène for video conferencing, were topics of particular note.  I have been using both and while they have shortcomings, we would benefit hugely from having a single official platform for communication.  The fragmentation induced by having IRC, Slack, mailing lists, Discord, GitHub, Bugzilla, Phabricator, and other channels is a significant barrier to new contributors trying to get involved in the project.

The final presentation of the morning was by me, about some work I had done to provide an end-to-end testing utility for the FreeBSD src tree.  In particular I presented a script, bricoler, which effectively glues together a number of steps required to build virtual machine images, boot them, run various test suites, and collect some sort of result.  This script is still a work-in-progress, but for the past while I have been using it in my own day-to-day work and hope to make it more usable over the next several months.  I got a lot of good feedback during the talk and in the days following, and it seems quite clear to me that I’m not the only one who perceives a significant gap in our development tooling.

The last scheduled talk of the day was by Mitchell Horne, giving us a tour of the state of RISC-V support in FreeBSD.  He mainly focused on the level of support we have for various generally available RISC-V SOCs, as well as our level of support for a number of optional extensions to the RISC-V ISA.  The picture painted by the talk was a bit somber; it seems that many boards are quite tricky to support, partially due to lack of open documentation, relatively high prices and low availability, and use non-standard platform interfaces. These factors make it hard to attract contributors and support building packages for RISC-V, which has further negative effects.  However, development of new platform features is progressing nonetheless; Mitchell discussed some of his ongoing work on RISC-V memory management and the state of a RISC-V bhyve port, which at the time of this writing is close to being committed upstream to FreeBSD.

A large chunk of the second day of the developer summit was taken up by planning for FreeBSD 15.0, wherein developers and contributors presented their “haves” “needs” and “wants” for the next major FreeBSD release.  These refer to continuously maintained lists of features that someone either:

  • Has already implemented and needs to integrate into FreeBSD,
  • Needs in order to make progress on a project using FreeBSD, or
  • Things FreeBSD would benefit by having.

Quite a few different projects and features were mentioned.  I brought up one that I was planning to work on later in the summer, only to find out that another developer had already implemented a working prototype along similar lines.  This kind of immediate feedback highlights the usefulness and importance of in-person planning sessions.  Also of note for me was discussion of a 9pfs implementation that several developers have worked on; it had been in review for a long time and was somewhat stalled, but I think the interest in it helped ensure it was committed shortly after the conference.

The final talk of the day was by an engineer from a company called Antithesis, which develops a software testing product, a key component of which is the “Deterministic Hypervisor”, based on FreeBSD’s bhyve hypervisor.  Alex’s talk described the product, which effectively simulates an adversarial, reproducible testing environment for software, especially systems software such as databases. The software under test runs in a virtual machine which uses a proprietary system to take snapshots, inject faults, and reproduce software failures.  A key component of this system is the ability to deterministically trigger and reproduce bugs in the software, and the talk described many intricacies of the amd64 architecture relating to these goals, and how Antithesis’ product works around them.  This talk and the product were quite fascinating, and I’d strongly encourage any readers to watch the recording on YouTube.  Later during the conference, John Baldwin and I had some discussion with Alex regarding challenges porting Antithesis to other architectures, particularly AMD (whose virtualization extensions are quite different from Intel’s) and arm64.

During the conference proper I attended several talks that were of particular interest to me:

  • Warner Losh described his efforts to make it easier to contribute to FreeBSD via GitHub.  For   the past several months he has done a huge amount of work reviewing pull requests, improving tooling for landing them, and establishing guidelines for contributors.  The talk presented the current state of affairs. While the process of contributing via GitHub is not yet fully streamlined, so far it has been quite successful relative to contributions via Phabricator or Bugzilla.
  • Rob Norris presented a development framework, quiz, he uses for OpenZFS development.  While his requirements are somewhat different from my own, it was very useful to see how he sets up a ZFS testing environment, particularly with respect to speed: quiz is able to boot a VM with a new ZFS build in several seconds, which is a fair bit better than I can do.  After the talk, there was some side discussion of what it would take to support quiz on FreeBSD.  Rob already has patches for bhyve to support direct loading of Linux and FreeBSD kernels, which gets us most of the way there, and we talked for a while about how best to get them integrated into upstream FreeBSD.

As usual, on the night of the last day of the conference, there was a social event at a bar in Ottawa’s Byward Market.  There I had a chance to catch up with a couple of OpenZFS developers and hear some of their thoughts regarding OS support (i.e., how ports of OpenZFS to other operating systems determine the “shape” of various internal platforms interfaces), ZFS performance on Linux, and the general state of the project.  I also had a long chat with Mike Karels, who very sadly passed away the next day while on his way home from the conference. We talked for a while about how to advance some patches he had written for the ULE scheduler to support heterogeneous CPU topologies, and about his work as member of the release engineering team, in particular his role as the release engineering lead for the forthcoming FreeBSD 13.4 release.  I’ve had many fruitful interactions with him over the past several years and will deeply miss his presence on the mailing lists, in our commit logs, and at conferences.

Thank you to the FreeBSD Foundation for sponsoring my attendance at the conference.

– Contributed by Mark Johnston