October 24, 2023

The FreeBSD Foundation kindly helped sponsor my trip to Coimbra, where EuroBSDCon 2023 was held. As usual, the conference consisted of a two-day FreeBSD developer summit followed by the conference proper, also two days long. The conference was held in a couple of buildings belonging to the University of Coimbra, which itself was well worth the visit: the university buildings are spectacular, and being situated on a hill they offer a great view of the city. Climbing up the hill each morning was a bit arduous, but well worth it.

The first day of the developer summit started somewhat late due to technical difficulties, but it provided a good opportunity to chat and catch up with other attendees. I talked a bit with Ruslan Bukin (br@) about his plans to port some software to Morello, a hardware implementation of CHERI from ARM, and with Bojan Novković, my Google Summer of Code (GSOC) student from this year, about some patches he’s been working on. He has continued to contribute to FreeBSD beyond the end of GSOC. I also finally met Olivier Certner, who recently started contributing high-quality patches and code reviews for the kernel.

Once the technical difficulties were cleared up, some talks went on as planned. Several FreeBSD Foundation staff members presented an update on Foundation activities; it was nice to see updates on the numerous ongoing intern projects. Following that, Justin Gibbs gave a talk subtitled, “What would you do if you weren’t afraid?”, describing the history of technical accomplishments and “firsts” of the FreeBSD developer community. The unifying theme was projects that required almost blind ambition and the willingness to try something without worrying much about all the ways it could fail. Quite a few examples of this were highlighted, CHERI and the original port of ZFS to FreeBSD being just a couple. Justin then invited us to consider what we would make if we didn’t have to think about failure, and there was some discussion as folks shared various ideas.

A number of us had lunch together in the student cafeteria and caught up. After lunch, Greg Wallace from the Foundation presented the idea of a SWOT analysis, in which an organization’s strengths, weaknesses, opportunities and threats are characterized, so as to provide an objective context for planning future work. The room split into groups and each performed their own SWOT analyses of FreeBSD. During the afternoon I finally met Christos Margiolis (christos@), who has recently finished an internship for the Foundation and who had been my GSOC student in 2022. This was our first time meeting face-to-face despite having worked together on several projects, so it was great to meet him finally. We and several others chatted about various FreeBSD topics for quite a while. At the end of the day, Jordan Hubbard gave a talk describing his career path and describing how he thought the emergence of large language models and other machine learning innovations would transform the software landscape in the near
term.

Day two of the developer summit started with a couple of technical talks. First, Ruslan Bukin presented some of his recent work on CPU tracing, which enables the use of technologies such as ARM CoreSight and Intel PT. These facilities allow the CPU to record its operations in real-time, enabling fine-grained tracing of the CPU’s behavior with relatively little overhead. Ruslan’s implementation, HWT, currently targets ARM but is designed to be extensible. Ruslan also gave some interesting demos in which he ran small command-line utilities (such as uname(1)) with tracing enabled. This demonstrated HWT’s symbol resolution capabilities. HWT is spiritually similar to Christos’ recent Foundation-sponsored work on DTrace, which enables tracing of individual CPU instructions, but has substantially different tradeoffs.

Bojan then presented some work he started for GSOC, specifically a framework for performance benchmarking of the kernel. This was motivated by the challenge of summarizing the performance improvements and impacts of the main feature he developed, a kernel subsystem that performs online memory defragmentation. In particular, such a system necessarily has some performance overhead since it consumes CPU resources, but it (hopefully) confers some benefits by providing more contiguous physical memory to the rest of the system, which can improve the performance of some workloads. Determining the net effect of the feature is thus a tricky task, hence the benchmarking framework.

The schedule for the rest of the day was relatively light, so I spent some time hacking on a personal project which aims to make it easy to run FreeBSD’s regression test suite, and also talking with some other FreeBSD developers about various gaps and the lack of consistency in our development tools.

EuroBSDCon itself took place over the next few days. I attended quite a few talks; some highlights included:

  • Hiroki Sato’s talk on his patches to FreeBSD’s USB stack to enable the use of
    the USB debug capability. This is a hardware feature present in some USB
    controllers which lets one create a communication channel over USB. This
    effectively allows one to get a debugging console for any device with a
    compatible USB controller. This should be very handy for debugging issues
    related to suspend/resume of laptops, which otherwise lack an out-of-band
    channel that one could use for debugging. This work requires a bit of special
    hardware, specifically a USB crossover cable; fortunately, Hiroki had brought
    several which were available for sale after the talk. I had to move pretty
    quickly but managed to snag one.
  • Christos Margiolis’ talk on kinst, a new DTrace provider that he developed
    over the course of GSOC 2022 and his 2023 Foundation internship. I had worked
    with him on this in the past so was already familiar with the technical
    background, but it was great to see him present it to a broader audience. kinst
    is quite a low-level piece of technology and requires some knowledge of CPU
    internals to understand. Christos did an admirable job explaining it to a
    diverse audience. There were many interesting questions and
    discussions after the talk which I participated in.
  • Michael Chiu’s work on xc, a new container runtime targeted at FreeBSD. This
    is a very impressive piece of work that provides several convenient
    features around the management of FreeBSD jails. The talk included quite a
    few demonstrations, for instance using xc to start running MariaDB in a Linux
    jail on FreeBSD. I was inspired by the talk to try using xc for some of my
    own workloads; in particular, it seems like it will be quite useful as a way
    to run automated tests on FreeBSD with minimal setup. I have been working on
    an xc configuration which can be used to run syzkaller on FreeBSD. Overall
    I’m quite excited about Michael’s work.

I also had some chats with Warner Losh about some of his work to enable booting FreeBSD via LinuxBoot. LinuxBoot simplifies the boot process on some platforms by replacing traditional boot firmware with a minimal Linux kernel which is responsible for initializing hardware and then booting the main Linux kernel. Booting FreeBSD this way involves implementing a binary interface called kexec(2); this work was the subject of Warner’s talk. Recently I have been working on a spiritually similar project with quite a bit of technical overlap, so it was helpful to hear about his work and try to find ways to combine efforts
when it comes time to commit our respective projects to FreeBSD.

I would like to thank the FreeBSD Foundation for sponsoring my trip; I had a lot of very productive conversations and learned quite a lot about what others in the FreeBSD community are working on.