MeetBSD 2018 Trip Report: Mark Johnston

Thanks to the FreeBSD Foundation, I was able to travel to the Bay Area to attend MeetBSD 2018, and the attached FreeBSD developer summit.  This was my second time attending the conference after the last one in Berkeley in 2016; MeetBSD 2018 was held at Intel’s Santa Clara campus.  As usual, the conference was a welcome opportunity to see colleagues and meet newcomers to the FreeBSD community, and I’m grateful to the Foundation for having made it possible for me to attend.

Upon arriving I quickly discovered that Santa Clara is rather inhospitable without a car – my motel was about a thirty-minute walk from the conference venue.  As a result, I didn’t end up doing any exploring, but the weather was a pleasant change from the intensifying cold in Toronto.  My first day, I attended the one-day FreeBSD developer summit that immediately preceded the conference itself.  The summit opened with an introduction by John Baldwin, and the bulk of the rest of the day was devoted to talks by a few FreeBSD developers.  Ravi Pokala gave the first talk, about the history of power failure handling in Panasas’ FreeBSD-based storage appliance.  All such appliances need to solve the problem of avoiding data loss when power to the device is lost, and until recently storage vendors all had to have custom, proprietary hardware to do so.  In the past few years, however, NVDIMM technology has become increasingly standard, and it provides a mechanism to save the contents of memory to NVRAM following a power loss.  When the system resumes operation, it can recover the previously saved contents of memory and continue without having lost anything.  Ravi’s talk described the components of a Panasas filesystem which makes use of NVDIMMs, and he went into detail about the software interface between NVDIMMs and the operating system.  Having worked for a storage vendor myself, I found the details of Ravi’s description of the filesystem internals to be quite interesting.  Following Ravi was Ed Maste’s talk about the considerations involved in moving the FreeBSD project from Subversion to Git, a topic that’s arisen many times in the past few years and is generally guaranteed to generate some contentious debate.  Regardless of one’s personal taste in version control systems, it seems pretty clear that Git has come to dominate the open-source landscape, and many FreeBSD developers, myself included, use Git to manage their work.  Ed’s talk explored various commonly raised arguments and counter-arguments for and against the transition to Git, in many cases either debunking them or constructively proposing mechanisms to minimize the disruption that would be caused by a switch to Git.  One point that resonated with me was the claim on svnvsgit.com that many large open-source projects continue to use Subversion; the site lists LLVM and FreeBSD, as of 2016, as prime examples, but LLVM is currently switching to Git, and FreeBSD is clearly considering doing so.  Ed also considered a hybrid model in which Git becomes the official front-end for FreeBSD development, with various supported workflows for contributors, while the Subversion repository remains the source of truth.  Such a model would seem to be a practical compromise, at least for some time, but has the potential to blur the distinction between FreeBSD contributors and committers; Ed thoughtfully considered the ramifications of this in his talk.  Following Ed’s talk was my own, on recent work done on the FreeBSD kernel’s memory management subsystem to improve performance on NUMA systems and many-CPU systems in general.  I won’t describe it here, but would like to encourage anyone interested in these topics to watch the recording on YouTube and email me with questions or comments.  The developer summit ended with the usual Have/Need/Want session, which delivered a mix of very useful and interesting project ideas and implementations as well as some less practical and rather entertaining proposals.  The list can be found here (https://wiki.freebsd.org/DevSummit/201810).

Friday morning saw the beginning of the conference proper.  As is my tendency during conferences, I alternated between listening to the talks and discussing projects with other developers.  This time, the latter consisted mostly of helping debug some issues with a new PowerPC pmap (page table management) implementation for use on POWER9 systems, done by Matt Macy; the new pmap is a port of amd64’s pmap, and as such required quite a few non-obvious tweaks to get working on a foreign CPU architecture.  At the beginning of the conference he had it at a state where the kernel would panic shortly after stating init, and by the end it was possible to boot to multi-user mode, get a shell, and verify that transparent superpage support was apparently working.  Although I did little other than provide a few tips, it was neat to learn a bit about page table management on the POWER architecture and to see steady progress as various bugs were squashed.

The first talk that I attended was by Ben Widawsky, a FreeBSD committer and developer at the Intel Open Source Technology Center (OTC).  This year, the OTC created an internal team specifically devoted to FreeBSD, with the goal of “Making Intel products run better on FreeBSD.”  Ben’s talk gave an overview of Intel’s past contributions to FreeBSD and how, in light of the Meltdown and Spectre vulnerabilities, it has become increasingly clear that Intel needs to be more proactive in collaborating with the FreeBSD community.  To that end, the aforementioned team was created, and since the FreeBSD Foundation (and in particular Konstantin Belousov) have been handling the task of mitigating the various security holes that have surfaced in Intel CPUs in the past year, the team has been focusing more on improving platform support.  In the second half of his talk Ben discussed his efforts towards improving power management and suspend/resume support on FreeBSD and gave thorough technical descriptions of some of the stumbling blocks that he had run into thus far.  After lunch I listened to Kamil Rytarowski’s talk on the implementation and use of LLVM’s sanitizers in NetBSD.  This is a topic I’ve long been meaning to learn more about, so the in-depth description of the pitfalls and required infrastructure was quite helpful; I’ve since referred to Kamil’s slides a number of times while experimenting with UBSan on FreeBSD.  After the talks ended I had dinner and chatted with a few other FreeBSD developers.  In particular, I talked with John Baldwin about a number of ongoing projects: RISC-V platform support and an effort to add live migration support to bhyve.

The second and final day of the conference consisted of more talks.  (In addition to some more pmap debugging!)  The first that I attended was by Devin Teske, who presented her creation, dwatch(1), and some rather novel methods of visualizing the data that it can extract from a running FreeBSD system using DTrace. I was already somewhat familiar with dwatch, having spent some time reviewing its code and using it myself occasionally. Devin’s demo-based talk gave examples using Vizceral, to visualize network traffic graphs in real-time using dwatch as a backend; the results were quite dazzling, and I’d encourage any readers to look up the talk on YouTube.  I tend to gravitate towards text-based interfaces and displays in general, but the talk gave me some ideas on how I might apply these tools to illustrate activity in various subsystems of the kernel, both to gain insight into their behavior and as a teaching tool.  After lunch I attended Mariusz Zaborski’s talk on Capsicum, the userspace sandboxing framework in FreeBSD.  This is a topic near to my heart, especially seeing as I’ve been working with it myself for the past couple of months.  The talk was mostly an introduction to various pieces of the framework, but Mariusz also discussed some recent work of his that would be landing in FreeBSD head following the conference; I was quite glad to hear about it, since it was directly related to some problems that I’ve been working on, and following the talk I got to further discuss some of that work with him.  Over dinner we talked about the planned demise of GCC 4.2 and various other plans once FreeBSD 12.0 ships. A couple of people had questions about work I had done to support so-called early microcode loading on Intel platforms a few months ago, so we were able to have a productive discussion about that.  Following dinner I made the long trek back to my motel, and I flew out of SFO the following morning.

— contributed by Mark Johnston