As Core Team 11 readies to take over management of the FreeBSD Project, we interviewed members of Core Team 10 for their observations on a productive two years.

“Democracy is a slow process of stumbling to the right decision instead of going straight forward to the wrong one.” — Anonymous

For 20 years, the FreeBSD Project has elected a group of 9 project Committers to serve two-year terms on the Core Team (CT for short). All active developers in the project are eligible to vote for the self-nominated candidates. The project website describes the role of the Core Team as follows:

The FreeBSD Core Team constitutes the project’s “Board of Directors”, responsible for deciding the project’s overall goals and direction as well as managing specific areas of the FreeBSD project landscape. 

Core.10, which concluded in July ‘20, made several process decisions to tackle what they perceived to be a heavy workload:

The following people served on Core.10. The number represents the times each has served on Core, with bold indicating that Core.10 represents a consecutive term:

The composition of Core.10 reflects the balance between new ideas and institutional knowledge that contribute to the project’s steady innovation. Warner Losh, who served on the first Core Team from 2000-2002, and on a total of six, put it this way:

It’s important to have a mix of people with new ideas asking “what should we do?”, and people who have been involved longer to reflect why things are done the way they are now. Having this balance protects against allowing the way things are done now getting in the way of improvement. 

The FreeBSD Foundation interviewed several members of Core.10 to understand what made it tick and some of the strategies the team pursued to successfully tackle challenging topics like migrating to Git and updating the CoC. What emerges is that this team’s focus, collaboration, and transparency were essential to it achieving as much as it did. 

Multiple Core.10 members we interviewed, including some that have served on Core before, observed that this team’s decision to focus on a small number of big initiatives was key. 

Representing opposite ends of the new and long-serving spectrum, Jeff Roberson (1) and John Baldwin (6) offered strikingly-similar takes on the importance of prioritization. “Figure out early the small set of things you are going to drive to completion,” said Jeff. John adds “Don’t stuff the agenda, instead pick one to three big things per term. Otherwise, it’s too easy to get spread too thin and not make progress.”

Of course, focusing on a smaller set of big projects requires consensus onwhich things to pursue. Which brings us to one of Core.10’s other big initiatives – the annual community survey. Echoing comments of several other members, Allan Jude notes “the annual user survey, now in its second round, has been very useful to discover how people are using FreeBSD. For instance, it helps us answer long-debated questions like ‘do people actually want more frequent releases?’” 

The Core.10 Secretary, Joseph Mingrone adds “the community and developer surveys were a big step towards becoming more data-driven and transparent.” For purposes of focusing on the small number of bigger projects to undertake, Jeff Roberson found that the survey revealed “some nearly universal common themes. There was lots of common ground.”

In other cases, Core.10 members felt that the election had provided them with a mandate. Observing that this is uncommon in his experience on other Cores, John Baldwin remarked “rarely does Core come into their term with a mandate. This time we did. The CoC was a clear mandate.”

Let’s unpack the move to Git and the CoC update to understand how Core.10 operated.

The forthcoming report from the Git Working Group will provide the definitive rationale for this move, but the general consensus from Core.10 interviews and conference talks is that the lack of Git was an impediment to attracting new developers to the project. SVN also makes code collaboration more difficult, many Core.10 members shared. In Allan Jude’s assessment, the latter was the dominant motivator. 

Git is seen as a better collaboration tool, and that’s why we need it. With Git, you can add an extra remote anywhere on the internet and collaborate with anyone, even if they don’t have a FreeBSD account. This makes it much easier to collaborate than with the SVN-based system we have today. Git also allows you to create a series of commits that you can move around as a unit and rebase and so on. SVN is more monolithic. Git also has advantages for CI because it takes patches in context.

Core.10 formalized the Working Group structure to charter people to undertake significant projects that don’t fit neatly into existing teams. The most notable instance of WGs in action is Git. 

From Warner’s perspective, “the Git Working Group, with Ed Maste leading the charge, really did the work and Core provided oversight, asking a lot of questions. This turned out to be a good model. With Git, the hard part was figuring out how to do the transition. Phase 1 is a big leap sideways that allows us to continue to innovate, but now we do it on Git. We can add capabilities and features as we go. As with all community decisions, the art of politics is the art of the possible. We asked, and answered, ‘what’s possible NOW? What’s version 1?”

Revealing an advantage of an engaged community with mature use cases, Brooks Davis shared that the CHERI project he works on served as an excellent, and challenging, use case to smoke out SVN —> Git migration issues. Kris Moore also shared that when iXsystems changed their TrueNAS product from subversion to Git, community engagement and code submissions from users increased. FreeBSD is a pragmatic community, and with “perfect the enemy of good,” committing to a transition plan from SVN to Git was agreed upon as the next logical step. After the initial migration to Git, the Core Team expects future Working Groups to spring up to further enhance the developer and community contribution experience.

A significant portion of the community was dissatisfied with the existing Code of Conduct, several interviewees observed. As one CT member put it, “FreeBSD strives to be both an inclusive and a fair community. The CoC sits at the heart of these goals as a way to establish community norms for interaction that are easy to understand, follow, and enforce.” Others observed that interpersonal issues tend to dominate the work of Core, and this seemed to be particularly so for Core.10. Although this kind of work can be draining, the fact that CT has nine members ensures that there is enough of a bench to alleviate burn out. 

CT undertook a thorough process of researching how the project arrived at the current CoC, soliciting feedback from the community, discussing the desired end state, and agreeing that the project would be best served by tailoring an existing “field-proven” CoC from another project. To one Core.10 member, the CoC work felt like democracy in action. “The CT came in with a community mandate to address an issue, they committed to it, pursued a transparent and inclusive process, and followed through.” At the time of this writing, the community was in the process of voting for their preferred CoC among two options put forward by CT. 

It’s been said that strategy is as much about what you don’t do as what you do. For Core.10, delegating work with good processes around accountability and follow-through has been essential to its successful term. This CT formalized the Working Group structure and put it into action with the Git WG. 

Allan Jude offers his take: “For a while, Core has tried to improve the process for someone to propose a bigger idea and get momentum, similar to Python PIP. An earlier effort around the FreeBSD Community Proposal didn’t catch on, I think because it put too much burden on the person requesting the change. This Core adopted Working Groups, where a person or group proposes the WG, the person or group creates a charter, then an operating plan, and Core reviews/approves the proposal giving the group an implicit blessing to pursue a potentially disruptive change to improve the community. This seems to be the right model.”

The other benefit of the WG structure, noted some Core.10 members we interviewed, is that it moves the project away from a so-called “culture of heroes.” While WGs might not eliminate the need for occasional acts of heroism, a community that depends entirely on heroes for big change leaves itself unnecessarily exposed to the risk of burn-out as well as the “tyranny of the minority.”

Recognizing the amount of work ahead of them, early on in the term and at the behest of Secretary Joseph Mingrone, the CT decided to double meeting frequency from once to twice a month. And for a time while the size of the CT backlog was high, the team met weekly. 

Admitting that this was a significant investment, Brooks Davis notes “the higher meeting frequency helped us make a lot of progress. And it wasn’t just about efficiency, it also helped the team bond. We certainly didn’t always agree, but the team was very collegial – we could and did disagree without being disagreeable. And for a CT working on things like the CoC, our collegiality was important to set the tone for the project.”

Encapsulating the importance of collaboration and collegiality, Warner offers this management observation:

A big reason why this Core was so effective was because we understood we needed to work together, even though we held different perspectives and different priorities. In terms of working with other teams in the project, this means dealing with politics between different groups. The goal is not to prevent politics – that’s impossible. Rather, the goal is to prevent toxic politics. And despite tackling a highly-charged topic like the CoC, we succeeded. As a group, we were able to facilitate constructive engagement with one another and across the project. In a management role like FreeBSD Core where you are working with volunteers, something you learn quickly is that giving ultimatums doesn’t work. Instead, it’s about using the power of persuasion to help shape direction.

In his BSDCan 2019 Core.10 Status Presentation, Sean Chittendon highlighted the following examples of strong collaboration:

In addition to meeting more frequently, Core.10 also published more detailed and more frequent notes. This task falls to the Secretary. In addition, the Secretary plays a vital role as a project manager and helps Core stay organized and efficient.

With few exceptions, Core.10 Secretary Joseph Mingrone sent out detailed reports to all FreeBSD Committers, and also provided public Quarterly reports. Joseph, who has been a FreeBSD user since the 1990s and a Ports Committer for a few years, felt the more frequent and detailed reports were important to remove any perception of secrecy that might diminish trust in the CT and the project. CT members we spoke with unanimously hailed the efforts of Joseph as essential to the team’s success and counseled future Core Teams to recruit a Secretary with Joseph’s commitment. 

Some Core.10 members we spoke to expressed interest in exploring ways to make the work of CT and Working Groups even more transparent. One option offered was to potentially open Core meetings, at least in part, to a broader audience. Some noted that this model, in place at similar projects like OpenZFS, where as much of the meeting as possible is open, and only the portion dealing with matters for which privacy is needed are closed, may be worthwhile to explore.

Other Core.10 members suggested that opening Working Groups to greater participation among contributors and users could be beneficial. 

Reaching twenty years for any technology organization is a remarkable feat, especially when one considers the average lifespan of a business is 10 years. Enduring technology companies – whether its Apple, IBM, or even Microsoft – all have radically reinvented themselves, in many cases multiple times, to stay relevant as the world around us changes. In researching this article, there emerges a clear feeling among some that the FreeBSD project and community stand on the precipice of a similar reinvention. One can look at Core.10’s embrace of Git as an important example of the tenacity of a strong democratic community determined to remain relevant for years to come.