Implementing xlocale APIs Project Update

The following project update was written by David Chisnall who received a grant from us to implement xlocale APIs to enable porting libc++. We’re pleased that the project is almost completed! It’s traditional to start this sort of thing by telling you who I am. I started using FreeBSD around 2001. At the time, I’d used Linux but switched to FreeBSD because it sounded like it worked correctly – I could have xmms playing music, my IM and email clients notifying me of new messages, and BZFLag making gunshot noises all at the same time. Apparently, ten years later, this still doesn’t work reliably on Linux…

I got involved with clang via a somewhat indirect route. I’m a member of the core team of the Étoilé project, which aims to build a (BSD licensed) desktop environment on top of GNUstep. I grew increasingly frustrated with the level of Objective-C support in GCC, which included shipping one release with Objective-C completely broken and displaying no progress towards supporting the Objective-C 2 extensions that were about 5 years old at the time. I looked at the code, but it was an incomprehensible mess of spaghetti.

Apple had just released a new compiler front end (clang) that had Objective-C parsing mostly finished, but code generation missing. I started poking the clang code to try to support the GCC Objective-C runtime, and a few weeks later had a working Objective-C 2 compiler. I then grew frustrated with the limitations (including the license) of the GCC Objective-C runtime and wrote a more modern (MIT licensed) replacement. Clang now supports both and with the new runtime is at feature parity with Apple’s implementation of the language.

While hacking on clang – which I do on FreeBSD – I fixed various FreeBSD-related bugs. This put me in contact with Roman Divacky, who had been working on importing clang into the base system. This is an important task, because FreeBSD currently uses the last GPLv2 version of GCC as the system compiler. Although this release seems less buggy than subsequent ones, it is now over 4 years old and is no longer supported upstream. It won’t, for example, get any of the features of C1x or C++11.

The compiler is only half of the problem. The other half is the standard library. For C, this isn’t an issue: FreeBSD has had its own C standard library implementation since before it was FreeBSD. For C++, it’s a bigger problem. FreeBSD currently ships with GNU libstdc++, which has undergone the same sort of license change as GCC, leaving FreeBSD stuck with an old version.

A good candidate for replacement is libc++, developed as part of the LLVM project and available under UIUC and MIT licenses. This has a few dependencies. One is a low-level C++ ABI library, which implements the dynamic parts of C++ such as exception handling and run-time type information. I’d written an implementation of this for PathScale, and the FreeBSD and NetBSD Foundations jointly paid for it to be open sourced. I’ve since extended it with some additional features required to support C++11, which includes an std::exception_ptr object that allows exceptions to be passed between threads.

The other dependency is the C standard library. Libc++ was written by Apple developers (Apple is in the same situation as FreeBSD with regards to the GPLv3) and so uses some features of Darwin libc that are not portable. Specifically, Darwin libc has a convenient set of extensions to localisation: xlocale. This extends the POSIX 2008 per-thread locale APIs (missing in FreeBSD) to provide a set of _l variants of locale-aware libc functions that use a specific locale, rather than the global one.

My recent work, sponsored by the FreeBSD Foundation, has been to implement the missing xlocale APIs. This is now mostly done and pending code review. With this and the new tweaks to libcxxrt, it’s now possible to build libc++ on FreeBSD and most of the tests pass.

Most of the remaining test failures are in the header. This defines a lot of complex atomic operations and requires a lot of compiler support. Eli Friedman has been working on adding this support in clang, and with his latest patch applied 25 of the 52 atomic tests pass. There are still a few remaining failures:

– 27 caused by clang not fully supporting the atomic operations yet
– 3 caused by clang not fully supporting the C++11 type-trait intrinsics
– 20 that I don’t think are real failures – they’re caused by the VM where I’m running the tests not having sufficiently fine-grained time reporting for the thread operation timeout tests to work properly
– 1 is caused by FreeBSD lacking the C1x quick_exit() APIs.
– 2 caused by FreeBSD lacking the uchar.h header

For comparison, Howard Hinnant, the libc++ lead developer, just sent me a list of the failures on OS X. On FreeBSD, 4271 tests pass, 53 fail. On OS X 4253 pass, 71 fail. This is looking very promising for an entirely GNU-free C++ stack.