July 2017 – Siva’s Capsicum Test Suite Project
As a co-op student at the FreeBSD Foundation, I have the unique opportunity of being encouraged to contribute to other open source projects that help make progress on FreeBSD and its components. One such project that I worked on in July is an analysis of the Capsicum test suite. This test suite targets the Capsicum capability and sandbox framework, which is included in FreeBSD since version 10.0. As described on the University of Cambridge Computer Laboratory website, “Capsicum extends the POSIX API, providing several new OS primitives to support object-capability security on UNIX-like operating systems”. Furthermore, the test suite was initially created by David Drysdale at Google while porting FreeBSD’s reference implementation of the Capsicum framework to Linux.
For this project, I was first assigned to investigate a test case for
pdfork(2) that had been failing for some time. However, upon running the test suite on a FreeBSD 12 CURRENT machine, I noticed that there were several test cases failing with non-trivial error messages and status codes. I discussed these results with Ed Maste, my supervisor, and we determined that the following test results were of interest:
OpenatTest.WithCapability: Open files using pathnames that are relative to a directory with restricted capabilities and contain “..”. The test was failing due to an assertion that every call to
openat()with a pathname containing “..” is invalid. Based on changes made to FreeBSD after the test was written, the assertion should instead be that calling
openat(base_fd, path, ...)with a
pathcontaining “..” should be forbidden only if
pathpoints to a file outside of the base directory represented by
WithFiles.DisallowedFileSyscalls: Execute the
mknod(2)syscall in capability mode with invalid arguments. This test case expected the syscall to return with
ECAPMODE; however, the FreeBSD implementation of the syscall first ran validations against the given arguments before checking for capability. Since the arguments were invalid, it returned with
EINVAL. The POSIX specification confirms that this behavior is permissible, so we decided that the test case must be changed.
Pdfork.OtherUser: Kill a process with
pdkill(2)after changing the current process’s user identifier (UID). This test is currently failing because the kernel disallows sending signals to processes that are not of the same UID as the current process. After a short discussion, we determined that this behavior is desired, and we decided that the test case should be modified to remove the call to
pdkill()after changing the current process’s UID.
Pdfork.Simple(related tests omitted): Test basic operations using process descriptors on processes created with the
pdfork(2)syscall. These tests were failing because of checks to determine if a certain process is a zombie. These tests used the
ps(1)tool, but upon further inspection, I noticed that
ps -p <zombie-pid>does not report zombie processes. The call to
ps(1)was modified in the test suite to ensure that the
Pdfork.*test cases pass, but the kernel’s behavior, with respect to reporting zombie processes, remains the same until further discussion.
PosixMqueue.CapMode: Test basic operations using
kmq_*()syscalls on POSIX message queues in capability mode. In order to run this test case, I needed to dynamically load the
mqueuefs(5)kernel module. After doing this, I noticed that the newly-loaded syscalls returned
ECAPMODEwhile in capability mode. Remote debugging these syscalls with GDB showed that their respective syscall entries (
sysents) were not being initialized with the correct flags, so I submitted a kernel patch (which has now been committed to FreeBSD) and the tests now pass.
PipePdfork.Close: Test the behavior of a closed process descriptor. This test is currently failing because of a
waitpid(2)call to a process that had its last process descriptor closed. I noticed that the kernel reparents a process to the
initprocess after the last
close(2)on its process descriptor. Ed and I decided that this behavior should be further discussed at the biweekly Capsicum conference call, and the verdict was that the kernel’s behavior is appropriate. The test case must be changed to reflect this behavior.
To run the Capsicum test suite, follow these steps on a FreeBSD machine:
Clone the Capsicum test suite git repository and navigate into the directory
$ git clone https://github.com/google/capsicum-test.git && cd capsicum-test
Build the test suite
Run all tests that do not require you to be root
Run all tests that require root
$ sudo ./capsicum-test
Run a specific test case with verbose log messages
$ ./capsicum-test -v --gtest_filter="Pdfork.Simple"
For more help on how to use the test suite, run
$ ./capsicum-test --help.
 Full discussion can be found here
– Contributed by Siva Mahadevan