Contributing FreeBSD Documentation

Quality documentation is crucial to the success of FreeBSD, submitting documentation is one of the easiest ways to contribute to the project and anyone is welcome to submit! Willingness to contribute is the only membership requirement.

1. First Steps

The first step to contribute is to subscribe to the FreeBSD documentation project mailing list. This is a fantastic resource for any questions or problems involving the documentation.

Install the textproc/docproj meta-package and Subversion. This meta-package installs all of the software needed to edit and build FreeBSD documentation. The Subversion package is needed to obtain a working copy of the documentation and generate patches with.
# pkg install docproj subversion
Optional: to generate PDF documentation, install the textproc/fop package as it is not installed by default by textproc/docproj.
# pkg install fop
Install a local working copy of the documentation from the FreeBSD repository in ~/doc (see Chapter 3, The Working Copy).
% svn checkout https://svn.FreeBSD.org/doc/head ~/doc
Configure the text editor:
- Word wrap set to 70 characters.
- Tab stops set to 2.
- Replace each group of 8 leading spaces with a single tab. Specific editor configurations are listed in Chapter 15, Editor Configuration.
Update the local working copy:
% svn up ~/doc
Edit the documentation files that require changes. If a file needs major changes, consult the mailing list for input. References to tag and entity usage can be found in Chapter 8, XHTML Markup and Chapter 9, DocBook Markup.
After editing, check for problems by running:
% igor -R filename.xml | less -RS
Review the output and edit the file to fix any problems shown, then rerun the command to find any remaining problems. Repeat until all of the errors are resolved.
Always build-test changes before submitting them. Running make in the top-level directory of the documentation being edited will generate that documentation in split HTML format. For example, to build the English version of the Handbook in HTML, run make in the en_US.ISO8859-1/books/handbook/ directory.
When changes are complete and tested, generate a “diff file”:
% cd ~/doc % svn diff > bsdinstall.diff.txt
Give the diff file a descriptive name. In the example above, changes have been made to the bsdinstall portion of the Handbook.
Submit the diff file using the web-based Problem Report system. If using the web form, enter a Summary of [Patch] short description of problem. Select the Component Documentation. In the Description field, enter a short description of the changes and any important details about them. Use the [ Add an attachment ] button to attach the diff file. Finally, use the [ Submit Bug ] button to submit your diff to the problem report system.

2. Installing Tools

Required Tools

Install textproc/docproj from the Ports Collection. This meta-port installs all the applications required to do useful work with the FreeBSD documentation. Some further notes on particular components are given below.
FreeBSD documentation uses several Document Type Definitions (DTDs) and sets of XML entities. These are all installed by the textproc/docproj port.XHTML DTD (textproc/xhtml)
- XHTML is the markup language of choice for the World Wide Web, and is used throughout the FreeBSD web site.
- DocBook is designed for marking up technical documentation.

Optional Tools

These applications are not required, but can make working on the documentation easier or add capabilities.
- Vim (editors/vim)
- Emacs (editors/emacs), A popular editor for working with XML and derived documents, like DocBook XML.

3. Editing and Maintaining a Working Copy

The working copy is a copy of the FreeBSD repository documentation tree downloaded onto the local computer. Changes are made to the local working copy, tested, and then submitted as patches to be committed to the main repository. A full copy of the documentation tree can occupy 700 megabytes of disk space. Allow for a full gigabyte of space to have room for temporary files and test versions of various output formats.

In order to manage the FreeBSD documentation files, Subversion should be installed from packages.
# pkg install subversion

Manual Pages

FreeBSD documentation for commands and configuration files can be found within the manual page collection. These consist of two repositories: doc for books and articles, and base for operating system and manual pages. These repositories can contain multiple versions of documentation and source code. New modifications are almost always made to the latest version: head.

In order to edit manual pages, base will need to be checked out separately.

Choosing a Directory

FreeBSD documentation is traditionally stored in /usr/doc/, and system source code with manual pages in /usr/src/. These directory trees are relocatable, and users may want to put the working copies in other locations to avoid interfering with existing information in the main directories. The examples that follow use ~/doc and ~/src, both subdirectories of the user’s home directory.

Check Out a Copy

A download of a working copy from the repository is called a checkout, this can be done with svn checkout. For example, to checkout the latest version (head) of the documentation tree.

% svn checkout https://svn.FreeBSD.org/doc/head ~/doc

And to checkout the source code for manual pages:

% svn checkout https://svn.FreeBSD.org/base/head ~/src

Updating the Working Copy

The FreeBSD repository is frequently updated, even a short time after the initial checkout, changes may have already been made creating differences between the local working copy and the main FreeBSD repository. To update the local version, use svn update on the directory containing the local working copy. Getting into the habit of doing this often before editing document files will be helpful.

% svn update ~/doc

Reverting Changes

Sometimes it turns out that changes were not necessary after all, or the writer just wants to start over. Files can be “reset” to their unchanged form with svn revert. For example, to erase the edits made to chapter.xml and reset it to unmodified form:

% svn revert chapter.xml

Making a Diff

After edits to a file or group of files are completed, the differences between the local working copy and the version on the FreeBSD repository must be collected into a single file for submission. These diff files are produced by redirecting the output of svn diff into a file:

% cd ~/doc
% svn diff > doc-fix-spelling.diff

Give the file a meaningful name that identifies the contents. The example above is for spelling fixes to the whole documentation tree.

If the diff file is to be submitted with the web “Submit a FreeBSD problem report” interface, add a .txt extension to give the earnest and simple-minded web form a clue that the contents are plain text.

Be careful: svn diff includes all changes made in the current directory and any subdirectories. If there are files in the working copy with edits that are not ready to be submitted yet, provide a list of only the files that are to be included:

% cd ~/doc
% svn diff disks/chapter.xml printers/chapter.xml > disks-printers.diff

4. Documentation Build Process

Rendering DocBook into Output

Different types of output can be produced from a single DocBook source file. The type of output desired is set with the FORMATS variable. A list of known formats is stored in KNOWN_FORMATS:

% cd ~/doc/en_US.ISO8859-1/books/handbook % make -V KNOWN_FORMATS

`FORMATS` Value	File Type	Description
`html`	HTML, one file	A single `book.html` or `article.html`.
`html-split`	HTML, multiple files	Multiple HTML files, one for each chapter or section, for use on a typical web site.
`pdf`	PDF	Portable Document Format

The default output format can vary by document, but is usually html-split. Other formats are chosen by setting FORMATS to a specific value. Multiple output formats can be created at a single time by setting FORMATS to a list of formats.

FreeBSD Documentation Build Toolset

These are the tools used to build and install the FDP documentation.

The primary build tool is make(1), specifically Berkeley Make.
Package building is handled by FreeBSD’s pkg-create(8).
gzip(1) is used to create compressed versions of the document. bzip2(1) archives are also supported. tar(1) is used for package building.
install(1) is used to install the documentation.

5. Understanding Makefiles in the Documentation Tree

There are three main types of Makefiles in the FreeBSD Documentation Project tree.

Subdirectory Makefiles simply pass commands to those directories below them.
Documentation Makefiles describe the documents that are produced from this directory.
Make includes are the glue that perform the document production, and are usually of the form doc.xxx.mk.

Subdirectory `Makefile`s

These Makefiles usually take the form of:

SUBDIR =articles
SUBDIR+=books

COMPAT_SYMLINK = en

DOC_PREFIX?= ${.CURDIR}/..
.include "${DOC_PREFIX}/share/mk/doc.project.mk"

The first four non-empty lines define the make(1) variables SUBDIR, COMPAT_SYMLINK, and DOC_PREFIX.

The SUBDIR statement and COMPAT_SYMLINK statement show how to assign a value to a variable, overriding any previous value.

The second SUBDIR statement shows how a value is appended to the current value of a variable. The SUBDIR variable is now articles books.

The DOC_PREFIX assignment shows how a value is assigned to the variable, but only if it is not already defined. This is useful if DOC_PREFIX is not where this Makefile thinks it is – the user can override this and provide the correct value.

What does it all mean? SUBDIR mentions which subdirectories below this one the build process should pass any work on to.

COMPAT_SYMLINK is specific to compatibility symlinks (amazingly enough) for languages to their official encoding (doc/en would point to en_US.ISO-8859-1).

DOC_PREFIX is the path to the root of the FreeBSD Document Project tree. This is not always that easy to find, and is also easily overridden, to allow for flexibility. .CURDIR is a make(1) builtin variable with the path to the current directory.

The final line includes the FreeBSD Documentation Project’s project-wide make(1) system file doc.project.mk which is the glue which converts these variables into build instructions.

Documentation `Makefile`s

These Makefiles set make(1) variables that describe how to build the documentation contained in that directory.

Here is an example:

MAINTAINER=nik@FreeBSD.org

DOC?= book

FORMATS?= html-split html

INSTALL_COMPRESSED?= gz
INSTALL_ONLY_COMPRESSED?=

# SGML content
SRCS=  book.xml

DOC_PREFIX?= ${.CURDIR}/../../..

.include "$(DOC_PREFIX)/share/mk/docproj.docbook.mk"

The MAINTAINER variable allows committers to claim ownership of a document in the FreeBSD Documentation Project, and take responsibility for maintaining it.

DOC is the name (sans the .xml extension) of the main document created by this directory. SRCS lists all the individual files that make up the document. This should also include important files in which a change should result in a rebuild.

FORMATS indicates the default formats that should be built for this document. INSTALL_COMPRESSED is the default list of compression techniques that should be used in the document build. INSTALL_ONLY_COMPRESS, empty by default, should be non-empty if only compressed documents are desired in the build.

The DOC_PREFIX and include statements should be familiar already.

6. FreeBSD Documentation Project Make Includes

make(1) includes are best explained by inspection of the code. Here are the system include files:

doc.project.mk is the main project include file, which includes all the following include files, as necessary.
doc.subdir.mk handles traversing of the document tree during the build and install processes.
doc.install.mk provides variables that affect ownership and installation of documents.
doc.docbook.mk is included if DOCFORMAT is docbook and DOC is set.

`doc.project.mk`

By inspection:

DOCFORMAT?=	docbook
MAINTAINER?=	doc@FreeBSD.org

PREFIX?=	/usr/local
PRI_LANG?=	en_US.ISO8859-1

.if defined(DOC)
.if ${DOCFORMAT} == "docbook"
.include "doc.docbook.mk"
.endif
.endif

.include "doc.subdir.mk"
.include "doc.install.mk"

Variables

DOCFORMAT and MAINTAINER are assigned default values, if these are not set by the document make file.

PREFIX is the prefix under which the documentation building tools are installed. For normal package and port installation, this is /usr/local.

PRI_LANG should be set to whatever language and encoding is natural amongst users these documents are being built for. US English is the default.

Note:

PRI_LANG does not affect which documents can, or even will, be built. Its main use is creating links to commonly referenced documents into the FreeBSD documentation install root.

Conditionals

The .if defined(DOC) line is an example of a make(1) conditional which, like in other programs, defines behavior if some condition is true or if it is false. defined is a function which returns whether the variable given is defined or not.

.if ${DOCFORMAT} == "docbook", next, tests whether the DOCFORMAT variable is "docbook", and in this case, includes doc.docbook.mk.

The two .endifs close the two above conditionals, marking the end of their application.

`doc.subdir.mk`

This file is too long to explain in detail. These notes describe the most important features.

Variables

SUBDIR is a list of subdirectories that the build process should go further down into.
ROOT_SYMLINKS is the name of directories that should be linked to the document install root from their actual locations, if the current language is the primary language (specified by PRI_LANG).
COMPAT_SYMLINK is described in the Subdirectory Makefile section.

Targets and Macros

Dependencies are described by target: dependency1 dependency2 ... tuples, where to build target, the given dependencies must be built first.

After that descriptive tuple, instructions on how to build the target may be given, if the conversion process between the target and its dependencies are not previously defined, or if this particular conversion is not the same as the default conversion method.

A special dependency .USE defines the equivalent of a macro.

_SUBDIRUSE: .USE
.for entry in ${SUBDIR}
	@${ECHO} "===> ${DIRPRFX}${entry}"
	@(cd ${.CURDIR}/${entry} && \
	${MAKE} ${.TARGET:S/realpackage/package/:S/realinstall/install/} DIRPRFX=${DIRPRFX}${entry}/ )
.endfor

In the above, _SUBDIRUSE is now a macro which will execute the given commands when it is listed as a dependency.

What sets this macro apart from other targets? Basically, it is executed after the instructions given in the build procedure it is listed as a dependency to, and it does not adjust .TARGET, which is the variable which contains the name of the target currently being built.

clean: _SUBDIRUSE
	rm -f ${CLEANFILES}

In the above, clean will use the _SUBDIRUSE macro after it has executed the instruction rm -f ${CLEANFILES}. In effect, this causes clean to go further and further down the directory tree, deleting built files as it goes down, not on the way back up.

Provided Targets

install and package both go down the directory tree calling the real versions of themselves in the subdirectories (realinstall and realpackage respectively).
clean removes files created by the build process (and goes down the directory tree too). cleandir does the same, and also removes the object directory, if any.

7. The Website

The FreeBSD website is included in the FreeBSD documentation and is stored in the en_US.ISO8859-1/htdocs subdirectory of the document tree directory, ~/doc.

Environmental Variables

DOCDIR
- This specifies where web site files are to be installed.
- Variable can be set with env(1) or using the default method of setting variables (setenv for csh(1) or export for sh(1))
ENGLISH_ONLY
- Default is undefined, will build and include all translations. Can be set to ENGLISH_ONLY=YES to only use english documents.
WEB_ONLY
- Default is undefined, will build books and articles in addition to the web site. Can be set to WEB_ONLY=YES to build only HTML pages from the en_US.ISO8859-1/htdocs directory
WEB_LANG
- Default is undefined, will build and include all translations. Can be set to a space-separated list of languages to be included. Use the directory names in the document root directory.
- If a user wanted to include the German and French documents, for example:

      WEB_LANG="de_DE.ISO8859-1 fr_FR.ISO8859-1"

Building and Installing the Web Pages

After obtaining the documentation and web-site source files, the web site can be built.

An actual installation of the web site is run as the root user because the permissions on the web server directory will not allow files to be installed by an unprivileged user. For testing, it can be useful to install the files as a normal user to a temporary directory.

In these examples, the web site files are built by user jru in their home directory, ~/doc, with a full path of /usr/home/jru/doc.

Build the Full Web Site and All Documents

% cd ~/doc/en_US.ISO8859-1/htdocs/
% make all

Build Only the Web Site in English

% cd ~/doc/en_US.ISO8859-1/htdocs/
% env DOCDIR=/tmp/www make ENGLISH_ONLY=yes WEB_ONLY=yes all install

After the site has been built as shown above, a modified main page can be viewed with:

% firefox /tmp/www/data/index.html

After building the site as shown above, this /usr/local/etc/apache24/httpd.conf can be used with www/apache24:

# httpd.conf for testing the FreeBSD website
Define TestRoot "/tmp/www/data"

# directory for configuration files
ServerRoot "/usr/local"

Listen 80

# minimum required modules
LoadModule authz_core_module libexec/apache24/mod_authz_core.so
LoadModule mime_module libexec/apache24/mod_mime.so
LoadModule unixd_module libexec/apache24/mod_unixd.so
LoadModule cgi_module libexec/apache24/mod_cgi.so
LoadModule dir_module libexec/apache24/mod_dir.so

# run the webserver as user and group
User www
Group www

ServerAdmin you@example.com
ServerName fbsdtest

# deny access to all files
<Directory />
    AllowOverride none
    Require all denied
</Directory>

# allow access to the website directory
DocumentRoot "${TestRoot}"
<Directory "${TestRoot}">
    Options Indexes FollowSymLinks
    AllowOverride None
    Require all granted
</Directory>

# prevent access to .htaccess and .htpasswd files
<Files ".ht*">
    Require all denied
</Files>

ErrorLog "/var/log/httpd-error.log"
LogLevel warn

# set up the CGI script directory
<Directory "${TestRoot}/cgi">
    AllowOverride None
    Options None
    Require all granted
    Options +ExecCGI
    AddHandler cgi-script .cgi
</Directory>

Include etc/apache24/Includes/*.conf

Start the web server with

# service apache24 onestart

Build and Install the Web Site

Build the web site and all documents as user jru. Install the resulting files as root into the default directory, /root/public_html:

% cd ~/doc/en_US.ISO8859-1/htdocs
% make all
% su -
Password:
# cd /usr/home/jru/doc/en_US.ISO8859-1/htdocs
# make install

The install process does not delete any old or outdated files that existed previously in the same directory. If a new copy of the site is built and installed every day, this command will find and delete all files that have not been updated in three days:

# find /usr/local/www -ctime 3 -delete

Goal: $1,250,000

Current Donations: $93,860

Donors:285

Contributing FreeBSD Documentation

1. First Steps

2. Installing Tools

Required Tools

Optional Tools

3. Editing and Maintaining a Working Copy

Manual Pages

Choosing a Directory

Check Out a Copy

Updating the Working Copy

Reverting Changes

Making a Diff

4. Documentation Build Process

Rendering DocBook into Output

FreeBSD Documentation Build Toolset

5. Understanding Makefiles in the Documentation Tree

Subdirectory `Makefile`s

Documentation `Makefile`s

6. FreeBSD Documentation Project Make Includes

`doc.project.mk`

Variables

Note:

Conditionals

`doc.subdir.mk`

Variables

Targets and Macros

Provided Targets

7. The Website

Environmental Variables

Building and Installing the Web Pages

Build the Full Web Site and All Documents

Build Only the Web Site in English

Build and Install the Web Site

1. First Steps

2. Installing Tools

Required Tools

Optional Tools

3. Editing and Maintaining a Working Copy

Manual Pages

Choosing a Directory

Check Out a Copy

Updating the Working Copy

Reverting Changes

Making a Diff

4. Documentation Build Process

Rendering DocBook into Output

FreeBSD Documentation Build Toolset

5. Understanding Makefiles in the Documentation Tree

Subdirectory Makefiles

Documentation Makefiles

6. FreeBSD Documentation Project Make Includes

doc.project.mk

Variables

Note:

Conditionals

doc.subdir.mk

Variables

Targets and Macros

Provided Targets

7. The Website

Environmental Variables

Building and Installing the Web Pages

Build the Full Web Site and All Documents

Build Only the Web Site in English

Build and Install the Web Site

Subdirectory `Makefile`s

Documentation `Makefile`s

`doc.project.mk`

`doc.subdir.mk`