Quality documentation is crucial to the success of FreeBSD, submitting documentation is one of the easiest ways to contribute to the project and anyone is welcome to submit! Willingness to contribute is the only membership requirement.
1. First Steps
The first step to contribute is to subscribe to the FreeBSD documentation project mailing list. This is a fantastic resource for any questions or problems involving the documentation.
- Install the textproc/docproj meta-package and Subversion. This meta-package installs all of the software needed to edit and build FreeBSD documentation. The Subversion package is needed to obtain a working copy of the documentation and generate patches with.
# pkg install docproj subversion
- Optional: to generate PDF documentation, install the textproc/fop package as it is not installed by default by textproc/docproj.
# pkg install fop
- Install a local working copy of the documentation from the FreeBSD repository in
~/doc
(see Chapter 3, The Working Copy).% svn checkout https://svn.FreeBSD.org/doc/head ~/doc
- Configure the text editor:
- Word wrap set to 70 characters.
- Tab stops set to 2.
- Replace each group of 8 leading spaces with a single tab. Specific editor configurations are listed in Chapter 15, Editor Configuration.
- Update the local working copy:
% svn up ~/doc
- Edit the documentation files that require changes. If a file needs major changes, consult the mailing list for input. References to tag and entity usage can be found in Chapter 8, XHTML Markup and Chapter 9, DocBook Markup.
- After editing, check for problems by running:
% igor -R filename.xml | less -RS
Review the output and edit the file to fix any problems shown, then rerun the command to find any remaining problems. Repeat until all of the errors are resolved. - Always build-test changes before submitting them. Running
make
in the top-level directory of the documentation being edited will generate that documentation in split HTML format. For example, to build the English version of the Handbook in HTML, runmake
in theen_US.ISO8859-1/books/handbook/
directory. - When changes are complete and tested, generate a “diff file”:
% cd ~/doc
% svn diff > bsdinstall.diff.txt
Give the diff file a descriptive name. In the example above, changes have been made to thebsdinstall
portion of the Handbook. - Submit the diff file using the web-based Problem Report system. If using the web form, enter a Summary of
[Patch]
short description of problem. Select the Component Documentation. In the Description field, enter a short description of the changes and any important details about them. Use the[ Add an attachment ]
button to attach the diff file. Finally, use the[ Submit Bug ]
button to submit your diff to the problem report system.
2. Installing Tools
Required Tools
- Install textproc/docproj from the Ports Collection. This meta-port installs all the applications required to do useful work with the FreeBSD documentation. Some further notes on particular components are given below.
- FreeBSD documentation uses several Document Type Definitions (DTDs) and sets of XML entities. These are all installed by the textproc/docproj port.XHTML DTD (textproc/xhtml)
- XHTML is the markup language of choice for the World Wide Web, and is used throughout the FreeBSD web site.
- DocBook is designed for marking up technical documentation.
Optional Tools
- These applications are not required, but can make working on the documentation easier or add capabilities.
- Vim (editors/vim)
- Emacs (editors/emacs), A popular editor for working with XML and derived documents, like DocBook XML.
3. Editing and Maintaining a Working Copy
The working copy is a copy of the FreeBSD repository documentation tree downloaded onto the local computer. Changes are made to the local working copy, tested, and then submitted as patches to be committed to the main repository. A full copy of the documentation tree can occupy 700 megabytes of disk space. Allow for a full gigabyte of space to have room for temporary files and test versions of various output formats.
In order to manage the FreeBSD documentation files, Subversion should be installed from packages.# pkg install subversion
Manual Pages
FreeBSD documentation for commands and configuration files can be found within the manual page collection. These consist of two repositories: doc
for books and articles, and base
for operating system and manual pages. These repositories can contain multiple versions of documentation and source code. New modifications are almost always made to the latest version: head
.
In order to edit manual pages, base
will need to be checked out separately.
Choosing a Directory
FreeBSD documentation is traditionally stored in /usr/doc/
, and system source code with manual pages in /usr/src/
. These directory trees are relocatable, and users may want to put the working copies in other locations to avoid interfering with existing information in the main directories. The examples that follow use ~/doc
and ~/src
, both subdirectories of the user’s home directory.
Check Out a Copy
A download of a working copy from the repository is called a checkout, this can be done with svn checkout
. For example, to checkout the latest version (head
) of the documentation tree.
% svn checkout https://svn.FreeBSD.org/doc/head ~/doc
And to checkout the source code for manual pages:
% svn checkout https://svn.FreeBSD.org/base/head ~/src
Updating the Working Copy
The FreeBSD repository is frequently updated, even a short time after the initial checkout, changes may have already been made creating differences between the local working copy and the main FreeBSD repository. To update the local version, use svn update
on the directory containing the local working copy. Getting into the habit of doing this often before editing document files will be helpful.
% svn update ~/doc
Reverting Changes
Sometimes it turns out that changes were not necessary after all, or the writer just wants to start over. Files can be “reset” to their unchanged form with svn revert
. For example, to erase the edits made to chapter.xml
and reset it to unmodified form:
%
svn revert chapter.xml
Making a Diff
After edits to a file or group of files are completed, the differences between the local working copy and the version on the FreeBSD repository must be collected into a single file for submission. These diff files are produced by redirecting the output of svn diff
into a file:
%
cd
~/doc
%
svn diff >
doc-fix-spelling.diff
Give the file a meaningful name that identifies the contents. The example above is for spelling fixes to the whole documentation tree.
If the diff file is to be submitted with the web “Submit a FreeBSD problem report” interface, add a .txt
extension to give the earnest and simple-minded web form a clue that the contents are plain text.
Be careful: svn diff
includes all changes made in the current directory and any subdirectories. If there are files in the working copy with edits that are not ready to be submitted yet, provide a list of only the files that are to be included:
%
cd
~/doc
%
svn diff
disks/chapter.xml printers/chapter.xml
>disks-printers.diff
4. Documentation Build Process
Rendering DocBook into Output
Different types of output can be produced from a single DocBook source file. The type of output desired is set with the FORMATS
variable. A list of known formats is stored in KNOWN_FORMATS
:
% cd ~/doc/en_US.ISO8859-1/books/handbook
% make -V KNOWN_FORMATS
FORMATS Value | File Type | Description |
---|---|---|
html | HTML, one file | A single book.html or article.html . |
html-split | HTML, multiple files | Multiple HTML files, one for each chapter or section, for use on a typical web site. |
pdf | Portable Document Format |
The default output format can vary by document, but is usually html-split
. Other formats are chosen by setting FORMATS
to a specific value. Multiple output formats can be created at a single time by setting FORMATS
to a list of formats.
FreeBSD Documentation Build Toolset
These are the tools used to build and install the FDP documentation.
- The primary build tool is make(1), specifically Berkeley Make.
- Package building is handled by FreeBSD’s pkg-create(8).
- gzip(1) is used to create compressed versions of the document. bzip2(1) archives are also supported. tar(1) is used for package building.
- install(1) is used to install the documentation.
5. Understanding Makefiles in the Documentation Tree
There are three main types of Makefile
s in the FreeBSD Documentation Project tree.
- Subdirectory
Makefile
s simply pass commands to those directories below them. - Documentation
Makefile
s describe the documents that are produced from this directory. - Make includes are the glue that perform the document production, and are usually of the form
doc.
.xxx
.mk
Subdirectory Makefile
s
These Makefile
s usually take the form of:
SUBDIR =articles
SUBDIR+=books
COMPAT_SYMLINK = en
DOC_PREFIX?= ${.CURDIR}/..
.include "${DOC_PREFIX}/share/mk/doc.project.mk"
The first four non-empty lines define the make(1) variables SUBDIR
, COMPAT_SYMLINK
, and DOC_PREFIX
.
The SUBDIR
statement and COMPAT_SYMLINK
statement show how to assign a value to a variable, overriding any previous value.
The second SUBDIR
statement shows how a value is appended to the current value of a variable. The SUBDIR
variable is now articles books
.
The DOC_PREFIX
assignment shows how a value is assigned to the variable, but only if it is not already defined. This is useful if DOC_PREFIX
is not where this Makefile
thinks it is – the user can override this and provide the correct value.
What does it all mean? SUBDIR
mentions which subdirectories below this one the build process should pass any work on to.
COMPAT_SYMLINK
is specific to compatibility symlinks (amazingly enough) for languages to their official encoding (doc/en
would point to en_US.ISO-8859-1
).
DOC_PREFIX
is the path to the root of the FreeBSD Document Project tree. This is not always that easy to find, and is also easily overridden, to allow for flexibility. .CURDIR
is a make(1) builtin variable with the path to the current directory.
The final line includes the FreeBSD Documentation Project’s project-wide make(1) system file doc.project.mk
which is the glue which converts these variables into build instructions.
Documentation Makefile
s
These Makefile
s set make(1) variables that describe how to build the documentation contained in that directory.
Here is an example:
[email protected] DOC?= book FORMATS?= html-split html INSTALL_COMPRESSED?= gz INSTALL_ONLY_COMPRESSED?= # SGML content SRCS= book.xml DOC_PREFIX?= ${.CURDIR}/../../.. .include "$(DOC_PREFIX)/share/mk/docproj.docbook.mk"
The MAINTAINER
variable allows committers to claim ownership of a document in the FreeBSD Documentation Project, and take responsibility for maintaining it.
DOC
is the name (sans the .xml
extension) of the main document created by this directory. SRCS
lists all the individual files that make up the document. This should also include important files in which a change should result in a rebuild.
FORMATS
indicates the default formats that should be built for this document. INSTALL_COMPRESSED
is the default list of compression techniques that should be used in the document build. INSTALL_ONLY_COMPRESS
, empty by default, should be non-empty if only compressed documents are desired in the build.
The DOC_PREFIX
and include statements should be familiar already.
6. FreeBSD Documentation Project Make Includes
make(1) includes are best explained by inspection of the code. Here are the system include files:
doc.project.mk
is the main project include file, which includes all the following include files, as necessary.doc.subdir.mk
handles traversing of the document tree during the build and install processes.doc.install.mk
provides variables that affect ownership and installation of documents.doc.docbook.mk
is included ifDOCFORMAT
isdocbook
andDOC
is set.
doc.project.mk
By inspection:
DOCFORMAT?= docbook MAINTAINER?= [email protected] PREFIX?= /usr/local PRI_LANG?= en_US.ISO8859-1 .if defined(DOC) .if ${DOCFORMAT} == "docbook" .include "doc.docbook.mk" .endif .endif .include "doc.subdir.mk" .include "doc.install.mk"
Variables
DOCFORMAT
and MAINTAINER
are assigned default values, if these are not set by the document make file.
PREFIX
is the prefix under which the documentation building tools are installed. For normal package and port installation, this is /usr/local
.
PRI_LANG
should be set to whatever language and encoding is natural amongst users these documents are being built for. US English is the default.
Note:
PRI_LANG
does not affect which documents can, or even will, be built. Its main use is creating links to commonly referenced documents into the FreeBSD documentation install root.
Conditionals
The .if defined(DOC)
line is an example of a make(1) conditional which, like in other programs, defines behavior if some condition is true or if it is false. defined
is a function which returns whether the variable given is defined or not.
.if ${DOCFORMAT} == "docbook"
, next, tests whether the DOCFORMAT
variable is "docbook"
, and in this case, includes doc.docbook.mk
.
The two .endif
s close the two above conditionals, marking the end of their application.
doc.subdir.mk
This file is too long to explain in detail. These notes describe the most important features.
Variables
SUBDIR
is a list of subdirectories that the build process should go further down into.ROOT_SYMLINKS
is the name of directories that should be linked to the document install root from their actual locations, if the current language is the primary language (specified byPRI_LANG
).COMPAT_SYMLINK
is described in the Subdirectory Makefile section.
Targets and Macros
Dependencies are described by
tuples, where to build target
: dependency1 dependency2 ...
target
, the given dependencies must be built first.
After that descriptive tuple, instructions on how to build the target may be given, if the conversion process between the target and its dependencies are not previously defined, or if this particular conversion is not the same as the default conversion method.
A special dependency .USE
defines the equivalent of a macro.
_SUBDIRUSE: .USE .for entry in ${SUBDIR} @${ECHO} "===> ${DIRPRFX}${entry}" @(cd ${.CURDIR}/${entry} && \ ${MAKE} ${.TARGET:S/realpackage/package/:S/realinstall/install/} DIRPRFX=${DIRPRFX}${entry}/ ) .endfor
In the above, _SUBDIRUSE
is now a macro which will execute the given commands when it is listed as a dependency.
What sets this macro apart from other targets? Basically, it is executed after the instructions given in the build procedure it is listed as a dependency to, and it does not adjust .TARGET
, which is the variable which contains the name of the target currently being built.
clean: _SUBDIRUSE rm -f ${CLEANFILES}
In the above, clean
will use the _SUBDIRUSE
macro after it has executed the instruction rm -f ${CLEANFILES}
. In effect, this causes clean
to go further and further down the directory tree, deleting built files as it goes down, not on the way back up.
Provided Targets
install
andpackage
both go down the directory tree calling the real versions of themselves in the subdirectories (realinstall
andrealpackage
respectively).clean
removes files created by the build process (and goes down the directory tree too).cleandir
does the same, and also removes the object directory, if any.
7. The Website
The FreeBSD website is included in the FreeBSD documentation and is stored in the en_US.ISO8859-1/htdocs
subdirectory of the document tree directory, ~/doc
.
Environmental Variables
- DOCDIR
- ENGLISH_ONLY
- Default is undefined, will build and include all translations. Can be set to ENGLISH_ONLY=YES to only use english documents.
- WEB_ONLY
- Default is undefined, will build books and articles in addition to the web site. Can be set to WEB_ONLY=YES to build only HTML pages from the
en_US.ISO8859-1/htdocs
directory
- Default is undefined, will build books and articles in addition to the web site. Can be set to WEB_ONLY=YES to build only HTML pages from the
- WEB_LANG
- Default is undefined, will build and include all translations. Can be set to a space-separated list of languages to be included. Use the directory names in the document root directory.
- If a user wanted to include the German and French documents, for example:
WEB_LANG="de_DE.ISO8859-1 fr_FR.ISO8859-1"
Building and Installing the Web Pages
After obtaining the documentation and web-site source files, the web site can be built.
An actual installation of the web site is run as the root
user because the permissions on the web server directory will not allow files to be installed by an unprivileged user. For testing, it can be useful to install the files as a normal user to a temporary directory.
In these examples, the web site files are built by user jru
in their home directory, ~/doc
, with a full path of /usr/home/jru/doc
.
Build the Full Web Site and All Documents
%
cd ~/doc/en_US.ISO8859-1/htdocs/
%
make all
Build Only the Web Site in English
%
cd ~/doc/en_US.ISO8859-1/htdocs/
%
env DOCDIR=/tmp/www make ENGLISH_ONLY=yes WEB_ONLY=yes all install
After the site has been built as shown above, a modified main page can be viewed with:
%
firefox /tmp/www/data/index.html
After building the site as shown above, this /usr/local/etc/apache24/httpd.conf
can be used with www/apache24:
# httpd.conf for testing the FreeBSD website Define TestRoot "/tmp/www/data" # directory for configuration files ServerRoot "/usr/local" Listen 80 # minimum required modules LoadModule authz_core_module libexec/apache24/mod_authz_core.so LoadModule mime_module libexec/apache24/mod_mime.so LoadModule unixd_module libexec/apache24/mod_unixd.so LoadModule cgi_module libexec/apache24/mod_cgi.so LoadModule dir_module libexec/apache24/mod_dir.so # run the webserver as user and group User www Group www ServerAdmin [email protected] ServerName fbsdtest # deny access to all files <Directory /> AllowOverride none Require all denied </Directory> # allow access to the website directory DocumentRoot "${TestRoot}" <Directory "${TestRoot}"> Options Indexes FollowSymLinks AllowOverride None Require all granted </Directory> # prevent access to .htaccess and .htpasswd files <Files ".ht*"> Require all denied </Files> ErrorLog "/var/log/httpd-error.log" LogLevel warn # set up the CGI script directory <Directory "${TestRoot}/cgi"> AllowOverride None Options None Require all granted Options +ExecCGI AddHandler cgi-script .cgi </Directory> Include etc/apache24/Includes/*.conf
Start the web server with
#
service apache24 onestart
Build and Install the Web Site
Build the web site and all documents as user jru
. Install the resulting files as root
into the default directory, /root/public_html
:
%
cd ~/doc/en_US.ISO8859-1/htdocs
%
make all
%
su -
Password:#
cd /usr/home/jru/doc/en_US.ISO8859-1/htdocs
#
make install
The install process does not delete any old or outdated files that existed previously in the same directory. If a new copy of the site is built and installed every day, this command will find and delete all files that have not been updated in three days:
#
find
/usr/local/www
-ctime 3 -delete