Documentation management

This chapter describes all the processes related to the documentation of the POP-Java project.

Documentation types

The POP-Java project currently has two main types of documentation:

Prose documentation
The documentation you are reading right now. It contains manuals, guides and how-tos about the project itself. The prose documentation is written by humans for humans. This documentation is currently edited using Sphinx.
API reference
The API Reference contains the documentation for each functional unit of the project (packages, classes, methods, …) and is generated by parsing the source code. The API reference is currently being generated using Doxygen.

Editing the prose documentation

As mentioned in the previous section, the prose documentation is managed by Sphinx. Sphinx is a tool to create documentation from reStructuredText sources and can produce a variety of output formats (e.g. HTML, LaTeX -> PDF, ePub, Texinfo, man pages, plain text, …).

The prose documentation resides in the docs/ directory at the root of the repository and can be edited with any text editor. The complete reference for the reStructuredText syntax and the addons added by Sphinx are available on the reStructuredText (see also http://sphinx-doc.org/rest.html) and Sphinx Markup Constructs pages, respectively.

For quick and small edits, it is possible to use the GitHub editing interface instead of a complete local clone + edit + commit + push process. The documentation files are available on GitHub in the gridgroup/pop-java repository.

Building and publishing the prose documentation

The building process transforms the input documents in reStructuredText format into one of the output formats supported by Sphinx (HTML, PDF, ePub, …).

There are two different ways to build the documentation:

  • Locally on the development machine, by installing the necessary tools and executing the right command.
  • Remotely, via a custom post-commit hook on a CI server or through a specialized service, such as http://readthedocs.org/.

In the next two subsections we will discuss these two possibilities. In the second case we will limit ourselves to building through http://readthedocs.org/.

Locally

To build the documentation locally we need a working Sphinx installation.

Sphinx requires at least Python 2.5 or 3.1; you can read more about additional details regarding the requirements on the Sphinx introduction page.

To check which version of python you have installed, run the following command:

python -V

If python is not installed or does not meet Sphinx’s version requirements, you can either install or update it by using your distribution package manager (apt-get on Debian/Ubuntu, yum on CentOS/Fedora, emerge on Gentoo, …).

Below we report some easy steps to install Sphinx on your system. For a more complete walk-through (including platforms such as OS X and Windows), you can always refer to the Sphinx installation instructions.

To install Sphinx, you can use either easy_install:

easy_install sphinx

or pip (recommended):

pip install sphinx

Depending on the platform and your setup, you probably have to run these commands with administration privileges:

sudo pip install sphinx

You can check if Sphinx was successfully installed by running the following command:

python -c 'import sphinx'

In addition to Sphinx itself, this documentation makes use of one additional required and one additional optional packages:

  • The bibtex extension allows to use BibTeX databases to manage references. You can install it by executing:

    pip install sphinxcontrib-bibtex
    
  • The sphinx_rtd_theme is the theme used on Read The Docs. If the corresponding package is installed, it will be used for local builds as well. The theme can be installed by running:

    pip install sphinx_rtd_theme
    

Note

If you had to use sudo When installing Sphinx, then the commands to install these additional packages will have to be prefixed with it as well.

Once you have a working Sphinx installation on your system, it’s time to start building your documentation. As the initial setup is already done for the POP-Java project, you don’t have to configure anything.

Different ways exist to build the documentation. The easiest is by far to use the generated Makefile present in the docs/ directory:

cd path/to/pop-java/docs/
make html

If successfull, the last line of the command output will indicate the location of the build. In this case, the HTML files will be located in _build/html.

To publish the generated documentation, it suffices to upload the html directory (or the equivalent artifact for a different output format) to a publicly accessible web server.

In order to support the other output formats managed by Sphinx, the Makefile has different additional targets. You can find out more about them by running the help target, whose output is shown below:

Please use `make <target>' where <target> is one of
  html       to make standalone HTML files
  dirhtml    to make HTML files named index.html in directories
  singlehtml to make a single large HTML file
  pickle     to make pickle files
  json       to make JSON files
  htmlhelp   to make HTML files and a HTML help project
  qthelp     to make HTML files and a qthelp project
  devhelp    to make HTML files and a Devhelp project
  epub       to make an epub
  latex      to make LaTeX files, you can set PAPER=a4 or PAPER=letter
  latexpdf   to make LaTeX files and run them through pdflatex
  latexpdfja to make LaTeX files and run them through platex/dvipdfmx
  text       to make text files
  man        to make manual pages
  texinfo    to make Texinfo files
  info       to make Texinfo files and run them through makeinfo
  gettext    to make PO message catalogs
  changes    to make an overview of all changed/added/deprecated items
  xml        to make Docutils-native XML files
  pseudoxml  to make pseudoxml-XML files for display purposes
  linkcheck  to check all external links for integrity
  doctest    to run all doctests embedded in the documentation (if enabled)

readthedocs.org

Read the Docs (RTD) is a free service to build and host Sphinx documentation sets. It supports polling any GIT or Mercurial repository and re-running a build each time a new commit is detected. The POP-Java documentation is currently available on RTD at the following link: http://pop-java.readthedocs.org/en/latest/.

In order to update the documentation hosted on RTD, it suffices to commit the changes to the GIT repository and push them to the GitHub remote:

git commit -m 'Documentation update'
git push origin master

The repository hosted on GitHub is configured with a post-commit hook to trigger a new RTD build and the updated version should be available in a short time (usually < 2 minutes).

Thanks to RTD’s integration with GitHub, an even easier way to carry out small, self-contained edits to the documentation is directly through the GitHub editing interface:

  • Each HTML page generated on RTD contains a GitHub edit link in the right corner which brings up the GitHub interface browsing.

    ../../_images/ghedit1.png
  • We can then click on the edit button to enter the editing interface.

    ../../_images/ghedit2.png
  • From there we can carry out the desired changes, commit them directly (if the account with which we are logged in to GitHub allows it; fork and open a pull request otherwise) and have the documentation hosted on RTD updated automatically.

    ../../_images/ghedit3.png

Building and publishing the API reference

In the introduction to the present chapter we mentioned that the API Reference is currently being generated using Doxygen. In the following paragraphs we will describe how the documentation can be rebuilt and how the resulting artifact can be published on the GitHub pages service.

Building

To build the documentation you need a working installation of Doxygen on your system. Extensive documentation about the installation process is available directly from the Doxygen manual.

Once installed, the steps needed to build the documentation are very simple:

doxygen doxygen.conf

The resulting build will be available in the doxygen/html and doxygen/latex directories. The HTML version is ready to use, while the LaTeX version needs and additional build to get to the PDF version:

cd doxygen/latex
make

The resulting PDF will be available at doxygen/latex/refman.pdf.

Publishing on GitHub pages

GitHub supports a basic kind of static website hosting, based on GIT repositories. This service is called GitHub pages; you can find out more about it on http://pages.github.com/.

Currently, the latest build of the HTML version of the API reference is made available for browsing at http://gridgroup.github.io/pop-java/api/ and the PDF version at http://gridgroup.github.io/pop-java/api/POP-Java.pdf.

The steps involved in the update process of the gh-pages branch (the branch from which the static website is made available on GitHub) are summarized as follows:

git checkout gh-pages                   # Checkout the 'gh-pages' branch
mv doxygen/html api                     # Move the built HTML reference into place
git commit -am 'API reference update'   # Commit everything
git push origin gh-pages                # Push to GitHub (publish)
git checkout <oldbranch>                # Go back to the branch we were working on

To make it easier to build and publish the documentation, a script to automate the process is made available at scripts/api-reference from the root of the repository. In order to build, commit, and publish a new version of the API reference, it suffices to execute it with no arguments:

./scripts/api-reference