Skip to content

Creating Man Pages Using optparse and distutils

March 17, 2009

This blog post describes how to generate a simple man page during build time for Python applications using distutils and optparse. Technically this post describes how to write a custom HelpFormatter and a custom build command.

For GUI applications command line options usually bother me not much. Just in some rare cases, mainly when debugging or feeling unlucky with the startup behavior, I’m interested in these options. Even then I prefer to use “–help”. BTW, for the “real” command line tools like sed, ls or grep this is totally different.

So, due to my lack of interest in man pages for GUI applications I’ve shipped the previous release of CrunchyFrog along with a totally out-dated man page. Moreover I really don’t like writing man pages or keeping them in sync with the actual command line options already documented by OptionParser’s help features…

Did I say “already documented”? I’m not a friend of doing things twice, reusing that “already documented” should be very pleasant to go to get rid of my problem.

What To Put On A Man Page?

The short answer is: It should mention the application’s name, it’s purpose, how to start it, who wrote it, where to find additional information and what command line options it provides. Or, in terms of man page sections: “NAME”, “SYNOPSIS”, “DESCRIPTION”, “OPTIONS”, “SEE ALSO” and “AUTHOR” (as you may already have noticed, I’m focussing just on man page (1) in this post). Of course, there are more possible sections, but after inspecting some randomly choosen man pages for GUI applications on my system, it seems that those sections are the most common ones.

(If you are interested in the full answer, read the Linux Man Page Howto.)

Now back to our Python application that uses optparse to provide command line options.

optparse comes with a pretty good out-of-the-box help feature. As stated in the docs the primary purpose is to assist in creating user-friendly command-line interfaces and not to generate help documents (like man pages). But the ability to write a custom HelpFormatter makes it easy to generate such documents.

Writing a Custom HelpFormatter

Let’s have a look at the custom formatter:

class ManPageFormatter(optparse.HelpFormatter):

    def __init__(self, indent_increment=2, max_help_position=24,
	         width=None, short_first=1):
        """Constructor. Unfortunately HelpFormatter is no new-style class."""
        optparse.HelpFormatter.__init__(self, indent_increment,
                                        max_help_position, width, short_first)

    def _markup(self, txt):
        """Prepares txt to be used in man pages."""
        return txt.replace('-', '\\-')

    def format_usage(self, usage):
        """Formate the usage/synopsis line."""
        return self._markup(usage)

    def format_heading(self, heading):
        """"Format a heading.
        If level is 0 return an empty string. This usually is the string "Options".
        """"
        if self.level == 0:
            return ''
        return '.TP\n%s\n' % self._markup(heading.upper())

    def format_option(self, option):
        """Format a single option.
        The base class takes care to replace custom optparse values.
        """"
        result = []
        opts = self.option_strings[option]
        result.append('.TP\n.B %s\n' % self._markup(opts))
        if option.help:
            help_text = '%s\n' % self._markup(self.expand_default(option))
            result.append(help_text)
        return ''.join(result)

The three methods starting with format_* do all the formatting for us. The base class has additional formatting methods, but there’s no need to overwrite them as they do nothing special except wrapping lines to the desired width.

To use a custom formatter either give it as an parameter to the OptionParser constructor or use

parser.formatter = ManPageFormatter()
parser.formatter.set_parser(parser)

if your option parser is already instantiated.

Now, when you call parser.format_option_help() you’ll see a man page fragment with all options (including nicely formatted option groups). Maybe you wonder why not use parser.print_help() here, which normally adds a nice usage line and the description if given to the output. The reason for that is quite simple: Many of the documentation features of optparse are optional. If there’s no usage defined, no usage line will be formatted. So using it for the SYNOPSIS section and eventually the header of the man page isn’t a good idea. Everything surrounding the OPTIONS section of the man page should be generated by something else, that takes care to include a proper header and footer. We are using the custom formatter just for the options themself.

Adding a Custom Build Command

To write the rest of the man page we’re using a custom build command for distutils. This command is executed when you run python setup.py build or install that depends on build. There’s a nice side-effect when generating the man page on build time: You can simply skip this step when the setup.py commands are run on an OS that doesn’t know what a man pages is ,-)

The skeleton of a distutils command looks like this:

# -*- coding: utf-8 -*-

"""build_manpage command -- Generate man page from optparse/dist metadata."""

from distutils.command.build import build
from distutils.core import Command

class build_manpage(Command):

    description = 'Generate man page.'

    user_options = []

    def initialize_options(self):
	pass

    def finalize_options(self):
	pass

    def run(self):
	# Do something useful!

build.sub_commands.append(('build_manpage', None))

As you can see, “build_manpage” is added as a sub-command to the build command and the build_manpage class has some methods for option handling and a run() method.

To use this command add the following to your setup.py:

from distutils.core import setup
from build_manpage import build_manpage
setup(
    [...]
    cmdclass={'build_manpage': build_manpage}
)

Adding Command Options

Generating the man page needs some options. The build_manpage commands needs to know where to write the generated man page and which option parser it should use. To achieve this we have to modify the user_options attribute and the two corresponding methods in our skeleton class:

    user_options = [
        ('output=', 'O', 'output file'),
        ('parser=', None, 'module path to optparser (e.g. mymod:func'),
        ]                                                                       

    def initialize_options(self):
        self.output = None
        self.parser = None                                                      

    def finalize_options(self):
        if self.output is None:
            raise DistutilsOptionError('\'output\' option is required')
        if self.parser is None:
            raise DistutilsOptionError('\'parser\' option is required')
        mod_name, func_name = self.parser.split(':')
        fromlist = mod_name.split('.')
        try:
            mod = __import__(mod_name, fromlist=fromlist)
            self._parser = getattr(mod, func_name)()
        except ImportError, err:
            raise
        self._parser.formatter = ManPageFormatter()
        self._parser.formatter.set_parser(self._parser)
        self.announce('Writing man page %s' % self.output)
        self._today = datetime.date.today()

user_options is a list of 3-tuples (long option, short option, help),  initialize_options() sets default values and finalize_options() makes sure that the given options are sane (nb, the Command class has some validation functions, unfortunately you’ll have to read the source to find them…). Run setup.py build_manpage --help to see how to use your options on the command line. Besides setting these options on the command line you can set them in a file called setup.cfg too:

[build_manpage]
output=data/mymanpage.1
parser=myapp.somemod:get_parser

Both options are required. output tells the command to which file it should write the generated man page, parser is a string pointing to a function that returns an OptionParser instance (a dotted module path with a function name separated by a single colon).

Putting It All Together

This is done by the run() method, of course. In finalize_options() the option parser was imported and our ManPageFormatter was attached to it. The run() method now does the following:

    def run(self):
        manpage = []
        manpage.append(self._write_header())
        manpage.append(self._write_options())
        manpage.append(self._write_footer())
        stream = open(self.output, 'w')
        stream.write(''.join(manpage))
        stream.close()

Let’s have a look at the _write_header() helper function (_write_options() is basically a call to the formatter and _write_footer() just adds the author and a link to the homepage):

    def _write_header(self):
        appname = self.distribution.get_name()
        ret = []
        ret.append('.TH %s 1 %s\n' % (self._markup(appname),
                                      self._today.strftime('%Y\\-%m\\-%d')))
        description = self.distribution.get_description()
        if description:
            name = self._markup('%s - %s' % (self._markup(appname),
                                             description.splitlines()[0]))
        else:
            name = self._markup(appname)
        ret.append('.SH NAME\n%s\n' % name)
        synopsis = self._parser.get_usage()
        if synopsis:
            synopsis = synopsis.replace('%s ' % appname, '')
            ret.append('.SH SYNOPSIS\n.B %s\n%s\n' % (self._markup(appname),
                                                      synopsis))
        long_desc = self.distribution.get_long_description()
        if long_desc:
            ret.append('.SH DESCRIPTION\n%s\n' % self._markup(long_desc))
        return ''.join(ret)

As you can see, this method makes use of another advantage of using a distutils command. It fetches some information not covered by optparse from distutils’ meta data and in addition takes care of optparse’s optional usage and description attributes.

The complete implementation of the build_manpage command is available here.

This is how the resulting man page for CrunchyFrog looks like:

Generated Man Page for CrunchyFrog

Generated Man Page for CrunchyFrog

I’ll have to admit, it’s a very simple approach to have a man page without writing it and – even worse – maintaining it. But at least it solved my problem to distribute a GUI application with an up-to-date man page. This approach has no extra requirements as everything needed can be found in Python’s stdlib and the man page is generated when needed. Luckily the build command is extensible, so it’s no problem to add additional sections (what about connection the script to a bug tracker and add a hopefully not too long BUGS section?).

P.S.: In case your interested in another formatter example, I’ve recently written another HelpFormatter that creates a Google Code Wiki page for Rietveld‘s upload.py. The source are available here.

Edit (2009-03-25): There’s a nice patch written by René Neumann that adds a SEE ALSO option to the distutils command.

Edit (2016-03-17): The script including the patch mentioned above is also available on github: https://github.com/andialbrecht/build_manpage

4 Comments
  1. May 6, 2009 12:24 pm

    Hi ! I just discovered help2man, a utility that can be easily installed on Ubuntu/Debian GNU/Linux. I guess it should work on Mac too. It works well with optparse –help output. It outputs the text of a man page.

  2. October 5, 2009 2:17 pm

    good post! i was looking to do something very similar, and wasn’t sure how to integrate it with distutils properly. the magic i was missing was: build.sub_commands.append((‘build_manpage’, None))

    thanks!

Trackbacks

  1. Portato Manpage « Necoro’s Blog

Comments are closed.

%d bloggers like this: