What is it?

Apigen is a tool for automatically generating API reference documentation for Python projects. It works by examining code at runtime rather than at compile time. This way it is capable of displaying information about the code base after initialization. A drawback is that you cannot easily document source code that automatically starts server processes or has some other irreversible effects upon getting imported.

The apigen functionality is normally triggered from py.test, and while running the tests it gathers information such as code paths, arguments and return values of callables, and exceptions that can be raised while the code runs (XXX not yet!) to include in the documentation. It's also possible to run the tracer (which collects the data) in other code if your project does not use py.test but still wants to collect the runtime information and build the docs.

Apigen is written for the py lib, but can be used to build documentation for any project: there are hooks in py.test to, by providing a simple script, build api documentation for the tested project when running py.test. Of course this does imply py.test is actually used: if little or no tests are actually ran, the additional information (code paths, arguments and return values and exceptions) can not be gathered and thus there will be less of an advantage of apigen compared to other solutions.

Features

Some features were mentioned above already, but here's a complete list of all the niceties apigen has to offer:

• source documents

  Apigen not only builds the API documentation, but also a tree of syntax-colored source files, with links from the API docs to the source files.

• abundance of information

  compared to other documentation generation tools, apigen produces an abundant amount of information: it provides syntax-colored code snippets, code path traces, etc.

• linking

  besides links to the source files, apigen provides links all across the documentation: callable arguments and return values link to their definition (if part of the documented code), class definition to their base classes (again, if they're part of the documented code), and everywhere are links to the source files (including in traces)

• (hopefully) improves testing

  because the documentation is built partially from test results, developers may (especially if they're using the documentation themselves) be more aware of untested parts of the code, or parts can use more tests or need attention

Using apigen

To trigger apigen, all you need to do is run the py/bin/py.test tool with an --apigen argument, as such:

$ py.test --apigen=<path>

where <path> is a path to a script containing some special hooks to build the documents (see below). The script to build the documents for the py lib can be found in py/apigen/apigen.py, so building those documents can be done by cd'ing to the 'py' directory, and executing:

$ py.test --apigen=apigen/apigen.py

The documents will by default be built in the parent directory of the package dir (in this case the 'py' directory). Be careful that you don't overwrite anything!

Other projects

To use apigen from another project, there are three things that you need to do:

Using apigen from code

If you want to avoid using py.test, or have an other idea of how to best collect information while running code, the apigen functionality can be directly accessed. The most important classes are the Tracer class found in py/apigen/tracer/tracer.py, which holds the information gathered during the tests, and the DocStorage and DocStorageAccessor classes from py/apigen/tracer/docstorage.py, which (respectively) store the data, and make it accessible.

>>> import py
>>> from py.__.apigen.tracer.docstorage import DocStorage, DocStorageAccessor
>>> from py.__.apigen.tracer.tracer import Tracer
>>> toregister = {'py.path.local': py.path.local,
...               'py.path.svnwc': py.path.svnwc}
>>> ds = DocStorage().from_dict(toregister)
>>> t = Tracer(ds)
>>> t.start_tracing()
>>> p = py.path.local('.')
>>> p.check(dir=True)
True
>>> t.end_tracing()

>>> dsa = DocStorageAccessor(ds)

Comparison with other documentation generation tools

Apigen is of course not the only documentation generation tool available for Python. Although we knew in advance that our tool had certain features the others do not offer, we decided to investigate a bit so that we could do a proper comparison.

Questions, remarks, etc.

For more information, questions, remarks, etc. see http://codespeak.net/py. This website also contains links to mailing list and IRC channel.