Boost.Python - Building and Testing

###

# Boost.Python

## Building and Testing

## Contents

Requirements
Building Boost.Python
  Configuration
  Configuration for Cygwin GCC from a Windows prompt
  Results
  Notes for Cygwin GCC Users
  Notes for MinGW (and Cygwin with -mno-cygwin) GCC Users
  Testing
Building your Extension Module
Build Variants
Building Using the Microsoft Visual Studio IDE

## Requirements

Boost.Python version 2 requires Python 2.2 or newer. An unsupported archive of Boost.Python version 1, which works with versions of Python since 1.5.2, is available here.

## Building Boost.Python

Normally, Boost.Python extension modules must be linked with the `boost_python` shared library. In special circumstances you may want to link to a static version of the `boost_python` library, but if multiple Boost.Python extension modules are used together, it will prevent sharing of types across extension modules, and consume extra code space. To build `boost_python`, use Boost.Build in the usual way from the `libs/python/build` subdirectory of your boost installation (if you have already built boost from the top level this may have no effect, since the work is already done).

### Basic Configuration

You may need to configure the following variables to point Boost.Build at your Python installation:

Variable Name | Semantics | Default | Notes
--- | --- | --- | ---
`PYTHON_ROOT` | The root directory of your Python installation | Windows: `c:/tools/python` Unix: `/usr/local` | On Unix, this is the `--with-prefix=` directory used to configure Python
`PYTHON_VERSION` | The The 2-part python Major.Minor version number | `2.2` | Be sure not to include a third number, e.g. not "`2.2.1`", even if that's the version you have.
`PYTHON_INCLUDES` | path to Python `#include` directories | Autoconfigured from `PYTHON_ROOT`. Try the default before attempting to set it yourself.
`PYTHON_LIB_PATH` | path to Python library object. | Autoconfigured from `PYTHON_ROOT`. Try the default before attempting to set it yourself.

### Configuration for Cygwin GCC from a Windows prompt

The following settings may be useful when building with Cygwin GCC (not MinGW) from a Windows command shell using a Windows build of `bjam`. If "`bjam -v`" does not report "`OS=NT`", these settings do not apply to you; you should use the normal configuration variables instead. They are only useful when building and testing with multiple toolsets on Windows using a single build command, since Cygwin GCC requires a different build of Python.

Variable Name | Semantics | Default
--- | --- | ---
`CYGWIN_PYTHON_[DEBUG_]VERSION` | The version of python being used under Cygwin. | $(PYTHON_VERSION)
`CYGWIN_PYTHON_[DEBUG_]ROOT` | unix-style path containing the `include/` directory containing `python$(CYGWIN_PYTHON_[DEBUG_]VERSION)/python.h`. | $(PYTHON_ROOT)
`CYGWIN_PYTHON_[DEBUG_]LIB_PATH` | path containing the user's Cygwin Python import lib `libpython$(CYGWIN_PYTHON_[DEBUG_]VERSION).dll.a` | Autoconfigured from `CYGWIN_PYTHON_ROOT`
`CYGWIN_PYTHON_[DEBUG_]DLL_PATH` | path containing the user's Cygwin Python dll (`libpython$(CYGWIN_PYTHON_[DEBUG_]VERSION).dll`) | `/bin`

### Notes for Cygwin GCC Users

If you are using Cygwin GCC to build extension modules, you must use a Cygwin build of Python. The regular Win32 Python installation that you can download from python.org will not work with your compiler because the dynamic linking conventions are different (you can use MinGW GCC if you want to build extension modules which are compatible with a stock Win32 Python). The Cygwin installer may be able to install an appropriate version of Python, or you can follow the traditional Unix installation process to build Python from source.

The special build configuration variables listed above make it possible to use a regular Win32 build of bjam to build and test Boost.Python and Boost.Python extensions using Cygwin GCC and targeting a Cygwin build of Python.

### Notes for MinGW (and Cygwin with -mno-cygwin) GCC Users

You will need to create a MinGW-compatible version of the Python library; the one shipped with Python will only work with a Microsoft-compatible linker. Follow the instructions in the "Non-Microsoft" section of the "Building Extensions: Tips And Tricks" chapter in Installing Python Modules to create `libpythonXX.a`, where `XX` corresponds to the major and minor version numbers of your Python installation.

### Results

The build process will create a `libs/python/build/bin-stage` subdirectory of the boost root (or of `$(ALL_LOCATE_TARGET)`, if you have set that variable), containing the built libraries. The libraries are actually built to unique directories for each toolset and variant elsewhere in the filesystem, and copied to the `bin-stage` directory as a convenience, so if you build with multiple toolsets at once, the product of later toolsets will overwrite that of earlier toolsets in `bin-stage`.

### Testing

To build and test Boost.Python, start from the `libs/python/test` directory and invoke

>     bjam -sTOOLS=toolset test

This will update all of the Boost.Python v1 test and example targets. The tests are relatively verbose by default. To get less-verbose output, you might try

>     bjam -sTOOLS=toolset -sPYTHON_TEST_ARGS= test

By default, `PYTHON_TEST_ARGS` is set to `-v`.

## Building your Extension Module

Though there are other approaches, the smoothest and most reliable way to build an extension module using Boost.Python is with Boost.Build. If you have to use another build system, you should use Boost.Build at least once with the "`-n`" option so you can see the command-lines it uses, and replicate them. You are likely to run into compilation or linking problems otherwise.

The files required to build a Boost.Python extension module using bjam are the "local" files `Jamfile`, `Jamrules`, and `boost_build.jam`, and the `boost/` and `tools/build/v1/` subdirectories of your Boost tree. The latter directory contains the source code of the Boost.Build system, which is used to generate the correct build commands for your extension module. The '`v1`' refers to Boost.Build version 1. Version 2 is pre-release and currently not ready for general use.

The `libs/python/example/` project we're going to build is set up to automatically rebuild the Boost.Python library in place whenever it's out-of-date rather than just reusing an existing library, so you'll also need the Boost.Python library sources in `boost/python/src/`.

> Note: Third-party package and distribution maintainers for various operating systems sometimes split up Boost's structure or omit parts of it, so if you didn't download an official Boost release you might want to browse our CVS structure to make sure you have everything you need, and in the right places.

The `libs/python/example` subdirectory of your boost installation contains a small example which builds and tests two extensions. To build your own extensions copy the example subproject and make the following two edits:

1. `boost-build.jam` - edit the line which reads

   >     boost-build ../../../tools/build/v1 ;

   so that the path refers to the `tools/build/v1` subdirectory of your Boost installation.
2. `Jamrules` - edit the line which reads

   >     path-global BOOST_ROOT : ../../.. ;

   so that the path refers to the root directory of your Boost installation.

The instructions above for testing Boost.Python apply equally to your new extension modules in this subproject.

## Build Variants

Three variant configurations of all python-related targets are supported, and can be selected by setting the `BUILD` variable:

- `release` (optimization, `-DNDEBUG`)
- `debug` (no optimization `-D_DEBUG`)
- `debug-python` (no optimization, `-D_DEBUG -DBOOST_DEBUG_PYTHON`)

The first two variants of the `boost_python` library are built by default, and are compatible with the default Python distribution. The `debug-python` variant corresponds to a specially-built debugging version of Python. On Unix platforms, this python is built by adding `--with-pydebug` when configuring the Python build. On Windows, the debugging version of Python is generated by the "Win32 Debug" target of the `PCBuild.dsw` Visual C++ 6.0 project in the `PCBuild` subdirectory of your Python distribution. Extension modules built with Python debugging enabled are not link-compatible with a non-debug build of Python. Since few people actually have a debug build of Python (it doesn't come with the standard distribution), the normal `debug` variant builds modules which are compatible with ordinary Python.

On many windows compilers, when extension modules are built with `-D_DEBUG`, Python defaults to force linking with a special debugging version of the Python DLL. Since this debug DLL isn't supplied with the default Python installation for Windows, Boost.Python uses `boost/python/detail/wrap_python.hpp` to temporarily undefine `_DEBUG` when `Python.h` is `#include`d - unless `BOOST_DEBUG_PYTHON` is defined.

If you want the extra runtime checks available with the debugging version of the library, `#define BOOST_DEBUG_PYTHON` to re-enable python debuggin, and link with the `debug-python` variant of `boost_python`.

If you do not `#define BOOST_DEBUG_PYTHON`, be sure that any source files in your extension module `#include <boost/python/detail/wrap_python.hpp>` instead of the usual `Python.h`, or you will have link incompatibilities.

## Building Using the Microsoft Visual Studio IDE

For the those of you who feel more comfortable in the IDE world, a workspace and project file have been included in the libs/python/build/VisualStudio subdirectory. It builds release and debug versions of the Boost.Python libraries and places them and the same directory as Jamfile build does, though the intermediate object files are placed in a different directory. The files have been created using Microsoft Visual C++ version 6, but they should work for later versions as well. You will need to tell the IDE where to find the Python `Include/` and `Libs/` directories. Under Tools>Options>Directories, add an entry for the Python include dir (i.e. `c:/Python22/Include`), and one for the Lib (i.e. `c:/Python/Libs`. Make sure it is `Libs` with an "`s`" and not just `Lib`).

### Using the IDE for your own projects

Building your own projects using the IDE is slightly more complicated. Firstly, you need to make sure that the project you create as the right kind. It should be a "Win32 Dynamic-Link Library". The default one that Visual Studio 6 creates needs some modifications: turn on RTTI, and change the debug and release builds to use the respective debug and release Multithreaded DLL versions. You should probably turn off incremental linking too -- I believe it a bit flaky. If you do this, then change the "Debug Info" to "Program Database" to get rid of the Edit and Continue warning.

You'll need to add the Boost root directory under Tools>Options>Directories to get your code compiling. To make it link, add the above `boost_python.dsp` file to your workspace, and make your project depend upon it (under Project>Dependencies). You should be able to build now.

Lastly, go to the Project Settings>Debug Page and add the `Python.exe` as the executable for the project. Set a startup directory, and make sure that your current project's output dll, the `boost_python.dll` and the `python22.dll` are on the current `PATH`. If you have a python script that tests your dll, then add it in the "Program Arguments". Now, if all went well, you should be able to hit the Run (F5) button, and debug your code.

> The Visual Studio project files are graciously contributed and maintained by Brett Calcott.

© Copyright David Abrahams 2002-2004. Permission to copy, use, modify, sell and distribute this document is granted provided this copyright notice appears in all copies. This document is provided ``as is'' without express or implied warranty, and with no claim as to its suitability for any purpose.

Updated: 13 April 2004 (David Abrahams)