Understanding OpenStack developer dependencies

While reviewing the OpenStack keystone codebase on an existing VM used with devstack I came across a dependency problem with Python pbr. Python Build Reasonableness (pbr) is actually a result of work on OpenStack. Additional info can be found at Openstack pbr.

On one server machine I had this package installed. At this time I do not know what process actually installed the pbr package.

$ sudo dpkg -l | grep pbr
ii  python-pbr        0.7.0-0ubuntu2      all          inject useful and sensible default behaviors into setuptools - Python 2.x

This is incompatible with current code from several OpenStack projects, keystone and python-openstackclient being two I am working with when reviewing the projects requirements in requirements.txt.

$ grep pbr requirements.txt

As seen here, 0.7 is specifically excluded. When updating this machine with the required versions to run the checked out code I ran into the following problem.

$ sudo -H pip install -r requirements.txt

  Found existing installation: pbr 0.7.0
    Uninstalling pbr-0.7.0:
     OSError: [Errno 13] Permission denied: '/usr/lib/python2.7/dist-packages/pbr/version.py'

This lead me to determine I need to run multiple separate VMs. Dedicated VMs for devstack installations when I'm testing things, and a dedicated VM for source development. I later determined the best action was to do development on my host machine installing these developer dependencies and always running any deployed versions in VMs.

Minimum requirements

Using a stock Ubuntu 14.04 LTS server installation I took the time to iteratively check the needed dependencies

# Git needed to retrieve OpenStack code
sudo apt-get install -y git-core

# Python is installed by default on an Ubuntu Server

# install easy_install
sudo apt-get install python-setuptools

# install pip - Package Management System   Uses Python Package Index (PyPI)
sudo easy_install pip

# Install tox - Python automated and standardized testing
sudo -H pip install tox

# Python Developer Libraries
sudo apt-get install -y python-dev

# Openstack developer dependencies
sudo apt-get install -y libffi-dev libssl-dev libldap2-dev libffi-dev libsasl2-dev libxslt1-dev libxml2-dev

With the necessary dependencies met, the following builds a working keystone developer virtual environment.

git clone git://git.openstack.org/openstack/keystone
cd keystone
tox -e py27 --notest

Required Dependencies

Certain projects do a good job of defining the required OS dependencies such as keystone.

To validate these requirements the following is an iterative process of determining the compilation error message and needed package dependency.

  • For missing #include <ffi.h> install libffi-dev
  • For missing #include <openssl/aes.h> install libssl-dev
  • For missing #include "lber.h" install libldap2-dev
  • For missing #include <ffi.h> install libffi-dev
  • For missing #include <sasl.h> install libsasl2-dev
  • For missing #include "libxml/xmlversion.h" install libxslt1-dev which requires libxml2-dev

For setting up a development environment libsqlite3-dev was not initially needed. This does not mean it's needed later for testing purposes.