Understanding OpenStack developer dependencies

While reviewing the OpenStack keystone codebase on an existing VM used with devstack I came across a dependency problem with Python pbr . Python Build Reasonableness (pbr) is actually a result of work on OpenStack. Additional info can be found at Openstack pbr .

On one server machine I had this package installed. At this time I do not know what process actually installed the pbr package.

$ sudo dpkg -l | grep pbr
ii  python-pbr        0.7.0-0ubuntu2      all          inject useful and sensible default behaviors into setuptools - Python 2.x

This is incompatible with current code from several OpenStack projects, keystone and python-openstackclient being two I am working with when reviewing the projects requirements in requirements.txt.

$ grep pbr requirements.txt
pbr>=0.6,!=0.7,<1.0

As seen here, 0.7 is specifically excluded. When updating this machine with the required versions to run the checked out code I ran into the following problem.

$ sudo -H pip install -r requirements.txt

...
  Found existing installation: pbr 0.7.0
    Uninstalling pbr-0.7.0:
...
     OSError: [Errno 13] Permission denied: '/usr/lib/python2.7/dist-packages/pbr/version.py'

This lead me to determine I need to run multiple separate VMs. Dedicated VMs for devstack installations when I’m testing things, and a dedicated VM for source development. I later determined the best action was to do development on my host machine installing these developer dependencies and always running any deployed versions in VMs.

Minimum requirements

Using a stock Ubuntu 14.04 LTS server installation I took the time to iteratively check the needed dependencies

# Git needed to retrieve OpenStack code
sudo apt-get install -y git-core

# Python is installed by default on an Ubuntu Server

# install easy_install
sudo apt-get install python-setuptools

# install pip - Package Management System   Uses Python Package Index (PyPI)
sudo easy_install pip

# Install tox - Python automated and standardized testing
sudo -H pip install tox

# Python Developer Libraries
sudo apt-get install -y python-dev

# Openstack developer dependencies
sudo apt-get install -y libffi-dev libssl-dev libldap2-dev libffi-dev libsasl2-dev libxslt1-dev libxml2-dev

With the necessary dependencies met, the following builds a working keystone developer virtual environment.

git clone git://git.openstack.org/openstack/keystone
cd keystone
tox -e py27 --notest

Required Dependencies

Certain projects do a good job of defining the required OS dependencies such as keystone .

To validate these requirements the following is an iterative process of determining the compilation error message and needed package dependency.

• For missing #include <ffi.h> install libffi-dev
• For missing #include <openssl/aes.h> install libssl-dev
• For missing #include "lber.h" install libldap2-dev
• For missing #include <ffi.h> install libffi-dev
• For missing #include <sasl.h> install libsasl2-dev
• For missing #include "libxml/xmlversion.h" install libxslt1-dev which requires libxml2-dev

For setting up a development environment libsqlite3-dev was not initially needed. This does not mean it’s needed later for testing purposes.