Using your devstack cloud

April 5, 2016 by ronald

You have setup and installed devstack. Now what!

The Horizon UI will allow you to administer your running cloud from a web interface. We are not going to discuss the web UI in this post.

Using the command line will provide you access to the following initial developer/operator capabilities.

Duplicating the features of the UI with the client tools
Observing the running services
Understanding the logging of OpenStack services
Understanding the configuration of OpenStack services
Understanding the source code of OpenStack services

This is not an exhaustive list or explanation of each point but an intro into navigating around the running OpenStack services.

Duplicating UI features

OpenStack has a number of individual command line clients for many services, and a common client openstack.

To get started:

$ openstack user list
Missing parameter(s): 
Set a username with --os-username, OS_USERNAME, or auth.username
Set an authentication URL, with --os-auth-url, OS_AUTH_URL or auth.auth_url
Set a scope, such as a project or domain, set a project scope with --os-project-name, OS_PROJECT_NAME or auth.project_name, set a domain scope with --os-domain-name, OS_DOMAIN_NAME or auth.domain_name

By default you will need to provide applicable authentication details via arguments or environment variables.
Using the output of the devstack setup, we can obtain applicable details needed for most parameters.

$ ./stack.sh
...
...
...
This is your host IP address: 192.168.56.101
This is your host IPv6 address: ::1
Horizon is now available at http://192.168.56.101/dashboard
Keystone is serving at http://192.168.56.101:5000/
The default users are: admin and demo
The password: passwd

We can now retrieve a summary list of users defined in your project with:

$ openstack --os-username=admin --os-password=passwd --os-auth-url=http://192.168.56.101:5000/ --os-project-name=demo user list
+----------------------------------+----------+
| ID                               | Name     |
+----------------------------------+----------+
| a531ea1011af43bb8277f3e5edfea34b | admin    |
| d6ce303e83b64a2998228c55ebd274c3 | demo     |
| fe7301aa4d2b44b482cd6ba19c24f6b8 | alt_demo |
| e18ae48148df4593b4067785c5e72820 | nova     |
| 9a49deabb7b64454abf411de87c2862c | glance   |
| 1315257f265740f8a32988b014c9e693 | cinder   |
+----------------------------------+----------+

One parameter that is required but no information was available in the devstack installation output was project. There are a number of projects defined in the installation which you can obtain with:

$ openstack --os-username=admin --os-password=passwd --os-auth-url=http://192.168.56.101:5000/ --os-project-name=admin project list
+----------------------------------+--------------------+
| ID                               | Name               |
+----------------------------------+--------------------+
| 3b9f48af38ac40a495ca7b22d4d5c036 | demo               |
| 42c574962a114974bfe35e4a3467df60 | service            |
| 7af69c571e764d5f99688ed2e59930d5 | alt_demo           |
| 893b8954952c4319abd6596b587bba5f | admin              |
| da71fdc9c88f4eddac38937dfef542a2 | invisible_to_admin |
+----------------------------------+--------------------+

By defining authentication with environment variables you can easily simply CLI command usage. For example:

$ export OS_USERNAME=admin
$ export OS_PASSWORD=passwd
$ export OS_AUTH_URL=http://192.168.56.101:5000/
$ export OS_PROJECT_NAME=demo
$ openstack user list
...

devstack pre-packages a few source files that enable you to avoid specifying these arguments or environment variables manually. For example to duplicate this example:

$ source accrc/admin/demo
$ openstack user list

The openstack command provides a --help option to list the available options. You can also inquire as to commands with the command list option.

$ openstack --help
$ openstack command list

With the openstack command line interface you can perform all the operations needed to configure, administer and run your cloud services.

Observing the running services

OpenStack is made up of a number of services, those key services in devstack start with nova, keystone, glance, cinder and horizon. devstack conveniently packages the individual running services into separate screen processes, leveraging a cursors based view of services via the output of log files.

You can view the running screen sessions by reattaching with.

$ screen -r

If you get the following error when attempting to reattach “Cannot open your terminal ‘/dev/pts/0′ – please check.”, you have likely tried reconnecting in a different shell session. You can address this with:

$ script /dev/null
$ screen -r

Commands in screen are driven by a key combination starting with ^a (ctrl-A). ^a d will detach from your screen session you just reattached to. This is what gets you out of screen. See the later section for the full list screen help commands.

On the command line you can run the following command to list the available images via the glance service.

$ openstack image list
+--------------------------------------+---------------------------------+--------+
| ID                                   | Name                            | Status |
+--------------------------------------+---------------------------------+--------+
| 864bad45-d0de-4031-aea6-80b6af72cf2a | cirros-0.3.4-x86_64-uec         | active |
| 75e8b1ef-ae84-41aa-b0a0-7ea785771f14 | cirros-0.3.4-x86_64-uec-ramdisk | active |
| f694bdb1-4bb0-4f18-a7c9-290ad26b1fc8 | cirros-0.3.4-x86_64-uec-kernel  | active |
+--------------------------------------+---------------------------------+--------+

Within screen you can look at the glance api screen log (^a 5) and can observe the logging that occurs in relation to this command. For example we can see an INFO message to get the images (GET /v2/images), and we can see several DEBUG messages. We will use these DEBUG messages in a later post to describe handling logging output.

The INFO message will look like:

2016-04-04 16:24:00.139 INFO eventlet.wsgi.server [req-acf98429-60de-4d18-a69c-36a7d80bed7c a531ea1011af43bb8277f3e5edfea34b 3b9f48af38ac40a495ca7b22d4d5c036] 192.168.1.60 - - [04/Apr/2016 16:24:00] "GET /v2/images HTTP/1.1" 200 2202 0.116774

While we will discuss logging formats in another post, the standard format (in devstack) includes:

Date/Time
Logging Level
Package
Request context. this is made up of
- req-acf98429-60de-4d18-a69c-36a7d80bed7c a request-id, useful for grouping logging records
- a531ea1011af43bb8277f3e5edfea34b refers to the user id (as seen in user list above, i.e. admin)
- 3b9f48af38ac40a495ca7b22d4d5c036 refers to the project id (as seen in the project list above, i.e. demo)
The actual log message

In order to page back in screen output, you enter copy mode “^a [” and then you can use ^b (page back) and ^f (page forward) keys.

Understanding the logging of OpenStack services

What is actually observed in the screen output is what is being logged for the Glance API service. We can verify this with the log file logged in /opt/stack/logs.

$ tail -f /opt/stack/logs/g-api.log

NOTE: You may see that there are colors within both the screen and log output. This is an optional configuration setup used by devstack (not an OpenStack default for logging). We will use this later to show a change in the logging of the service.

We can verify the details of the command used within the screen session (^a 5) by killing the running process with ^c.

Using the bash history, you can up arrow to observe the last running command, and restart this.

/usr/local/bin/glance-api --config-file=/etc/glance/glance-api.conf & echo $! >/opt/stack/status/stack/g-api.pid; fg || echo "g-api failed to start" | tee "/opt/stack/status/stack/g-api.failure"

The actual log file is produced by the screen configuration defined in devstack/stack-screenrc.

screen -t g-api bash
"tuff "/usr/local/bin/glance-api --config-file=/etc/glance/glance-api.conf
logfile /opt/stack/logs/g-api.log.2016-04-04-110956
log on

In a running OpenStack environment you would configure logging output to file as per the log_file option.

Understanding the configuration of OpenStack services

This command indicated a configuration file /etc/glance/glance-api.conf. Glance like other services may contain several configuration files. These are by default defined in the individual projects namespace under /etc.

$ ls -l /etc/glance/
total 152
-rw-r--r-- 1 stack stack 65106 Apr  4 11:12 glance-api.conf
-rw-r--r-- 1 stack stack  3266 Mar 11 12:22 glance-api-paste.ini
-rw-r--r-- 1 stack stack 13665 Apr  4 11:12 glance-cache.conf
-rw-r--r-- 1 stack stack 51098 Apr  4 11:12 glance-registry.conf
-rw-r--r-- 1 stack stack  1233 Mar 11 12:22 glance-registry-paste.ini
drwxr-xr-x 2 stack root   4096 Apr  4 11:12 metadefs
-rw-r--r-- 1 stack stack  1351 Mar 11 12:22 policy.json
-rw-r--r-- 1 stack stack  1380 Mar 11 12:22 schema-image.json

This is an appropriate time to point to several documentation sources including the Glance Developer Documentation – Configuration Options and the Configuration Guide Image Service options which describe in more detail these listed configuration files and the possible options available. You can find similar documentation for other services.

To demonstrate just how the configuration and logging work with a running service the following will modify the logging of the Glance API service by commenting out the logging configuration lines, and then reverting to the oslo.log configuration defaults.

$ sudo vi /etc/glance/glance-api.conf

Comment out the four logging_ options in the [DEFAULT] section.

[DEFAULT]
#logging_exception_prefix = %(color)s%(asctime)s.%(msecs)03d TRACE %(name)s ^[[01;35m%(instance)s^[[00m
#logging_debug_format_suffix = ^[[00;33mfrom (pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d^[[00m
#logging_default_format_string = %(asctime)s.%(msecs)03d %(color)s%(levelname)s %(name)s [^[[00;36m-%(color)s] ^[[01;35m%(instance)s%(color)s%(message)s^[[00m
#logging_context_format_string = %(asctime)s.%(msecs)03d %(color)s%(levelname)s %(name)s [^[[01;36m%(request_id)s ^[[00;36m%(user)s %(tenant)s%(color)s] ^[[01;35m%(instance)s%(color)s%(message)s^[[00m

Now, repeating the earlier steps within the g-api screen window, kill and restart the service.
The first thing you will observe is that the logging no longer contains color (this helps greatly for log file analysis). Repeat the CLI option to list the images, and you will notice a slightly modified logging message occur.

2016-04-05 11:38:57.312 17696 INFO eventlet.wsgi.server [req-1e66b7e5-3429-452e-a9b7-e28ee498f772 a531ea1011af43bb8277f3e5edfea34b 3b9f48af38ac40a495ca7b22d4d5c036 - - -] 192.168.1.60 - - [05/Apr/2016 11:38:57] "GET /v2/images HTTP/1.1" 200 2202 11.551233

The request context now is a modified format (containing additional - - - values) as a result of using the default value of logging_context_format_string. We will discuss the specifics of logging options in a later post.

There are a reasonable number of log files for a minimal devstack installation, some services have multiple log files.

$ cd /opt/stack/logs; ls -l *.log
lrwxrwxrwx 1 stack stack       27 Apr  5 12:49 c-api.log -> c-api.log.2016-04-05-124004
lrwxrwxrwx 1 stack stack       27 Apr  5 12:49 c-sch.log -> c-sch.log.2016-04-05-124004
lrwxrwxrwx 1 stack stack       27 Apr  5 12:49 c-vol.log -> c-vol.log.2016-04-05-124004
-rw-r--r-- 1 stack stack 16672591 Apr  5 14:01 dstat-csv.log
lrwxrwxrwx 1 stack stack       27 Apr  5 12:42 dstat.log -> dstat.log.2016-04-05-124004
lrwxrwxrwx 1 stack stack       27 Apr  5 12:48 g-api.log -> g-api.log.2016-04-05-124004
lrwxrwxrwx 1 stack stack       27 Apr  5 12:48 g-reg.log -> g-reg.log.2016-04-05-124004
lrwxrwxrwx 1 stack stack       29 Apr  5 12:50 horizon.log -> horizon.log.2016-04-05-124004
lrwxrwxrwx 1 stack stack       32 Apr  5 12:42 key-access.log -> key-access.log.2016-04-05-124004
lrwxrwxrwx 1 stack stack       25 Apr  5 12:42 key.log -> key.log.2016-04-05-124004
lrwxrwxrwx 1 stack stack       27 Apr  5 12:48 n-api.log -> n-api.log.2016-04-05-124004
lrwxrwxrwx 1 stack stack       29 Apr  5 12:49 n-cauth.log -> n-cauth.log.2016-04-05-124004
lrwxrwxrwx 1 stack stack       28 Apr  5 12:48 n-cond.log -> n-cond.log.2016-04-05-124004
lrwxrwxrwx 1 stack stack       27 Apr  5 12:49 n-cpu.log -> n-cpu.log.2016-04-05-124004
lrwxrwxrwx 1 stack stack       27 Apr  5 12:48 n-crt.log -> n-crt.log.2016-04-05-124004
lrwxrwxrwx 1 stack stack       28 Apr  5 12:42 n-dhcp.log -> n-dhcp.log.2016-04-05-124004
lrwxrwxrwx 1 stack stack       27 Apr  5 12:48 n-net.log -> n-net.log.2016-04-05-124004
lrwxrwxrwx 1 stack stack       29 Apr  5 12:49 n-novnc.log -> n-novnc.log.2016-04-05-124004
lrwxrwxrwx 1 stack stack       27 Apr  5 12:49 n-sch.log -> n-sch.log.2016-04-05-124004
lrwxrwxrwx 1 stack stack       46 Apr  5 12:40 stack.sh.log -> /opt/stack/logs/stack.sh.log.2016-04-05-124004

To turn off color in logging across service, you can configure this in the devstack local.conf file before starting the stack.

# local.conf
LOG_COLOR=False

Understanding the source code of OpenStack services

devstack installs the OpenStack code in two ways, via packaging and via source.

Generally all libraries are installed via packaging. You can discern these via looking at the python packages via pip with:

$ pip freeze
...
oslo.cache==1.5.0
oslo.concurrency==3.6.0
oslo.config==3.9.0
oslo.context==2.2.0
oslo.db==4.6.0
oslo.i18n==3.4.0
oslo.log==3.2.0
oslo.messaging==4.5.0
oslo.middleware==3.7.0
oslo.policy==1.5.0
oslo.reports==1.6.0
oslo.rootwrap==4.1.0
oslo.serialization==2.4.0
oslo.service==1.7.0
oslo.utils==3.7.0
oslo.versionedobjects==1.7.0
oslo.vmware==2.5.0
...
python-barbicanclient==4.0.0
python-ceilometerclient==2.3.0
python-cinderclient==1.6.0
python-designateclient==2.0.0
python-glanceclient==2.0.0
python-heatclient==1.0.0
python-ironicclient==1.2.0
python-keystoneclient==2.3.1
python-magnumclient==1.1.0
python-manilaclient==1.8.0
python-memcached==1.57
python-mimeparse==1.5.1
python-mistralclient==2.0.0
python-neutronclient==4.1.1
python-novaclient==3.3.0
python-openstackclient==2.2.0
python-saharaclient==0.13.0
python-senlinclient==0.4.0
python-subunit==1.2.0
python-swiftclient==3.0.0
python-troveclient==2.1.1
python-zaqarclient==1.0.0
...

This is a list of all Python packages so it’s not possible to determine which are OpenStack specific, and which are dependencies. These installed packages are actually Python source that you can view and even modify.

$ ls -l /usr/local/lib/python2.7/dist-packages/

You can approximate the installed OpenStack packages via source by looking at the base source directory:

$ ls -l /opt/stack
total 92
drwxr-xr-x 10 stack stack 4096 Mar 11 12:23 cinder
drwxr-xr-x  6 stack root  4096 Apr  5 12:42 data
-rw-r--r--  1 stack stack  440 Apr  5 12:52 devstack.subunit
drwxr-xr-x  4 stack stack 4096 Mar 11 12:27 dib-utils
drwxr-xr-x 10 stack stack 4096 Mar 11 12:22 glance
drwxr-xr-x 15 stack stack 4096 Mar 11 12:26 heat
drwxr-xr-x  7 stack stack 4096 Mar 11 12:27 heat-cfntools
drwxr-xr-x 10 stack stack 4096 Mar 11 12:27 heat-templates
drwxr-xr-x 11 stack stack 4096 Mar 11 14:13 horizon
drwxr-xr-x 13 stack stack 4096 Mar 11 11:57 keystone
drwxr-xr-x  2 stack stack 4096 Apr  5 12:50 logs
drwxr-xr-x 12 stack stack 4096 Mar 11 15:45 neutron
drwxr-xr-x 13 stack stack 4096 Mar 11 12:25 nova
drwxr-xr-x  8 stack stack 4096 Mar 11 12:24 noVNC
drwxr-xr-x  4 stack stack 4096 Mar 11 12:27 os-apply-config
drwxr-xr-x  4 stack stack 4096 Mar 11 12:27 os-collect-config
drwxr-xr-x  5 stack stack 4096 Mar 11 12:27 os-refresh-config
drwxr-xr-x  7 stack stack 4096 Apr  5 12:51 requirements
drwxr-xr-x 13 stack stack 4096 Mar 11 15:47 solum
drwxr-xr-x  3 stack stack 4096 Apr  4 11:13 status
drwxr-xr-x 10 stack stack 4096 Mar 11 12:22 swift

devstack enables you to configure which packages you want to install via source. Checkout plugins for more information. For example, the following added to the local.conf would install solum.

# local.conf
...
enable_plugin solum git://git.openstack.org/openstack/solum

You have complete flexibility of which branch and version of each package using devstack. This enables you to use devstack as a testing tool for code changes.

At this time to understand more about how software is installed check out devstack documentation and review the stack.sh script.

What’s next

This is only a cursory introduction into what devstack sets up during the installation process. Subsequent posts will talk more on topics including the configuration options, the different logging capabilities and how to test code changes.

screen help

^a ? will provide the following help output.

                                                                                     Screen key bindings, page 1 of 2.

                                                                                     Command key:  ^A   Literal ^A:  a

  break       ^B b         dumptermcap .            info        i            meta        a            pow_detach  D            reset       Z            title       A            xoff        ^S s      
  clear       C            fit         F            kill        K k          monitor     M            prev        ^H ^P p ^?   screen      ^C c         vbell       ^G           xon         ^Q q      
  colon       :            flow        ^F f         lastmsg     ^M m         next        ^@ ^N sp n   quit        \            select      '            version     v         
  copy        ^[ [         focus       ^I           license     ,            number      N            readbuf     <            silence     _            width       W         
  detach      ^D d         hardcopy    h            lockscreen  ^X x         only        Q            redisplay   ^L l         split       S            windows     ^W w      
  digraph     ^V           help        ?            log         H            other       ^A           remove      X            suspend     ^Z z         wrap        ^R r      
  displays    *            history     { }          login       L            pow_break   B            removebuf   =            time        ^T t         writebuf    >         

^]   paste .
"    windowlist -b
-    select -
0    select 0
1    select 1
2    select 2
3    select 3
4    select 4
5    select 5
6    select 6
7    select 7
8    select 8
9    select 9
I    login on
O    login off
]    paste .
|    split -v
:kB: focus prev

Running a devstack virtual machine with limited memory

April 4, 2016 by ronald

If you have a system with only 4GB of RAM, you need to assign at least 2.5GB (2560M) to a virtual machine to install devstack. Even with this limited RAM there are times the devstack installation will fail.

One way to give the installation process an opportunity to complete is to configure your virtual machine to have swap space. The amount of swap space you can configure may be limited to the size of your initial disk partition configuration (which is 8GB). The following steps add a 2GB swap file to your virtual machine.

sudo swapon -s
free -m
sudo fallocate -l 2G /swapfile
ls -lh /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
sudo swapon -s
free -m
echo "/swapfile   none    swap    sw    0   0" | sudo tee -a /etc/fstab
cat /etc/fstab

The use of swap space by your virtual machine instead of available RAM will cause a significant slowdown of any software. For the purposes of a minimal installation this option provides a means to observe a running minimal OpenStack cloud.

Downloading and installing devstack

April 2, 2016 by ronald

The following instructions assume you have a running Linux virtual machine that can support the installation of devstack to demonstrate a simple working OpenStack cloud.

For more information about the preparation needed for this step, see these pre-requisite instructions:

Pre-requisites

You will need to login to your Linux virtual machine as a normal user (e.g. stack if you followed these instructions).

To verify the IP address of your machine you can run:

$ ifconfig eth1

NOTE: This assumes you configured a second network adapter as detailed.

You need to determine the IP address assigned. If this is your first-time using VirtualBox and this was configured with default settings, the value will be 192.168.56.101

eth1      Link encap:Ethernet  HWaddr 08:00:27:db:42:6e  
          inet addr:192.168.56.101  Bcast:192.168.56.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:fedb:426e/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:398500 errors:0 dropped:0 overruns:0 frame:0
          TX packets:282829 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:35975184 (35.9 MB)  TX bytes:59304714 (59.3 MB)

Verify that you have applicable sudo privileges.

$ sudo id

If you are prompted for a password, then your privileges are not configured correctly. See here.

Download devstack

After connecting to the virtual machine the following commands will download the devstack source code:

$ sudo apt-get install -y git-core
# NOTE: You will not be prompted for a password
#       This is important for the following installation steps
$ git clone https://git.openstack.org/openstack-dev/devstack

Configure devstack

The following will create an example configuration file suitable for a default devstack installation.

$ cd devstack
# Use the sample default configuration file
$ cp samples/local.conf .
$ HOST_IP="192.168.56.101"
$ echo "HOST_IP=${HOST_IP}" >> local.conf

NOTE: If your machine has different IP address you should specify this alternative value.

Install devstack

$ ./stack.sh

Depending on your physical hardware and network connection, this takes approximately 20 minutes.

When completed you will see the following:

...
This is your host IP address: 192.168.56.101
This is your host IPv6 address: ::1
Horizon is now available at http://192.168.56.101/dashboard
Keystone is serving at http://192.168.56.101:5000/
The default users are: admin and demo
The password: nomoresecrete
While the installation of devstack is happening, you should read Configuration section, and look at the devstack/samples/local.conf sample configuration file being used.

Accessing devstack

You now have a running OpenStack cloud. There are two easy ways to access the running services to verify.

Connect the Horizon dashboard in your browser with the URL (e.g. http://192.168.56.101/), and use the user and password described (e.g. admin and nomoresecrete).
Use the OpenStack client that is installed with devstack, for example:

$ source accrc/admin/admin
$ openstack image list

See Using your devstack cloud for more information about analyzing your running cloud, restarting services, configuration files and how to demonstrate a code change.

Other devstack commands

There are some useful commands to know about with your devstack setup.

If you restart your virtual machine, you reconnect to devstack by re-running the installation (there is no longer a rejoin-stack.sh):

$ ./stack.sh

To shutdown a running devstack.

$ ./unstack.sh

To cleanup your VM of devstack installed software.

$ ./clean.sh

Setting up Ubuntu on VirtualBox for devstack

March 30, 2016 by ronald

As discussed, devstack enables a software developer to run a standalone minimal OpenStack cloud on a virtual machine (VM). In this tutorial we are going to step through the installation of an Ubuntu VM using VirtualBox manually. This is a pre-requisite to installing devstack.

NOTE: There are several different ways in creating a base Ubuntu VM image. These steps are the more manual approach, however they are provided for completeness in understanding varying options.

Pre-requisites

You will need a computer running a 64 bit operating system on Mac OSX, Windows, Linux or Solaris with at least 4GB of RAM and 10GB of available disk drive space.
You will need to have a working VirtualBox on your computer. See Setting up VirtualBox to run virtual machines as a pre-requisite for these steps.
You will need an Ubuntu server .iso image. Download the Ubuntu Server 14.04 (Trusty) server image (e.g. ubuntu-14.04.X-server-amd64.iso) to your computer. This will be the base operating system of your virtual machine that will run devstack.

If using Mac OS X or Linux you can obtain a recent .iso release with the command:

$ wget http://releases.ubuntu.com/14.04/ubuntu-14.04.4-server-amd64.iso

NOTE: devstack can be installed on different operating systems. As a first time user, Ubuntu 14.04 is used as this is a more common platform (and used by OpenStack infrastructure). Other operating systems include Ubuntu (14.10, 15.04, 15.01), Fedora (22, 23) and CentOS/RHEL 7.

Create an Ubuntu Virtual Machine

To create a virtual machine in VirtualBox select the New icon. This will prompt you for some initial configuration. Use these recommendations:

Name and operating System
- Name: devstack
- Type: Linux
- Version: Ubuntu (64-bit)
Memory Size
- If you have 8+GB use 4GB.
- If you have only 4GB use 2.5GB. (Note. Testing during the creation of this guide found that 2048M was insufficient, and that a minimum of 2560M was needed)
Hard Disk
- Use the default settings including 8.0GB, VDI type, dynamically allocated, File location and size.

By default your virtual machine is ready to install however by making the following network recommendation it will be easier to access your running virtual machine and devstack from your host computer.

Click Settings
Select Network
Enable Adapter 2 and attach to a Host-only Adapter and select vboxnet0
Ok

You are now ready to install the Operating System on the virtual machine with the following instructions.

Click Start
Open the Ubuntu .iso file you just downloaded.
You will be prompted for a number of options, select the default provided and use the following values when prompted.
Install Ubuntu Server
English (or your choice)
United States (or your location)
No for configure the keyboard
English (US) for keyboard (or your preference)
English (US) for keyboard layout (or your preference)
Select eth0 as your primary network interface
Select default ubuntu for hostname
Enter stack for full username/username
Enter Openstack for password (or your own preference)
Select No to encrypt home directory
Select Yes for time zone selected
Select Guided – use entire disk for partition method
Select highlighted partition
Select Yes to partition disks
Select Continue for package manager proxy
Select No automatic updates
Select OpenSSH Server in software to install
Select Yes to install GRUB boot loader
Select Continue when installation complete

The new virtual machine will now restart and you will be able to login with the username and password specified (i.e. stack and Openstack).

Post Installation

After successfully logging in run the following commands to complete the Ubuntu installation setup needed as pre-requisites to install devstack.

$ sudo su -
# Enter your stack user password
$ umask 266 & echo "stack ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/stack
$ apt-get update && apt-get upgrade -y
$ echo "auto eth1
iface eth1 inet dhcp" >> /etc/network/interfaces
$ ifup eth1

You are now ready to download and install devstack.

You can also setup an Ubuntu virtual machine via vagrant which simplifies these instructions.

More information

This blog is a series for the software developer with no experience in OpenStack to experience just the tip of functionality and features to become more interested in the project.

Installing VirtualBox for OpenStack development

March 30, 2016 by ronald

Download VirtualBox for your operating system

VirtualBox is an open source virtualization product that will allow you to create virtual machines on a computer using Linux, Mac OS X or Windows. While the current version is 5.x, older versions will also work if you are already using this software.

NOTE: There are different products that can provide virtualization on your computer. As a first time user with virtualization, VirtualBox is a common open source product used by developers.

Install VirtualBox on your system

Just follow the default prompts.

Recommended VirtualBox Networking

To provide for a better experience for installing and accessing devstack or RDO the following VirtualBox configuration setup is recommended to create a host-only adapter network on your host machine.

Start VirtualBox
Open Preferences (e.g. File|Preferences)
Select Network
Select Host-only Networks
Add Network (accept all defaults)

This additional step will create a network configuration in VirtualBox that is called vboxnet0. This will define a network in the 192.168.56.X range, and will configure a DHCP server that will issue IP addresses starting at 192.168.56.101. This will enable you to more easily access your VMs from your host computer as discussed in VirtualBox networking for beginners. ‎

Installing Openstack with devstack, a first-time guide

March 29, 2016 by ronald

This guide will enable the reader to install a minimal OpenStack cloud using devstack for the first time.

This guide will assume you have never installed virtualization software, used or configured devstack or even observed a running OpenStack cloud. This guide does assume that you can perform some basic software development instructions as documented.

This guide is targeted towards the software developer that may want to review the Python code and contribute to the open source project or the system architect that wants to evaluate some of the features of OpenStack. If you are an end user should try a public cloud that runs OpenStack such as OVH, Rackspace or other public cloud providers listed in the OpenStack Marketplace).

There are some hardware requirements and various copy/paste command line instructions on a Linux virtual machine. While it would be possible to publish a completed virtual machine you could download and click to run, understanding the underpinnings of the most basic installation and configuration of devstack will provide an appreciation of the complexity of the product and the software development capabilities.

At the end of this process you will have a running OpenStack cloud on your computer that is running on a Linux virtual machine. You will be able to access this with your browser and be able to perform basic cloud infrastructure tasks, such as creating a compute instance. This guide is not intended to talk about the benefits or usages of a cloud.

You will need a computer running Mac OS X, Windows, Linux (see supported list) or Solaris with at least 4GB of RAM and 10GB of available disk drive space in order to complete the following steps.

NOTE: These steps will provide one means of installing devstack with one type of virtualization software on a specific Linux operating system. This is only meant as a first-time users guide, and some pre-defined decisions have been made. There are multiple ways to implement and use devstack with different software and operating systems.

What’s next?

Without knowing the purpose of following this first-time guide what’s next depends on your. As a software developer you may be interested in looking OpenStack Bugs or contributing to new features of one of the many projects. As an architect you may want to understand a more complex configuration setup as you plan to determine what may be necessary to utilize a cloud infrastructure in your organization. This guide is only intended as the first introduction and hopefully has provided the intended result for the reader to consider what OpenStack can possibly provide.

More references

We will assume you have never installed virtualization software on your computer and have not installed devstack, or even seen an OpenStack interface. The devstack documentation does not make this assumption and so these more generic instructions are useful to the uninitiated. While some (including this author) feel these are instructions worthy of the official devstack documentation, others (with valid reasons) do not and hence the democracy of a large distributed open source project. For more information see review #290854. This guide joins the many others searchable by Internet search engines.

devstack, your personal OpenStack Cloud

March 28, 2016 by ronald

As a software developer or system architect that is interested in looking at the workings of OpenStack, devstack is one of several different ways to start a personal cloud using the current OpenStack code base.

In it’s most basic form, you can run devstack in a virtual machine and be able to manage your personal cloud via the Horizon web interface (known as the Dashboard), or via several CLI APIs such as the OpenStack client (OSC). You can use this to launch compute services, manage boot images and disk volumes, define networking and configure administrative users, projects and roles.

The benefit of devstack is for the developer and deployer. You can actually see the running cloud software, interact and engage with individual services. devstack is a valuable tool to debug and bugfix services. devstack is used by the OpenStack CI/CD system for testing so it is robust enough to evaluate the core projects and many of the available projects that can be configured to be installed with devstack. You can also configure to use trunk (i.e. master) code, or specific branches or tags for individual services. The CI system for example will install the trunk of services, and the specific branch of a new feature or bug fix for one given project in order to perform user and functional testing.

devstack also enables more complex configuration setups. You can setup devstack with LXC containers, you can run a multi-node setup, you can run with Neutron networking. While devstack installs a small subset of projects including keystone, nova, cinder, glance and horizon, you can use devstack to run other OpenStack projects such as Manila, Trove, Magnum, Sahara, Solum and Heat.

The benefit of devstack is for evaluation of capabilities. devstack is not a product to use to determine a path for production deployment of OpenStack. This process includes many more complex considerations of determining why you want to implement an infrastructure for demand for your organization, and considerations of the most basic technical needs such as uptime and SLA requirements, high availability, monitoring and alerting, security management and upgrade paths of your software.

If you are ready to see what OpenStack could provide and want to run a local cloud, you can start with installing Openstack with devstack, a first-time guide.

Additional References

devstack documentation

Disabling the temporary authorization token in devstack keystone

May 5, 2015 by ronald

While building my own OpenStack cloud on physical servers I realized that Keystone uses a temporary authorization token in the Create the service entity and API endpoint and Create projects, users, and roles steps.

The Verify operation step makes reference to removing this mechanism however my current devstack installations have not done this.

To verify this I use the SERVICE_TOKEN as defined in my devstack/local.conf and the Keystone Admin URL.

$ openstack --os-token=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee --os-url=http://controller:35357/v2.0 user list
+----------------------------------+----------------------------------+
| ID                               | Name                             |
+----------------------------------+----------------------------------+
| 554209509f5b47e286e0379bcbf66762 | admin                            |
| 59ac0457a80d449c9dac3b66848f2b5b | demo                             |
| 8aab962698f9460692efb8d3aab35886 | verify_tempest_config-1304647972 |
| 8b602467cd9045888687987067cbd3f6 | alt_demo                         |
| a134c3b33e94475fb5398643dd816053 | glance                           |
| c68c68579ec0437094a14dfbc4728224 | cinder                           |
| e65bd34ca85a429ea5c56bf980f77d67 | nova                             |
+----------------------------------+----------------------------------+

Removing the configuration settings as documented from /etc/keystone/keystone-paste.ini as documented DOES NOT disable this level of access.

NOTE: This edit removes the admin_token_auth option from the pipeline setting in the [pipeline:public_api], [pipeline:admin_api] and [pipeline:api_v3] sections.

$ sudo sed -ie "s/ admin_token_auth / /" /etc/keystone/keystone-paste.ini

$ openstack --os-token=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee --os-url=http://controller:35357/v2.0 user list
+----------------------------------+----------------------------------+
| ID                               | Name                             |
+----------------------------------+----------------------------------+
| 554209509f5b47e286e0379bcbf66762 | admin                            |
| 59ac0457a80d449c9dac3b66848f2b5b | demo                             |
| 8aab962698f9460692efb8d3aab35886 | verify_tempest_config-1304647972 |
| 8b602467cd9045888687987067cbd3f6 | alt_demo                         |
| a134c3b33e94475fb5398643dd816053 | glance                           |
| c68c68579ec0437094a14dfbc4728224 | cinder                           |
| e65bd34ca85a429ea5c56bf980f77d67 | nova                             |
+----------------------------------+----------------------------------+

An additional (and not presently documented step) of restarting apache is needed to invalidate this access.

$ sudo service apache2 restart

$ openstack --os-token=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee --os-url=http://controller:35357/v2.0 user list
ERROR: openstack Could not find token: aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee (Disable debug mode to suppress these details.) (HTTP 401) (Request-ID: req-617961b7-012a-4d61-bdfb-aa738b8f788f)

The results for the command as shown can be produced by using the user/password credentials with the Keystone public URL.

$ openstack --os-username=admin --os-password=passwd --os-project-name=admin --os-auth-url=http://localhost:5000/ user list
+----------------------------------+----------------------------------+
| ID                               | Name                             |
+----------------------------------+----------------------------------+
| 554209509f5b47e286e0379bcbf66762 | admin                            |
| 59ac0457a80d449c9dac3b66848f2b5b | demo                             |
| 8aab962698f9460692efb8d3aab35886 | verify_tempest_config-1304647972 |
| 8b602467cd9045888687987067cbd3f6 | alt_demo                         |
| a134c3b33e94475fb5398643dd816053 | glance                           |
| c68c68579ec0437094a14dfbc4728224 | cinder                           |
| e65bd34ca85a429ea5c56bf980f77d67 | nova                             |
+----------------------------------+----------------------------------+

Writing re-runable shell script

February 24, 2015 by ronald

I recently started playing with devstack again (An all-in-on OpenStack developer setup). Last time was over 3 years ago because I remember a pull request for a missing dependency at the time.

The installation docs provide information to bootstrap your system with a necessary user and privileges, however like many docs for software setup they contain one off instructions.

adduser stack
echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers

When you write operations code you need to always be thinking about “testability” and “automation”. It is important to write re-runable code. You should always write parameterized code when possible, which can be refactored into usable functions at any time.

This is a good example to demonstrate a simple test condition for making the initial instructions re-runable.

sudo su -
NEW_USER="stack"
# This creates default group of same username
# This creates user with default HOME in /home/stack
[ `grep ${NEW_USER} /etc/passwd | wc -l` -eq 0 ] && useradd -s /bin/bash -m ${NEW_USER}
NEW_USER_SUDO_FILE="/etc/sudoers.d/${NEW_USER}"
[ ! -s ${NEW_USER_SUDO_FILE} ] && umask 226 && echo "${NEW_USER} ALL=(ALL) NOPASSWD: ALL" > ${NEW_USER_SUDO_FILE}
ls -l ${NEW_USER_SUDO_FILE}