Moving to OpenStackClient CLI

In working with the keynote CLI within the TripleO scripts I came across the following deprecation warning message.

$ keystone token-get
/usr/local/lib/python2.7/dist-packages/keystoneclient/shell.py:65: DeprecationWarning: The keystone CLI is deprecated in favor of python-openstackclient. For a Python library, continue using python-keystoneclient.
  'python-keystoneclient.', DeprecationWarning)

Time to switch to using the OpenStackClient , historically also called the unified CLI.

Very easy to install.

$ sudo pip install python-openstackclient

$ openstack help
usage: openstack help [-h] [cmd [cmd ...]]

print detailed help for another command

positional arguments:
  cmd         name of the command

optional arguments:
  -h, --help  show this help message and exit

I would also suggest you add the following alias to your startup shell rc.

alias os='openstack'

The --help option also provides a much detailed list of available argument options.

$ os --help
usage: openstack [--version] [-v] [--log-file LOG_FILE] [-q] [--debug]
                 [--os-region-name ]
                 [--os-cacert ] [--verify | --insecure]
                 [--os-default-domain ] [--timing]
                 [--os-compute-api-version ]
                 [--os-network-api-version ]
                 [--os-image-api-version ]
                 [--os-volume-api-version ]
                 [--os-identity-api-version ]
                 [--os-auth-type ] [--os-username ]
                 [--os-identity-provider ]
                 [--os-project-domain-name ]
                 [--os-project-domain-id ]
                 [--os-project-name ]
                 [--os-auth-url ]
                 [--os-trust-id ]
                 [--os-service-provider-endpoint ]
                 [--os-user-domain-id ]
                 [--os-domain-name ]
                 [--os-identity-provider-url ]
                 [--os-token ] [--os-domain-id ]
                 [--os-url ]
                 [--os-user-domain-name ]
                 [--os-user-id ] [--os-password ]
                 [--os-project-id ]
                 [--os-object-api-version 


  ] [-h]
  
  Command-line interface to the OpenStack APIs
  
  optional arguments:
    --version             show program's version number and exit
    -v, --verbose         Increase verbosity of output. Can be repeated.
  ...
  

The new CLI provides a number of benefits above the consolidation of syntax into a single client. There is the flexibility in formatting output, both selecting columns and output format.

Working with nova for more simple initial examples.

$ nova image-list
+--------------------------------------+---------------------------------+--------+--------+
| ID                                   | Name                            | Status | Server |
+--------------------------------------+---------------------------------+--------+--------+
| a8672506-049f-4fda-bc58-e64a646d587e | cirros-0.3.2-x86_64-uec         | ACTIVE |        |
| 557dec3a-f912-430c-bda1-ace9c669b78d | cirros-0.3.2-x86_64-uec-kernel  | ACTIVE |        |
| 1d9b6c2e-96d9-4426-a98d-7fa378c26189 | cirros-0.3.2-x86_64-uec-ramdisk | ACTIVE |        |
+--------------------------------------+---------------------------------+--------+--------+
$ openstack image list
+--------------------------------------+---------------------------------+
| ID                                   | Name                            |
+--------------------------------------+---------------------------------+
| a8672506-049f-4fda-bc58-e64a646d587e | cirros-0.3.2-x86_64-uec         |
| 1d9b6c2e-96d9-4426-a98d-7fa378c26189 | cirros-0.3.2-x86_64-uec-ramdisk |
| 557dec3a-f912-430c-bda1-ace9c669b78d | cirros-0.3.2-x86_64-uec-kernel  |
+--------------------------------------+---------------------------------+

As you can see the Status and Server columns are not in the default format. You can access a list of columns, however both the column order, and even the contents (e.g. ACTIVE v active) means you need to adjust your existing scripts. In this case Status exists, Server does not.

$ openstack image list --long
+--------------------------------------+---------------------------------+-------------+------------------+----------+--------+------------+-----------+----------------------------------+-----------------------------------------------------------------------------------------------------+
| ID                                   | Name                            | Disk Format | Container Format |     Size | Status | Visibility | Protected | Owner                            | Properties                                                                                          |
+--------------------------------------+---------------------------------+-------------+------------------+----------+--------+------------+-----------+----------------------------------+-----------------------------------------------------------------------------------------------------+
| a8672506-049f-4fda-bc58-e64a646d587e | cirros-0.3.2-x86_64-uec         | ami         | ami              | 25165824 | active | public     | False     | 0087dccc995e4ea3aaf209ddc8ad33e2 | kernel_id='557dec3a-f912-430c-bda1-ace9c669b78d', ramdisk_id='1d9b6c2e-96d9-4426-a98d-7fa378c26189' |
| 1d9b6c2e-96d9-4426-a98d-7fa378c26189 | cirros-0.3.2-x86_64-uec-ramdisk | ari         | ari              |  3723817 | active | public     | False     | 0087dccc995e4ea3aaf209ddc8ad33e2 |                                                                                                     |
| 557dec3a-f912-430c-bda1-ace9c669b78d | cirros-0.3.2-x86_64-uec-kernel  | aki         | aki              |  4969360 | active | public     | False     | 0087dccc995e4ea3aaf209ddc8ad33e2 |                                                                                                     |
+--------------------------------------+---------------------------------+-------------+------------------+----------+--------+------------+-----------+----------------------------------+-----------------------------------------------------------------------------------------------------+

Column Options

openstack provides the ability to pass column names, so we can try to simulate what we seen in the legacy nova client.

$ openstack image list -c ID
+--------------------------------------+
| ID                                   |
+--------------------------------------+
| 1b79bbb2-e8b6-430f-8cb7-ced8f3589807 |
| e52bd241-db30-40e2-9230-40f21ff3e4c7 |
| 59e28cdf-c4af-4b02-bdeb-3779ec4c306d |
+--------------------------------------+

$ openstack image list -c ID -c Name
+--------------------------------------+---------------------------------+
| ID                                   | Name                            |
+--------------------------------------+---------------------------------+
| 1b79bbb2-e8b6-430f-8cb7-ced8f3589807 | cirros-0.3.2-x86_64-uec         |
| e52bd241-db30-40e2-9230-40f21ff3e4c7 | cirros-0.3.2-x86_64-uec-ramdisk |
| 59e28cdf-c4af-4b02-bdeb-3779ec4c306d | cirros-0.3.2-x86_64-uec-kernel  |
+--------------------------------------+---------------------------------+

$ openstack image list -c ID -c Name -c Status
+--------------------------------------+---------------------------------+
| ID                                   | Name                            |
+--------------------------------------+---------------------------------+
| 1b79bbb2-e8b6-430f-8cb7-ced8f3589807 | cirros-0.3.2-x86_64-uec         |
| e52bd241-db30-40e2-9230-40f21ff3e4c7 | cirros-0.3.2-x86_64-uec-ramdisk |
| 59e28cdf-c4af-4b02-bdeb-3779ec4c306d | cirros-0.3.2-x86_64-uec-kernel  |
+--------------------------------------+---------------------------------+

The Status column which is in both nova and openstack output above seems to provide no output or even an error. However if specified as the only individual column it draws an expected error message.

stack@ubuntu:~$ openstack image list -c Status
ERROR: openstack No recognized column names in ['Status']

Not knowing what the actual list of valid columns are, as it does not match the --long output we move on.

Update 4/23/15
It pays to read the source code, specifically openstackclient/image/v2/image.py. In order to list additional columns you must also specify --long.

$ python openstackclient/shell.py image list --long -c ID -c Name -c Status
+--------------------------------------+---------------------------------+--------+
| ID                                   | Name                            | Status |
+--------------------------------------+---------------------------------+--------+
| c5153c2b-df9c-488c-995e-5cb347c0ee35 | cirros-0.3.2-x86_64-uec         | active |
| ac7e0c04-e47c-43da-ba28-b0d47d293eb7 | cirros-0.3.2-x86_64-uec-ramdisk | active |
| 52cc99f5-a9aa-4f27-98b8-d8ec762a246d | cirros-0.3.2-x86_64-uec-kernel  | active |
+--------------------------------------+---------------------------------+--------+

I should also point out that column names are Case Sensitive. Id is not valid.

$ python openstackclient/shell.py image list --long -c Id -c Name -c Status
+---------------------------------+--------+
| Name                            | Status |
+---------------------------------+--------+
| cirros-0.3.2-x86_64-uec         | active |
| cirros-0.3.2-x86_64-uec-ramdisk | active |
| cirros-0.3.2-x86_64-uec-kernel  | active |
+---------------------------------+--------+

Format Options

One cool feature is the formatting options. There are currently 6 types.

TABLE

$ openstack image list --format table
+--------------------------------------+---------------------------------+
| ID                                   | Name                            |
+--------------------------------------+---------------------------------+
| a8672506-049f-4fda-bc58-e64a646d587e | cirros-0.3.2-x86_64-uec         |
| 1d9b6c2e-96d9-4426-a98d-7fa378c26189 | cirros-0.3.2-x86_64-uec-ramdisk |
| 557dec3a-f912-430c-bda1-ace9c669b78d | cirros-0.3.2-x86_64-uec-kernel  |
+--------------------------------------+---------------------------------+

CSV

$ openstack image list --format csv
"ID","Name"
"a8672506-049f-4fda-bc58-e64a646d587e","cirros-0.3.2-x86_64-uec"
"1d9b6c2e-96d9-4426-a98d-7fa378c26189","cirros-0.3.2-x86_64-uec-ramdisk"
"557dec3a-f912-430c-bda1-ace9c669b78d","cirros-0.3.2-x86_64-uec-kernel"

JSON

$ openstack image list --format json | python -m json.tool
[
    {
        "ID": "1b79bbb2-e8b6-430f-8cb7-ced8f3589807",
        "Name": "cirros-0.3.2-x86_64-uec"
    },
    {
        "ID": "e52bd241-db30-40e2-9230-40f21ff3e4c7",
        "Name": "cirros-0.3.2-x86_64-uec-ramdisk"
    },
    {
        "ID": "59e28cdf-c4af-4b02-bdeb-3779ec4c306d",
        "Name": "cirros-0.3.2-x86_64-uec-kernel"
    }
]

HTML

$ openstack image list --format html
<table>
<thead>
<tr><th>ID</th>
<th>Name</th></tr>
</thead>
<tr><td>a8672506-049f-4fda-bc58-e64a646d587e</td>
<td>cirros-0.3.2-x86_64-uec</td></tr>
<tr><td>1d9b6c2e-96d9-4426-a98d-7fa378c26189</td>
<td>cirros-0.3.2-x86_64-uec-ramdisk</td></tr>
<tr><td>557dec3a-f912-430c-bda1-ace9c669b78d</td>
<td>cirros-0.3.2-x86_64-uec-kernel</td></tr>
</table>

YAML

$ openstack image list --format yaml
- {ID: a8672506-049f-4fda-bc58-e64a646d587e, Name: cirros-0.3.2-x86_64-uec}
- {ID: 1d9b6c2e-96d9-4426-a98d-7fa378c26189, Name: cirros-0.3.2-x86_64-uec-ramdisk}
- {ID: 557dec3a-f912-430c-bda1-ace9c669b78d, Name: cirros-0.3.2-x86_64-uec-kernel}

SHELL

$ openstack image list --format shell
usage: openstack image list [-h] [-f {csv,html,json,table,yaml}] [-c COLUMN]
                            [--max-width <integer>]
                            [--quote {all,minimal,none,nonnumeric}]
                            [--public | --private] [--property <key=value>]
                            [--long] [--sort <key>[:<direction>]]
openstack image list: error: argument -f/--format: invalid choice: 'shell' (choose from 'csv', 'html', 'json', 'table', 'yaml')

As you can see shell is invalid for the list argument, however it is valid for the show argument.

$ openstack image show a8672506-049f-4fda-bc58-e64a646d587e
+------------------+-----------------------------------------------------------------------------------------------------------------+
| Field            | Value                                                                                                           |
+------------------+-----------------------------------------------------------------------------------------------------------------+
| checksum         | 4eada48c2843d2a262c814ddc92ecf2c                                                                                |
| container_format | ami                                                                                                             |
| created_at       | 2015-03-31T19:42:53.000000                                                                                      |
| deleted          | False                                                                                                           |
| disk_format      | ami                                                                                                             |
| id               | a8672506-049f-4fda-bc58-e64a646d587e                                                                            |
| is_public        | True                                                                                                            |
| min_disk         | 0                                                                                                               |
| min_ram          | 0                                                                                                               |
| name             | cirros-0.3.2-x86_64-uec                                                                                         |
| owner            | 0087dccc995e4ea3aaf209ddc8ad33e2                                                                                |
| properties       | {u'kernel_id': u'557dec3a-f912-430c-bda1-ace9c669b78d', u'ramdisk_id': u'1d9b6c2e-96d9-4426-a98d-7fa378c26189'} |
| protected        | False                                                                                                           |
| size             | 25165824                                                                                                        |
| status           | active                                                                                                          |
| updated_at       | 2015-03-31T19:42:54.000000                                                                                      |
+------------------+-----------------------------------------------------------------------------------------------------------------+
$ openstack image show a8672506-049f-4fda-bc58-e64a646d587e --format=shell
checksum="4eada48c2843d2a262c814ddc92ecf2c"
container_format="ami"
created_at="2015-03-31T19:42:53.000000"
deleted="False"
disk_format="ami"
id="a8672506-049f-4fda-bc58-e64a646d587e"
is_public="True"
min_disk="0"
min_ram="0"
name="cirros-0.3.2-x86_64-uec"
owner="0087dccc995e4ea3aaf209ddc8ad33e2"
properties="{u'kernel_id': u'557dec3a-f912-430c-bda1-ace9c669b78d', u'ramdisk_id': u'1d9b6c2e-96d9-4426-a98d-7fa378c26189'}"
protected="False"
size="25165824"
status="active"
updated_at="2015-03-31T19:42:54.000000"

Interactive

The CLI also provides an interactive shell.

$ openstack
(openstack) image list
+--------------------------------------+---------------------------------+
| ID                                   | Name                            |
+--------------------------------------+---------------------------------+
| 1b79bbb2-e8b6-430f-8cb7-ced8f3589807 | cirros-0.3.2-x86_64-uec         |
| e52bd241-db30-40e2-9230-40f21ff3e4c7 | cirros-0.3.2-x86_64-uec-ramdisk |
| 59e28cdf-c4af-4b02-bdeb-3779ec4c306d | cirros-0.3.2-x86_64-uec-kernel  |
+--------------------------------------+---------------------------------+
(openstack) quit
Tagged with: Cloud Computing OpenStack

Producing IQR and Outlier statistics with SQL

The interquartile range (IQR) measures the spread of the middle 50% of a distribution — the distance between the first quartile (Q1) and the third quartile (Q3). Combined with Tukey’s 1.

Producing Mode statistics with SQL

The mode is the value or values that appear most frequently in a dataset. Unlike the mean or median, it applies naturally to categorical and ordinal data — star ratings, product codes, survey responses — and reveals what is most common, not what is average.

Extending MySQL Capabilies with UDFs, Plugins and Components - Part 2

MySQL offers three different approaches to extending the SQL capabilities with the default product you download and install. These are: User Defined Function (UDF) MySQL Manual MySQL Plugin MySQL Manual MySQL Component MySQL Manual In my prior post I provided a new uuidv function that accepted a numeric argument to return a string of the version of UUID specified.