Tag Archives: devops

This post shows you how to install Rundeck with the Ansible plugin on Centos 7. The installation is done with nginx as the web server and using SSL with a self signed certificate. Please read the Ansible installation script, and modify anything that should be different for your situation. You will be amazed how well readable the installation script is!

Rundeck is a web based user interface that allows you to run commands against a group of hosts. Rundeck has an ansible plugin. Using that plugin, it could perform the similar functionality as Ansible Tower (commercial product) or Semaphore (open source).

After a fresh installation of Centos 7, do the following as root:

yum -y localinstall
yum -y install ansible git
git clone

This adds the EPEL (extra packages for Enterprise Linux) repository to Centos, which contains ansible and git. The next yum command installs ansible (the orchestration software) and git (distributed version control). The git clone command pulls the ansible orchestration script from my install_rundeck git repository at gitlab.

It is important to open the install_rundeck/install_rundeck.yml script and modify anything that is different in your situation. The public hostname will most likely be different than the in the script. You might want to change the locality information with the certificate generation (unless you live in Lutjebroek like me :-). If you do a corporate installation, you might want to skip this step altogether and get a certificate pair signed by your company’s CA.

Please mind I ran into issues with ansible inventories:
– The hosts in the inventory need to have ansible run against them to pick up their properties and become visible in rundeck in the nodes tab. For being able to have ansible run against the hosts in the inventory, the host need to exist in the ssh known hosts file of the rundeck user, otherwise ansible can’t run and the host or hosts will not be visible in rundeck. The solution is to log in as the rundeck user and logon to the hosts in the inventory once manually and accept the host or hosts. From a security perspective it’s understandable that you careful need to assess the hosts to trust, but from an automation standpoint this is annoying. Outside of essentially filling out the known hosts file as I described, there are several other workarounds.
– I created an ansible inventory file in the rundeck project directory and entered the hosts in it. Rundeck picked up the hosts (after solving the above point they became visible in the nodes tab), however when executing something using ansible via rundeck it would say ‘[WARNING]: provided hosts list is empty, only localhost is available’. This means ansible was not pointed explicitly to an inventory, so it used the default one. In order to solve this, I symlinked my (rundeck) project inventory to the /etc/ansible/hosts to make it centrally available. Apparently, using a central inventory for ansible using the plugin is by design. I would rather have rundeck generate an inventory per execution, and pointing to it when the plugin executes ansible.

Now install rundeck:

ansible-playbook install_rundeck/install_rundeck.yml



This is a short blogpost meant as both an introduction for those who don’t know Ara and a guide on how to install Ara.
Ara means ‘Ansible Runtime Analysis’, and is a tool for storing metadata that Ansible uses during execution. It is very valuable, because it takes a lot of guesswork and entering debug statements in your playbook away.

This is a guide on how to install Ara on Oracle Linux 7. I assume ansible is already installed. If want to start fresh, add EPEL and yum install ansible and git. That’s all you need to begin!

First, become root and install ara using a playbook:

mkdir ansible
cd $_
mkdir roles
git clone roles/ara
echo "- hosts: localhost
  - ara" > install_ara.yml
ansible-playbook install_ara.yml
useradd ansible
systemctl stop ara

Ara and a systemd unit (service) is created and started by the playbook. I create an ‘ansible’ user and stop the service. The ansible ara role creates a service running as root, I don’t like running daemons as root if it’s not necessary.

The next part is executed as the user that is supposed to run ansible, which I created above:

echo "[defaults]
local_tmp = /home/ansible/.ansible/tmp
callback_plugins = /usr/lib/python2.7/site-packages/ara/plugins/callbacks
dir = /home/ansible/.ara " > ~/.ansible.cfg

What this does, is define the callback for ara so executions are recorded. The default way of recording is entering the details in a sqlite3 database. This database (which is a file) is stored in /home/ansible/.ara, which is defined in the [ara] section using ‘dir’.

The next part is executed as root again:

pip install --upgrade ara
echo "[Unit]

ExecStart=/bin/ara-manage runserver -h -p 9191

[Install]" > /etc/systemd/system/ara.service
systemctl daemon-reload
systemctl start ara

This first executes pip (which is installed by the playbook above) and does an upgrade of ara and it’s depended packages. If this is not done, the website run by ara-manage will throw a 500 internal error (!!). Then the echo overwrites the current ara webserver config run as root by a modified version that runs it as ansible. Next we inform systemd it needs to reload the config, and then we start the ara service again.

Now run an ansible playbook as the ansible user, and go to port 9191 with your browser. It will detail the entire run, and list all the plays and tasks, with all information! For the CLI die-hards, there is a CLI tool ‘ara’ which provides the same information as the website, but entirely text based. It can generate static html files for sending through though.

If you don’t want to record everything, remove or comment the callback_plugins setting in ~/.ansible.cfg. Uncomment or add back if you want to record again. You can quite simply purge the history by stopping the ara service and remove the ~/.ara/ansible.sqlite file. Ara has an option to use mysql or postgresql instead of sqlite.

The CLI provides easy access to the results once you figured out how to use it. This is an example:
First list all the playbook run (I ran one playbook):

$ ara playbook list
| ID                                   | Path                           | Time Start          | Duration | Complete | Ansible Version |
| a152d1e5-fdab-4f50-b84a-de2d0b336124 | /home/ansible/ansible/test.yml | 2017-05-11 16:57:25 | 0:00:00  | True     |         |

Then show the tasks in this playbook:

$ ara task list -b a152d1e5-fdab-4f50-b84a-de2d0b336124
| ID                                   | Name            | Path | Line | Action  | Time Start          | Duration |
| bf0bbc33-b0a4-4ac3-8242-58f5c8f1c4e5 | Gathering Facts | None | None | setup   | 2017-05-11 16:57:25 | 0:00:00  |
| 2b952b48-611f-424d-ba06-b94e22e87b1c | command         | None | 3    | command | 2017-05-11 16:57:26 | 0:00:00  |

Once you got the task IDs, you can list the task results:

$ ara result list --task 2b952b48-611f-424d-ba06-b94e22e87b1c
| ID                                   | Host      | Action  | Status  | Ignore Errors | Time Start          | Duration |
| d9278719-209e-41f8-8ecc-4ad2681dcdf6 | localhost | command | changed | False         | 2017-05-11 16:57:26 | 0:00:00  |

And then show the result in long format:

$ ara result show d9278719-209e-41f8-8ecc-4ad2681dcdf6 --long
| Field         | Value                                       |
| ID            | d9278719-209e-41f8-8ecc-4ad2681dcdf6        |
| Playbook ID   | a152d1e5-fdab-4f50-b84a-de2d0b336124        |
| Playbook Path | /home/ansible/ansible/test.yml              |
| Play ID       | 683dbe2b-84cc-4e17-b2ec-9b013e7d7612        |
| Play Name     | localhost                                   |
| Task ID       | 2b952b48-611f-424d-ba06-b94e22e87b1c        |
| Task Name     | command                                     |
| Host          | localhost                                   |
| Action        | command                                     |
| Status        | changed                                     |
| Ignore Errors | False                                       |
| Time Start    | 2017-05-11 16:57:26                         |
| Time End      | 2017-05-11 16:57:26                         |
| Duration      | 0:00:00                                     |
| Result        | {                                           |
|               |     "changed": true,                        |
|               |     "cmd": "echo \"test\"",                 |
|               |     "delta": "0:00:00.002718",              |
|               |     "end": "2017-05-11 16:57:26.616876",    |
|               |     "rc": 0,                                |
|               |     "start": "2017-05-11 16:57:26.614158",  |
|               |     "stderr": "",                           |
|               |     "stderr_lines": [],                     |
|               |     "stdout": "test",                       |
|               |     "stdout_lines": [                       |
|               |         "test"                              |
|               |     ]                                       |
|               | }                                           |

This is how to install ara on OSX:

sudo pip install ara --user

Not sure if I need to use sudo when installing with ‘–user’, I ended up with directories in my local user Library directory for use with python that were not readable to my user ‘frits.hoogland’. In order to correct that, I ran:

sudo chown -R frits.hoogland Library 

Now fetch the ara directory in the python site-packages directory:

python -c "import os,ara; print(os.path.dirname(ara.__file__))"

Now create a ansible config file to be able to active the ara callback for running ansible-playbook and point the ara server to the sqlite database:

echo "[defaults]
local_tmp = /Users/frits.hoogland/.ansible/tmp
callback_plugins = /Users/frits.hoogland/Library/Python/2.7/lib/python/site-packages/ara/plugins/callbacks
dir = /Users/frits.hoogland/.ara" > .ansible.cfg

At this moment the ara callback is activated. If you don’t want the overhead of ara writing all information to the sqlite database, comment ‘callback_plugins’.

If you want to use the ara webserver, active it using:

nohup ara-manage runserver -h localhost -p 9191 &

If you want to deactivate the ara webserver, stop it using:

pkill -f ara-manage

This is a blogpost about how I setup my test virtual machines. The seasoned sysadmin and DBA will notice that the techniques used here are perfectly usable for real production environments. The most important thing is there is no need to download or stage any software for installing the virtual machine, everything is downloaded when needed during installation. Obviously this works best when you have got reasonable bandwidth available for connecting to the internet.

The main infrastructure software components of this setup are:
Virtualbox as the virtualisation software.
Ansible as the configuration and management engine.
Vagrant as the virtualisation manager.

Installation (Mac OSX specific).
– Virtual box is installed by downloading and installing the installation image in the normal way.
– Ansible requires a few steps. Ansible relies on python.
First install pip using easy_install:

$ sudo easy_install pip

Then install ansible for your current user:

$ sudo pip install ansible --user

Now to pick up the ansible local install, add the following to .bash_profile:

export PATH=$PATH:~/Library/Python/2.7/bin

– Vagrant is installed by downloading and installing the installation image in the normal way.

My versions:

$ VBoxManage -v
$ ansible --version
  config file = /Users/fritshoogland/.ansible.cfg
  configured module search path = Default w/o overrides
$ vagrant version
Installed Version: 1.8.5
Latest Version: 1.8.5

You're running an up-to-date version of Vagrant!

Okay! Now all the software is installed, up to the next step!

SSH keys issue for vagrant (OSX specific).
For ssh certificate based authentication to work, the private key file needs to protected by mode 600 (rw-|—|—). Vagrant uses a ssh key to access the virtual machine. I am running the vagrant directory from a share, which forces the mode of the files to be 700. In order to workaround that issue, I setup my own keys in ~/.vagrant_ssh:

$ mkdir .vagrant_ssh
$ cd .vagrant_ssh
$ ssh-keygen -q -N "" -f id_rsa

Later on some settings are made in the Vagrantfile to use the private key just generated.

SSH keys for normal authentication (OSX specific).
The Ansible scripts copy your ssh public key and put it in the authorised_keys file of both the oracle and the root user. By putting your public key in the authorized_keys file of a user, the authentication part of logging on is done via the public key.

However, for this to work, you first need to have a private and public key pair. There are many ways of doing that, this is an example of that:

Check if you have a dsa key pair:

$ cd ~
$ find .ssh

In this case, as you can see, there are two id_dsa files, one with the extension “.pub”, which is the public key, and the other one without an extension is the private key. The private key should be kept private (at all times). If you don’t have id_dsa keys, set them up in the following way:

$ cd ~
$ mkdir .ssh
$ chmod 700 .ssh
$ cd .ssh
$ ssh-keygen -q -N "" -t dsa -f id_dsa

Setting the Virtualbox images directory.
Virtualbox will put the virtual machines in it’s default machine folder. You can see where the default machine folder is set to using the VBoxManage utility:

$ VBoxManage list systemproperties | grep machine
Default machine folder:          /Volumes/VirtualBox

Because the virtual machines are disk space intensive, you might want to change the location. At least, I later decided to dedicate an external disk to it. The way to get the disk images in another location, is change the virtualbox settings:

$ VBoxManage setproperty machinefolder /Volumes/Virtualbox

The vagrant ‘box’ location.
Vagrant uses operating system images that it is calling ‘boxes’. These ‘boxes’ are the base image that is copied when a virtual machine is created. It is stored to save it from downloading over and over. The location of these images or ‘boxes’ is ~/.vagrant.d/boxes. If you want to store the ‘boxes’ somewhere else, you can set the environment variable VAGRANT_HOME to make vagrant use a different location for storing its global state and the ‘boxes’.

Setting up the vagrant directory.
The next step is to set up the directory where the virtual machine will be run. The simplest way of doing so is cloning the files from github:

$ git clone

This will create a ‘Vagrant-oracle-setup’ directory.
Now you need to fill out the Vagrantfile in the directory:
a) line 12: set the hostname. (change optional)
b) line 14: set an IP address. (do not choose an already existing network, like the default virtualbox 10.0.2/24 network)
c) line 18/19: set the memory and CPU capacity for the VM to use.
d) line 41/42: set your My Oracle Support credentials to download the installation media.
e) line 43: set the IP address for ansible, as set at b)/line 14.
f) line 46: set the database name to create. (change optional)

Install the virtual machine with linux, the oracle database and setup a database.
Now the cool part: install linux, install the oracle database software and create a database without any manual intervention! For this you need to go into the Vagrant-oracle-setup directory, and execute:

$ vagrant up

That’s it.

You can follow the steps vagrant and ansible are executing. You’ll see vagrant first fetching (downloading) the OS image (which it will save to prevent from downloading it again), and then copy this image to be the image for the virtual machine to run. Vagrant then makes some settings, mostly network related, and hands over the control to ansible.

Ansible executes the playbook as set in the Vagrantfile, which I’ve put in the ‘ansible’ directory. Ansible sets up linux, configures the extra block devices using LVM for the software home and the database, gets the installation media from My Oracle Support, installs the software and then runs DBCA to create a database.

Further things.
Handling vagrant to administer the virtual machine is easy. Go into the directory with the Vagrantfile and use:
– vagrant halt: stop the virtual machine.
– vagrant up: for a new machine, get the base image and provision the machine. For an existing machine: startup the virtual machine.
– vagrant destroy: stop the virtual machine and remove all the information it has used like configuration/provisioning status and the disks.
– vagrant provision: run the ansible playbook again.

Anything else? Well, yes, I didn’t setup a listener for example. It should be really simple and straightforward: just start the listener. The listener will start listening on all network devices, and the database will register itself. There might be much more, depending on your specific need.

Thanks to Maris Elsins for getMOSPatch.

%d bloggers like this: