Archive

Tag Archives: ansible-playbook

This blogpost is about using ansible vault. Vault is a way to encrypt sensitive information in ansible scripts by encrypting it. The motivation for this blogpost is the lack of a description that makes sense to me of what the possibilities are for using vault, and how to use the vault options in playbooks.

The basic way ansible vault works, is that when ansible-playbook reads a yaml file, it encounters $ANSIBLE_VAULT;1.1;AES256 indicating ansible vault is used to encrypt the directly following lines, it will use a password to decrypt it, and then uses the decrypted version in memory only. This way secrets can be hidden from being visible. Obviously, the password will allow decrypting it, and the password must be used in order for ansible-playbook to decrypt it.

The original use of vault is to encrypt an entire yaml file. As of Ansible version 2.3, ansible allows the encryption of single values in a yaml file.

1. File encryption
A very silly example of ansible-vault, which shouldn’t be used (!) is this:

$ echo "---
- hosts: localhost
  vars:
    a: aaa
    b: bbb
  tasks:

  - name: execute a shell command
    shell: >
      ls" > test1.yml

When this runs, it will execute on localhost, set to ansible variables (a and b) and then use the shell module to execute “ls”. Again, this is just an example, and is silly on itself. This is how it’s run:

$ ansible-playbook test1.yml
PLAY [localhost] ***********************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************
ok: [localhost]

TASK [execute a shell command] *********************************************************************************************************
changed: [localhost]

PLAY RECAP *****************************************************************************************************************************
localhost                  : ok=2    changed=1    unreachable=0    failed=0

This can be encrypted using ansible vault in the following way:

$ ansible-vault encrypt test1.yml
New Vault password:
Confirm new Vault password:
Encryption successful

Now the yaml file is entirely encrypted:

$ cat test1.yml
$ANSIBLE_VAULT;1.1;AES256
32663536343532643864353135643431373830333236636230613264383531336562323530386632
3933353561316530333538363032633731336135396434630a626631303839646161363432396230
32336130633264343766393562656138363132393835646137633065393739336234336463336138
6564613730333436650a336537326233343566633761383436356136666533326561346530613566
64316336393036346538656338303734623161386435373138656532663737666662633765656438
37303666393939356161393134666130303435333463636630386434626335383164663761383035
35613437356361366665633931346664386535393936396637363561386164613337373562313264
63333530353263363437386536393765393930653234353733313433643637333932353638353339
31623366376566613766366236356163303837666664646465363439313265383562656637623266
6463663065383030626232643365623437666332313631376262

It still can be run, but you need to specify the password. In this case it asks to enter a password:

$ ansible-playbook test1.yml --ask-vault-pass
Vault password:

PLAY [localhost] ***********************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************
ok: [localhost]

TASK [execute a shell command] *********************************************************************************************************
changed: [localhost]

PLAY RECAP *****************************************************************************************************************************
localhost                  : ok=2    changed=1    unreachable=0    failed=0

Of course this is not practical, the most common and probable way you want to use this is to have variables with secret information, like passwords, tokens, private keys, personal data, etc. encrypted, but the playbook still be readable. You should use a version control tool like git, and use functionality like diff!

The first thing is to move the variables out of the playbook into a dedicated section. In this case I chosen to assign them based on host, in a directory called ‘host_vars’, and then for localhost ‘host_vars/localhost’:

$ cat host_vars/localhost/vars.yml
---
a: aaa
b: bbb

And then of course remove them from the playbook itself:

$ cat test2.yml
---
- hosts: localhost
  tasks:

  - name: execute a shell command
    shell: >
      ls
  - debug: msg="{{ a }} / {{ b }}"

If this is run, it will pick up the variables (for which I put in a debug task):

$ ansible-playbook test2.yml
PLAY [localhost] ***********************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************
ok: [localhost]

TASK [execute a shell command] *********************************************************************************************************
changed: [localhost]

TASK [debug] ***************************************************************************************************************************
ok: [localhost] => {
    "msg": "aaa / bbb"
}

PLAY RECAP *****************************************************************************************************************************
localhost                  : ok=3    changed=1    unreachable=0    failed=0

Okay, now let’s say I need to encrypt the variable ‘a’. I can bluntly encrypt vars.yml, and that will work, however, then you don’t know where the variable comes from. The solution to that is ‘indirect referencing’. This is done by adding a second file to host_vars/localhost called ‘vault.yml’, which will be the encrypted file. In the case of encrypting variable a, do the following:
1. Change the vars.yml file to assign {{ vault_a }} to a:

$ cat host_vars/localhost/vars.yml
---
a: "{{ vault_a }}"
b: bbb

2. Define the value of a to vault_a in vault.yml:

$ cat host_vars/localhost/vault.yml
---
vault_a: aaa

3. Encrypt vault.yml:

$ ansible-vault encrypt host_vars/localhost/vault.yml
New Vault password:
Confirm New Vault password:
Encryption successful

Now it can be used by specifying the password again:

$ ansible-playbook test2.yml --ask-vault-pass
Vault password:
PLAY [localhost] ***********************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************
ok: [localhost]

TASK [execute a shell command] *********************************************************************************************************
changed: [localhost]

TASK [debug] ***************************************************************************************************************************
ok: [localhost] => {
    "msg": "aaa / bbb"
}

PLAY RECAP *****************************************************************************************************************************
localhost                  : ok=3    changed=1    unreachable=0    failed=0

This should not be a surprise, as I said, ansible will recognise an encrypted file, and decrypt it automatically. However, if you looked carefully you saw something else that is important to realise: I encrypted the variable a containing ‘aaa’, and it was revealed to me using the debug statement. I think this is reasonable, because it’s been explicitly programmed to do so. You should write your playbooks not to reveal any information that you encrypted previously.

2. Variable encryption
If you have larger sets of variables to be encrypted, the above method makes sense. However, if you want to encrypt only a few variables it’s probably simpler to encrypt only the variable you want to be encrypted. The ansible-vault executable provides a facility for that called ‘encrypt_string’. The way this works is that you simply specify the variable and the variable value with ansible-vault, which then outputs the encrypted value, which you need to put in a playbook yourself. For example your ‘a’ variable with the value ‘aaa’:

$ ansible-vault encrypt_string --name 'a' 'aaa'
New Vault password:
Confirm New Vault password:
a: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          37363334323763633562313934656631396431636139653036333036356636326163623238626132
          6138343534396232643638323830326462383831366165630a393335303939663138613437633835
          65396461383565316135316339383035626262343664643563373631366338393361623230666634
          3637363232313935310a323564663863336565623366633838616337393366313831396637386432
          3035
Encryption successful

Now use this to input to a playbook:

---
- hosts: localhost
  vars:
    a: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          37363334323763633562313934656631396431636139653036333036356636326163623238626132
          6138343534396232643638323830326462383831366165630a393335303939663138613437633835
          65396461383565316135316339383035626262343664643563373631366338393361623230666634
          3637363232313935310a323564663863336565623366633838616337393366313831396637386432
          3035
    b: bbb
  tasks:

  - name: execute a shell command
    shell: >
      ls
  - debug: msg="{{ a }} / {{ b }}"

3. Vault passwords
The next thing to look at is how to specify the decryption password. One way or another, you need a password specified to decrypt. The first option is to specify the password at the CLI, which is done in the previous examples by specifying ‘–ask-vault-pass’ as an argument to ansible-playbook. This is a good option for running playbooks ad-hoc.

If you want playbooks to run from a facility like ansible tower, rundeck or semaphore, you need to specify the password in non-interactive, in a file. This is often implemented as a hidden file, like ‘.vault_pass.txt’. Of course this then means the password is readable if you got command line access. The main idea here is this gives you an option to encrypt sensitive data in a playbook and store it in a version control system, and do NOT store the password (file) in version control.

$ echo "mypassword" > .vault_pass.txt
$ ansible-playbook --vault-password-file .vault_pass.txt test2.yml
PLAY [localhost] **************************************************************************************************************************************************************

TASK [Gathering Facts] ********************************************************************************************************************************************************
ok: [localhost]

TASK [execute a shell command] ************************************************************************************************************************************************
changed: [localhost]

TASK [debug] ******************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": "aaa / bbb"
}

PLAY RECAP ********************************************************************************************************************************************************************
localhost                  : ok=3    changed=1    unreachable=0    failed=0

There is a caveat here too, into which I ran because my removable HDD enforces 777 on my files, which means the execute bit is set:

$ ansible-playbook --vault-password-file .ttt test2.yml
 [WARNING]: Error in vault password file loading (default): Problem running vault password script /Volumes/Samsung_T3/ansible/.ttt ([Errno 8] Exec format error). If this
is not a script, remove the executable bit from the file.

ERROR! Problem running vault password script /Volumes/Samsung_T3/ansible/.ttt ([Errno 8] Exec format error). If this is not a script, remove the executable bit from the file.

The message is self-explanatory: if ansible finds the passwordfile specified to be executable, it will execute it. In my case, I created a little shell script to do that:

#!/bin/bash
echo "mypassword"

3.a. Vault-id for passwords
Since ansible version 2.4 the official way to use encryption is by using vault-id. Vault-id is more or less the same as the above usage, however, it adds a new feature that could be important: the option to use multiple vault passwords in the same playbook. The way this works is quite simple: ansible tries all the passwords it has been given until it finds one that works. Please mind vault-id only works with encrypted files, not with encrypted variables! Below is an example of the use of vault-id with multiple passwords:

$ echo "---
- hosts: localhost
  tasks:

  - name: execute a shell command
    shell: >
      ls
  - debug: msg="{{ a }} / {{ b }}" > test3.yml
$ echo '---
a: "{{ vault_a }}"
b: "{{ vault_b }}"' > host_vars/localhost/vars.yml
$ echo "password_a" > .a.vault_pass.txt
$ echo "password_b" > .b.vault_pass.txt
$ echo "---
vault_a: aaa" > host_vars/localhost/vault_a.yml
$ echo "---
vault_b: bbb" > host_vars/localhost/vault_b.yml
$ ansible-vault --vault-id a@.a.vault_pass.txt encrypt host_vars/localhost/vault_a.yml
Encryption successful
$ ansible-vault --vault-id b@.b.vault_pass.txt encrypt host_vars/localhost/vault_b.yml
Encryption successful
$ ansible-playbook --vault-id a@.a.vault_pass.txt --vault-id b@.b.vault_pass.txt test3.yml
PLAY [localhost] **************************************************************************************************************************************************************

TASK [Gathering Facts] ********************************************************************************************************************************************************
ok: [localhost]

TASK [execute a shell command] ************************************************************************************************************************************************
changed: [localhost]

TASK [debug] ******************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": "aaa / bbb"
}

PLAY RECAP ********************************************************************************************************************************************************************
localhost                  : ok=3    changed=1    unreachable=0    failed=0
Advertisements

This is a short blogpost meant as both an introduction for those who don’t know Ara and a guide on how to install Ara.
Ara means ‘Ansible Runtime Analysis’, and is a tool for storing metadata that Ansible uses during execution. It is very valuable, because it takes a lot of guesswork and entering debug statements in your playbook away.

This is a guide on how to install Ara on Oracle Linux 7. I assume ansible is already installed. If want to start fresh, add EPEL and yum install ansible and git. That’s all you need to begin!

First, become root and install ara using a playbook:

mkdir ansible
cd $_
mkdir roles
git clone https://git.openstack.org/openstack/ansible-role-ara roles/ara
echo "- hosts: localhost
  roles:
  - ara" > install_ara.yml
ansible-playbook install_ara.yml
useradd ansible
systemctl stop ara

Ara and a systemd unit (service) is created and started by the playbook. I create an ‘ansible’ user and stop the service. The ansible ara role creates a service running as root, I don’t like running daemons as root if it’s not necessary.

The next part is executed as the user that is supposed to run ansible, which I created above:

echo "[defaults]
local_tmp = /home/ansible/.ansible/tmp
callback_plugins = /usr/lib/python2.7/site-packages/ara/plugins/callbacks
[ara]
dir = /home/ansible/.ara " > ~/.ansible.cfg

What this does, is define the callback for ara so executions are recorded. The default way of recording is entering the details in a sqlite3 database. This database (which is a file) is stored in /home/ansible/.ara, which is defined in the [ara] section using ‘dir’.

The next part is executed as root again:

pip install --upgrade ara
echo "[Unit]
Description=ARA
After=network.target

[Service]
Type=simple
User=ansible
Group=ansible
TimeoutStartSec=0
Restart=on-failure
RestartSec=10
RemainAfterExit=yes
ExecStart=/bin/ara-manage runserver -h 0.0.0.0 -p 9191

[Install]
WantedBy=multi-user.target" > /etc/systemd/system/ara.service
systemctl daemon-reload
systemctl start ara

This first executes pip (which is installed by the playbook above) and does an upgrade of ara and it’s depended packages. If this is not done, the website run by ara-manage will throw a 500 internal error (!!). Then the echo overwrites the current ara webserver config run as root by a modified version that runs it as ansible. Next we inform systemd it needs to reload the config, and then we start the ara service again.

Now run an ansible playbook as the ansible user, and go to port 9191 with your browser. It will detail the entire run, and list all the plays and tasks, with all information! For the CLI die-hards, there is a CLI tool ‘ara’ which provides the same information as the website, but entirely text based. It can generate static html files for sending through though.

If you don’t want to record everything, remove or comment the callback_plugins setting in ~/.ansible.cfg. Uncomment or add back if you want to record again. You can quite simply purge the history by stopping the ara service and remove the ~/.ara/ansible.sqlite file. Ara has an option to use mysql or postgresql instead of sqlite.

Addendum:
The CLI provides easy access to the results once you figured out how to use it. This is an example:
First list all the playbook run (I ran one playbook):

$ ara playbook list
+--------------------------------------+--------------------------------+---------------------+----------+----------+-----------------+
| ID                                   | Path                           | Time Start          | Duration | Complete | Ansible Version |
+--------------------------------------+--------------------------------+---------------------+----------+----------+-----------------+
| a152d1e5-fdab-4f50-b84a-de2d0b336124 | /home/ansible/ansible/test.yml | 2017-05-11 16:57:25 | 0:00:00  | True     | 2.3.0.0         |
+--------------------------------------+--------------------------------+---------------------+----------+----------+-----------------+

Then show the tasks in this playbook:

$ ara task list -b a152d1e5-fdab-4f50-b84a-de2d0b336124
+--------------------------------------+-----------------+------+------+---------+---------------------+----------+
| ID                                   | Name            | Path | Line | Action  | Time Start          | Duration |
+--------------------------------------+-----------------+------+------+---------+---------------------+----------+
| bf0bbc33-b0a4-4ac3-8242-58f5c8f1c4e5 | Gathering Facts | None | None | setup   | 2017-05-11 16:57:25 | 0:00:00  |
| 2b952b48-611f-424d-ba06-b94e22e87b1c | command         | None | 3    | command | 2017-05-11 16:57:26 | 0:00:00  |
+--------------------------------------+-----------------+------+------+---------+---------------------+----------+

Once you got the task IDs, you can list the task results:

$ ara result list --task 2b952b48-611f-424d-ba06-b94e22e87b1c
+--------------------------------------+-----------+---------+---------+---------------+---------------------+----------+
| ID                                   | Host      | Action  | Status  | Ignore Errors | Time Start          | Duration |
+--------------------------------------+-----------+---------+---------+---------------+---------------------+----------+
| d9278719-209e-41f8-8ecc-4ad2681dcdf6 | localhost | command | changed | False         | 2017-05-11 16:57:26 | 0:00:00  |
+--------------------------------------+-----------+---------+---------+---------------+---------------------+----------+

And then show the result in long format:

$ ara result show d9278719-209e-41f8-8ecc-4ad2681dcdf6 --long
+---------------+---------------------------------------------+
| Field         | Value                                       |
+---------------+---------------------------------------------+
| ID            | d9278719-209e-41f8-8ecc-4ad2681dcdf6        |
| Playbook ID   | a152d1e5-fdab-4f50-b84a-de2d0b336124        |
| Playbook Path | /home/ansible/ansible/test.yml              |
| Play ID       | 683dbe2b-84cc-4e17-b2ec-9b013e7d7612        |
| Play Name     | localhost                                   |
| Task ID       | 2b952b48-611f-424d-ba06-b94e22e87b1c        |
| Task Name     | command                                     |
| Host          | localhost                                   |
| Action        | command                                     |
| Status        | changed                                     |
| Ignore Errors | False                                       |
| Time Start    | 2017-05-11 16:57:26                         |
| Time End      | 2017-05-11 16:57:26                         |
| Duration      | 0:00:00                                     |
| Result        | {                                           |
|               |     "changed": true,                        |
|               |     "cmd": "echo \"test\"",                 |
|               |     "delta": "0:00:00.002718",              |
|               |     "end": "2017-05-11 16:57:26.616876",    |
|               |     "rc": 0,                                |
|               |     "start": "2017-05-11 16:57:26.614158",  |
|               |     "stderr": "",                           |
|               |     "stderr_lines": [],                     |
|               |     "stdout": "test",                       |
|               |     "stdout_lines": [                       |
|               |         "test"                              |
|               |     ]                                       |
|               | }                                           |
+---------------+---------------------------------------------+

This is how to install ara on OSX:

sudo pip install ara --user

Not sure if I need to use sudo when installing with ‘–user’, I ended up with directories in my local user Library directory for use with python that were not readable to my user ‘frits.hoogland’. In order to correct that, I ran:

cd
sudo chown -R frits.hoogland Library 

Now fetch the ara directory in the python site-packages directory:

python -c "import os,ara; print(os.path.dirname(ara.__file__))"
/Users/frits.hoogland/Library/Python/2.7/lib/python/site-packages/ara

Now create a ansible config file to be able to active the ara callback for running ansible-playbook and point the ara server to the sqlite database:

echo "[defaults]
local_tmp = /Users/frits.hoogland/.ansible/tmp
callback_plugins = /Users/frits.hoogland/Library/Python/2.7/lib/python/site-packages/ara/plugins/callbacks
[ara]
dir = /Users/frits.hoogland/.ara" > .ansible.cfg

At this moment the ara callback is activated. If you don’t want the overhead of ara writing all information to the sqlite database, comment ‘callback_plugins’.

If you want to use the ara webserver, active it using:

nohup ara-manage runserver -h localhost -p 9191 &

If you want to deactivate the ara webserver, stop it using:

pkill -f ara-manage

This blogpost is about how to install the semaphore user-interface for running ansible. Ansible is an automation language for automating IT infrastructures. It consists of command-line executables (ansible, ansible-playbook for example) that can run a single task using a module (using the ansible executable), or can run multiple tasks using multiple modules in order to perform more complex setup requirements (using the ansible-playbook executable). The downside of running IT tasks via the command-line is that there is no logging by default, unless someone decides to save the standard out to a file, which, if multiple people start doing that by hand will probably lead to a huge collection of text files which are hard to navigate. Also, when tasks are run via a common place, it’s an all or nothing situation: everybody has access to all the scripts, or to nothing.

This is where semaphore comes in: semaphore is a web-based user interface for running Ansible playbooks, for which you can define users, grant users access to certain templates (groups of playbooks, inventories (=groups of hosts) and ssh keys), which they can run, for which the output is saved in a database.

Semaphore is an alternative to Ansible Tower, which is a non-free product from RedHat. Semaphore is an open source project under active development, and is relatively “young”, which means it can (still) have “sharp edges”. Also Ansible Tower has more functionality.

An other feature from semaphore (and tower for that matter), is that any manipulation done via the web interface can also be done via a REST API. This means that if you use another language or product for the planning and/or deployment of (virtual, cloud, etc) machines which can do REST calls, or if you can craft REST calls with it, you can hand over the management of running Ansible for provisioning to a machine or a group of machines to semaphore, so tracking and logging of running Ansible is taken care of.

However, I did find the documentation of how to install semaphore in a real-life usable way not as verbose as I would like it. For that reason, I decided to write it down in this blogpost. Actually, most of the tasks are done using an Ansible playbook, so the installation is quite simple.

I am using a machine running Centos 7.3, I think this instruction is usable on any version of enterprise linux 7.

1. Install EPEL
EPEL has additional packages to the packages in the distribution repositories.

# rpm -i https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

2. Install Ansible and git

# yum install -y ansible git

Git is necessary for being able to clone from a git repository, and ansible is necessary to run a playbook. The playbook will actually update git to version 2.x because that is stated as a requirement on the semaphore repository.

3. Clone install_semaphore repository

# git clone https://gitlab.com/FritsHoogland/install_semaphore.git 

You might want to change the admin_(username|email|desc|password), which are the credentials for the semaphore superuser in the install_semaphore.yml file. If you like you can change the mysql credentials, however I do think the default settings are okay, because the mysql database will be hidden behind the firewall.

4. Run the semaphore install playbook from the repository

# ansible-playbook install_semaphore/install_semaphore.yml

Please mind this requires root to be allowed to sudo to any user.

Done!

%d bloggers like this: