All about ansible vault
This blogpost is about using ansible vault. Vault is a way to encrypt sensitive information in ansible scripts by encrypting it. The motivation for this blogpost is the lack of a description that makes sense to me of what the possibilities are for using vault, and how to use the vault options in playbooks.
The basic way ansible vault works, is that when ansible-playbook reads a yaml file, it encounters $ANSIBLE_VAULT;1.1;AES256 indicating ansible vault is used to encrypt the directly following lines, it will use a password to decrypt it, and then uses the decrypted version in memory only. This way secrets can be hidden from being visible. Obviously, the password will allow decrypting it, and the password must be used in order for ansible-playbook to decrypt it.
The original use of vault is to encrypt an entire yaml file. As of Ansible version 2.3, ansible allows the encryption of single values in a yaml file.
1. File encryption
A very silly example of ansible-vault, which shouldn’t be used (!) is this:
$ echo "---
- hosts: localhost
vars:
a: aaa
b: bbb
tasks:
- name: execute a shell command
shell: >
ls" > test1.yml
When this runs, it will execute on localhost, set to ansible variables (a and b) and then use the shell module to execute “ls”. Again, this is just an example, and is silly on itself. This is how it’s run:
$ ansible-playbook test1.yml PLAY [localhost] *********************************************************************************************************************** TASK [Gathering Facts] ***************************************************************************************************************** ok: [localhost] TASK [execute a shell command] ********************************************************************************************************* changed: [localhost] PLAY RECAP ***************************************************************************************************************************** localhost : ok=2 changed=1 unreachable=0 failed=0
This can be encrypted using ansible vault in the following way:
$ ansible-vault encrypt test1.yml New Vault password: Confirm new Vault password: Encryption successful
Now the yaml file is entirely encrypted:
$ cat test1.yml $ANSIBLE_VAULT;1.1;AES256 32663536343532643864353135643431373830333236636230613264383531336562323530386632 3933353561316530333538363032633731336135396434630a626631303839646161363432396230 32336130633264343766393562656138363132393835646137633065393739336234336463336138 6564613730333436650a336537326233343566633761383436356136666533326561346530613566 64316336393036346538656338303734623161386435373138656532663737666662633765656438 37303666393939356161393134666130303435333463636630386434626335383164663761383035 35613437356361366665633931346664386535393936396637363561386164613337373562313264 63333530353263363437386536393765393930653234353733313433643637333932353638353339 31623366376566613766366236356163303837666664646465363439313265383562656637623266 6463663065383030626232643365623437666332313631376262
It still can be run, but you need to specify the password. In this case it asks to enter a password:
$ ansible-playbook test1.yml --ask-vault-pass Vault password: PLAY [localhost] *********************************************************************************************************************** TASK [Gathering Facts] ***************************************************************************************************************** ok: [localhost] TASK [execute a shell command] ********************************************************************************************************* changed: [localhost] PLAY RECAP ***************************************************************************************************************************** localhost : ok=2 changed=1 unreachable=0 failed=0
Of course this is not practical, the most common and probable way you want to use this is to have variables with secret information, like passwords, tokens, private keys, personal data, etc. encrypted, but the playbook still be readable. You should use a version control tool like git, and use functionality like diff!
The first thing is to move the variables out of the playbook into a dedicated section. In this case I chosen to assign them based on host, in a directory called ‘host_vars’, and then for localhost ‘host_vars/localhost’:
$ cat host_vars/localhost/vars.yml --- a: aaa b: bbb
And then of course remove them from the playbook itself:
$ cat test2.yml
---
- hosts: localhost
tasks:
- name: execute a shell command
shell: >
ls
- debug: msg="{{ a }} / {{ b }}"
If this is run, it will pick up the variables (for which I put in a debug task):
$ ansible-playbook test2.yml
PLAY [localhost] ***********************************************************************************************************************
TASK [Gathering Facts] *****************************************************************************************************************
ok: [localhost]
TASK [execute a shell command] *********************************************************************************************************
changed: [localhost]
TASK [debug] ***************************************************************************************************************************
ok: [localhost] => {
"msg": "aaa / bbb"
}
PLAY RECAP *****************************************************************************************************************************
localhost : ok=3 changed=1 unreachable=0 failed=0
Okay, now let’s say I need to encrypt the variable ‘a’. I can bluntly encrypt vars.yml, and that will work, however, then you don’t know where the variable comes from. The solution to that is ‘indirect referencing’. This is done by adding a second file to host_vars/localhost called ‘vault.yml’, which will be the encrypted file. In the case of encrypting variable a, do the following:
1. Change the vars.yml file to assign {{ vault_a }} to a:
$ cat host_vars/localhost/vars.yml
---
a: "{{ vault_a }}"
b: bbb
2. Define the value of a to vault_a in vault.yml:
$ cat host_vars/localhost/vault.yml --- vault_a: aaa
3. Encrypt vault.yml:
$ ansible-vault encrypt host_vars/localhost/vault.yml New Vault password: Confirm New Vault password: Encryption successful
Now it can be used by specifying the password again:
$ ansible-playbook test2.yml --ask-vault-pass
Vault password:
PLAY [localhost] ***********************************************************************************************************************
TASK [Gathering Facts] *****************************************************************************************************************
ok: [localhost]
TASK [execute a shell command] *********************************************************************************************************
changed: [localhost]
TASK [debug] ***************************************************************************************************************************
ok: [localhost] => {
"msg": "aaa / bbb"
}
PLAY RECAP *****************************************************************************************************************************
localhost : ok=3 changed=1 unreachable=0 failed=0
This should not be a surprise, as I said, ansible will recognise an encrypted file, and decrypt it automatically. However, if you looked carefully you saw something else that is important to realise: I encrypted the variable a containing ‘aaa’, and it was revealed to me using the debug statement. I think this is reasonable, because it’s been explicitly programmed to do so. You should write your playbooks not to reveal any information that you encrypted previously.
2. Variable encryption
If you have larger sets of variables to be encrypted, the above method makes sense. However, if you want to encrypt only a few variables it’s probably simpler to encrypt only the variable you want to be encrypted. The ansible-vault executable provides a facility for that called ‘encrypt_string’. The way this works is that you simply specify the variable and the variable value with ansible-vault, which then outputs the encrypted value, which you need to put in a playbook yourself. For example your ‘a’ variable with the value ‘aaa’:
$ ansible-vault encrypt_string --name 'a' 'aaa'
New Vault password:
Confirm New Vault password:
a: !vault |
$ANSIBLE_VAULT;1.1;AES256
37363334323763633562313934656631396431636139653036333036356636326163623238626132
6138343534396232643638323830326462383831366165630a393335303939663138613437633835
65396461383565316135316339383035626262343664643563373631366338393361623230666634
3637363232313935310a323564663863336565623366633838616337393366313831396637386432
3035
Encryption successful
Now use this to input to a playbook:
---
- hosts: localhost
vars:
a: !vault |
$ANSIBLE_VAULT;1.1;AES256
37363334323763633562313934656631396431636139653036333036356636326163623238626132
6138343534396232643638323830326462383831366165630a393335303939663138613437633835
65396461383565316135316339383035626262343664643563373631366338393361623230666634
3637363232313935310a323564663863336565623366633838616337393366313831396637386432
3035
b: bbb
tasks:
- name: execute a shell command
shell: >
ls
- debug: msg="{{ a }} / {{ b }}"
3. Vault passwords
The next thing to look at is how to specify the decryption password. One way or another, you need a password specified to decrypt. The first option is to specify the password at the CLI, which is done in the previous examples by specifying ‘–ask-vault-pass’ as an argument to ansible-playbook. This is a good option for running playbooks ad-hoc.
If you want playbooks to run from a facility like ansible tower, rundeck or semaphore, you need to specify the password in non-interactive, in a file. This is often implemented as a hidden file, like ‘.vault_pass.txt’. Of course this then means the password is readable if you got command line access. The main idea here is this gives you an option to encrypt sensitive data in a playbook and store it in a version control system, and do NOT store the password (file) in version control.
$ echo "mypassword" > .vault_pass.txt
$ ansible-playbook --vault-password-file .vault_pass.txt test2.yml
PLAY [localhost] **************************************************************************************************************************************************************
TASK [Gathering Facts] ********************************************************************************************************************************************************
ok: [localhost]
TASK [execute a shell command] ************************************************************************************************************************************************
changed: [localhost]
TASK [debug] ******************************************************************************************************************************************************************
ok: [localhost] => {
"msg": "aaa / bbb"
}
PLAY RECAP ********************************************************************************************************************************************************************
localhost : ok=3 changed=1 unreachable=0 failed=0
There is a caveat here too, into which I ran because my removable HDD enforces 777 on my files, which means the execute bit is set:
$ ansible-playbook --vault-password-file .ttt test2.yml [WARNING]: Error in vault password file loading (default): Problem running vault password script /Volumes/Samsung_T3/ansible/.ttt ([Errno 8] Exec format error). If this is not a script, remove the executable bit from the file. ERROR! Problem running vault password script /Volumes/Samsung_T3/ansible/.ttt ([Errno 8] Exec format error). If this is not a script, remove the executable bit from the file.
The message is self-explanatory: if ansible finds the passwordfile specified to be executable, it will execute it. In my case, I created a little shell script to do that:
#!/bin/bash echo "mypassword"
3.a. Vault-id for passwords
Since ansible version 2.4 the official way to use encryption is by using vault-id. Vault-id is more or less the same as the above usage, however, it adds a new feature that could be important: the option to use multiple vault passwords in the same playbook. The way this works is quite simple: ansible tries all the passwords it has been given until it finds one that works. Please mind vault-id only works with encrypted files, not with encrypted variables! Below is an example of the use of vault-id with multiple passwords:
$ echo "---
- hosts: localhost
tasks:
- name: execute a shell command
shell: >
ls
- debug: msg="{{ a }} / {{ b }}" > test3.yml
$ echo '---
a: "{{ vault_a }}"
b: "{{ vault_b }}"' > host_vars/localhost/vars.yml
$ echo "password_a" > .a.vault_pass.txt
$ echo "password_b" > .b.vault_pass.txt
$ echo "---
vault_a: aaa" > host_vars/localhost/vault_a.yml
$ echo "---
vault_b: bbb" > host_vars/localhost/vault_b.yml
$ ansible-vault --vault-id a@.a.vault_pass.txt encrypt host_vars/localhost/vault_a.yml
Encryption successful
$ ansible-vault --vault-id b@.b.vault_pass.txt encrypt host_vars/localhost/vault_b.yml
Encryption successful
$ ansible-playbook --vault-id a@.a.vault_pass.txt --vault-id b@.b.vault_pass.txt test3.yml
PLAY [localhost] **************************************************************************************************************************************************************
TASK [Gathering Facts] ********************************************************************************************************************************************************
ok: [localhost]
TASK [execute a shell command] ************************************************************************************************************************************************
changed: [localhost]
TASK [debug] ******************************************************************************************************************************************************************
ok: [localhost] => {
"msg": "aaa / bbb"
}
PLAY RECAP ********************************************************************************************************************************************************************
localhost : ok=3 changed=1 unreachable=0 failed=0
Frits Hoogland Weblog 