Using Ansible for provisioning
Due to the recent increase of personnel at Grio, it emerged the need of having an automated way of setting up new employers’ machines.
Starting with a brand new machine is always a pain for a developer, and setting it takes at least a couple of days if not the whole first week, resulting in big waste of valuable time. Besides, when a developer starts on a new technology it is not always clear which tools are suggested and which ones the rest of the team are using. Therefore, I have been asked to work as a side project on a way to solve such issues.
Provisioning is a well known problem and there are a lot of tools out there.The leaders are definitely Chef and Puppet, both well established and widely used by a lot of companies. They both have their pros and cons, Chef is more flexible and has great versioning capabilities, Puppet is easier to use and has great reporting capabilities. However, both of them require a non trivial setup and have a quite steep learning curve. Since Grio is still a fairly small company and we do not require extremely fancy setup on our machines, I opted for a third tool: Ansible.
Ansible is probably less powerful than Chef and Puppet, but is very easy to learn, it does not require any client, and it is very quick to have a running setup. For these reasons it was particularly fitting the bill. Ansible is based on documents called playbooks, written in YAML. A playbook defines the configuration specifications that will be set up on the machine. You can have different roles for a playbook, in order to define different set ups. In particular, in our case it means defining different configurations for iOS, Android, and web developers as much as for designers.
Usually big companies use a master-client model where there is a centralized server with the configurations and the client machines retrieving and deploying the setup needed. However, since I wanted to have quickly a first prototype to hand to my colleagues and start focusing on the actual configurations needed in our company, I opted for a stand alone setup: the user needs to download the provisioning repository and run a bootstrap file locally.
Since we all use a MacBook this particular provisioning is focused on OS X, but Ansible can be used against other operative systems if properly configured. The bootstrap file is a very simple shell script installing brew and ansible and then run the Ansible playbook:
#!/bin/bash
#Bootstrap machine to prepare it for Ansible
set -e
# Download and install Homebrew
if [ ! `which brew` ]; then
echo "Install Homebrew"
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
fi
# Download and install Ansible
if [ ! `which ansible` ]; then
echo "Install Ansible"
brew update
brew install ansible
fi
ansible-playbook local.yml -K -i ./inventory/localhost.yml
It is possible to run a playbook against a particular or many roles, or against some tags which can be defined on each task. In this particular case, I just defined one role ‘base’.
The actual playbook (local.yml) is a YAML file containing a few basic information: the type of hosts and connection, the variables prompted, the environment variables needed and the roles in the playbook. You can define as many playbook as needed and have different bootstrap files or a configurable one.
---
- hosts: localhost
connection: local
gather_facts: false
vars_prompt:
- name: "githubname"
prompt: "Enter your name"
private: no
- name: "githubemail"
prompt: "Enter your github email"
private: no
vars:
home: "{{ lookup('env','HOME') }}"
roles:
- base
As you may noticed Ansible needs very basic variables, like HOME, to be defined in the playbook in order to be used later in tasks. Tasks will not be able to have access to such variables otherwise.
Roles are defined in particular folders that follow a well defined schema. Luckily you do not need to create all the needed folders. This is done by Ansible running the ansible-galaxy command. Ansible Galaxy (https://galaxy.ansible.com/) is a very useful website for finding, downloading, and sharing community developed Ansible roles. You can easily download roles from Galaxy to jumpstart your automation project.
Of course, you can create a role from scratch as well using the command ansible-galaxy init rolename. It will create a folder for you with the role name, and a few subfolders and files where to define your tasks, variables, handlers, and tests. In particular it will create these folders:
- handlers
- meta
- tasks
- tests
- vars
and a README.md file.
Each folder will contain an empty main.yml file which will need to be modified according to the desired configurations.
My vars/main.yml file is very simple and defines the lists of packages and apps that will be installed through brew and brew-cask:
---
# vars file for base
brew_base_packages:
- name: git
brew_cask_applications:
- name: iterm2
For simplicity I show only an element per list. Of course, the lists can contain as many elements as needed.
My tasks/main.yml file contains the actual core of the configuration. I will go only through some parts of it in order to give some examples of what you can actually do with Ansible, and show the most useful Ansible modules (http://docs.ansible.com/ansible/modules_by_category.html)
The simplest module is the command one (http://docs.ansible.com/ansible/command_module.html). It allows to run a shell command, like it would be done in a script or command line. Ansible allows you to define tags for each task, so it is possible to run only a subset of the tasks in a playbook matching the defined tags. This is very handy when you are dealing with a big playbook and you are interested in updating only part of your configuration. In this example, I am updating brew.
- name: Update homebrew
command: brew update
tags:
- base
- brew
- update
Ansible comes with brew (http://docs.ansible.com/ansible/homebrew_module.html) and brew cask modules as well, allowing to install packages and apps very simply. In the example, I am using the list of packages/apps I defined in the vars file, and Ansible will run through such list and install them one by one. It is possible to define also the state you want the package to be (latest version in this case), and the install options for each one.
- name: Install base homebrew packages
homebrew:
name={{ item.name }}
install_options={{ item.install_options | default("") }}
state=latest
with_items: brew_base_packages
tags:
- base
- brew
- name: Install brew cask applications
homebrew_cask:
name={{ item.name }}
state={{ item.state | default("present") }}
with_items: brew_cask_applications
tags:
- base
- brew
- brew-cask
Another very useful module is the copy one (http://docs.ansible.com/ansible/copy_module.html): it simply allows you to copy a file/folder. It is not much more complex than a simple ‘mv’ from command line but it handles a few nice options that avoid you to run multiple commands. In particular you can define the access permission of the copied file (mode option), if overriding the file if existing already (yes by default), and many more useful options.
- name: Copy brew-update-notifier.sh files
copy: src=./base/brew-update-notifier/brew-update-notifier.sh dest=/usr/local/bin/brew-update-notifier.sh mode="u+rwx"
In general Ansible has a very wide file module, handling almost all operations you can do on files.
For each task it is possible to define the user that will run it (by default is the user that ran the bootstrap script), using the become and become_user options. In this example, I needed it in order to copy a file into a system folder and needed root privileges.
- name: Copy plist brew-update-notifier file
copy: src=./base/brew-update-notifier/com.grio.brewUpdateNotifier.plist dest=/Library/LaunchAgents/com.grio.brewUpdateNotifier.plist
become: yes
become_user: root
Ansible also allows you to add lines to files through the lineinfile module (http://docs.ansible.com/ansible/lineinfile_module.html). In the example, I source homeshick to avoid typing all the time the whole path to homeshick.sh. I also force to create the .bash_profile file in case it is not existing (which does not on a brand new machine).
- name: Update bash_profile to have homeshick initialization
lineinfile: dest={{home}}/.bash_profile line="source $HOME/.homesick/repos/homeshick/homeshick.sh" regexp='homeshick.sh' create=yes
Of course, Ansible has a git module as well (http://docs.ansible.com/ansible/git_module.html). You simply need to define the repo and the destination folder. It comes with a lot of options, like adding the hostkey for the repo url (accept_hostkey) and discarding modified files in the working repository (force). Ansible provides many source control modules, besides this one. In the example, I use it to install Homeshick.
- name: Install homeshick
git: repo=git://github.com/andsens/homeshick.git dest={{home}}/.homesick/repos/homeshick accept_hostkey=true force=true
Homeshick is a dot files management tool, and it is very useful for setting up or exporting configurations defined in a dotfiles. Homeshick creates regular git repositories with just one special condition: they have in their root a directory named home, inside of which you will put your dotfiles. Managed repository are called castles. Later, you can tell homeshick to symlink the contents of that directory into your actual home directory. Of course, applications using your dotfiles won’t notice the difference. It is very simple to use, as you can see in the example where I refresh the castles and link the dotfiles.
- name: Refresh all castles
command: $HOME/.homesick/repos/homeshick/bin/homeshick refresh
- name: Link dotfiles
command: $HOME/.homesick/repos/homeshick/bin/homeshick link dotfiles -f -q
As you can see picking up Ansible is very simple, and it is very fast to have an initial basic configuration running. It is definitely a tool to take into consideration if you need provisioning in your company, especially if it is not a very large one and you do not need very fancy configurations. It can also come handy to export configurations to your new computer, or if you have multiple and want to maintain the same settings. So no excuses anymore for not automating configurations on your machine!
