ansible, automation, servers, and python

Ansible to Manage Personal Servers

Background

I have a lot of hobbies. In the Venn diagram of my hobbies, the circle labelled "Nerdy" is disproportionately large. What that means, for me, is that I have a large number of computing systems to manage. I am a principal member of the Pwn9.com gaming community, I have a home NAS that runs a number of services, such as my SageTV installation, a Dirvish backup server (which backs up all the other servers) A Jenkins CI server, A number of web servers, Minecraft servers, and a handful of other things.

Managing all of these as one-off’s is cumbersome. Sometimes, I don’t touch some of these services for months, or even a year. Others, I’m messing with daily. I needed a system to handle Administration of the services, without adding too much to my workload. My other motivation was the fact that I wanted to become more familiar with Server Automation options, which have developed significantly since I was a Sysadmin in the 90’s using CFEngine. (Kudos to the CFEngine folks for still going strong!)

Initially, I spent a lot of time working with the Open Source version of Puppet. It looks like a great tool for large-scale deployments, but trying to run Puppet "masterless" just proved too difficult in my environment. I wanted something that didn’t require a server to be 24/7 accessible. I tried running the Puppet Master on my FreeNAS system in a jail, but I just kept running into obstacles.

He said we wouldn’t get it. He said we wouldn’t get the treasure we seek on account of our obstacles.

— Pete, O Brother, Where Art Thou?

I looked briefly at Chef, but it seemed more like Puppet, in terms of required infrastructure. Ansible got my attention not just because it was written in Python, one of my preferred languages. It was a lot lighter-weight, it didn’t require a centralized trusted server, and it had a lot fewer rules. So all of the ad-hoc tasks I need to run are no problem. In fact, Ansible encourages me to keep working in a very ad-hoc way, but allows me to add automation and process to the things I was already doing.

Setup

I spent a lot of time just playing around with Ansible, and becoming familiar with it’s philosophy, and function. As you would expect, the Ansible Docs are a great place to start. I also heavily rely on the Ansible docsets in Dash.app which are indispensable while learning the ins and outs, and in day to day use.

Installation

I decided to use the development version, so I could get all the latest and greatest modules, etc. Since nothing in my infrastructure is particularly critical, I figured I could handle any bugs without much issue. (Incidentally, I haven’t found any in almost a year of using the dev version.) Instructions below are taken from the Ansible Docs.

$ git clone git://github.com/ansible/ansible.git --recursive
$ cd ./ansible

# Bash environment setup (adapt if you are using a different shell)
$ source ./hacking/env-setup

# Make sure python dependencies are installed
$ sudo pip install paramiko PyYAML Jinja2 httplib2 six

That should install ansible on your system. We can test by setting up our local host in a hosts file, and running the ansible "ping" module:

# If you were in the ansible checkout directory, go back up to the main dir.
$ cd ..

$ echo "127.0.0.1" > ansible_hosts
$ ansible -i ansible_hosts -m ping all --ask-pass
SSH password:
127.0.0.1 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

The "SUCCESS" message indicates that Ansible can talk to our host (127.0.0.1) via ssh, successfully logged in as the current user, and the password we provided at the "SSH password:" prompt was correct.

This will give you a basic ansible setup to start hacking on. I have set up a git repo with a copy of my setup, which you can use to follow along, if you wish. You can get it by running the following command:

git clone --recursive git://github.com/ptoal/ansible-example

It is important to include the "--recursive" option to get a copy of the ansible repo.

I will create Git "Tags" for each tutorial post in this blog. The first tag, matching this post, is called "Intro". It will give you a directory with a single file, ansible_hosts, with one host entry (127.0.0.1). Additionally, with the --recurse flag, it will clone the ansible repo into an 'ansible' subdirectory.

In the next post, I’ll talk about how I use Vagrant and VirtualBox to mock up all of my servers before I actually deploy them to a hosting provider. This combination allows you to iterate over your ansible changes quickly, allowing you to identify errors in your ansible setup before pushing out to a production server.

Until next time.

-Sage