One of the things that initially attracted me to PHP was that I could write all of my apps in pure PHP, and they would run anywhere I deployed them.
I was young and dumb back then, and I’ve learned a lot in the meantime. One of the things that I’ve learned is that your environment is incredibly important. A misconfigured setting is probably the biggest reason that code deployments fail. It’s so important that it’s almost universally recognized that if your code has any complexity, you need a staging server to test your code in a production-like configuration before you push out your releases.
There are a plethora of tools available to help you manage these jumps
from development to staging to production. There are all sorts of problems
to deal with when doing this, from preventing the execution of
partially-uploaded files, to ensuring that all of the files inside a user request are
from the same revision, to managing your load balancer configuration as
nodes go down for updates. There are also all sorts of software packages to
help you deal with this, ranging from simple shell scripts to
code, to post-commit hooks, to complex, environment-specific homebrewed
What is examined less often — and in much less detail — is how to manage
configuration files. At first glance, maintaining configurations on a
cluster of nodes seems easy. They’re all the same, right? Yes, that is true
until you start to notice that they need slightly different hosts files, or
have slightly different roles. Your perfect world, where you just
a single directory to all of your hosts, quickly starts to fall apart. Pretty
soon, you start writing Bash scripts to do different things to different
types of nodes, and it all gets complicated. Of course, there are other
software packages to help you with this problem, too. For example, Puppet is
excellent at configuration management, and you can truly manage a massive
installation of machines very easily with it. However, you really only have
a handful of machines right now, you’re going live with the beta tomorrow,
and you don’t have the time or the energy left to learn an advanced
configuration management system. Plus, maybe you’re just a really good PHP
developer, and don’t know much else about Unix-like systems. That’s nothing to
be ashamed of.
Luckily, there’s a convenient file archive management system in front of you right now. You’ve been using it for a while, and it lets you do all sorts of cool stuff. You can use it to just copy files, to run bash scripts before or after installation, and to keep track of which versions you need of all of your assets. It’s a system that you’re familiar with that lets you tell your servers that you need Apache 2.2, PHP 5.3, MySQL 5.1, the current release of Postfix, and even custom modules.
This system is your Linux distribution’s package manager. What, you thought you were the first person to have these problems? Package managers keep you safe every time you upgrade Apache, and they can keep your web app safe, too! In this example, I am going to use a Debian-based system, because I love Aptitude. However, most every distribution has a tool with similar capabilities, and a little Google research will yield quick rewards. So, let’s dig in.
In our example, we’ve thought about what we need need managed, and we’ve come to the conclusion that we need the following on every node in the cluster:
- Customized Apache, MySQL, and PHP configuration files
- A default document root directory which contains a small check script for the load balancer to poll
hostsfile entry for our four load-balanced MySQL servers
- A firewall (
- An SSH configuration
To begin with, let’s create a directory named after our package file,
l33tbox. This directory represents what we want to copy to
every system, relative to root. In this directory, we also need a
DEBIAN, which will contain all the package
At this point, our directory structure is pretty bare:
panda@pandastation:~/phpadvent$ find . -type d . ./l33tbox ./l33tbox/DEBIAN panda@pandastation:~/phpadvent$
The next thing we need to do is build a control file for our package.
This will be be located at
l33tbox/DEBIAN/control with the following
Package: l33tbox Version: 0.1 Section: main Priority: standard Architecture: all Depends: vim, apache2-mpm-prefork, php5, php5-cli, mysql-server, openssh-server, php-db, php-apc, iptables, php5-geoip, php5-imagick, php5-mcrypt, php5-memcache, php5-suhosin, php5-uuid, php5-curl, php5-dev, php5-gd, php5-mysql, php5-sqlite, libapache2-mod-php5, rsync Maintainer: Marcel Esser <marcel dot esser at gmail dot com> Description: My web nodes!
Let me give a brief explanation of the fields above:
- An arbitrary package name. Make it something easy to type.
- The package’s version number. This is also arbitrary but relevant to the package manager for figuring out which packages are more recent than others.
- Which section the package lives in. We’re a plain vanilla ’main’ package.
- What you might expect. We have normal priority.
- Which hardware architectures this
package is designed for. We’re just making a bunch of configuration
files for cross-platform applications, so we’ve chosen
all. You can also choose
amd64, or whatever your heart — or your app — desires.
- A list of packages on which this package
depends. This is where it starts to get good. This takes exactly the
same input for a name as, say,
apt-get installwould take. In this case, we’re just going to very generally specify our LAMP stack.
- Something to tell users what your package contains.
That is all we need to establish our package. Now, we move on to the next
issue, the configuration files we actually want to push. There is
one small problem we need to figure out; the files we want
to push (
apache2.conf, &c.) are all
files that are already controlled in the package manager by another package,
and it will complain if files collide (as it should). There are lots of ways
around this, but for the purpose of simplicity, I have chosen to put the
files in another directory, and
rsync them into their actual locations in
our post-installation script (which we will discuss in a moment). This gives
us the added bonus of putting a reference copy of our files on the machines.
I have chosen the location
/opt/l33tbox for the tree of
“override” files, and provide an example below.
Additionally, we have two files that are perfectly okay to let the package manager handle: our firewall script and our load balancer check script. We’ve placed these files in the normally-correct location. Our directories now look like this:
# debian control ./l33tbox/DEBIAN ./l33tbox/DEBIAN/control # files that would collide ./l33tbox/opt/l33tbox/etc/php5/php.ini ... lots more php config files ./l33tbox/opt/l33tbox/etc/ssh/sshd_config ... more ssh config files ... ./l33tbox/opt/l33tbox/etc/mysql/my.cnf ./l33tbox/opt/l33tbox/etc/apache2/apache2.conf ... lots more apache config files ... #legit files ./l33tbox/etc/network/if-up.d/00-firewall ./l33tbox/var/www/webapp/docroot/alive.php
Now comes the fun part. We need to write a post-installation script that does the following:
- Copies our config files into the right place
- Adds our MySQL cluster IP to the
hostsfile, if it’s not already there
- Starts up our firewall
Fortunately, this is also quite simple with
To write a post-installation script for a package, just name it
DEBIAN/postinst and give it an executable flag. You can write it
in any language you want, including PHP, but here is a small shell script to get
#!/bin/sh # copy over our sensitive config files /usr/bin/rsync -av /opt/l33tbox/etc/ /etc/ # does the host file contain our mysql cluster host name? MYSQLDONE=`/bin/grep mysql-cluster /etc/hosts | /usr/bin/wc -l` if [ $MYSQLDONE = "0" ]; then echo "Adding mysql-cluster to /etc/hosts..." echo "\n\n10.69.1.3 mysql-cluster\n\n" >> /etc/hosts fi # start the firewall echo "Starting firewall..." /etc/network/if-up.d/00-firewall
We are now ready to build our package. To do
so, we’ll use
dpkg-deb, which is in the
package. The usage is simple; just type:
dpkg-deb -b directory packagename. So, for us that would be:
panda@pandastation:~/phpadvent$ dpkg-deb -b l33tbox l33tbox-0.1.deb dpkg-deb: building package `l33tbox' in `l33tbox-0.1.deb'. panda@pandastation:~/phpadvent$
That’s it! We’re now ready to install our package. To do so, we again,
panda@pandastation:~/phpadvent$ sudo dpkg -i l33tbox-0.1.deb Selecting previously deselected package l33tbox. (Reading database ... 336942 files and directories currently installed.) Unpacking l33tbox (from l33tbox-0.1.deb) ... Setting up l33tbox (0.1) ... sending incremental file list ./ apache2/ apache2/conf.d/ apache2/mods-available/ apache2/mods-enabled/ apache2/sites-available/ apache2/sites-enabled/ mysql/ network/ network/if-up.d/ php5/ php5/apache2/ php5/apache2filter/ php5/cgi/ php5/cli/ php5/conf.d/ ssh/ sent 4891 bytes received 205 bytes 10192.00 bytes/sec total size is 71833 speedup is 14.10 Starting firewall... panda@pandastation:~/phpadvent$
From here, you can manage your package just like any other Debian package. You can also expand it! We’ve barely scratched the surface of what you can do, and there is plenty left to learn, but this will get you ready for your launch tomorrow. For more information, I strongly urge you to research building packages for your system’s package manager. A good exercise is to set up your own Apt repository.
I’ve included the example we discussed today, plus a little script to make building packages slightly easier.
Thanks for reading, and happy holidays!