Cluster Management with FreePanel

I haven't had much time to continue work on FreePanel, but the Perl modules I've written have proven to be useful in cluster management. Take a look at the following diagram as an example scenario:

Web Cluster Example

There's two key points to take away from that diagram.

  • physical servers are load balanced using a single VIP (Virtual IP)
  • the servers share access to a single storage mount

The idea behind load balancing is that requests go to the individual servers based on which can handle the traffic. A hardware appliance takes care of the actual load balancing work, but this could be done using some software just as easily.

So what does this have to do with FreePanel?

If you're constantly needing to add additional domains to your cluster, doing this manually becomes tiresome, especially when in most cases your application needs the same Apache configuration per domain. Take the below basic Virtual Host config:


        ServerAdmin webmaster@example.com
        ServerName example.com
        ServerAlias www.example.com
        ErrorLog /home/sites/logs/example.com.error
        CustomLog /home/sites/logs/example.com.access combined
        DocumentRoot /home/sites/example.com/web

        
                Options Indexes FollowSymLinks
                AllowOverride None
                Order allow,deny
                Allow from all
        

Each of your 50 or so hosted applications needs a similar configuration, just with a different domain name. That's where FreePanel can make your life easier. Take the above example and we'll form a template that can be used with FreePanel's Apache module.

>
        ServerAdmin webmaster@
        ServerName 
        ServerAlias www.
        ErrorLog /home/sites/logs/.error
        CustomLog /home/sites/logs/.access combined
        DocumentRoot /home/sites//web

        /web/>
                Options Indexes FollowSymLinks
                AllowOverride None
                Order allow,deny
                Allow from all
        

As you can expect, this module can parse the template file and replace certain items with what arguments are being passed. We'll take a look at the addSite method of FreePanel::Admin::Apache.

sub addSite {
        my ($self, $domain, $ip_addr) = @_;
        $self->logger("function: addSite($domain, $ip_addr) called.", $self->FUNC_CALL);

        my $vhost_dir = $self->getVhostDir();
        my $vhost_templ = $self->getVhostTemplate();
        my $check = $self->getValidateObj();

        if ($check->is_active($domain, $vhost_dir)) {
                $self->logger("vhost config already exists for $domain.", $self->ERROR);
                return 0;
        }

        open (VHOST, '>', $vhost_dir."/$domain");
        open (VHOST_T, ') {
                        s//$domain/;
                        s//$ip_addr/;
                        print VHOST;
                }
        }
        else { 
                while () {
                        s//$domain/;
                        s//\*/;
                        print VHOST;
                }
        }

        close VHOST_T;
        close VHOST;

        return 1;
}

Nothing really that crazy going on here, it just opens up the template file -- defined in your config file -- then replaces <DOMAIN> and <IP_ADDR> with the values you pass. This file is written out to a directory -- also defined in the config file -- and if no IP address is passed, we default to use *. This could be easily modified to include additional values, but I won't go into that now.

Okay, how do I use it?

#!/usr/bin/perl
use strict;
use FreePanel::Admin::Apache;

if ($#ARGV  [ip address]\n";
        exit;
}

my $domain = $ARGV[0];
my $ip_addr = $ARGV[1];

my $apache = new FreePanel::Admin::Apache;

$apache->addWebDir($domain)
        or die "Error adding web directory for $domain";
$apache->logger("web directory for $domain has been created.", $apache->INFO);

if ($ip_addr) {
        $apache->addSite($domain, $ip_addr)
                or die "Error creating vhost with $ip_addr for $domain";
        $apache->logger("vhost config created for $domain using $ip_addr.", $apache->INFO);
}
else {
        $apache->addSite($domain)
                or die "Error creating vhost for $domain";
        $apache->logger("vhost config craeted for $domain.", $apache->INFO);
}

# restart apache, still implementing FreePanel::Admin::Apache->restart
system('apachectl restart');

With the above you have a basic script for adding new domains fairly quickly. This will also allow you to keep track of what has been added and when since everything you do using FreePanel is logged to a file you chose. Here's a sample log output:

03-07-2010 11:59:36 :: [ DEBUG ] function: addWebDir(example.com)
03-07-2010 11:59:36 :: [ INFO ] web directory for example.com has been created.
03-07-2010 11:59:36 :: [ DEBUG ] function: addSite(example.com) called.
03-07-2010 11:59:36 :: [ INFO ] vhost config craeted for example.com.

Your level of logging is also defined by the configuration file. So with this basic script you get a log keeping track of what sites have been added and when as well as taking on the tedious work of copying a configuration file and switching a few values to meet the new domain's needs. There's a bit of documentation lacking, but some will soon be added to the FreePanel Wiki.

A case study: three web servers (clustered) using SSL for all domains

Recently, I was dealing with a client that had three web servers all behind a load balancer (Cisco CSS). This was a basic CRM software that required SSL for each virtual host (domain). This was troublesome because you need a unique IP address for each SSL site, and since all of their clients were using SSL, this meant every single domain had to have its own IP. This also means that each virtual host will be listening on a private IP behind the CSS. What this means is you will see something like this:

Web01:


        ServerAdmin webmaster@example.com
        ServerName example.com
        ...

Web02:


        ServerAdmin webmaster@example.com
        ServerName example.com
        ...

Web03:


        ServerAdmin webmaster@example.com
        ServerName example.com
        ...

Being a lazy systems admin, I certainly did not want to manually create three configurations each time I wanted to add a new domain. So I used FreePanel::Admin::Apache and some ssh keys to carry out the creation of each. Using the addSite method, I passed the last octet of each domain rather than the full IP, and had the template using something similar to this:

>

Of course, changing the first three octets based on which server this template was being used on. I had a simple script similar to the one above that took the input for domain name and IP address, and a main script which I ran from an admin box. The main script was in charge of creating SSH connections and launching the helper script on each server with the proper arguments. This main script also remotely restarted Apache after creating each site.

While this example certainly isn't very elegant as password-less root SSH keys were required, it did however prove to be a very useful use-case for our Apache module. Solutions to problems such as this are why I work on FreePanel, because we all know that lazy syadmins are usually the best sysadmins.

Tags: