Sometimes when you're trying to automate all the things, you have to revisit old technologies that you have long forgotten. In the networking world, SNMP is king for monitoring and metric gathering. I haven't seen a lot of information out there about setting up SNMP traps in a way that allows you to do what you want. Sure there's plenty of Network Management Systems out there, but I don't need all the bells and whistles.
I started working at a company (Shutterstock) that had MCollective implemented a few months ago. We use it to perform various functions across a wide range of servers. Things such as rolling the application servers to pick up new code, clearing caches, checking replication lag on databases, telling our puppetmaster to sign certificates, and even doing code deployment.
Inspired by Etsy's post on keeping track of everything, I set out to achieve similar functionality on a Rails application. Essentially, you run Graphite somewhere and send UDP packets from your application to StatsD.
Amazon recently announced their new CloudFormation API. This allows you to create what they call "stacks," which will bring up and provision various AWS resources. You are able to configure security groups, EC2 instances, RDS instances, and elastic load balancers just to name a few. With this level of support and ease of declaration, it makes you wonder how much longer things like RightScale are going to last.
Recently, I've been standing up a lot of infrastructure in Amazon's EC2. While auto-scaling the infrastructure aspects is fairly simple using configuration automation utilities such as Chef or Puppet, getting developers to understand these concepts may be an interesting task. I will try to outline most of the common issues I see when uplifting applications to an elastic cloud environment.
Sometimes companies are a bit wary of using an external service such as Github for their revision control. Especially with recent outages on their service. It also costs money if you want your repository to be entirely private. You can set up your own repositories pretty easily, and even do collaboration between multiple users without much issue.
Using nothing but open source software, I've come up with a pretty cool solution that will perform builds, tests, and deployment. All of this is done once a commit to your git repository is made.