The Codebelay Blog

This is a quick and dirty EC2 backup script for virtual unix servers that works just fine when crontabbed:

This is just a quick summary of what it was like implementing Amazon’s EC2 in an enterprise environment.

1. You’ll need to write your own LDAP plug-ins to interface with any access control lists. E.G. where I work WordPress is used for corporate communications so an LDAP plug-in had to be written to make sure the right people saw the right information.

2. Migration can be expensive if you’re using EBS on the first go. On windows, and I’m not sure why, it can cost about $50 to migrate 2GB of data into EBS. In linux, it happens at a fraction of that cost and as advertised.

3. Windows can be very expensive. Although they say it’s 12 cents per hour per small instance beware of hidden costs like authentication services and SQL server. With both, you are using a server at the cost of $1.35 / hour, which IMHO could be run cheaper with just a small linux instance and do the same thing at 10 cents per hour.

I’m pretty sure that with the right Amazon EC2 set up you could run a cluster of servers for a Fortune 500 company for under $1000.00 (one thousand dollars) per month without the CapEX costs associated with new hardware.

If you have any more questions about Amazon EC2 in the enterprise I’d be happy to answer them. Please ask them in the comments below.

This blog post is a quick introduction to load balancing and auto scaling on with Amazon’s EC2.

I was kinda amazed about how easy it was.

Prelims: Download the load balancer API software, auto scaling software, and cloud watch software. You can get all three at a download page on Amazon.

Let’s load balancer two servers.

elb-create-lb lb-example --headers \
--listener "lb-port=80,instance-port=80,protocol=http" \
--availability-zones us-east-1a

The above creates a load balancer called “lb-example,” and will load balance traffic on port 80, i.e. the web pages that you serve.

To attach specific servers to the load balancer you just type:

elb-register-instances-with-lb lb-example --headers \
--instances i-example,i-example2

where i-example and i-example2 are the instance id’s of the servers you want added to the load balancer.

You’ll also want to monitor the health of the load balanced servers, so please add a health check:

elb-configure-healthcheck lb-example --headers \
--target "HTTP:80/index.html" --interval 30 --timeout 3 \
--unhealthy-threshold 2 --healthy-threshold 2

Now let’s set up autoscaling:

as-create-launch-config example3autoscale --image-id ami-mydefaultami \
--instance-type m1.small

as-create-auto-scaling-group example3autoscalegroup  \
--launch-configuration example3autoscale \
--availability-zones us-east-1a \
--min-size 2 --max-size 20 \
--load-balancers lb-example

as-create-or-update-trigger example3trigger \
--auto-scaling-group example3autoscalegroup --namespace "AWS/EC2" \
--measure CPUUtlization --statistic Average \
--dimensions "AutoScalingGroupName=example3autoscalegroup" \
--period 60 --lower-threshold 20 --upper-threshold 40 \
--lower-breach-increment=-1 --upper-breach-increment 1 \
--breach-duration 120

With the 3 commands above I’ve created an auto-scaling scenario where a new server is spawned and added to the load balancer every two minutes if the CPU Utlization is above 20% for more than 1 minute.

Ideally you want to set –lower-threshold to something high like 70 and –upper-threshold to 90, but I set both to 20 and 40 respectively just to be able to test.

I tested using siege.

Caveats: the auto-termination part is buggy, or simply didn’t work. As the load went down, the number of the server on-line remained the same. Anybody have thoughts on this?

What does auto-scaling and load balancing in the cloud mean? Well, the total cost of ownership for scalable, enterprise infrastructure just went down by lots. It also means that IT departments can just hire a cloud expert and deploy solutions from a single laptop instead of having to figure out the cost for hardware load balancers and physical servers.

The age of Just-In-Time IT just got ushered in with auto-scaling and load balancing in the cloud.

If you don’t fail fast enough, you’re on the slow road to success.

One idea that I recently failed was using a screen and sitebeagle to monitor sites.

It’s not a complete failure… it works okay.

Due to budget constraints, I put my screen and sitebeagle set up on a production server.

For some reason that production server ran out of space and became unresponsive. Screen no doubt caused this. I was alerted of the issue and did a reboot.

After the reboot, although Amazon’s monitoring tools told me the server was okay, the server was not. The MySQL database was in an EBS volume and needed to be re-mounted.

The solution I now have in place is still screen and sitebeagle. But I use another server with screen and sitebeagle on it to monitor the production server that gave me the issue in the first place.

It’s a question of who will monitor the monitors… in a world of web sites with few site users the answers pretty bleak. In the world of super popular commercial sites, the answer’s clear. The wisdom of crowds will monitor the web sites.

I recently created a cross platform browser, Windows 2003 EC2 AMI: ami-69739500

It has the following pre-installed:

• gvim
• IE 7
• Firefox 3 with Web Developer, yslow & Firebug
• opera
• Putty SSH
• Putty SCP

Pretty much with that list you’re all set to do troubleshooting for cross platform browser issues.

There’s IIS 6.0 and SQL Server, too.

I’ve linked the password to this ami at http://www.codebelay.com/ami-69739500.txt . It’s a short-coming of Windows AMIs on EC2 that I have to link the password, so please change it once you get into the instance.

For checking up on sites in the enterprise, I use Alertsite. It was suggested to me by a VP I work with at McCann, Ed Recinto. It’s been a great tool.

For personal websites that I manage, I’ve been using something I rolled in newLISP, sitebeagle. Why? Because beagles are great watchdogs.

Very often, most problems can be solved with tweaking code, changing permissions, or upgrading and apache or mysql.

Very often, it’s the weekend, I’m sitting in a cafe, and get an alert from Nagios or Alertsite. With iSSH, on the iPhone, I can ssh into a LAMP server and do the work I need.

I can see things getting a bit more complex. What tools do you use to sysadmin from an iPhone?