The Codebelay Blog

The print_GeoCache_Url function came across my email today from a self-described local designer and geek, but after a little research, I found out it only works up to WordPress 1.2 . Thank goodness for the WordPress WP-Geo Plug-in which I’m using right now.

More info here: WPGeo.com

This is a quick and dirty EC2 backup script for virtual unix servers that works just fine when crontabbed:

I couldn’t sleep last night. I’m worried we’ll lose this client.

So just to be clear. I wasn’t part of the crew responsible for scaling this site. I had already set up a scalable architecture for the site, that would automatically and horizontally scale at Amazon. That idea got shot down for legal reasons that to my surprise haven’t been in play for awhile. Can we say, “Office politics?”

I totally recommend Amazon’s Autoscaling to anybody that’s new to this.

Instead of auto-scaling, the site was architected by a local San Francisco firm who I won’t mention here.

Let’s just hope enough people read this so that they won’t even have to know the name of the company and will just know the smell of an un-scaleable architecture.

Scalability requirement: 100,000 concurrent users

This is how they set it up:

• two web servers
• one database
• four video transcoders that hits the master database
• one more app server that hits the master database
• no slave db

If they had even googled ‘building scalable websites’ they would have come across a book that would have avoided all of this, Cal Henderson’s Building Scalable Websites. It should be mandatory reading for anybody working on a large website, and it just scratches the surface.

So, how did we get to 600 concurrent users?

We tweaked mysql by putting this in /etc/m.cnf:

We ran siege and were able to get to about 300 concurrent users without breaking a sweat, but now apache was dying.

So we tweaked apache. We started out with this:

StartServers 8
MinSpareServers 5
MaxSpareServers 20
ServerLimit 256
MaxClients 256
MaxRequestsPerChild 4000

And ended up with this:

RAM and CPU were doubled.

Well, it looks pretty bad right now. A vendor just ceded control for web application architecture. Initial tests say that the site won’t do no more 100 users concurrently.

Who the hell makes a web app without a slave database and calls themselves website architects? Apparently these guys did.

Please start following if you want to see if this web app can make it to launch.

This is one of those things they don’t mention in the docs.

When you create a provisioning profile for an iPhone app you get an app identifier. It looks like this:
KED2IOCNA.com.codebelay.barceftw

In order to get your iPhone app in development into your iPhone you got to have a bundle identifier. The docs that I’ve found so far tell you that your bundle identifier is the same as your app identifier. After struggling for close to 4 hours, I found out that your bundle identifier really is:
com.codebelay.barceftw

Just strip out the encrypted looking part and the dot (.) before com and you’re set.

Your team will sabotage your career worse than any other nemesis at work, if you let them.

Here’s what you need to know to protect yourself and your company from sabotage:

Who’s popular? Yeah, I know. It sounds like highschool, but like then it’s still in important and socially real factor that’s now kept track of on social media sites.

What is your team’s weakness as perceived by those outside? By the team itself? A good manager can appease the two.

Whose skills are the most respected? Yup you have to get along with this douchebag, if she or he is one. Just create enough space between the two of you.

Any others?

This blog post is a quick introduction to load balancing and auto scaling on with Amazon’s EC2.

I was kinda amazed about how easy it was.

Prelims: Download the load balancer API software, auto scaling software, and cloud watch software. You can get all three at a download page on Amazon.

Let’s load balancer two servers.

elb-create-lb lb-example --headers \
--listener "lb-port=80,instance-port=80,protocol=http" \
--availability-zones us-east-1a

The above creates a load balancer called “lb-example,” and will load balance traffic on port 80, i.e. the web pages that you serve.

To attach specific servers to the load balancer you just type:

elb-register-instances-with-lb lb-example --headers \
--instances i-example,i-example2

where i-example and i-example2 are the instance id’s of the servers you want added to the load balancer.

You’ll also want to monitor the health of the load balanced servers, so please add a health check:

elb-configure-healthcheck lb-example --headers \
--target "HTTP:80/index.html" --interval 30 --timeout 3 \
--unhealthy-threshold 2 --healthy-threshold 2

Now let’s set up autoscaling:

as-create-launch-config example3autoscale --image-id ami-mydefaultami \
--instance-type m1.small

as-create-auto-scaling-group example3autoscalegroup  \
--launch-configuration example3autoscale \
--availability-zones us-east-1a \
--min-size 2 --max-size 20 \
--load-balancers lb-example

as-create-or-update-trigger example3trigger \
--auto-scaling-group example3autoscalegroup --namespace "AWS/EC2" \
--measure CPUUtlization --statistic Average \
--dimensions "AutoScalingGroupName=example3autoscalegroup" \
--period 60 --lower-threshold 20 --upper-threshold 40 \
--lower-breach-increment=-1 --upper-breach-increment 1 \
--breach-duration 120

With the 3 commands above I’ve created an auto-scaling scenario where a new server is spawned and added to the load balancer every two minutes if the CPU Utlization is above 20% for more than 1 minute.

Ideally you want to set –lower-threshold to something high like 70 and –upper-threshold to 90, but I set both to 20 and 40 respectively just to be able to test.

I tested using siege.

Caveats: the auto-termination part is buggy, or simply didn’t work. As the load went down, the number of the server on-line remained the same. Anybody have thoughts on this?

What does auto-scaling and load balancing in the cloud mean? Well, the total cost of ownership for scalable, enterprise infrastructure just went down by lots. It also means that IT departments can just hire a cloud expert and deploy solutions from a single laptop instead of having to figure out the cost for hardware load balancers and physical servers.

The age of Just-In-Time IT just got ushered in with auto-scaling and load balancing in the cloud.

I’ve pretty much come to rely on git to pull me out of any bad jams in the chaotic environment I work in.

One thing I’ve had to learn to do is cherry pick commits and package them under a tag in git.

Here’s how to do it if you were working with my newLISP project called Sitebeagle:

fork sitebeagle on this page

cd sitebeagle

git fetch –tags

git checkout 8f5bb33a771f7811d21b8c96cec67c28818de076

git checkout -b sample_cherry_pick

git cherry-pick 22aab7

git cherry-pick b1334775

git diff sample_cherry_pick..master

git tag leaving_out_one_commit

git push origin –tags

At this point, you should have a tagged branch that doesn’t have the commit with the change to the “2nd file.” The diff should look exactly like this:

diff –git a/test.lsp b/test.lsp
index 9cf1667..158b625 100755
— a/test.lsp
+++ b/test.lsp
@@ -1,6 +1,7 @@
#!/usr/bin/newlisp

; test tag test_a
+; cherry pick test 2

(load “sitebeagle.lsp”)
(load “twitter.lsp”);

Ever see characters any of these characters:

1) ^M

2) <feff>

3) A black diamond with a white question mark in it.

4) What<92>s going on?

???

Does your version control system tell everything has changed when it doesn’t?

Does your web app break because one of the above characters?

If you answered yes to any of these questions, you are not alone!

If you’ve dealt with this and figured out it please definitely shared your findings in the comments below.

I believe it is *the* number one issue in working in a hybrid environment, e.g. with Windows, Linux, and OS X.

In the next blog posts, I’ll talk about how I’ve struggled with these issues with other developers.

How fast can you get on the web? With newLISP it’s about as fast as typing:

How fast can you create a backdoor with newLISP?

If you telnet into port 1234 in localhost, you’ll see something that looks like this:

This opens up a lot of possibilities for distributed computing.

For example, you can set up a newLISP server that’s ready to respond to a newLISP client with this command:

Your newLISP client can have code that sends a computing problem to be solved to the server:

Or let’s say you had a farm of newLISP servers:

If the above example reminds you of Gearman, you get +12 points.