Monthly ArchiveNovember 2009
How-To & php 27 Nov 2009 08:25 pm
WP Geo Plugin
The print_GeoCache_Url function came across my email today from a self-described local designer and geek, but after a little research, I found out it only works up to WordPress 1.2 . Thank goodness for the WordPress WP-Geo Plug-in which I’m using right now.
More info here: WPGeo.com
AWS & How-To & sysadmin 14 Nov 2009 06:01 pm
EC2 Backup Script
This is a quick and dirty EC2 backup script for virtual unix servers that works just fine when crontabbed:
#!/bin/bash
DATE=`date +%m%d%Y-%H%m%M`
BUCKET=”codebelay-$DATE”
PRIVATE_KEY=’pk-codebelay.pem’
PRIVATE_CERT=’cert-codebelay.pem’
USERID=’555555555555′
AWS_ACCESS_ID=’AKIA0000000000000′
AWS_SECRET=’asdf+asdf+asdf+asdf’
s3cmd mb s3://$BUCKET
cd /mnt
mkdir img
ec2-bundle-vol -d /mnt/img -k /mnt/$PRIVATE_KEY -c /mnt/$PRIVATE_CERT -u $USERID -s 9999 –arch i386
cd /dev
mkdir loop
cd loop
mknod 0 b 7 0
ec2-upload-bundle -b $BUCKET -m /mnt/img/image.manifest.xml -a $AWS_ACCESS_ID -s $AWS_SECRET
# rm -rf /mnt/img
echo “please register $BUCKET/image.manifest.xml” >> /mnt/registerbackups.txt
Databases & MySQL & WebApps & scalability hacking 10 Nov 2009 03:56 pm
Notes on adding more MySQL databases
Just notes for myself on adding more MySQL databases without shutting down the master database.
on existing slave:
/etc/init.d/mysqld stop
copy data dir from /var/lib/mysql and data from /var/run/mysqld to new slave database:
cd /var/lib
tar cvf Mysql_slave.tar mysql/*
scp Mysql_slave.tar root@new-db.com:/var/lib/.
cd /var/run
tar cvf Mysqld_slave.tar mysqld/*
scp Mysqld_slave.tar mysqld/*
scp Mysqld_slave.tar root@new-db.com:/var/run/.
tar cvf Mysql_slave.tar mysql/*
scp Mysql_slave.tar root@new-db.com:/var/lib/.
cd /var/run
tar cvf Mysqld_slave.tar mysqld/*
scp Mysqld_slave.tar mysqld/*
scp Mysqld_slave.tar root@new-db.com:/var/run/.
copy /etc/my.cnf from old slave to new slave
add entry for new server-id
start existing slave:
cd /var/lib
tar xvf Mysql_slave.tar
cd /var/run
tar xvf Mysqld_slave.tar
/etc/init.d/mysqld start
tar xvf Mysql_slave.tar
cd /var/run
tar xvf Mysqld_slave.tar
/etc/init.d/mysqld start
start new slave:
/etc/init.d/mysqld start
mysql
start slave;
mysql
start slave;
on masterdb:
e.g.:
grant replication slave on *.* to ‘repl’@'192.168.107.33′ identified by ‘password’;
test on master:
create database repl;
check on slave:
show databases; /* should show new database */
test on master:
drop database repl;
check on slave:
show databases; /* new database should be dropped */
Now it’s time to turn this into an automated shell script with Expect in there.
How-To & scalability hacking 08 Nov 2009 10:42 am
Part II: Getting to 600 Concurrent Users
I couldn’t sleep last night. I’m worried we’ll lose this client.
So just to be clear. I wasn’t part of the crew responsible for scaling this site. I had already set up a scalable architecture for the site, that would automatically and horizontally scale at Amazon. That idea got shot down for legal reasons that to my surprise haven’t been in play for awhile. Can we say, “Office politics?”
I totally recommend Amazon’s Autoscaling to anybody that’s new to this.
Instead of auto-scaling, the site was architected by a local San Francisco firm who I won’t mention here.
Let’s just hope enough people read this so that they won’t even have to know the name of the company and will just know the smell of an un-scaleable architecture.
Scalability requirement: 100,000 concurrent users
This is how they set it up:
- two web servers
- one database
- four video transcoders that hits the master database
- one more app server that hits the master database
- no slave db
If they had even googled ‘building scalable websites’ they would have come across a book that would have avoided all of this, Cal Henderson’s Building Scalable Websites. It should be mandatory reading for anybody working on a large website, and it just scratches the surface.
So, how did we get to 600 concurrent users?
We tweaked mysql by putting this in /etc/m.cnf:
[mysqld]
max_connections=10000
query_cache_size=50000000
thread_cache_size=16
thread_concurrency=16 # only works on Solaris and is ignored on other OSes
max_connections=10000
query_cache_size=50000000
thread_cache_size=16
thread_concurrency=16 # only works on Solaris and is ignored on other OSes
We ran siege and were able to get to about 300 concurrent users without breaking a sweat, but now apache was dying.
So we tweaked apache. We started out with this:
StartServers 8
MinSpareServers 5
MaxSpareServers 20
ServerLimit 256
MaxClients 256
MaxRequestsPerChild 4000
And ended up with this:
StartServers 150
MinSpareServers 50
MaxSpareServers 200
ServerLimit 256
MaxClients 256
MaxRequestsPerChild 4000
MinSpareServers 50
MaxSpareServers 200
ServerLimit 256
MaxClients 256
MaxRequestsPerChild 4000
RAM and CPU were doubled.
Databases & How-To & scalability hacking 06 Nov 2009 10:21 am
Scaling from 100 to 100000 concurrent users in a day?
Well, it looks pretty bad right now. A vendor just ceded control for web application architecture. Initial tests say that the site won’t do no more 100 users concurrently.
Who the hell makes a web app without a slave database and calls themselves website architects? Apparently these guys did.
Please start following if you want to see if this web app can make it to launch.
