Today I was comparing the performance of Tokyo Cabinet, MySQL and Memcached using PHP. The result of Tokyo Cabinet is really impressive!

(Note that these results are generated from my computer. If you perform your own benchmarks, then you might find different results due to different system configurations. In fact, I encourage you to perform your own benchmarks. )

1. Quick Result

Total time used to write 10000 records and retrieve them back by each candidate  (The lower the better):
Tokyo Cabinet: 18.87s
Memcached: 11.309s
MySQL: 562.21s

A completed result will be given below.

2. Testing environment

Hardware

  • Intel Pentium II 450MHz
  • 160 MB RAM

Software

  • FreeBSD 7.2
  • Apache 2.2.11
  • PHP 5.2.9
  • Tokyo Cabinet 1.4.20
  • Tokyo Tyrant 1.1.26
  • MySQL 5.1.34
  • Memcached 1.2.6

3. Test scenario

For each candidate, I measure its performance using the following strategies through PHP:

1.) The time used to create all of the necessary objects.

2.) The time used to connect to the database.

3.) The time used to store 10000 randomly generated data.

4.) The time used to read 10000 randomly generated data.

5.) The time used to close the connection.

4. Test results

Tokyo Cabinet

time index ex time %
Start 1243136772.72818600 0.00%
create 1243136772.72838700 0.000201 0.00%
connect 1243136772.73224200 0.003855 0.02%
vanish 1243136772.73526800 0.003026 0.02%
put 1243136781.53287200 8.797604 46.60%
get 1243136791.60574800 10.072876 53.35%
close 1243136791.60755900 0.001811 0.01%
Stop 1243136791.60782300 0.000264 0.00%
total 18.879637 100.00%


Memcached

time index ex time %
Start 1243131185.86078100 0.00%
create 1243131185.86092500 0.000144 0.00%
connect 1243131185.86688300 0.005958 0.05%
put 1243131191.49702900 5.630146 49.78%
get 1243131197.16868100 5.671652 50.15%
close 1243131197.16982400 0.001143 0.01%
Stop 1243131197.17003400 0.000210 0.00%
total 11.309253 100.00%

MySQL

time index ex time %
Start 1243136138.19049000 0.00%
create 1243136138.19064700 0.000157 0.00%
connect 1243136138.22529900 0.034652 0.01%
put 1243136150.70760600 12.482307 2.22%
get 1243136700.40063700 549.693031 97.77%
close 1243136700.40078900 0.000152 0.00%
Stop 1243136700.40104500 0.000256 0.00%
total 562.210555 100.00%

The overhead time (create objects, making connections, closing connections etc) of all candidates are about the same. The only difference is storing the record and retrieving the record. Both Tokyo Cabinet and MySQL took about 10 seconds to store 10000 records in the database. However, Tokyo Cabinet took about 10 seconds to retrieve the records while MySQL took about 550 seconds! That’s about 31 times longer!

benchmark-tokyocabinet-memcached-mysql

5. Materials

Here is the material I used to make this benchmark test.
Click here to download the package.

Please feel free to let me know if you have any question or comment.

Our sponsors:

When Oracle acquired Berkeley DB, the quality of Berkeley DB has gone worse after that.

Now Oracle acquired Sun who owns MySQL, will the history repeat again?

(By the way, I am going to fall in love with Tokyo Cabinet.)

Tokyo Cabinet is now the new black.

Our sponsors:

Recently I developed several projects using Ruby on Rails.  One of the cool features of it is the built-in test tools. With this amazing tool, I can test my web page such as data validation in few seconds by myself. It reminds me the time how a software was tested in the company I worked for previously.

In the formal company I worked for, they had a very well Quality Assurance Team for testing the software functionality(The ratio of developer to QA is about 3:2). Usually we (developer) need to come up many test cases for them (e.g., testing the data field with positive, negative or invalid value) because the QA team does not have any technical background. Other than that, the QA team would only focus on the GUI level testing, such as whether this control is aligned with the label or not etc. One of the key components that has been ignored is the stress test.

I still remember that I developed a resource-consuming feature, and I was worrying about its impact of overall performance when many uses are using this feature at the same time. So I talked with my QA and see what he think. Initially I expected that he would run a stress test (e.g., simulating 100 thousands of users to use the feature at the same time), and guess what his answer?

I can schedule an appointment with the rest of the QA team, we can sit down together and try to use your feature at the same time.

No, this is not a joke. In fact, about a month later, someone from other application team really does something fancy about it. They sent out a mass email and ask people to sign up. They end up having about 900 uses to use the application at the same time. That’s about 1/3 of the whole population in the company!

I think it is kind of stupid to ask 1/3 of the whole company to do a simple test, which can be done using computer automatically! Think about how expensive is the test:

900 people x $40/hr x 2 hour = $72,000

With this amount of money, they can probably hire one to two QAs to work on the test for a year!

Our sponsors:

If you cannot decide whether you should go with FreeBSD or Ubuntu (or Linux), check out this article: FreeBSD or Linux in 6 Simple Questions

Recently I am working on building a web farm which is formed by three ancient time computers (The sum of their ages is older than me). Therefore, I need to pick an operating system to make them running efficient enough. Basically here are my criterias:

  • It must be secure. (Sorry, Microsoft Windows, I tried installing Windows Server and the OS got infected by virus in 20 minutes after connecting to Web.)
  • It must be stable and require no reboot. (I have a very good experience with FreeBSD. My longest record of uptime is 6 months so far.)
  • It must be efficient and effective on performance.

So after trying different operating systems, I ran down to three choices: FreeBSD 8, Ubuntu 10 and OS X.

And I finally go with FreeBSD 8. Before I talked about why I kick the Ubuntu and OS X away, let me discuss what methodology I used.

First Test: Extreme Computation

I write a program to calculate the value of pi using Monte-Carlo simulations. For those who are not familiar with this simulation method, basically it try to repeat a calculation for n times and try to determine the value of pi based on the results of calculations. This calculation will use all available CPU resources.

Here is the result from a computer: Pentium II 400MHz + 160MB

  • FreeBSD 8.1 ~ 5 seconds
  • FreeBSD 4.11~ 10 seconds
  • Ubuntu 10.04 ~ 8 seconds
  • OS X 10.3.9 ~ 11 seconds

The result is very obvious, FreeBSD 8.1 is the best candidate in this test. However, the result of my program only gives a very small portion of the picture. So I decide to run another test:

Second Test: Apache Benchmark

No test is better than the real world test drive. Since my primary purpose is for web server, so I decide to test the performance on how Apache perform on different operating systems. My test is pretty simple, I basically ask the Apache Benchmark client to download a page from the test server. The page does three things:

  1. Display a picture
  2. Insert a record into MySQL database
  3. Retrieve the current count from MySQL database

This way I can test the overall performance on how the system handles the file I/O, database I/O and computations.

So the result is pretty interesting (1000 total requests and 100 concurrent requests)

  • FreeBSD 8.1 ~ 200 requests/sec, no failure
  • FreeBSD 4.11~ 150 requests/sec, approx. 20% failure.
  • Ubuntu 10.04 ~ 180 requests/sec, no failure
  • OS X 10.3.9 ~ 160 requests/sec, approx. 50% failure.

I am not surprised with the result because FreeBSD 8 really does a very good job in kernel optimization. Since kernel is the heart(or brain) of all computations, an OS with good kernel is expected to give good results always.

So, I finally pick FreeBSD 8 as the core OS of my web farm, and I am loving it.

Our sponsors:

Recently I found that Wolfram Research (The maker of Mathematica) released an online Math search engine. It is way much more powerful than the Google Calculator. It gives you not only the math result, but also a graphical expression on the result.

Solving a math problem:
Suppose I want to find out the value of x, where sin(x)cos(x) = 0.5. Oh wow, I got the answer right the way!
worlframalpha-sinxcosx
Try to find out the result of sin(x)cos(x) = 0.5 here

Or let’s say I want to know the populations of California:
worlframalpha-californiapopulations

Try to find out the result of California population here

Now something more interesting. I want to know what’s the result of sin(x)=2:
worlframalpha-sinx2
I got the result in 2 seconds!!!! I still remember that I’ve spent over 2 hours to work on this problem back in freshman year. (See My Presentation on Sin(x)=2)

Try to find out the result of sin x = 2 here

Want to have more fun with Math? Try it yourself here!

Our sponsors:

I was experimenting the salary search engine at Indeed.com today, and I found some interesting result…

“The salaries of Shit in different locations in US”

Here is the result:

indeed-shit

Or you can try to click here to repeat the test….

What makes me couldn’t stop laughing is the statistical conclusion below…

Average shit salaries for job postings in New York, NY are 68% higher than average shit salaries for job postings in Hawaii.

Does it mean a shit worth more value in NYC than in Hawaii?

The reason why I want to put this interesting result is not for making fun on Indeed.com, but I want to understand why the QA at Indeed did not notice this bug, and why the engineer of this search engine is using such a poor algorithm? I am pretty sure NO job has the keyword “shit” in the job title! Where did the search engine get this information? How does it get these numbers?

Do you think your salary is high enough? Try to compare your salary with the result using these keywords: Unemployed and Do Nothing.

indeed-unemployed

Want to know the solution and prevent this embarrassing result? Shoot me an email and we can talk.
(It will require you sometime to find out my email but it is not that hard, huh?)

Our sponsors:

This is my note on how to to install Apache, PHP and MySQL on a an OS X box. Although some of these applications are come with the operating systems, it is always good to know how to upgrade them to the latest version.

1. Apple Developer Tools and Mac Port

Before installing Mac Port, you will need the Apple Developer Tools. You can download it from the Apple Developer’s Network.

After installing Apple Developer’s Tools, you can install Mac Port. You can get it from the here.

Update Mac Port

sudo port selfupdate

Add the Mac Port path in the profile: ”’~/.bash_profile”’:

export PATH=/opt/local/bin:$PATH

MySQL:Mac Port
Install MySQL from Mac Port:

sudo port install mysql5

MySQL:Package
Go to the [http://dev.mysql.com/downloads/mysql/5.0.html#macosx-dmg MySQL’s website] to download the package.

Setup the a master password for root user:

/usr/local/mysql/bin/mysqladmin -u root password 'new-password'

Add the MySQL path in the profile: ”’~/.bash_profile”’:

export PATH=/usr/local/mysql/bin:$PATH

Apache:Mac Port

Install Apache from Mac Port:

sudo port install apache2

Open the configuration file: ”’/opt/local/etc/rc.conf”’ and add the following at the end of the file:

APACHE2=-YES-

PHP:Mac Port

Install PHP from Mac Port:

sudo port install php5 +apache2 +mysql5 +pear 

When that is done, register PHP 5 with Apache 2:

cd /opt/local/apache2/modules
sudo /opt/local/apache2/bin/apxs -a -e -n "php5" libphp5.so

And create a php.ini file (which you can edit to configure PHP 5):

sudo cp /opt/local/etc/php.ini-dist /opt/local/etc/php.ini

httpd.conf

Edit the Apache configuration file ”’/opt/local/apache2/conf/httpd.conf”’ and add the following lines to the end of the file:

AddType application/x-httpd-php .php
AddType application/x-httpd-php-source .phps

Replace this line:

DirectoryIndex index.html

by the following:

DirectoryIndex index.php index.html

php.ini

If the MySQL is installed from package rather than from Mac Port, we need to modify the mysql.default_socket value.

Open ”’/opt/local/etc/php.ini”’:

mysql.default_socket = /tmp/mysql.sock

MemCached

Install MemCached from Mac Port:

 sudo port install memcached
 sudo port install php5-memcache

We need to enable MemCached in PHP5.

First, we need to add the following in ”’/opt/local/etc/php.ini:”’:

extension_dir=/opt/local/lib/php/extensions/no-debug-non-zts-20060613
extension = memcache.so

Run the following command to start MemCached:

sudo /opt/local/bin/memcached -d -u nobody

Our sponsors:

This is my note on how to to install Apache, PHP, MySQL and MemCached on a FreeBSD 7 Box.

1. MySQL

Go to the mysql52-server port directory by typing the command:

cd /usr/ports/databases/mysql51-server

Build the port by typing:

make BUILD_OPTIMIZED=yes BUILD_STATIC=yes

Using these two options will speed up the execution time.

Install the port by typing:

make install clean

Open the configuration file: ”’/etc/rc.conf”’ and add the following at the end of the file:

mysql_enable="YES"

Copy the default configuration file:

cp /usr/local/share/mysql/my-medium.cnf /etc/my.cnf

Start mysql manually to avoid having to reboot now by typing:

/usr/local/etc/rc.d/mysql-server start

Setup the a master password for root user:

/usr/local/bin/mysqladmin -u root password 'new-password'

2. Apache

Go to the apache22 port directory by typing the command:

cd /usr/ports/www/apache22

Install the port by typing:

make install clean

Open the configuration file: ”’/etc/rc.conf”’ and add the following at the end of the file:

apache22_enable="YES"

Add the following in: ”’/boot/loader.conf”’

accf_http_load=YES

3. PHP

3.1 PHP5

Go to the php5 port directory by typing the command:

cd /usr/ports/lang/php5

Install the port by typing:

make install clean

Make sure the APACHE (Build Apache module) option is checked.

3.2 PHP5 Extentions

Note: If you need to install PDF-Lite, please do the following:

fetch ftp://ftp.swin.edu.au/gentoo/distfiles/PDFlib-Lite-7.0.2.tar.gz
sudo mv PDFlib-Lite-7.0.2.tar.gz /usr/ports/distfiles/

Go to the php5 extensions port directory by typing the command:

cd /usr/ports/lang/php5-extensions

Make sure sockets is selected.

Install the port by typing:

make install clean

Install the php.ini file:

cp /usr/local/etc/php.ini-dist /usr/local/etc/php.ini

3.3 Enable PHP in Apache

3.3.1 httpd.conf

Edit the Apache configuration file ”’/usr/local/etc/apache22/httpd.conf”’ and add the following lines to the end of the file:

AddType application/x-httpd-php .php
AddType application/x-httpd-php-source .phps

Replace this line:

DirectoryIndex index.html

by the following:

DirectoryIndex index.php index.html
3.3.2 httpd-languages.conf

Edit the Languageconfiguration file ”’/usr/local/etc/apache22/extra/httpd-languages.conf”’ and add the following lines to the end of the file:

AddDefaultCharset On

Startup the Apache:

/usr/local/etc/rc.d/apache22 start

4. MEMCached

Go to the pecl-memcache port directory by typing the command:

cd /usr/ports/databases/pecl-memcache

Install the port by typing:

make install clean

Go to the memcached port directory by typing the command:

cd /usr/ports/databases/memcached

Install the port by typing:

make install clean

Open the configuration file: ”’/etc/rc.conf”’ and add the following at the end of the file:

memcached_enable="YES"

To start using MemCached without rebooting, type:

/usr/local/bin/memcached -d -u nobody

–Derrick

Our sponsors:

If you are looking for the classic del.icio.us to work on Firefox 3, then you are on the right page.

Download del.icio.us classic

Background

Why del.icio.us. classic add-on doesn’t work in Firefox 3?

The add-on was developed for Firefox 2, which is not compatible with Firefox 3. Since this add-on is no longer maintained by the original author, this add-on becomes an orphan. In fact, del.icio.us encourages users to upgrade to the modern version. Some users like me prefer to stick with the classic version.

Why del.icio.us classic, not modern?

Well, there are couple reasons, privacy and security. The classic version only uploads your public bookmark (The one you want to share), however the modern version will upload both your local and public bookmarks. Since I care about my privacy and security, I don’t want to see my personal information floating around the web at all.

Running del.icio.us Classic on Firefox 3

If you want to run the classic version on Firefox 3, you will need to download the add-on from the following link:

Download del.icio.us classic

…And if you want to use any Firefox Add-on on Firefox 3.5+ but it is not available, feel free to let me know! Don’t forget to include the link. Thanks!

Enjoy!

Our sponsors: