PHP Network Error

One of my PHP web applications stopped working today. After I investigated the issues, I noticed that this is a very famous, yet unsolved error. I have no idea why it happens, but I do have a work around solution for it. Basically, this article applies to you if you match all of the following:

  • You are using PHP
  • Your web applications talk to other servers via domain name (e.g., example.com rather than 123.1.1.3)
  • You use XAMPP (instead of native Apache, PHP).

Notice that I am not 100% sure whether this has anything to do with XAMPP. But most of problem I experienced happen on XAMPP platform.

And here are some example problems:

  • Getting a file connect using file_get_contents(‘http://example.com/somepage.html’), or CURL etc.
  • Sending emails (SMTP server: ‘mail.example.com’)
  • Connect to a database server via domain name (‘example.com:3306’)

Why this problem happens?

This problem has nothing to do with your PHP code. In fact, the problem happens when PHP tries to look up the IP address of your domain name. Let’s take a look to the following example. Suppose I have the following code:

$data = file_get_contents('http://example.com/test.html')

//process the $data here...

When PHP executes this code, it will try to get the IP address of example.com first, and talk to the server to retrieve the content. This problem happens because PHP is unable to get the IP address of example.com.

Initially, I thought it was my server issues, therefore I tried to ping example.com on the server, i.e.,

#ping example.com

and the result looks fine to me. So the problem has nothing to do with the OS / server. Then I run the following code in PHP

$IP = gethostbyname('example.com');
echo $IP;

Normally, I expect to see the IP address of example.com. If it return ‘example.com’, that means PHP is unable to determine the IP address. That explains why the web application stops working.

Solution #1: Restart XAMPP

Try to restart the XAMPP to see whether it resolves the problem or not:

sudo /opt/lampp/lampp stopapache
sudo /opt/lampp/lampp startapache

This method aims to resolve the situation that the XAMPP was already started but the network was not available. Restarting the Apache server helps to resolve this problem.

Solution #2: /etc/hosts

Since PHP was unable to lookup the IP address, I decided to give some hints to PHP by editing /etc/hosts:

123.1.1.1   example.com
123.1.1.2   anotherexample.com

This is a quick and easy solution. However, if example.com is moved a different IP address, you will need to update the file. It is pain in long term.

Solution #3: Stop using XAMPP

As I mentioned earlier, I notice that this problem happen in XAMPP environment only. I haven’t experienced this kind of problem with native Apache and native PHP. So I guess it may have something to do with XAMPP.

In fact, it is quite easy to switch from XAMPP to native Apache+PHP+MySQL etc. Native applications give you better performance and reliability and most importantly: the packages get upgraded automatically.

Hope it helps.

–Derrick

Our sponsors:

[PHP]imagecreatefrompng Error (After upgrading FreeBSD)

Today, I found that one of my PHP scripts stop working. After some investigations, I found that it was the function, imagecreatefrompng, which caused the problem.

Interestingly, other similar functions such as imagecreatefromgif and imagecreatefromjpeg were completely working fine. The program stopped working when calling imagecreatefrompng only.

Initially, I thought the problem was coming from the PHP side. Therefore, I tried to get the error messages as many as possible, such as PHP error (error_reporting), Apache error (error_log), and even system error (dmesg). Unfortunately, I couldn’t find anything.

The script was working completely fine until I upgraded the system. So I think the problem may come from a missing library. To give you some background, here is how imagecreatefrompng works. First, your program calls imagecreatefrompng, which involves the GD library. GD library calls a PNG function in your system (possibly libpng) to process the PNG file. Since the program comes with from the system library, PHP / Apache will not report anything about it.

Since it will be too complicated to identify which library caused the problem, I decide to reinstall all ports. It sounds a very long process but it wasn’t at all. It only took 20 minutes to finish the whole thing. So here is what I did:

#FreeBSD
#sudo portmaster -fa

After the installation is completed, I rebooted the machine and everything worked again!

Enjoy!

–Derrick

Our sponsors:

Natural Sort. MySQL. Sorting a 2D Array By Multiple Columns in PHP Naturally. Array_MultiSort with Natural Sort

MySQL has a biggest missing feature: It does not sort naturally. This has been an opened request since 2003, and MySQL team has no plan to fix this in a near future. I have looked for a solution to solve this problem for a while. Unfortunately, I found nothing useful. Here is a summary:

The + 0 Trick

Many developers suggest this solution:

SELECT names FROM your_table ORDER BY age + 0 ASC

This is not a good solution because it only works in some very specific situations. This query fails in most general cases.

Drupal Patch

Some developers suggest to use the patch from Drupal. Forget it, I could not even install it in my system.

My solution: Let PHP to handle the dirty work

So I decide to move the process to PHP. Before you say something about the performance issue, please keep reading my story first. I will talk about it in the later section.

MySQL is a relational database, it stores everything in a table. If we express that in an array, it will look something like this:

Array
(
    [1] => Array
        (
            [Name] => John
            [Identification] => 10
            [Address] => 100 Michigan Ave.
        )

    [2] => Array
        (
            [Name] => Peter
            [Identification] => 1000
            [Address] => 1 Michigan Ave.
        )

    [3] => Array
        (
            [Name] => Tom
            [Identification] => 2
            [Address] => 10 Michigan Ave.
        )

    [4] => Array
        (
            [Name] => Paul
            [Identification] => 1
            [Address] => 10 Michigan Ave.
        )

)

where the array key represents the record ID in the table.

Now, let say I want to sort the contents in the following orders:

  1. Sort by Identification: Ascending
  2. Then by Address: Descending
  3. Then by Name: Ascending

I specify the orders in an array:

$order = array('Identification' => 'ASC',
			   'Address' => 'DESC',
			   'Name' => 'ASC');

And include the following function in my code:

//array_multiSort with natural sort
function naturalSort2D(&$array){

   if (!function_exists(naturalSort2DCompare)){
	function naturalSort2DCompare($a, $b){
		global $order;

		foreach($order as $key => $value){
			
			if (!isset($a[$key])) continue;

			unset($compareResult);

                        //Case insensitive string comparisons using a "natural order" algorithm
			$compareResult = strnatcasecmp($a[$key], $b[$key]);
			
			if ($compareResult === 0) continue;
			
			$value = strtoupper(trim($value));
			
			if ($value === 'DESC'){
				$compareResult = $compareResult*-1;
			}
			
			return $compareResult;
		}
		
		return 0;
	}
   }
	
   //Maintain index association
   //Preserve array keys
   //Use usort if you don't care about key index
   uasort($array, 'naturalSort2DCompare');
	
   return true;
}

Now I sort the array:

naturalSort2D($array);

And I get my result like the following:

Array
(
    [4] => Array
        (
            [Name] => Paul
            [Identification] => 1
            [Address] => 10 Michigan Ave.
        )

    [3] => Array
        (
            [Name] => Tom
            [Identification] => 2
            [Address] => 10 Michigan Ave.
        )

    [1] => Array
        (
            [Name] => John
            [Identification] => 10
            [Address] => 100 Michigan Ave.
        )

    [2] => Array
        (
            [Name] => Peter
            [Identification] => 1000
            [Address] => 1 Michigan Ave.
        )

)

Performance

Before we talk about the performance issue, I like to talk about my database design first. In my application, the records are stored in a table. It is a multiple user system. Each user can access their own records only. Each user has about 3000 records, and we have 1000 users. Therefore we have about 3 millions of records in one table.

Here is a data structure:

#Information input by users
Name
Identification
Address

#Information input by system
ID
UserID

So the SQL quere is nothing more than something like:

Select * FROM `database`.`table` WHERE `UserID` = '1'

Previously, I let MySQL to handle the filter and sorting, i.e.,

Select * FROM `database`.`table` WHERE (`UserID` = '1') AND (`Address` LIKE `%Michigan%`) ORDER BY `Identification` ASC LIMIT 5, 10

After moving to PHP, I just let MySQL to handle the filter,

Select * FROM `database`.`table` WHERE (`UserID` = '1') AND (`Address` LIKE `%Michigan%`)

Notice that I will get all qualified records, could be few thousands. Then I let PHP to do the sorting:

$order = array('Identification' => 'ASC');
naturalSort2D($array);

Now I have a sorted array. Next I need to slice the records:

$array = array_slice($array, 5, 10);

Sounds like lots of work huh? Now let’s talk about the performance impact. Guess what, I see performance improvement for about 15%!

In the new approach, MySQL does lesser work. PHP does more work. Apparently, PHP works more effectively than MySQL. However, this may not true for all database model. Remember that in my situation, the system needs to handle at most few thousands records at a time? So I think PHP is doing better in this range. If your application needs to handle millions of records at a time, e.g., bookstore, then you better let MySQL to do the work.

By the way, the new approach allows to sort the dynamic generated content, which is a big plus in my situation. That is something I cannot do in MySQL approach.

Have fun.

–Derrick

Our sponsors:

[PHP]How to get the number of CPU cores in Fedora, Ubuntu, and FreeBSD

I am working on a PHP application which needs to be deploy to multiple places. One of the variables is the number of CPU core. Since each server has different number of CPU cores, I need to specify the number of CPU core in my application for each server. I think there should be a smarter way to do it.

PHP is a scripting language, it has limit support on accessing the hardware level information. In short, there is no library or function to do it. Fortunately, we can do it via shell command. In the other words, the following methods are not limited to PHP, it will work in any languages, as long as your language supports running the UNIX command and catch the return.

Getting the number of CPU Cores – Linux

(Tested on Fedora, Ubuntu Linux, should work on other Linuxs because they all use the same Linux kernel.)

cat /proc/cpuinfo | grep processor | wc -l

This will return something like this:

8

Getting the number of CPU Cores – FreeBSD

sysctl -a | grep 'hw.ncpu' | cut -d ':' -f2

which will return something like this (notice the extra space before the number):

8

Now, let’s put everything together. Run the command inside your application (Here I am using PHP for example):

//Linux
$cmd = "cat /proc/cpuinfo | grep processor | wc -l";

//FreeBSD
$cmd = "sysctl -a | grep 'hw.ncpu' | cut -d ':' -f2";

$cpuCoreNo = intval(trim(shell_exec($cmd)));

Of course, you can make the application to detect the system automatically:

$cmd = "uname";
$OS = strtolower(trim(shell_exec($cmd)));

switch($OS){
   case('linux'):
      $cmd = "cat /proc/cpuinfo | grep processor | wc -l";
      break;

   case('freebsd'):
      $cmd = "sysctl -a | grep 'hw.ncpu' | cut -d ':' -f2";
      break;

   default:
      unset($cmd);
}

if ($cmd != ''){
   $cpuCoreNo = intval(trim(shell_exec($cmd)));
}

That’s it! Happy PHPing.

–Derrick

Our sponsors:

Weird JavaScript Date Bug

Today I found a very weird bug in JavaScript – The date function. I never expected to see a bug in the standard JavaScript implementation. Here is my code. It does nothing more than adding 21 days on the reference date:

var referenceDateObject	= new Date(2011, 9, 10);
var deliveryDateObj 	= new Date(referenceDateObject.getTime() + 24 * 60 * 60 * 1000 * 21);
var deliveryDateMonth	= parseInt(deliveryDateObj.getUTCMonth());
var deliveryDateString	= deliveryDateObj.getUTCFullYear() + '-' + deliveryDateMonth + '-' +  deliveryDateObj.getUTCDate();
alert(deliveryDateString);

Given a reference date of September 10, 2011, I expect the date should be October 1, 2011. However, it ends up returning me September 31, which doesn’t exist!

I test that on Firefox and Google Chrome, both give the same problem. A quick fix will be moving the date calculation to the server side.

P.S. MySQL date function automatically convert 9/31 to 10/1, so the data on the server looks fine.

Update – That’s my fault. The Date object treats the month from 0-11 instead of 1-12. Here is a corrected version:

var referenceDateObject	= new Date(2011, 8, 10);   //That's September 10, 2011, not August 10.
var deliveryDateObj 	= new Date(referenceDateObject.getTime() + 24 * 60 * 60 * 1000 * 21);
var deliveryDateMonth	= parseInt(deliveryDateObj.getUTCMonth()) + 1;
var deliveryDateString	= deliveryDateObj.getUTCFullYear() + '-' + deliveryDateMonth + '-' +  deliveryDateObj.getUTCDate();
alert(deliveryDateString);

That will do the trick.

–Derrick

Our sponsors:

A letter to Microsoft Internet Explorer Development Team

Dear Microsoft Internet Explorer Development Team,

Would you like to take the source of Firefox (or any browser besides Internet Explorer) and call it Internet Explorer 11?

Thanks.

–Derrick

Our sponsors:

Enough “Rails Surprise”! Goodbye Ruby On Rails.

Couple months ago, I published an article about Ruby On Rails. I was wrong.

The idea of Ruby On Rails is really good. You can create a very simple database-driven web application (i.e., User can add, browse, remove and search records) in less than half an hour. There is no way to do something similar in such a short time using other programming languages. However – perhaps this is the tradition of the Rails development team, it comes with surprise (“Rails Surprise”) every time after upgrading the Ruby gems component. They either make your Rails application couldn’t start, or some features in your apps are not functioning after the upgrade.

What kind of people should stick with Ruby On Rails?

  • Someone is big fan of Ruby/Gems/Rails.
  • Someone has plenty of spare time.
  • Someone likes to sharpen his/her debugging skill – To debug the backward incompatibility mess introduced by the Ruby/Gems/Rails core development team.
  • Someone is patient and is willing to spend number of hours(could be days) to look for workaround.

I am neither. I think I already have enough “Rails Surprise”. And I will make my own web application framework from now on (The code name is called ICEPIZZA by the way). It will be simple, elegant and backward compatible.

–Derrick

Our sponsors:

Ruby On Rails gives no “surprise” in the latest version

I haven’t upgraded my Ruby On Rails and related Ruby gems for 5 months. One of the reasons is that Rails development team likes to break the backward compatibility, such as:

NameError: uninitialized constant ApplicationController

To avoid this kind of surprises, I tend to stick with a working, stable version.

Recently, I upgrade the entire Ruby On Rails family(2.3.2 -> 2.3.4), and surprisingly, my Ruby On Rails apps work perfectly fine! One exception is that the Mongrel server behaves a little bit slower than before. One way or the other, we need to use the load balancing technique to host the Rails App anyway. So it really doesn’t matter.

In case you want to try upgrading your Ruby Gems and other things:

Update the Ruby Gem Engine:

sudo gem update --system

Update other Ruby Gems:

sudo gem update

–Derrick

Our sponsors:

IE is “fxxking”, but it is not an excuse to leave bug in the codes

Today, my co-worker found that the web-application he developed worked only in Firefox, and it didn’t work in Internet Explorer(IE). He immediately asked me if we should ignore the IE users or not.

I know that he was working on JQuery(Javascript). It is not surprised that IE does not support the standard JQuery. However, it doesn’t mean that we can simply remove our support to IE users. In fact, IE still have more than 55% of market share today, while Firefox have around 30% only.(See here for more information) It’s too early(and not responsible) to ignore such a large user group.

So, after he complained about how fxxking the IE was for an hour, I tried to find out that the problem. I found that the problem came from his code rather than from IE. He had an extra tag in his code. Firefox was so smart to catch the error and ignore it. That’s why it only worked in Firefox, but not in IE.

So, here are what I’ve learned today:

  • The error catching function introduced in Firefox is so good. But at the same time, it makes some developers lazier by not verifying the quality of their works. They will think it is your problem if you are using IE.
  • IE sucks, but it is not an excuse of leaving the IE users behind.
  • Complaining has nothing to do to improve the problem. Talk is cheap, do it.

Next time before you complain to your manager about how fxxking the IE is, think twice.

Our sponsors:

Interviewing a software engineer / developer / programmer

I am going to have a face-to-face interview with a candidate of a PHP programmer position this afternoon. I was told to prepare some technical questions for the interview.

If a candidate earns an face-to-face interview opportunity, that means his resume should have a very good match to the position requirement. However, people can put anything on their resumes, there is no way to tell whether the information on the resume is true or not (and referencees guarantee will give you good feedback on the candidates). That’s why a face-to-face interview is really important.

I will have around 30 minutes to 1 hour to interview a candidate, how can I get the most information from him to help myself making this decision?

After googling on the web, I found tons of information about how to interviewing a programmer, but most of them are related to really technical questions, such as “How do you solve this math puzzle?”, or “What’s the meaning of double-dollar variable in PHP?” etc. I think these questions are not efficient and effective enough to judge the skill of the candidates. Personally, I don’t like these types of questions at all because I think it does not help to judge a realistic performance of a person. I prefer some more realistic questions.

So, I end up designing a test by myself. It is nothing more than a simple html form (e.g., a form to update users’ contact information). The candidate will be asked to implement the functions to update the records. I think this is pretty closed to what I do in my work, i.e., Create, View, Update and Delete record.

Well, this question sounds pretty simple, and I think there are plenty of hidden traps. Here are what I expect in the codes:

  • Core function, i.e., Updating the record – 40%
  • Validating the inputs(e.g., don’t store a negative number into age column, make sure the ID exists etc) – 10%
  • Performance (e.g., Update the record only if the records have been changed by the users.) – 10%
  • Handling simultaneous update (e.g., lock the record before update) – 10%
  • Security (e.g., storing the configuration out of the web server scope, prevent online attack etc) – 10%
  • Development time(e.g., spending 2 hours ends up with very few features is not acceptable) – 10%
  • Think outside the box(e.g., Suggesting to use Ruby on Rails to shorten the development time, using jQuery for client-side validations etc.) – 10%

I believe that if a person can do great in a simple task, he will do great in a complex task. Unfortunately, it is not easy to look for a good programmer.

Updates (August 31, 2009):

The answers I received from the candidates are pretty interesting and filled with tons of surprise. Some of the answers go beyond my imaginations. Here are few examples:

  • One candidate sent me a package with all of the source codes. Unfortunately, none of the codes works. Apparently, he forgot the golden rule of software development: A software must work.
  • Another candidate sent me a package with tons of files. After I read the codes, I found that the codes were copied from other projects with very minor modifications. This kind of candidates should be avoided because cleaning up the code is a basic responsibility of a programmer.
  • A candidate has spent a lot of efforts on defining the variables (More than 20 variables were defined in the configuration sections), but he only put one line of comment (and no code implementations) inside the validation function, i.e.,
    //Validation goes here...

    He would impress me if he included at least one validation example.

  • One of the submitted answers is even more interesting (and ridiculous). It was created via some PHP-MySQL wizard websites. Although the result works fine (of course!), he does not receive any credit because his programming skills could not be proven.

So far, the best answer scores 60%. I don’t think this is an exciting result. If you found this useful, please let me know, thanks!

–Derrick

Our sponsors: