I put my personal websites on a FreeBSD server. One of my websites is a photo album, which I want to read the content from a Dropbox. That Dropbox primarily runs on Mac, iPhone and iPad. I was trying to explore the possibilities to set up a Dropbox on FreeBSD. Since Dropbox doesn’t support FreeBSD officially, I need to use 3rd party tools, most of them are basically based on the Dropbox developer API.

So I have tried several 3rd party tools, as you expect, none of them works. The primary problem is the synchronization, i.e., if my wife adds or deletes a photo on the Dropbox, I expect that the Dropbox folder on FreeBSD will get updated as well. Another problem is the speed. Looks like the Dropbox API is not as fast comparing to its own native application. On the same network, it took few hours to download the content (around 1GB of jpeg files) from Dropbox on FreeBSD, versus 10 minutes on a Mac/Windows/Linux machine using the native application.

So I came up few alternative solutions:

  1. Hosting my website on CentOS Linux. Since Dropbox supports Linux, I can easily read the Dropbox without any problem.
  2. Push the Dropbox content from Mac/Linux to FreeBSD using Rsync periodically (e.g., every 5 mins, hourly etc). That way FreeBSD will have access the Dropbox files.
  3. Set up a NFS service on a Linux box with access to Dropbox, and let the FreeBSD to mount the corresponding NFS share. This solution is okay if both machines are on the same network. It may raise some security concerns if both machines are connected via the public.

Another solution I think it may work is to install the Dropbox native application on FreeBSD. FreeBSD supports running Linux application via Linux emulation. Back in the old days (FreeBSD 8), it was pretty easy to include the Linux support on FreeBSD (one click in the sysinstall). Since the recent releases, they’ve made it harder because not many people wants to run Linux binary on FreeBSD. Based on my previous experience, I think it should work on the latest FreeBSD, but it may require some works.

Another crazy idea will be running Dropbox with Wine on FreeBSD. But this goes way too far from my original purpose, and I am not a big fan of Wine because it adds too many libraries to the system.

Our sponsors:

FreeBSD or Linux

FreeBSD or Linux (Ubuntu/RHEL)? This is a very old question. It’s like asking iPhone or Android. There is no short answer. It all depends on your situation. To make things easier, I am going to break it down into six simple questions to help you making decision.

My Background
I have been a FreeBSD user since 2003. My usage on FreeBSD mainly on service-oriented stuffs such as web farm, database clusters and file system etc. In short, I mainly use my FreeBSD system via command line. In 2009, I jumped into Linux world (Ubuntu/RHEL) because of my job. As an advanced user on both Operating Systems (FreeBSD and Ubuntu/RHEL Linux), here is my guide on these two systems.

There is only one FreeBSD, but there are many different variants of Linux. The Linux I mention below refers to two popular distributions: RHEL and Ubuntu.

FreeBSD vs Linux: Q.1 How do you describe yourself?

I am a very demanding person. I like to control everything I manage. –> FreeBSD
I don’t care about how a system is run. I am okay as long as it just works. –> Linux / Ubuntu / RHEL

Comment:
FreeBSD gives you freedom to control every single thing. One of its coolest feature is the port tree. You can build every application from source using port tree. In Linux world, you usually install applications from pre-built/pre-compiled package (yum, apt-get etc), which may not be exactly what you need.

Example:
In RHEL and Ubuntu, HTTP load balancer module does not come with Apache by default. You will need to compile Apache from source. What about in FreeBSD? All you need is to check a box (pretty much like shopping cart) and you are done.

FreeBSD vs Linux: Q.2 Do you prefer Ferrari or Hyundai?

Ferrari / BMW / SLR Camera –> FreeBSD
Toyota / Hyundai / Point and Shoot Camera / Phone Camera –> Linux / Ubuntu / RHEL

Comment:
The technology uses by FreeBSD such as kernel, file system, architecture etc are way better and more advanced than Linux. It’s like comparing Ferrari and Hyundai (And no, I am not kidding).

Example 1:
I need to run some extreme applications (e.g., DNA Sequence Alignment) which use all available threads and memory. The default settings of memory management in Linux is very poor. Every time I run my application, the system becomes not usable to other users. However, FreeBSD does not have this server problem. In fact, FreeBSD is smart enough not to cause the system to freeze. Of course you can tweak the memory management settings in Linux, or even run the command along with the nice command. However, these settings are just not available out of the box, and most of the time, you learn these tricks after your system has problem.

Example 2:
I installed FreeBSD 11 and RHEL 7 on two identical computers respectively. Both use default settings and enable similar services, such as enabling the SSH server and disable booting to X-Windows (RHEL). I used these two machines for exact the same purposes: SSH tunneling, with exact the same work load (evenly distributed). After using them for a month, I checked the memory usage. With FreeBSD, the available memory is about 800MB (out of 1GB), while there are only 200MB left (out of 1GB) on RHEL. Yes, Linux (at least with RHEL 7) consumes lots of memory.

Example 3:
FreeBSD comes with ZFS (The next generation file system) by default. Although it has been ported to Linux world, it is definitely unstable. We’ve tried to use it in a production environment. One thing that we’ve learned is that the ZFS may stop working after upgrading to a newer Linux kernel. I’ve received countless of email alert about missing the files (ZFS is not working, of course the files are gone) in the middle of the night. I ended up disable the auto update and disable the reboot after the update. Sounds familiar? That’s a feature in Windows, and for some odd reasons, this feature is available in Linux, a server.

Example 4:
LVM+RAID is an the most advanced storage method in Linux world. Unfortunately it does not do what it promises, i.e., you may lose your data if a hard drive is failure, even if you follow its directions to detach your failed hard drive correctly. Not to mention that the data will get corrupted if the power is down (which has been taken care of in ZFS, available in FreeBSD).

That’s why I prefer ZFS over LVM+RAID here: Building a Super Large and Reliable File Server with Mixed Size of Harddisks. It solves my problem (yes, even the power is failed during writing to the disk, my data is still safe!)

FreeBSD vs Linux: Q.3 Do you have lots of free time?

Yes: FreeBSD
No: Linux / Ubuntu / RHEL

Comment:
Making a production-ready system using FreeBSD can take you days to weeks if you are not an experienced FreeBSD user, while everything works out of the box in Linux. Sometimes, the new upgrade from the port tree can drive you nut, such as package conflicts etc. However, working with Linux is a leisure thing.

Example:
Installing Apache + MySQL + PHP from FreeBSD port (compiling the source) can take at least half day on a computer with a dual core CPU (AMD Athlon 64 X2 Dual Core Processor 4600+), while it takes less than 30 minutes on Linux. That’s because you need to compile the code from source in FreeBSD, while you simply download the packages and extract them in Linux. The time difference is huge.

FreeBSD vs Linux: Q.4 Do you prefer simplicity or complexity?

Adding sugar into water –> FreeBSD
Taking sugar away from soda–> Linux / Ubuntu / RHEL

Comment:
FreeBSD is a very very simple system. Think of it as a bare-bone system that comes with no junk. By default, it comes with no graphic user interface, no unnecessary application. It likes a pure distilled water.

In Ubuntu / RHEL, everything is configured and ready to use. It comes with very attractive, beautiful graphical user interface. Everything just works out of the box, no tuning or tweaking is required. However, it also comes with lots of junk such as Ubuntu One (For Ubuntu Cloud service), SELinux (A program developed by Red Hat where many people don’t use it), etc. It makes your system very bulky and increase the computation resource consumption.

FreeBSD vs Linux: Q.5 Are you going to use the computer as desktop?

Yes: Linux / Ubuntu / RHEL
No: FreeBSD

Comment:
Setting up a desktop-ready system on FreeBSD can take a long time. Main reason is the drivers availability. A lot of hardware such as graphic cards, audio cards or webcams are not supported in FreeBSD natively. If you want to get them working as good as on Windows / Ubuntu, you will need to get the driver first (if available), build it (which may gives error during compiling the codes), and recompile the kernel to make it supports the new driver, which can take few days if you are not experienced with FreeBSD and debugging driver.

On the other side, the Linux driver community is very strong and well developed. Usually they develop drivers for most popular hardware.

Example:

#1: My Logitech Orbit MP webcam (Pan / Tilt / Zoom) is not working on FreeBSD but works like a charm on Linux.

#2: Some vendors such as Highpoint may stop developing drivers for their products for newest version of FreeBSD.

FreeBSD vs Linux: Q.6 Do you need to blame someone when something goes wrong?

Yes: Linux / Ubuntu / RHEL
No: FreeBSD

Comment:

When something goes wrong, you can blame Linux and you can’t blame FreeBSD.

FreeBSD is a community driven operating system, while some Linux distributions such as Ubuntu and RHEL are backed by commercial vendors. In the world of FreeBSD, it is not uncommon that some unskilled developers introduce bugs, troubles to the rest of the world. Yes, we know it is free, so we can’t complaint about it. However, as an IT administrator, you will need to use your judgement to judge whether those new stuffs are safe to use or not. In short, that will increase your workload. It’s more like Windows update. How many people actually read the change log before hitting the update button?

In Linux (e.g., RHEL), that’s a whole different story. Every patches, new updates have been screened by the vendor before hitting to public. So you can trust them in some degree. And the key thing is, you can blame them when something goes wrong.

Conclusion

In short, use FreeBSD for your personal purpose while Linux for your work. Be the top 5%, not the bottom 95%.

Our sponsors:

It is not easy to upgrade PHP 5.5 to 5.6 in FreeBSD. Without proper preparation, the upgrade process may drive you nut. Before you decide to get your hands wet, here are what I recommend you to do:

  1. Back up your files
  2. Test your website in a PHP 5.6 environment on a different server. It is because PHP 5.6 has introduced some backward incompatibilities. Some of the codes written in the prior versions may introduce run time error. See here for more information.
  3. Schedule a down time. Depending on your CPU speed / typing speed / trouble-shooting skill, it may take you an hour.

Background

I am assuming that you use PHP for web purposes (rather than command line / CLI only), and I am assuming that you are using PHP with Apache. Here are the ports you will need to touch:

  • Apache: /usr/ports/www/apache22 or /usr/ports/www/apache24
  • Apache-PHP: /usr/ports/www/mod_php56
  • PHP: /usr/ports/lang/php56
  • PHP Extensions: /usr/ports/lang/php56-extensions

1. Remove the old PHP and extensions

cd /usr/ports/lang/php55
sudo make deinstall clean


cd /usr/ports/lang/php55-extensions
sudo make deinstall clean

2. Install PHP 5.6

cd /usr/ports/lang/php56

#Don't forget enable ZTS if you have threaded Apache.
sudo make install clean

3. Install PHP 5.6 Extensions

cd /usr/ports/lang/php56-extensions
sudo make install clean

4. Test PHP and its extensions

php -v
php -m

Clean up the error by removing the duplicated entries in:
/usr/local/etc/php/extensions.ini

5. Rebuild the Apache-PHP Bridge

cd /usr/ports/www/mod_php55
sudo make deinstall clean

cd /usr/ports/www/mod_php56
#Don't forget enable ZTS if you have threaded Apache.
sudo make install clean

6. Restart Apache

sudo /usr/local/etc/rc.d/apache restart

7. Test PHP using phpinfo

Create a code called test.php to display phpinfo. Verify that everything is okay.

< ?php
phpinfo();
?>

8. Reinstall Apache (optional)

If you experience any problem, try to reinstall the following ports:

Apache: /usr/ports/www/apache22 or /usr/ports/www/apache24
Apache-PHP: /usr/ports/www/mod_php56

That’s it! Enjoy the new PHP!

–Derrick

Our sponsors:

When I tried to run rsync (3.1.1) on my FreeBSD box today, the following message caught my attention:

#rsync -avzr --rsh="ssh -c arcfour" --delete --compress-level=9  sourcemachine:/source/  /target/


This rsync lacks old-style --compress due to its external zlib.  Try -zz.
Continuing without compression.

Basically, rsync suggests that you should use the -zz option instead of the old style option. So I gave it a try, and of course, I got something like this:

#rsync -avr --rsh="ssh -c arcfour" --delete -zz  sourcemachine:/source/  /target/


#My target machine is CentOS / rsync (3.0.6)
rsync: on remote machine: --new-compress: unknown option
rsync error: syntax or usage error (code 1) at main.c(1422) [server=3.0.6]
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [Receiver=3.1.1]

The reason why we have all of these messy things because there is a bug in rsync 3.1.1. (rsync –version) To solve this problem, you will need to rebuild the rsync by skipping the ZLIB_BASE option:

cd /usr/ports/net/rsync
sudo make config

#Uncheck the ZLIB_BASE option

sudo make reinstall clean

Now the rsync should be happy.

–Derrick

Our sponsors:

First Edition: March 14, 2010
Last Revised: April 26, 2018

In this tutorial, I will show you how to improve the performance of your ZFS using the affordable consumer-grade hardware (e.g., Gigabit network card, standard SATA non-SSD hard drives, consumer-grade motherboard etc).

Many people found a problem on their ZFS system. The speed is slow! It is slow to read or write files to the system. In this article, I am going to show you some tips on improving the speed of your ZFS file system.

Notice that this article was originally based on ZFS on FreeBSD. Although most concepts can be applied to Linux, you may want to check out these two articles: ZFS: Linux VS FreeBSD and How to fix the mess created by ZFS on Linux after update or reboot.

This article is about how to build a single node ZFS server. If you are interested in implementing multiple-nodes ZFS system / ZFS clusters, please check here for details.

Table of Content
  1. A Good 64-bit CPU + Lots of Memory
  2. Tweaking the Boot Loader
  3. Use Disks with the Same Specifications
  4. Use a Powerful Power Supply
  5. Enable Compression
  6. Identify the Bottleneck
  7. Keep Your ZFS up to Date
  8. Understand How The ZFS Caching Works
  9. Two Drives Is Better than One Single Drive
  10. Use a Combination of Strip and RAIDZ If Speed Is Your First Concern.
  11. Distribute Your Free Space Evenly
  12. Make Your Zpool Expandable
  13. Backup the Data on a Different Machine, Not on the Same Zpool
  14. Rsync or ZFS Send?
  15. Did You Enable Dedup? Disable It!
  16. Reinstall Your Old System
  17. Connect Your Disks via High Speed Interface
  18. Do Not Use up All Spaces
  19. Use AHCI, Not IDE
  20. Refresh Your Zpool
  21. Great Performance Settings
  22. My Settings

Improve ZFS Performance: Step 1

A Good 64-bit CPU + Lots of Memory

Traditionally, we are told to use a less powerful computer for a file/data server. That’s not true for ZFS. ZFS is more than a file system. It uses a lot of resources to improve the performance of the input/output, such as compressing data on the fly. For example, suppose you need to write a 1GB file. Without enabling the compression, the system will write the entire 1GB file on the disk. With the compression being enabled, the CPU will compress the data first, and write the data on the disk after that. Since the compressed data is smaller, it takes shorter time to write to the disk, which results a higher writing speed. The same thing can be applied for reading. ZFS can cache the file for you in the memory, it will result a higher reading speed.

That’s why a 64-bit CPU and higher amount of memory is recommended. I recommended at least a Quad Core CPU with 4GB of memory (Personally I use Xeon and i7, with at least 20GB of memory).

Please make sure that the memory modules should have the same frequencies/speed. If you mix them with different speed, try to group the memories with same speed together., e.g., Channel 1 and Channel 2: 1333 MHz, Channel 3 and Channel 4: 1600 MHz.

Let’s do a test. Suppose I am going to create a 10GB file with all zero. Let’s see how long does it take to write on the disk:

#CPU: i7 920 (A 4 cores/8 threads CPU from 2009) + 24GB Memory + FreeBSD 9.3 64-bit
#dd if=/dev/zero of=./file.out bs=1M count=10k
10737418240 bytes transferred in 6.138918 secs (1749073364 bytes/sec)

That’s 1.6GB/s! Why is it so fast? That’s because it is a zero based file. After the compression, a compressed 10GB file may result in few bytes only. Since the performance of the compression is highly depended on the CPU, that’s why a fast CPU matters.

Now, let’s do the same thing on a not-so-fast CPU:

#CPU: AMD 4600 (2 cores) + 5GB Memory + FreeBSD 9.3 64-bit
#dd if=/dev/zero of=./file.out bs=1M count=10k
10737418240 bytes transferred in 23.672373 secs (453584362 bytes/sec)

That’s 434MB/s only. See the difference?

Improve ZFS Performance: Step 2

Tweaking the Boot Loader Parameters

Update: This section was written based on FreeBSD 8. As of today (February 12, 2017), the latest version is FreeBSD 11. I noticed that my FreeBSD/ZFS works very stable even without any tweaking! In the other words, you may skip this section if you are not using FreeBSD 8.2 or older.

Many people complain about ZFS for its stability issues, such as kernel panic, reboot randomly, crash when copying large files (> 2GB) at full speed etc. It may have something to do with the boot loader settings. By default, ZFS will not work smoothly without tweaking the system parameters system. Even FreeBSD (9.1 or earlier) claims that no tweaking is necessary for 64-bit system, my FreeBSD server crashes very often when writing large files to the pool. After trial and error for many times, I figure out few equations. You can tweak your boot loader (/boot/loader.conf) using the following parameters. Notice that I only tested the following on FreeBSD. Please let me know whether the following tweaks work on other operating systems.

If you experiences kernel panic, crash or something similar, it could be the hardware problem, such as memory. I encourage to test all memory modules by using Memtest86+ first. I wish someone told me about this few years ago. That would make my life a lot easier.

Warning: Make sure that you save a copy before doing anything to the boot loader. Also, if you experience anything unusual, please remove your changes and go back to the original settings.

#Assuming 8GB of memory

#If Ram = 4GB, set the value to 512M
#If Ram = 8GB, set the value to 1024M
vfs.zfs.arc_min="1024M"

#Ram x 0.5 - 512 MB
vfs.zfs.arc_max="3584M"

#Ram x 2
vm.kmem_size_max="16G"

#Ram x 1.5
vm.kmem_size="12G"

#The following were copied from FreeBSD ZFS Tunning Guide
#https://wiki.freebsd.org/ZFSTuningGuide

# Disable ZFS prefetching
# http://southbrain.com/south/2008/04/the-nightmare-comes-slowly-zfs.html
# Increases overall speed of ZFS, but when disk flushing/writes occur,
# system is less responsive (due to extreme disk I/O).
# NOTE: Systems with 4 GB of RAM or more have prefetch enabled by default.
vfs.zfs.prefetch_disable="1"

# Decrease ZFS txg timeout value from 30 (default) to 5 seconds.  This
# should increase throughput and decrease the "bursty" stalls that
# happen during immense I/O with ZFS.
# http://lists.freebsd.org/pipermail/freebsd-fs/2009-December/007343.html
# http://lists.freebsd.org/pipermail/freebsd-fs/2009-December/007355.html
# default in FreeBSD since ZFS v28
vfs.zfs.txg.timeout="5"

# Increase number of vnodes; we've seen vfs.numvnodes reach 115,000
# at times.  Default max is a little over 200,000.  Playing it safe...
# If numvnodes reaches maxvnode performance substantially decreases.
kern.maxvnodes=250000

# Set TXG write limit to a lower threshold.  This helps "level out"
# the throughput rate (see "zpool iostat").  A value of 256MB works well
# for systems with 4 GB of RAM, while 1 GB works well for us w/ 8 GB on
# disks which have 64 MB cache.

# NOTE: in v27 or below , this tunable is called 'vfs.zfs.txg.write_limit_override'.
vfs.zfs.write_limit_override=1073741824

Don’t forget to reboot your system after making any changes. After changing to the new settings, the writing speed improves from 60MB/s to 80MB/s, sometimes it even goes above 110MB/s! That’s a 33% improvement!

By the way, if you found that the system still crashes often, the problem could be an uncleaned file system.

After a system crashes, it may cause the file links to be broken (e.g., the system see the file tag, but unable to locate the files). Usually FreeBSD will automatically run fsck after the crash. However it will not fix the problem for you. In fact, there is no way to clean up the file system when the system is running (because the partition is mounted). The only way to clean up the file system is by entering the Single User Mode (a reboot is required).

After you enter the single user mode, make sure that each partition is cleaned. For example, here is my df result:

Filesystem    Size    Used   Avail Capacity  Mounted on
/dev/ad8s1a   989M    418M    491M    46%    /          
devfs         1.0k    1.0k      0B   100%    /dev       
/dev/ad8s1e   989M     23M    887M     3%    /tmp       
/dev/ad8s1f   159G     11G    134G     8%    /usr       
/dev/ad8s1d    15G    1.9G     12G    13%    /var        

Try running the following commands:

#-y: All yes
#-f: Force
fsck -y -f /dev/ad8s1a
fsck -y -f /dev/ad8s1d
fsck -y -f /dev/ad8s1e
fsck -y -f /dev/ad8s1f

These command will clean up the affected file systems.

After the clean up is done, type reboot and let the system to boot to the normal mode.

Improve ZFS Performance: Step 3

Use disks with the same specifications

A lot of people may not realize the importance of using exact the same hardware. Mixing different disks of different models/manufacturers can bring performance penalty. For example, if you are mixing a slow disk (e.g., green disk) and a fast disk(e.g., performance disk) in the same virtual device (vdev), the overall speed will depend on the slowest disk. Also, different hard drives may have different sector size. For example, Western Digital releases a harddrive with 4k sector, while the older models use 512 byte. Mixing hard drives with different sectors can bring performance penalty too. Here is a quick way to check the model of your harddrive:

sudo smartctl -a /dev/sda | grep 'Sector Size'

Here is an example output:

Sector Sizes:     512 bytes logical, 4096 bytes physical

If you don’t have enough budget to replace all disks with the same specifications, try to group the disks with similar specifications in the same vdev.

Here is an example: Support I have a group of hard drives with 4k sector (i.e., 4k = 4096 bytes = 2^12 bytes), which translate to ashift=12:

sudo zpool create myzpool -o ashift=12 raidz1 /dev/disk1 /dev/disk2 ...

You can also verify the ashift value using this command:

sudo zdb | grep shift

Let’s say we have a group of hard drives with 4k sector, we create two ZFS, one with ashift=12, and another one with the default value (ashift=9), here is the difference:

#Using default block size:
dd if=/myzpool/data/file.out of=/dev/null
40960000000 bytes (41 GB) copied, 163.046 s, 251 MB/s

#Using native block size:
dd if=/myzpool/data/file.out of=/dev/null
40960000000 bytes (41 GB) copied, 58.111 s, 705 MB/s

Improve ZFS Performance: Step 4

Use a Powerful Power Supply

I recently built a Linux-based ZFS file system with 12 hard disks. For some reasons, it was pretty unstable. When I tried filling the pool with 12TB of data, the ZFS system crashed randomly. When I rebooted the machine, the error was gone. However, when I resumed the copy process again, the error happened on a different disk. In short, the disks failed randomly. When I checked the dmesg, I found something like the following. Since I didn’t have the exact copy, I grabbed something similar from the web:

[  412.575724] ata10.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6
[  412.576452] ata10.00: BMDMA stat 0x64
[  412.577201] ata10.00: failed command: WRITE DMA EXT
[  412.577897] ata10.00: cmd 35/00:08:97:19:e4/00:00:18:00:00/e0 tag 0 dma 4096 out
[  412.577901]          res 51/84:01:9e:19:e4/84:00:18:00:00/e0 Emask 0x10 (ATA bus error)
[  412.579294] ata10.00: status: { DRDY ERR }
[  412.579996] ata10.00: error: { ICRC ABRT }
[  412.580724] ata10: soft resetting link
[  412.844876] ata10.00: configured for UDMA/133
[  412.844899] ata10: EH complete
[  980.304788] ata10.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6
[  980.305311] ata10.00: BMDMA stat 0x64
[  980.305817] ata10.00: failed command: WRITE DMA EXT
[  980.306351] ata10.00: cmd 35/00:08:c7:00:ce/00:00:18:00:00/e0 tag 0 dma 4096 out
[  980.306354]          res 51/84:01:ce:00:ce/84:00:18:00:00/e0 Emask 0x10 (ATA bus error)
[  980.307425] ata10.00: status: { DRDY ERR }
[  980.307948] ata10.00: error: { ICRC ABRT }
[  980.308529] ata10: soft resetting link
[  980.572523] ata10.00: configured for UDMA/133

Basically, this message means the disk was failed during writing the data. Initially I thought it could be the SMART/bad sectors. However since I observed the problem happend randomly on random disks, I think the problem could be something else. I have tried to replacing the SATA cable, power cable etc. None of them worked. Finally I upgraded my power supply (450W to 600W), and the error was gone.

FYI, here is the specs of my affected system. Notice that I didn’t use any component that required high-power such as graphic card etc.

  • CPU: Intel Q6600
  • Standard motherboard
  • PCI RAID Controller Card with 4 SATA ports
  • WD Green Drive x 12
  • CPU fan x 1
  • 12″ case fan x 3

And yes, you will need a 600W power supply for such a simple system. Also, another thing worth to check is the power cable. Sometimes, using 15 pin power cable (the one for SATA drive) is better than 4pin to 15 pin converter (IDE to SATA converter).

Improve ZFS Performance: Step 5

Compression

ZFS supports compressing the data on the fly. This is a nice feature that improves the I/O speed – only if you have a high speed CPU (such as Quad core or higher). If your CPU is not fast enough, I don’t recommend you to turn on the compression feature, because the benefit from reducing the file size is smaller than the time spent on the CPU computation. Also, the compression algorithm plays an important role here. ZFS supports two compression algorithms, LZJB and GZIP. I personally use lz4 (See LZ4 vs LZJB for more information) because it gives a better balance between the compression radio and the performance. You can also use GZIP and specify your own compression ratio (i.e., GZIP-N). FYI, I tried GZIP-9 (The maximum compression ratio available) and I found that the overall performance gets worse even on my i7 with 12GB of memory.

There is no solid answer here because it all depends on what kind of files you store. Different files such as large file, small files, already compressed files (such as Xvid movie) need different compression settings.

If you cannot decide, just go with lz4. It can’t be wrong:


#Try to use lz4 first.
sudo zfs set compression=lz4 mypool

#If you system does not support lz4, try to use lzjb
sudo zfs set compression=lzjb mypool

Improve ZFS Performance: Step 6

Identify the bottle neck

Sometimes, the limit of the ZFS I/O speed is closely related to the hardware. For example, I set up a network file system (NFS) that is based on ZFS. Most of the time, I mainly transfer the data in between the servers, rather than within the same server. Therefore, the maximum I/O speed I could get is the capacity of my network card, which is 125MB/s. On average, I can reach to 100-110 MB/s, which is pretty good for consumer grade network adapter.

One day, I decide to explore the options on network bonding, which combines multiple network adapters together on the same server. In theory, it multiplies the bandwidth, and it will increase the ZFS I/O speed.

I set up network bonding on a machine with two network cards. It was a CentOS 7 box with bonding mode: 6 (adaptive load balancing). I was able to double the I/O speed in both network and ZFS.

sudo cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: adaptive load balancing
Primary Slave: None
Currently Active Slave: em2
MII Status: up
MII Polling Interval (ms): 1
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: em1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 20:27:47:91:a3:a8
Slave queue ID: 0

Slave Interface: em2
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 20:27:47:91:a3:aa
Slave queue ID: 0

Improve ZFS Performance: Step 7

Keep your ZFS up to date

By default, ZFS will not update the file system itself even if a newer version is available on the system. For example, I created a ZFS file system on FreeBSD 8.1 with ZFS version 14. After upgrading to FreeBSD 8.2 (which supports ZFS version 15), my ZFS file system was still on version 14. I needed to upgrade it manually using the following commands:

sudo zfs upgrade my_pool
sudo zpool upgrade my_pool

Improve ZFS Performance: Step 8

Understand How The ZFS Caching Works

ZFS has three types of cache, ARC and L2ARC. ARC is a ram-based cache, and L2ARC is disk-based cache. If you want to have a super-fast ZFS system, you will need A LOT OF memory. How much? I have a RHEL 7 based data center running NFS on top of a ZFS file system. Its the storage capacity is 48TB (8 x 8TB, running RAIDZ1), and I have 96GB of memory. The I/O is about 1.5TB a day. Is the memory too much? I can tell you that sometimes it causes kernel problem because of running out of memory.

In general, you will need lots of memory for ARC cache (ram-based), and L2ARC (disk-based) is optional. Depending on which operating system you are using (in my case, I use RHEL 7 and FreeBSD), the settings can be very different. Let’s talk about the arc cache first.

By default, when you read a file from ZFS for the first time, ZFS will read the file from the disk. However if it is frequently used, ZFS will put the file in the ARC cache, which is your memory. Now we have something interesting. How much memory should you allow ZFS to use for ARC caching? You don’t want to use too little because it will increase accessing the disk, which will slow down the system. On the other hand, you don’t want to use too much because you want to reserve the memory for your operating system and other services. For FreeBSD, depend on which version you are using, it will either use 90% of the memory for ARC, or use all but 1GB of memory. For ZFS on Linux with RHEL 7, it will use about 50% of the memory.

You can monitor the ARC usuage here:

#FreeBSD
zfs-stats -A

------------------------------------------------------------------------
ZFS Subsystem Report                            Sun Feb 12 22:34:19 2017
------------------------------------------------------------------------

ARC Summary: (HEALTHY)
        Memory Throttle Count:                  0

ARC Misc:
        Deleted:                                247.52m
        Recycle Misses:                         0
        Mutex Misses:                           68.86k
        Evict Skips:                            778.88k

ARC Size:                               75.04%  16.76   GiB
        Target Size: (Adaptive)         74.94%  16.74   GiB
        Min Size (Hard Limit):          12.50%  2.79    GiB
        Max Size (High Water):          8:1     22.33   GiB

ARC Size Breakdown:
        Recently Used Cache Size:       73.41%  12.30   GiB
        Frequently Used Cache Size:     26.59%  4.46    GiB

ARC Hash Breakdown:
        Elements Max:                           1.40m
        Elements Current:               68.50%  956.10k
        Collisions:                             50.63m
        Chain Max:                              7
        Chains:                                 94.61k

------------------------------------------------------------------------



#ZFS on Linux
arcstat.py
    time  read  miss  miss%  dmis  dm%  pmis  pm%  mmis  mm%  arcsz     c
22:33:36     0     0      0     0    0     0    0     0    0    53G   53G


If you are using FreeBSD (the most advanced operating system in the world), you don’t need to tweak your system settings at all. ZFS is part of the FreeBSD kernel, and FreeBSD has excellent memory management. Therefore, you do not need to worry about ZFS using too much memory of your system. In fact, I’ve never crashed a FreeBSD system because of using too much memory. If you know how to do it, please show me how to do it.

If you want to tweak the ARC size, you can do it via /boot/loader.conf:

#Min: 10GB
vfs.zfs.arc_min="10000M"

#Max: 16GB
vfs.zfs.arc_max="16000M"

If you are using Linux, you may want to do some extra work to make your system stable. The default settings will get you to bottom of the mountain, and you will need to do some climbing to reach the peak.

First, you want to check the arc size:

arcstat.py

Now you will need to think about how much memory you want to reserve for ZFS. This is a tough question, and you can’t really expect Linux works like FreeBSD. Remember, Linux does not have a good memory management system like FreeBSD. It is very easy to crash the system if you do something wrong. That is one of the reasons why ZFS on Linux uses only 50% of the memory, because they know Linux, a car that may lose control if it is running slightly above the highway speed limit.

sudo nano /etc/modprobe.d/zfs.conf

#Min: 4GB
options zfs zfs_arc_min=4000000000

#Max: 8GB
options zfs zfs_arc_max=8000000000

Don’t forget to reboot the server.

FYI, tweaking the memory usage of ZFS ARC is more than entering some numbers. You need to think of the whole picture. Memory is a very valuable resource. You will need to decide how do use them effectively and efficiently. You can’t expect a single server running web server, database, NFS, virtual machine host all together on top of a ZFS system, and giving you a very high performance at the mean time. You need to divide the load across multiple servers. For example, here is what I use at work for production:

Server 1: A NFS host
– Running NFS only
– Have network bonding based on multiple network interfaces. This will multiply the bandwidth.
– Support other servers such as web server, databases, virtual machine host etc.

Server 2: A virtual machine host
– Running Virtual Box
– Virtual machine which has higher I/O requirement are hosted in a local SSD drive with standard partitions (non-ZFS).
– Virtual machine which has lower I/O requiremenet are hosted on a different ZFS server, connected via NFS.
– All of the system memory will be used for running virtual machines.

Server 3: A web server
– Running Apache + PHP + MySQL
– The PHP code is hosted locally.
– The large static files is hosted on the NFS server. In my case, I have over 20TB static content.
– All of the system memory will be used for running Apache / PHP / MySQL executions.

Of course, I am not talking about ZFS does not work well with Apache / PHP / MySQL. I am just saying that in some extreme environments, the services should be hosted across multiple machines instead of one single machine.

Now if you have extra resources (budget and extra SATA port), you may consider using L2ARC cache. Basically L2ARC cache is a very fast hard drive (i.e., SSD). Think of it as a hybrid hard drive, it is a buffer for reading and writing.

So what is the role of L2ARC to ARC? In general, the most frequently used files are stored in the ARC cache (ram). For the less frequently used files, or if the files are too large to fit in the arc cache, they will be stored in the L2ARC cache.

To improve the reading performance:

sudo zpool add myzpool cache 'ssd device name'

To improve the writing performance:

sudo zpool add myzpool log /dev/ssd_drive

It was impossible to remove the log devices without losing the data until ZFS v.19 (FreeBSD 8.3+/9.0+). I highly recommend to add the log drives as a mirror, i.e.,

sudo zpool add myzpool log mirror /dev/log_drive1 /dev/log_drive2

Now you may ask a question. How about using a ram disk as log / cache devices? First, ZFS already uses your system memory for I/O, so you don’t need to set up a dedicated ram disk by yourself. Also, using a ram disk for log (writing) devices is not a good idea. When somethings go wrong, such as power failure, you will end up losing your data during the writing.

Improve ZFS Performance: Step 9

Two drives is better than one single drive

Do you know ZFS works faster on multiple devices pool than single device pool, even they have the same storage size?

Improve ZFS Performance: Step 10

Use a combination of Striped and RAIDZ if speed is your first concern.

Striped design also gives the best performance. Since it offers no data protection at all, you may want to use RAIDZ (RAIDZ1, RAIDZ2, RAIDZ3) or mirror to handle the data protection. However, there are too many choices and each of them offer different degree of performance and protection level. If you want a quick answer, try to use a combination of striped and RAIDZ. I posted a very detail of comparison among Mirror, RAIDZ, RAIDZ2, RAIDZ3 and Striped here.

Here is an example of striped with RAIDZ:

#Command
zpool create -f myzpool raidz hd1 hd2 hd3 hd4 hd5 \
                        raidz hd6 hd7 hd8 hd9 hd10

#zpool status -v
        NAME        STATE     READ WRITE CKSUM
        storage     ONLINE       0     0     0
          raidz1-0  ONLINE       0     0     0
            hd1     ONLINE       0     0     0
            hd2     ONLINE       0     0     0
            hd3     ONLINE       0     0     0
            hd4     ONLINE       0     0     0
            hd5     ONLINE       0     0     0
          raidz1-1  ONLINE       0     0     0
            hd6     ONLINE       0     0     0
            hd7     ONLINE       0     0     0
            hd8     ONLINE       0     0     0
            hd9     ONLINE       0     0     0
            hd10    ONLINE       0     0     0

Improve ZFS Performance: Step 11

Distribute your free space evenly

One of the important tricks to improve ZFS performance is to keep the free space evenly distributed across all devices.

You can check it using the following command:

zpool iostat -v

The free space is show on the second column (available capacity)

               capacity     operations    bandwidth
pool         used  avail   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
storage     3.23T  1.41T      0      3  49.1K   439K
  ad4        647G   281G      0      0  5.79K  49.2K
  ad8        647G   281G      0      0  5.79K  49.6K
  ad10       647G   281G      0      0  5.82K  49.6K
  ad16       647G   281G      0      0  5.82K  49.6K
  ad18       647G   281G      0      0  5.77K  49.5K

When ZFS writes a new file to replace the old file in the system, it will first write the file in the free space first, then move the file pointer from the old one to the new one. In this case, even there is a power failure during writing the data, no data will be lost because the file pointer is still pointing to the old file. That’s why ZFS does not need fsck (file system check).

In order to keep the performance at a good level, we need to make sure that the free space is available in every device in the pool. Otherwise ZFS can only write the data to some of the devices only (instead of all). In the other words, the higher number of devices ZFS write, the better the performance.

Technically, if the structure of a zpool has not been modified or alternated, you should not need to worry about the free space distribution because ZFS will take care of that for you automatically. However, when you add a new device to an existing zpool, that will be a different story, e.g.,

               capacity     operations    bandwidth
pool         used  avail   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
storage     3.88T  2.33T      0      3  49.1K   439K
  ad4        647G   281G      0      0  5.79K  49.2K
  ad8        647G   281G      0      0  5.79K  49.6K
  ad10       647G   281G      0      0  5.82K  49.6K
  ad16       647G   281G      0      0  5.82K  49.6K
  ad18       647G   281G      0      0  5.77K  49.5K
  ad20          0   928G      0      0  5.77K  49.5K

In this example, I add a 1TB hard drive (ad20) to my existing pool, which gives about 928GB of free space. Let say I add a 6GB file, the free space will look something like this:

               capacity     operations    bandwidth
pool         used  avail   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
storage     4.48T  1.73T      0      3  49.1K   439K
  ad4        648G   280G      0      0  5.79K  49.2K
  ad8        648G   280G      0      0  5.79K  49.6K
  ad10       648G   280G      0      0  5.82K  49.6K
  ad16       648G   280G      0      0  5.82K  49.6K
  ad18       648G   280G      0      0  5.77K  49.5K
  ad20         1G   927G      0      0  5.77K  49.5K

In the other words, ZFS will still divide my 6GB file into six equal pieces and write each piece to each device. Eventually, ZFS will use up the free space in the older devices, and it can write the data to the new devices only (ad20), which will decrease the performance. Unfortunately, there is no way to redistribute the data / free space evenly without destroying the pool, i.e.,

1. Back up your data
2. Destroy the pool
3. Rebuild the pool
4. Put your data back

Depending on how much data do you have, it can take 2 to 3 days to copy 10TB of data from one server to another server over a gigabit network. You don’t want to use scp to do it because you will need to re-do everything again if the process is dropped. In my case, I use rsync:

(One single line)

#Run this command on the production server:
rsync -avzr --delete-before backup_server:/path_to_zpool_in_backup_server /path_to_zpool_in_production_server

Of course, netcat is a faster way if you don’t care about the security. (scp / rsync will encrypt the data during transfer).

See here for further information

Improve ZFS Performance: Step 12

Make your pool expandable

Setting up a ZFS system is more than a one-time job. Unless you take a very good care of your storage like how supermodels monitor their body weights, otherwise you will end up using all of the available space one day. Therefore it is a good idea to come up a good design that it can grow in the future.

Suppose we want to build a server with maximum storage capacity, how will we start? Typically we try to put as many hard drives on a single machine as possible, i.e., it will be be around 12 to 14 hard drives, which is what a typical consumer grade full tower computer case can hold. Let’s say we have 12 disks, here are couple setups which maximize storage capacity with a decent level of data safety:

In this design, we create a giant pool and let the ZFS to take care of the rest. This pool will offer n-2 storage capacity which will allow up to 2 hard drives fail without losing any data.

In the second design, it offers the same level of storage capacity and a similar level of data protection. It allows up to one failure disk in each vdev. Keep in mind that the first design offers a great data protection. However, the second design will offer a better performance and greater flexibility in terms of future upgrade. Check out this article if you want to learn more about the difference in ZFS design.

First, let’s talk about the good and bad of the first design. It offers a great data security because it allows ANY two disks in the zpool to fail. However, it has couple disadvantages. ZFS works great when the number of disk of vdev is small. Ideally, the number should be smaller than 8 (Personally, I will stick with 5). In the first design, we put 12 disks in one single vdev, which will be problematic when the storage is getting full (>90%). Also, when we talks about upgrading the entire zpool, we will need to upgrade each disk one by one first. We won’t be able to use the extra space until we replace all 12 disks. This may be an issue for those who do not have budget to get 12 new disks at a time.

For the second design, it does not have the problem mentioned in the first design. The number of disk in each vdev is small (6 disks in each vdev). For those who don’t have plenty of budgets, it is okay to get six disks at a time to expand the pool.

Here is how to create the second design:

sudo zpool create myzpoolname raidz /dev/ada1 /dev/ada2 ... /dev/ada6 raidz /dev/ada7 /dev/ada8 /dev/ada9 ... /dev/ada12

Here is how to expand the pool by replacing the hard drive one by one without losing any data:

1. Shutdown the computer, replace the hard drive and turn on the computer.

2. Tell ZFS to replace the hard drive. This will force it to fill in the new hard drive with the existing data based on the check sum.
zpool replace mypool /dev/ada1

3. Resilver the pool
zpool scrub mypool 

4 Shutdown the server and replace the second ard drive again. Repeat the steps until everything is done.

5. zpool set autoexpand=on mypool 

6. Resilve the pool if needed.
zpool scrub mypool 

Improve ZFS Performance: Step 13

Backup your data on a different machine, not on the same pool

ZFS comes with a very cool feature. It allows you to save multiple copies of the same data in the same pool. This adds an additional layer on data security. However, I don’t recommend using this feature for backup purpose because it adds more work when writing the data to the disks. Also, I don’t think this is a good way to secure the data. I prefer to set up a mirror on a different server (Master-Slave). Since the chance of two machines fail at the same time is much smaller than one machine fails. Therefore the data is safer in this settings.

Here is how I synchronize two machines together:

(Check out this guide on how to use rsyncd)

Create a script in the slave machine: getContentFromMaster.sh
(One single line)

rsync -avzr -e ssh --delete-before master:/path/to/zpool/in/master/machine /path/to/zpool/in/slave/machine

And put this file in a cronjob, i.e.,
/etc/crontab

@daily root /path/to/getContentFromMaster.sh

Now, you may ask a question. Should I go with strip-only ZFS (i.e., stripping only. No mirror, RAIDZ, RAIDZ2) when I set up my pool? Yes or no. ZFS allows you to mix any size of har ddrive in one single pool. Unlike RAID[0,1,5,10] and concatenation, it can be any size and there is no lost in the disk space, i.e., you can connect 1TB, 2TB, 3TB into one single pool while enjoying the data-stripping (Total usable space = 6TB). It is fast (because there is no overhead such as parity etc) and simple. The only down side is that the entire pool will stop working if at least one device fails.

Let’s come back to the question, should we employ simple stripping in production environment? I prefer not. Strip-only ZFS divides all data into all vdev. If each vdev is simply a hard drive, and if one fails, there is NO WAY to get the original data back. If something screws up in the master machine, the only way is to destroy and rebuild the pool, and restore the data from the backup. (This process can takes hours to days if you have large amount of data, say 6TB.) Therefore, I strongly recommend to use at least RAIDZ in the production environment. If one device fails, the pool will keep working and no data is lost. Simply replace the bad hard drive with a good one and everything is good to go.

To minimize the downtime when something goes wrong, go with at least RAIDZ in a production environment (ideally, RAIDZ or strip-mirror).

For the backup machine, I think using simple stripping is completely fine.

Here is how to build a pool with simple stripping, i.e., no parity, mirror or anything

zpool create mypool /dev/dev1 /dev/dev2 /dev/dev3

And here is how to monitor the health

zpool status

Some websites suggest to use the following command instead:

zpool status -x

Don’t believe it! This command will return “all pools are healthy” even if one device is failed in a RAIDZ pool. In the other words, your data is healthy doesn’t mean all devices in your pool are healthy. So go with “zpool status” at any time.

FYI, it can easily takes few days to copy 10TB of data from one machine to another through a gigabit network. In case you need to restore large amount of data through the network, use rsync, not scp. I found that scp sometimes fail in the middle of transfer. Using rsync allows me to resume it at any time.

Improve ZFS Performance: Step 14

rsync or ZFS send?

So what’s the main difference between rsync and ZFS send? What’s the advantage of one over the other?

Rsync is a file level synchronization tool. It simply goes through the source, find out which files have been changed, and copy the corresponding files to the destination. Also rsync is portable/cross-platform. Unlike ZFS, rsync is available in most Unix platforms. If your backup platform does not support ZFS, you may want to go with rsync.

ZFS send is doing something similar. First, it takes a snapshot on the ZFS pool first:

zfs snapshot mypool/vdev@20120417

After that, you can generate a file that contains the pool and data information, copy to the new server to restore it:

#Method 1: Generate a file first
zfs send mypool/vdev@20120417 > myZFSfile
scp myZFSfile backupServer:~/
zfs receive mypool/vdev@20120417 < ~/myZFSfile

Or you can do everything in one single command line:

#Method 2: Do everything over the pipe (One command)
zfs send pool/vdev@20120417 | ssh backupServer zfs receive pool/vdev@20120417

In general, the preparation time of ZFS send is much shorter than rsync, because ZFS already knows which files have been modified. Unlike rsync, a file-level tool, ZFS send does not need to go though the entire pool and find out such information. In terms of the transfer speed, both of them are similar.

So why do I prefer rsync over ZFS send (both methods)? It's because the latter one is not practical! In method #1, the obvious issue is the storage space. Since it requires generating a file that contains your entire pool information. For example, suppose your pool is 10TB, and you have 8TB of data (i.e., 2TB of free space), if you go with method #1, you will need another 8TB of free space to store the file. In the other words, you will need to make sure that at least 50% of free space is available all the time. This is a quite expensive way to run ZFS.

What about method #2? Yes, it does not have the storage problem because it copies everything over the pipe line. However, what if the process is interrupted? It is a common thing due to high traffic in the network, high I/O to the disk etc. Worst worst case, you will need to re-do everything again, say, copying 8TB over the network, again.

rsync does not have these two problems. In rsync, it uses relatively small space for temporary storage, and in case the rsync process is interrupted, you can easily resume the process without copying everything again.

Improve ZFS Performance: Step 15

Disable dedup if you don't have enough memory (5GB memory per 1TB storage)

Deduplication (dedup) is a space-saving technology. It works at the block level (a file can have many blocks). To explain it in simple English, if you have multiple copies of the same file in different places, it will store only one copy instead of multiple copies. Notice that dedup is not the same as compression. Check out this article: ZFS: Compression VS Deduplication(Dedup) in Simple English if you wan to learn more.

The idea of dedup is very simple. ZFS maintains an index of your files. Before writing any incoming files to the pool, it checks whether the storage has a copy of this file or not. If the file already exists, it will skip the file. With dedup enabled, instead of store 10 identical files, it stores one only copy. Unfortunately, the drawback is that it needs to check every incoming file before making any decision.

After upgrading my ZFS pool to version 28, I enabled dedup for testing. I found that it really caused huge performance hit. The writing speed over the network dropped from 80MB/s to 5MB/s!!! After disabling this feature, the speed goes up again.

sudo zfs set dedup=off your-zpool

In general, dedup is an expensive feature that requires a lot of hardware resources. You will need 5GB memory per 1TB of storage (Source). For example, if zpool is 10TB, I will need 50GB of memory! (Which I only have 12GB). Therefore, think twice before enabling dedup!

Notice that it won't solve all the performance problem by disabling the dedup. For example, if you enable dedup before and disable it afterward, all files stored during this period are dedup dependent, even dedup is disabled. When you need to update these files (e.g., delete), the system still needs to check again the dedup index before any processing your file. Therefore, the performance issue still exists when working with these affected files. For the new files, it should be okay. Unfortunately, there is no way to find out the affected dedup files. The only way is to destroy and re-build the ZFS pool, which will clear the list of dedup files.

Improve ZFS Performance: Step 16

Reinstall Your Old System

Sometimes, reinstalling your old system from scratch may help to improve the performance. Recently, I decided to reinstall my FreeBSD box. It was an old FreeBSD box that was started with FreeBSD 6 (released in 2005, about 8 years ago from today). Although I upgraded the system every release, it already accumulated many junk and unused files. So I decide to reinstall the system from scratch. After the installation, I can tell that the system is more responsive and stable.

Before you wipe out the system, you can export the ZFS tank using the following command:

sudo zpool export mypool

After the work is done, you can import the data back:

sudo zpool import mypool

Improve ZFS Performance: Step 17

Connect your disks via high speed interface

Recently, I found that my overall ZFS system is slow no matter what I have done. After some investigations, I noticed that the bottle neck was my RAID card. Here are my suggestions:

1. Connect your disks to the ports with highest speed. For example, my PCI-e RAID card deliveries higher speed than my PCI RAID card. One way to verify the speed is by using dmesg, e.g.,

dmesg | grep MB

#Connected via PCI card. Speed is 1.5Gb/s
ad4: 953869MB  at ata2-master UDMA100 SATA 1.5Gb/s

#Connected via PCI-e card. Speed is 3.0 Gb/s
ad12: 953869MB  at ata6-master UDMA100 SATA 3Gb/s

In this case, the overall speed limit is based on the slowest one (1.5Gb/s), even the rest of my disks are 3Gb/s.

2. Some RAID cards come with some advanced features such as RAID, linear RAID, compression etc. Make sure that you disable these features first. You want to minimize the workload of the card and maximize the I/O speed. It will only slow down the overall process if you enable these additional features. You can disable the settings in the BIOS of the card. FYI, most of the RAID cards in $100 ranges are "software RAID", i.e., they are using the system CPU to do the work. Personally, I think these fancy features are designed for Windows users. You really don't need any of these features in Unix world.

3. Personally, I recommend any brand except Highpoint Rocketraid because of the driver issues. Some of the Highpoint Rocketraid products are not supported by FreeBSD natively. You will need to download the driver from their website first. Their driver is version-specified, e.g., they have two different set of drivers for FreeBSD 7 and 8, and both of them are not compatible with each other. One day if they decide to stop supporting the device, then you either need to stick with the old FreeBSD, or buy a new card. My conclusion: Stay away from Highpoint Rocketraid.

Improve ZFS Performance: Step 18

Do not use up all spaces

Depending on the settings / history of your zpool, you may want to maintain the free space at a certain level to avoid speed-drop issues.

Recently, I found that my ZFS system is very slow in terms of reading and writing. The speed dropped from 60MB/s to 5MB/s over the network. After some investigations, I found that the available space was around 300GB (out of 10TB), which is 3% left. Someone suggest that the safe threshold is about 10%, i.e., the performance won't be impacted if you have at least 10% of the free space. I would say 5% is the bottom line, because I haven't noticed any performance issues until it hits 3%.

After I free up some spaces, the speed comes back again.

I think it doesn't make any sense not to use all of my space. So I decide to find out what caused this problem. The answer is the zpool structure.

In my old setup, I put set up a single RAIDZ vdev with 8 disks. This gives me basic data security (up to one disk fails), and maximum disk spaces (Usable space is 7 disks). However, I notice that the speed drops a lot when the available free space was 5%.

In my experiment setup, I decide to do the same thing with RAIZ2, i.e., it allows up to two disks fail, and the usable space is down to 6 disks. After filling up the pool, I found that it does not have the speed-drop problem. The I/O speed is still fast even the free space is 10GB (That's 0.09%).

My conclusion: RAIDZ is okay up to 6 devices. If you want to add more devices, either use RAIDZ2 or split them into multiple vdevs:

#Suppose I have 8 disks (/dev/hd1 ... /dev/hd8).

#One vdev
zpool create myzpool raidz2 /dev/hd1 /dev/hd2 ... /dev/hd8

#Two vdevs
zpool create myzpool raidz /dev/hd1 ... /dev/hd4 raidz /dev/hd5 ... /dev/hd8

Improve ZFS Performance: Step 19

Use AHCI, Not IDE

Typically there is a setting to control how the motherboard interacts with the hard drives: IDE or AHCI. If your motherboard has IDE ports (or manufactured before 2009), it is likely that the default value is set to IDE. Try to change to AHCI. Believe me, this litter tweak can save you countless of hours on debugging.

FYI, here is my whole story.

Improve ZFS Performance: Step 20

Refresh your pool

I had set up my zpool for five years. Over the past five years, I had performed lots of upgrade and changed a lot of settings. For example, during the initial set up, I didn't enable the compression. Later, I set the compression to lzjb and changed it to lz4. I also enabled and disabled the dedup. So you can imagine some part of the data is compressed using lzjb, some data has dedup enabled. In short, the data in my zpool has all kind of different settings. That's dirty.

The only thing I can clean up is to destroy the entire zpool and rebuild the whole thing. Depending on the size of your data, it can take 2-3 days to transfer 10TB of data from one to another server, i.e., 4-6 days round trip. However, you will see the performance gain in long run.

I recommend cleaning the pool every 3 years.

Improve ZFS Performance: Step 21

Great performance settings

The following settings will greatly improve the performance of your ZFS pool. However, each of them comes with a price tag. Its like removing the air bag from your car. Yes, it will save few pounds here and few pounds there. However, when something goes wrong, it could be a nightmare. Do it on your own risk.

Disable the sync option

sudo zfs set sync=disabled mypool

From the man page:

"File system transactions are only committed to stable storage periodically. This option will give the highest performance. However, it is very dangerous as ZFS would be ignoring the synchronous transaction demands of applications such as databases or NFS. Administrators should only use this option when the risks are understood."

Disable the checksum

sudo zfs set checksum=off mypool

Disabling the checksum will save some CPU computation time, but won't be a lot for each operation. However if you have a lot of data to write to the pull, saving a little bit in every write operation will end up a lot.

Disable the access time

sudo zfs set atime=off mypool

From the man page:

"Turning this property off avoids producing write traffic when reading files and can result in significant performance gains, though it might confuse mailers and other similar utilities."

Customizd the record size

Instead of creating one single filesystem to store everything, I recommend to create multiple filesystems for different purposes. For example, I usually create a seperate filesystem for MySQL databases with MyISAM engine, i.e.,

sudo zfs create -o recordsize=8k mypool/mysql_myisam

Of course, the drawback will be using more space comparing to the default settings.

Redundant Metadata

Set the redundant_metadata to most will improve the performance of random writes.

sudo zfs set redundant_metadata=most mypool

Enable the system attribute based xattrs

Storing xattrs as system attributes significantly decreases the amount of disk I/O.

sudo zfs set xattr=sa mypool

Improve ZFS Performance: Step 22

My Settings - Simple and Clean

I have set up over 50 servers based on ZFS. They all have different purposes. In general, they all share the same ZFS settings, and they can reach the hardware limit, e.g., ARC cache uses about 90% of the system memory, the network transfer speed max the limit of the network card etc.

sudo zpool history

#I like to use raidz, and each raidz vdev contains no more than 5 disks.
zpool create -f storage raidz /dev/hd1 /dev/hd2 ... raidz /dev/hd6 /dev/hd7 ... raidz /dev/hd11 /dev/hd12

#A partition for general purposes
zfs create storage/data

#A partition for general web
zfs create storage/web

#A partition for MySQL / MYISAM tables
zfs create -o recordsize=8k storage/mysql (MYISAM tables)

#Some common settings
zfs set compression=lz4 storage
zfs set atime=off storage
zfs set redundant_metadata=most storage

#For Linux server
zfs set xattr=sa storage

A simple nload when running rsync over a gigabit LAN:

(Bonding mode: 6, based on two gigabit network cards)
#nload -u M

Outgoing:
Curr: 209.48 MByte/s
Avg: 207.50 MByte/s
Min: 198.66 MByte/s
Max: 209.53 MByte/s
Ttl: 3755.46 GByte

I only uses two operating systems:

#Personal
FreeBSD 10.3-RELEASE-p11 FreeBSD 10.3-RELEASE-p11 #0: Mon Oct 24 18:49:24 UTC 2016     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64


#Work
CentOS Linux release 7.3.1611 (Core)
Linux 3.10.0-514.6.1.el7.x86_64 #1 SMP Wed Jan 18 13:06:36 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

I use only consumer-grade hard drives. Typically I buy the external one, and I remove the hard drive from the enclosures, because the cost is cheaper. Sometimes when I need to use all of the SATA ports, I will use PATA drive / USB flash drives for the operating systems. As long as all of the frequently used files are in the ZFS pool, the performance is not a problem. Another thing you can is to install card size SSD drive on the mSATA port, and connect a regular hard drive to the eSATA port on the back of the motherboard using a long eSATA-SATA cable. That will help to maximize the number of hard drives have. Most computer cases will fit 10-12 3.5" hard drives, some of them can go up to 15 for under USD 100. The most powerful one will fit 18 hard drives with a price tag of USD 200.

The amount of the memory is going to be tricky, as it depends on what applications you want to run on your server and how much memory will be consumed by the service. It also depends on how large is your ZFS pool capacity. Here are some of the server I have set up:

#A light weight web (Apache+MySQL+PHP), and file server (Samba)
CPU: Intel(R) Xeon(R) CPU E5-2440 v2 @ 1.90GHz (A mid-level server grade CPU from 2014)
Memory: 4GB
ZFS Pool Capacity: 1.6TB
OS: CentOS 7
Kernel: 3.10.0-514.6.1.el7.x86_64 #1 SMP Wed Jan 18 13:06:36 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux


#A server for nightly backup
CPU: Intel(R) Core(TM) i7 CPU K 875  @ 2.93GHz (A gaming-grade CPU from 2010)
Memory: 8GB
ZFS Pool Capacity: 43TB
OS: CentOS 7
Kernel: 3.10.0-514.6.1.el7.x86_64 #1 SMP Wed Jan 18 13:06:36 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux


#A server for analyzing genomic data, with a very high CPU usage
CPU: Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz (A gaming-grade CPU from 2015)
Memory: 64GB
ZFS Pool Capacity: 8.5TB
OS: CentOS 7
Kernel: 3.10.0-514.6.1.el7.x86_64 #1 SMP Wed Jan 18 13:06:36 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux


#A heavy weight network file server
CPU: Intel(R) Xeon(R) CPU E5-2430 0 @ 2.20GHz (A mid-level server grade CPU from 2012)
Memory: 96GB
ZFS Pool Capacity: 53TB
OS: CentOS 7
Kernel: 3.10.0-514.6.1.el7.x86_64 #1 SMP Wed Jan 18 13:06:36 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux


#A heavy weight web server
CPU: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz (An entry-level server grade CPU from 2013)
Memory: 16GB
ZFS Pool Capacity: 1.8TB
OS: CentOS 7
Kernel: 3.10.0-514.6.1.el7.x86_64 #1 SMP Wed Jan 18 13:06:36 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux


#A light weight web (Apache+MySQL+PHP), and file server (Samba)
CPU: Intel(R) Core(TM) i7 CPU 920  @ 2.67GHz  (A gaming-grade CPU from 2008)
Memory: 24GB
ZFS Pool Capacity: 25TB
OS: FreeBSD 10.3
Kernel: FreeBSD 10.3-RELEASE-p11 FreeBSD 10.3-RELEASE-p11 #0: Mon Oct 24 18:49:24 UTC 2016     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64


#A low-end backup server
CPU: Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz (An entry-level CPU from 2007)
Memory: 8GB
ZFS Pool Capacity: 15TB
OS: FreeBSD 10.3
Kernel: FreeBSD 10.3-RELEASE-p11 FreeBSD 10.3-RELEASE-p11 #0: Mon Oct 24 18:49:24 UTC 2016     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64

If you are interested in implementing network-based ZFS, please check here for details.

Enjoy ZFS.

--Derrick

Our sponsors:

I noticed some weird thing on my server today. When I ran some PHP code, I got the following error:

PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20121212-zts/pdo_sqlite.so' - Shared object "libsqliteo_sqlite.so" in Unknown on line 0

Warning: PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20121212-zts/pdo_sqlite.so' - Shared object "libsqlite3.so.ite.so" in Unknown on line 0
PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20121212-zts/sqlite3.so' - Shared object "libsqlite3.se3.so" in Unknown on line 0

Warning: PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20121212-zts/sqlite3.so' - Shared object "libsqlite3.so.8" " in Unknown on line 0
PHP 5.5.13 (cli) (built: Jun  3 2014 10:01:52)
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2014 Zend Technologies
    with Zend OPcache v7.0.4-dev, Copyright (c) 1999-2014, by Zend Technologies

Obviously, PHP SQLite extensions expect the library: libsqlite3.so.8, and it was missing or deleted after the upgrade. To solve this problem, we need to know where is the file.

sudo find / -name "libsqlite3.so*"

In my system, the file is here:

/usr/local/lib/libsqlite3.so.0
/usr/local/lib/libsqlite3.so.0.8.6

So I simply created the missing file by soft-linking it:

cd /usr/local/lib/
sudo ln -s libsqlite3.so libsqlite3.so.8

And the error is gone.

#php -v
PHP 5.5.13 (cli) (built: Jun  3 2014 10:01:52)
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2014 Zend Technologies
    with Zend OPcache v7.0.4-dev, Copyright (c) 1999-2014, by Zend Technologies

–Derrick

Our sponsors:

I upgraded the Apache to 2.2.27 on my FreeBSD box via portmaster. The upgrade went very smooth. After the upgrade, I decide to test the Apache by restarting the server. Oh well, I got the following error message:

[FreeBSD]Shared object "libaprutil-1.so.5" not found, required by "httpd"

Oh well, looks like libaprutil-1.so.5 is missing. How about I create this file by soft-linking from the newer version, such as libaprutil-1.so.6, libaprutil-1.so.7 etc. However, when I check the lib directory, I don’t see anything like that.

#/usr/local/lib
-rw-r--r--  1 root  wheel   251k Jun 11 10:58 libaprutil-1.a
-rwxr-xr-x  1 root  wheel   961B Jun 11 10:58 libaprutil-1.la
lrwxr-xr-x  1 root  wheel    21B Jun 11 10:58 libaprutil-1.so -> libaprutil-1.so.0.5.3
lrwxr-xr-x  1 root  wheel    21B Jun 11 10:58 libaprutil-1.so.0 -> libaprutil-1.so.0.5.3
-rwxr-xr-x  1 root  wheel   148k Jun 11 10:58 libaprutil-1.so.0.5.3

Looks like it is not that simple to solve the problem. After couple trials and errors, I came up a solution. First, I need to reinstall the Apache, and then I need to reinstall Apr1.

cd /usr/local/www/apache22
make

Notice that I compile the file without installing it. That’s because I want to check the library dependence.

cd /usr/ports/www/apache22/work/httpd-2.2.27
ldd ./httpd
./httpd:
        libm.so.5 => /lib/libm.so.5 (0x80087d000)
        libpcre.so.3 => /usr/local/lib/libpcre.so.3 (0x800a9e000)
        libaprutil-1.so.0 => /usr/local/lib/libaprutil-1.so.0 (0x800d02000)
        libdb-4.8.so.0 => /usr/local/lib/libdb-4.8.so.0 (0x800f27000)
        libgdbm.so.4 => /usr/local/lib/libgdbm.so.4 (0x801299000)
        libexpat.so.6 => /usr/local/lib/libexpat.so.6 (0x8014a4000)
        libiconv.so.3 => /usr/local/lib/libiconv.so.3 (0x8016c8000)
        libapr-1.so.0 => /usr/local/lib/libapr-1.so.0 (0x8019c4000)
        libcrypt.so.5 => /lib/libcrypt.so.5 (0x801bf5000)
        libthr.so.3 => /lib/libthr.so.3 (0x801e14000)
        libc.so.7 => /lib/libc.so.7 (0x802037000)
        libapr-1.so.5 => not found (0)
        libintl.so.9 => /usr/local/lib/libintl.so.9 (0x802392000)

So it is libaprutil-1.so.0, which is what we have in /usr/local/lib/. That’s good. Now we can hit the install button.

cd /usr/ports/www/apache22/
make reinstall

So let’s try to run Apache again. Looks like we got a different error message:

/usr/local/etc/rc.d/apache22 restart
Performing sanity check on apache22 configuration:
Shared object "libapr-1.so.5" not found, required by "libaprutil-1.so.0"

It’s okay. Let’s do something similar to the apr1:

cd /usr/ports/devel/apr1/
make reinstall

Try to restart Apache again. The problem should be gone:

/usr/local/etc/rc.d/apache22 restart
Performing sanity check on apache22 configuration:
Syntax OK
Stopping apache22.
Waiting for PIDS: 1022.
Performing sanity check on apache22 configuration:
Syntax OK
Starting apache22.

Hope my solutions help!

–Derrick

Our sponsors:

I upgraded the Apache to 2.2.27 on my FreeBSD box via portmaster. The upgrade went very smooth. After the upgrade, I found that Apache no longer rendered the PHP page correctly. In the other words, it displayed the source code of the PHP files instead of executing the code.

Before you start doing anything, please make sure that your website is not accessible from public. For example, most web applications like to include the password information in the source code, which is accessible by public if the PHP engine is failed. You may want to restrict the public access during the fix. The easiest way is to set up a .htaccess file and restrict the access by IP address.

Let’s come back to the fix. For some reasons, Apache/PHP team decide to make something fun to the FreeBSD sysadmin, because they think FreeBSD sysadmin have plenty of spare time. Here is what they decide to do:

Originally, we install Apache first, then we install PHP. When we install the PHP, we pick an option such that the PHP will install a PHP engine used by Apache. In the recent release, they decide to remove the engine from the standard PHP package. For example, if you simply use the portmaster to upgrade your PHP (In my case, PHP 5.4 and 5.5), the engine will be missing. That’s why Apache fails to render the PHP files.

Here is what you will need to do. I suggest you to finish reading the following before doing the fix. Trust me. It may save you couple hours.

First, make sure that Apache is working fine.

If you are using PHP 5.4, make sure that you install this port: /usr/ports/www/mod_php5. If you are using PHP 5.5, install this one instead: /usr/ports/www/mod_php55. Make sure that you select the ZTS option. You can do it by running:

sudo make config

Notice that installing this port will reinstall the PHP package (/usr/ports/lang/php5 or /usr/ports/lang/php55). Make sure that the ZTS option is selected in PHP package as well.

Now try to restart the Apache server. That should make the Apache to render the PHP files instead of dumping it.

#sudo /usr/local/etc/rc.d/apache stop
#sudo /usr/local/etc/rc.d/apache start

Test the PHP files again. If you see the PHP result instead of PHP source code, congratulations! You are 20% done.

Now go back to the command line and run the following:

#List the PHP extensions installed by PHP 
php -m

#Check whether the PHP extensions are loaded properly by PHP
php -v

For some reasons, I noticed that many extensions were missing during the reinstallation, such as sessions, json etc. Let’s install them back:

#For PHP 5.4
cd /usr/ports/lang/php5-extensions

#For PHP 5.5
cd /usr/ports/lang/php55-extensions


sudo make reinstall clean

Don’t forget the pecl and related packages as well. After that, try to restart your Apache server and clean up the php extension configuration:

nano /usr/local/etc/php/extensions.ini

You will see a mess. Try to clean up the duplicated extensions and run the php -m and php -v again.

During the fix, I notice that everything works fine except for MySQL package. Interestingly, when I ran php -m, the mysql was available. However when I ran phpinfo(), it was missing. I found that this problem only happens with PHP 5.4, but not PHP 5.5. Therefore, I decided to remove the PHP 5.4 and loaded PHP 5.5 instead.

Simply repeat the installation and restart the server. If possible, try to reboot the machine.

So here is the summary:

  1. Remove PHP, Extensions and PECL packages (sudo make deinstall).
  2. Restart Apache (sudo /usr/local/etc/rc.d/apache stop; sudo /usr/local/etc/rc.d/apache start).
  3. Verify installed extensions with php -m; php -v; and phpinfo().
  4. Verify the result by loading the pages on the web.
  5. If your page is failed, try to find out which extension is missing by checking the error log, which is typically available in in /var/log/

Hope my solutions help!

–Derrick

Our sponsors:

While I tried to upgrade both of my FreeBSD server (9 & 10) today, I got the following error:

install: /usr/ports/…/doc/: Inappropriate file type or format

Actually, here is the complete error message:

===>  Staging for memcached-1.4.17_1
===>   memcached-1.4.17_1 depends on shared library: libevent-2.0.so - found
===>   Generating temporary packing list
/usr/bin/make  install-recursive
Making install in doc
/usr/bin/make  install-am
test -z "/usr/local/man/man1" || /bin/mkdir -p "/usr/ports/databases/memcached/work/stage/usr/local/man/man1"
 install  -o root -g wheel -m 444 memcached.1 '/usr/ports/databases/memcached/work/stage/usr/local/man/man1'
test -z "/usr/local/bin" || /bin/mkdir -p "/usr/ports/databases/memcached/work/stage/usr/local/bin"
  install  -s -o root -g wheel -m 555 memcached '/usr/ports/databases/memcached/work/stage/usr/local/bin'
test -z "/usr/local/include/memcached" || /bin/mkdir -p "/usr/ports/databases/memcached/work/stage/usr/local/include/memcached"
 install  -o root -g wheel -m 444 protocol_binary.h '/usr/ports/databases/memcached/work/stage/usr/local/include/memcached'
install  -o root -g wheel -m 555 /usr/ports/databases/memcached/work/memcached-1.4.17/scripts/memcached-tool /usr/ports/databases/memcached/work/stage/usr/local/bin
install  -o root -g wheel -m 444 /usr/ports/databases/memcached/work/memcached-1.4.17/doc/ /usr/ports/databases/memcached/work/stage/usr/local/man/man1
install: /usr/ports/databases/memcached/work/memcached-1.4.17/doc/: Inappropriate file type or format
*** [post-install] Error code 71

Stop in /usr/ports/databases/memcached.
*** [install] Error code 1

Stop in /usr/ports/databases/memcached.

Initially, I thought the problem was caused by the syntax error of the code, therefore, I tried to do the following:

#Shut down the server
sudo /usr/local/etc/rc.d/memcached stop

#Remove Memcached from the system
sudo make deinstall

#Clean up the package
sudo make clean

#Compile the package
sudo make

#Install the package
sudo make install

Unfortunately, it still gave the same error message. I noticed that the error message has something to do with the documentation, so I decide to remove the documentation option in the installation page, i.e.,

#Enter the package setup
sudo make config

#Remove the "doc" option.

Unfortunately, it still gave the same response. I then tried to investigate the cause of the error. So I tried to copy the error command and re-ran it:

install  -o root -g wheel -m 444 /usr/ports/databases/memcached/work/memcached-1.4.17/doc/ /usr/ports/databases/memcached/work/stage/usr/local/man/man1

Which return the exact the same error message:

install: /usr/ports/databases/memcached/work/memcached-1.4.17/doc/: No such file or directory

So I came up an idea. Since I don’t really need the documentation, why not I disable the document option?

cd /usr/ports/databases/memcached

sudo nano Makefile

Change the following from:

post-install:
        ${INSTALL_SCRIPT} ${WRKSRC}/scripts/memcached-tool ${STAGEDIR}${PREFIX}/bin
        ${INSTALL_MAN} ${WRKSRC}/doc/${MAN1} ${STAGEDIR}${MAN1PREFIX}/man/man1
        @${MKDIR} ${STAGEDIR}${DOCSDIR}

To:

post-install:
        ${INSTALL_SCRIPT} ${WRKSRC}/scripts/memcached-tool ${STAGEDIR}${PREFIX}/bin
#        ${INSTALL_MAN} ${WRKSRC}/doc/${MAN1} ${STAGEDIR}${MAN1PREFIX}/man/man1
        @${MKDIR} ${STAGEDIR}${DOCSDIR}

That’s right, I just want to comment out the installation of the man/documentation part. Now, let’s try to reinstall the package again.

#sudo make install

Guess what, it works! Have fun!

–Derrick

Our sponsors:

There are many different ways to improve the MySQL performance. In general, it breaks down into two different categories: Server side and client side.

On the server side, we can optimize the database and table structure, such as indexing the columns etc. On the client side, we can optimize the queue to minimize the workload, or we can cache and share the result such that the traffic to the server will be minimized. However, these methods are doable if you have access to the source code, or you understand the logic of the software. If you are a system administrator, you may not want to touch the source code, because you never know what will happen after the modifications. Plus your modification may be overwritten in the next update.

I am going to show you a quick and easy way to solve this problem. First, I am assuming that you build the MySQL from source. In the other words, this article will not work if you install the MySQL through pkg_add, yum, apt-get etc.

My solution is very simple: Compiling the MySQL server with static option enabled

By default, we will compile the software from source, it is not built in static. According to MySQL documentation, building the binary using static will result a 13% improvement comparing to building the binary using dynamic. Here is an example how to build MySQL with static option enabled:

cd /usr/ports/databases/mysql56-server
make BUILD_OPTIMIZED=yes BUILD_STATIC=yes
make install clean

This method will work for the first time. It may be a problem in the long run. For example, I use portsnap to update the port tree, and I use portmaster to upgrade the application. By default, portmaster will use the default options to rebuild the port. In the other words, the MySQL will not be built using static.

To solve this problem, I will need to make the build static option as a default settings. First of all, try to include the following in /etc/make.conf

sudo nano /etc/make.conf

WITH_CHARSET=utf8
WITH_COLLATION=utf8_general_ci
BUILD_OPTIMIZED=yes
BUILD_STATIC=yes

We can update the port tree and ports again. This will make the system to use the new settings.

sudo portsnap fetch update
sudo portmaster -Day
#Don't forget to restart the MySQL server.
sudo /usr/local/etc/rc.d/mysql-server start

Now your MySQL server and other applications are built using static.

Please click here to learn more about building static versus building dynamics.

–Derrick

Our sponsors: