Tag: FreeBSD

ZFS Performance Boost/Improvement: How I push the I/O speed to 126MBps

Posted on by

This article is part of my main ZFS tutorial: How to improve ZFS performance. That article covers everything you need. If you already have the basic knowledge, or you just want to know how I push the I/O speed to 120+ MBps, you can skip that article and read this one.

Long story short, here is the result of my iostat:

sudo zpool iostat -v

               capacity     operations    bandwidth
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
storage     7.85T  12.1T      1  1.08K  2.60K   126M < ---126 MBps!
  raidz2    5.51T  8.99T      1    753  2.20K  86.9M
    ada0        -      -      0    142    102  14.5M
    ada1        -      -      0    143    102  14.5M
    ada3        -      -      0    146    307  14.5M
    ada4        -      -      0    145    409  14.5M
    ada5        -      -      0    144    204  14.5M
    ada6        -      -      0    143    204  14.5M
    ada7        -      -      0    172    409  14.5M
    ada8        -      -      0    171    511  14.5M
  raidz1    2.34T  3.10T      0    349    409  39.5M
    da0         -      -      0    169    204  13.2M
    da1         -      -      0    176      0  13.2M
    da2         -      -      0    168    102  13.2M
    da3         -      -      0    173    102  13.2M
----------  -----  -----  -----  -----  -----  -----

I measured the speed while I was transferring 10TB of data from one FreeBSD-based ZFS server to another FreeBSD-based ZFS server, over a consumer-level gigabit network. Basically, every components I used are consumer-level. The hard drives I used are standard PATA and SATA hard drives (i.e., not even the light speed SSD hard drive). The data I transferred are real data (rather than all zeros or ones generated through dd).

First of all, here are the settings I used. Both server and client have similar configurations.

sudo zpool history
History for 'storage':
2014-01-22.20:28:59 zpool create -f storage raidz2 /dev/ada0 /dev/ada1 /dev/ada3 /dev/ada4 /dev/ada5 /dev/ada6 /dev/ada7 /dev/ada8 raidz /dev/da0 /dev/da1 /dev/da2 /dev/da3
2014-01-22.20:29:07 zfs create storage/data
2014-01-22.20:29:15 zfs set compression=lz4 storage
2014-01-22.20:29:19 zfs set compression=lz4 storage/data
2014-01-22.20:30:19 zfs set atime=off storage

#where the ad* and da* are nothing more than standard SATA drives:
#ZFS Drives
#8 x 2TB standard SATA hard drives connected to the motherboard
#4 x 1.5TB standard SATA hard drives connected to a PCI-e RAID card

da0: 1430799MB (2930277168 512 byte sectors: 255H 63S/T 182401C)
da1: 1430799MB (2930277168 512 byte sectors: 255H 63S/T 182401C)
da2: 1430799MB (2930277168 512 byte sectors: 255H 63S/T 182401C)
da3: 1430799MB (2930277168 512 byte sectors: 255H 63S/T 182401C)
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 512bytes)
ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada3: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada4: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada5: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada5: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada6: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada6: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada7: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada7: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada8: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada8: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)


#System Drive
#A PATA 200MB drive from 2005
ada2: maxtor STM3200820A 3.AAE ATA-7 device
ada2: 100.000MB/s transfers (UDMA5, PIO 8192bytes)
ada2: 190782MB (390721968 512 byte sectors: 16H 63S/T 16383C)

Here are the corresponding hardware components:

#CPU
hw.machine: amd64
hw.model: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz
hw.ncpu: 8
hw.machine_arch: amd64


#Memory - 3 x 2GB
real memory  = 6442450944 (6144 MB)
avail memory = 6144352256 (5859 MB)

#Network - Gigabit network on the motherboard / PCI-e
#If your gigabit network card is PCI, that will be your bottleneck.
re0: realtek 8168/8111 B/C/CP/D/DP/E/F PCIe Gigabit Ethernet

Other than that, I didn’t really specify any special settings in my OS. Everything else are default:

#uname -a
FreeBSD 9.2-RELEASE-p3 FreeBSD 9.2-RELEASE-p3 #0: Sat Jan 11 03:25:02 UTC 2014     [email protected]:/usr/obj/usr/src/sys/GENERIC  amd64

The way how to transfer 10TB of data is through rsync and rsyncd. First of all, I set up a rsync as daemon on the server. Keep in mind that rsync as a daemon (service) is not the same as rsync. If you want to know how I set up the rsyncd, please visit How to Improve rsync Performance.

After you have set up the rsyncd on the server, I run the following on the client.

rsync -av server_ip::rsync_share_name /my_directory/

#Example
rsync -av 192.168.1.101::storage /mydata/

Notice that I didn’t enable the compression option. That’s because my files are pretty much compressed (jpeg, zip, tar, 7z etc). If your file types are different, you may want to enable the compression, i.e.,

rsync -avz server_name::rsync_share_name /my_directory/

Give it a try and see whether it improves the I/O speed.

Here are few things I have learned to improve the ZFS performance.

Keep the ZFS structure small and simple. For example, keep the number of disks in your vdev small. Previously, I set up a giant big pool with 14 disks RAIDZ in one single vdev. This is a bad idea. There is a maximum number of disks for RAIDZ. That’s why I split my disks into two groups:

#Group 1 - RAIDZ2
#8 x 2TB standard SATA hard drives connected to the motherboard

#Group2 - RAIDZ
#4 x 1.5TB standard SATA hard drives connected to a PCI-e RAID card

zpool create -f storage myzpool group1 raidz group2

Keep the ZFS settings clean. Enable the only thing you need, and disable the junk settings. In my case, I enable the best compression (lz4) and disable the access time:

2014-01-22.20:29:15 zfs set compression=lz4 storage
2014-01-22.20:30:19 zfs set atime=off storage

Keep the ZFS clean. I see the performance gain on a brand new ZFS than a few years old ZFS, even both of them contain the same data. Unfortunately, there is no easy way to clean the ZFS. The only way is to destroy the current ZFS first, create a new one and bring the data back. Typically, it takes about 2 days to transfer 10TB of data over a gigabit LAN. So it is really not too bad to spend about 5 days to rebuild your ZFS.

That’s it. Again, if you want to learn about tricks, please visit How to improve ZFS performance for more information.

–Derrick

Our sponsors:

[FreeBSD]mount: /dev/da0p2: R/W mount of / denied. Filesystem is not clean – run fsck. Forced mount will invalidate journal contents: Operation not permitted

Posted on by

When I tried to mount a hard drive on FreeBSD today, I got the following error message.

sudo mount -t ufs /dev/da0p2 /mnt/
mount: /dev/da0p2: R/W mount of / denied. Filesystem is not clean - run fsck. Forced mount will invalidate journal contents: Operation not permitted

Here is how to fix it:

sudo mount -r -t ufs /dev/da0p2 /mnt/

where ufs refers to the file system of FreeBSD.

That’s it.

–Derrick

Our sponsors:

[FreeBSD] ZFS FAULTED / kernel: (ada0:ahcich4:0:0:0): lost device

Posted on by

I am so happy that I finally solved a problem today. This problem had existed in my server farms for few months already. In the past few months, I had absolutely no idea what was causing the issues. Here is my story:

I built a file server using FreeBSD and ZFS. The hardware components are consumer grade hardware. Basically, it is a desktop computer with multiple hard drives. The hard drives are connected to the SATA ports on the motherboard. It is a very simple setup.

After I set up a ZFS pool, I started to load the data and stress tested the system. I noticed that the ZFS pool turned into a fault state after 15 mins, e.g., 

#sudo zpool status -v
  pool: storage
 state: FAULTED
status: One or more devices could not be opened.  There are insufficient
        replicas for the pool to continue functioning.
action: Attach the missing device and online it using 'zpool online'.
config:

        NAME        STATE     READ WRITE CKSUM
        storage     ONLINE       0     0     0
          raidz1-0  ONLINE       0     0     0
            ada4    ONLINE       0     0     0
            ada0    REMOVED      0     0     0 
            ada2    ONLINE       0     0     0
            ada5    ONLINE       0     0     0
            ada3    ONLINE       0     0     0

errors: Permanent errors have been detected in the following files:

In short, a drive is missing. So I tried to run dmesg 

Oct 17 20:24:00 kernel: (ada0:ahcich4:0:0:0): lost device
Oct 17 20:24:00 kernel: (ada0:ahcich4:0:0:0): removing device entry
Oct 19 20:42:14 kernel: (ada0:ahcich3:0:0:0): lost device
Oct 19 23:19:18 kernel: (ada0:ahcich4:0:0:0): lost device
Oct 19 23:19:18 kernel: (ada0:ahcich4:0:0:0): removing device entry
Oct 19 23:19:18 kernel: (ada0:ahcich4:0:0:0): lost device
Oct 19 23:19:18 kernel: (ada0:ahcich4:0:0:0): removing device entry
Oct 19 23:19:18 kernel: (ada0:ahcich4:0:0:0): lost device
Oct 19 23:19:18 kernel: (ada0:ahcich4:0:0:0): removing device entry
Oct 19 23:19:18 kernel: (ada0:ahcich4:0:0:0): lost device
Oct 19 23:19:18 kernel: (ada0:ahcich4:0:0:0): removing device entry
Oct 19 23:19:18 kernel: (ada0:ahcich4:0:0:0): lost device
Oct 19 23:19:18 kernel: (ada0:ahcich4:0:0:0): removing device entry
Oct 19 23:33:02 kernel: (ada0:ahcich4:0:0:0): Synchronize cache failed
Oct 20 22:53:40 kernel: (ada0:ahcich4:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 40 30 af 4e 40 7a 00 00 00 00 00
Oct 20 22:53:40 kernel: (ada0:ahcich4:0:0:0): CAM status: ATA Status Error
Oct 20 22:53:40 kernel: (ada0:ahcich4:0:0:0): ATA status: 61 (DRDY DF ERR), error: 04 (ABRT )
Oct 20 22:53:40 kernel: (ada0:ahcich4:0:0:0): RES: 61 04 00 00 00 40 00 00 00 00 00
Oct 20 22:53:40 kernel: (ada0:ahcich4:0:0:0): Retrying command

I also verified the content in /dev/, e.g.,

#ls /dev/ad0

crw-r-----  1 root  operator   0x6f Oct 18 22:10 /dev/ada0

Interestingly, the device is still available. This is very confusing because the system sees the hardware (with some error), but the application could not see the hardware. So I decided to perform the following tests:

  • Reboot
  • Shutdown and reboot
  • Replacing the SATA cables
  • Connecting the hard drive to a different SATA port
  • Replacing the hard drive with a new one that is certified by the manufacturer
  • Replacing the motherboard
  • Replacing the hard drive power cable
  • Replacing the power supply
  • Updating the firmware of the motherboard

Unfortunately, none of these fixed the problem. I also checked the S.M.A.R.T. status of the hard drives but I didn’t see any issues. One thing I noticed is that the error is not consistent. It happens randomly on different SATA port and hard drive. That doesn’t make any sense to me because I saw the same thing on 3 different motherboards. Finally, I decided to try one last thing: I connected the hard drives through an USB port. Guess what, my FreeBSD box stopped complaining.

So I decided to a different approach. I used the same hardware setup (the hard drives were connected using SATA instead of USB) and loaded a Windows. The system worked perfectly fine and stable. I didn’t experience any issue at all. So I believed the problem had nothing to do with the hardware. It must be the software settings issue.

Finally, I decided to try one last thing, something I hadn’t paid attention before: BIOS Settings

AHCI/IDE

Typically there is a setting to control how the motherboard interacts with the hard drives: IDE or AHCI. I usually stick with the default settings. In my case, the default value of my motherboard is IDE. After I changed the settings to AHCI, I found that the problem is gone. Yes, it’s gone and my headache is gone too.

That was easy!

–Derrick

Our sponsors:

How to Improve rsync Performance

Posted on by

I need to transfer 10TB of data from one machine to another machine. Those 10TB of files are living in a large RAID which span across 7 different disks. The target machine has another large RAID which span across 12 different disks. It is not easy to copying those files locally. Therefore, I decide to copy the files over the LAN.

There are four options popping up in my head: scp, rsync, rsyncd (rsync as daemon) and netcat.

scp

scp is handy, easy to use but comes with two disadvantages: slow and not fault-tolerant. Since scp comes with the highest security, all data are encrypted before the transfer. It will slow down the overall performance because of the extra encryption stuffs (which makes the data larger), and extra computational resource (which uses more CPU). If the transfer is interrupted, there is no easy way to resume the process other than transferring everything again. Here are some example commands:

#Source machine
#Typical speed is about 20 to 30MB/s
scp -r /data target_machine:/data

#Or you can enable the compression on the fly
#Depending on the type of your data, if your data is already compressed, you may see no or negative speed improvement
scp -rC /data target_machine:/data

rsync

rsync is similar to scp. It comes with the encryption (via SSH) such that the data is safe. It also allows you to transfer the newer files only. This will reduce the amount of data being transferred. However, it comes with few disadvantages: long decision time, encryption (which increase the size of overhead) and extra computational resource(e.g., data comparison, encryption and decryption etc). For example, if I use rsync to transfer 10TB of files from one machine to another machine (where the directory on the target machine is blank), it can easily take 5 hours to determine which files will need to be transferred before the actual data transfer is initialized.

#Run on the target machine
rsync -avzr -e ssh --delete-after source_machine:/data/ /data/

#Use a less secure encryption algorithm to speed up the process
rsync -avzr --rsh="ssh -c blowfish" --delete-after source_machine:/data/ /data/

#Use an even less secure algorithm to get the top speed
rsync -avzr --rsh="ssh -c arcfour" --delete-after source_machine:/data/ /data/

#By default, rsync compares the files using checksum, file size and modification date.
#Reduce the decision process by skipping the hash check
rsync -avzr --rsh="ssh -c arcfour" --delete-after --whole-file source_machine:/data/ /data/

Anyway, no matter what you do, the top speed of rsync in a consumer-grade gigabit network is around 45MB/s. On average, the speed is around 25-35MB/s. Keep in mind that this number does not include the decision time, which can be few hours.

rsyncd (rsync as a daemon)

Thanks for the comment of our reader. I got a chance to investigate the rsync as a daemon. Basically, the idea of running rsync as a daemon is similar to rsync. On the server, we run rsync as a service/daemon. We specify which directory we want to “export” to the clients (e.g., /usr/ports). When the files get changed on the server, it records the changes so that the when the clients talk to the server, the decision time will be faster. Here is how to set up rsync server on FreeBSD

sudo nano /usr/local/etc/rsyncd.conf

And this is my configuration file:

pid file = /var/run/rsyncd.pid

#Notice that I use derrick here instead of other systems users, such as nobody
#That's because nobody does not have permission to access the path, i.e., /data/
#Either you make the source directory available to "nobody", or you change the daemon user.
uid = derrick
gid = derrick
use chroot = no
max connections = 4
syslog facility = local5
pid file = /var/run/rsyncd.pid

[mydata]
   path = /data/
   comment = data

Don't forget to include the following in /etc/rc.conf, so that the service will be started automatically.

rsyncd_enable="YES"

#Let's start the rsync service:

sudo /usr/local/etc/rc.d/rsyncd start

To pull the files from the server to the clients, run the following:

rsync -av myserver::mydata /data/

#Or you can enable compression
rsync -avz myserver::mydata /data/

To my surprise, it works much better than running rsync alone. Here are some data I collected during transferring 10TB files from ZFS to ZFS:

Bandwidth measured on the client machine: 70MB/s

zpool IO speed on the client side: 75MB/s

P.S. Initially, the speed was about 45-60MB/s, after I tweak my Zpool, I can get the top speed to 75-80MB/s. Please check out here for references.

I notice that the decision time is much faster than running rsync alone. Also the process is much more stable, with zero interruption, i.e., 

rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at io.c(521) [receiver=3.1.0]
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(632) [generator=3.1.0]
rsync: [receiver] write error: Broken pipe (32)

NetCat

NetCat is similar to cat, except that it works at the network level. I decide to use netcat for the initial transfer. If it is interrupted, I will let rsync to kick in the process. Netcat does not encrypt the data, so the overhead is very small. If you transfer the file within a local network and you don’t care about the security, netcat is a perfect choice.

There is only one disadvantage of using netcat. It can only handle one file at a time. It doesn’t mean you need to run netcat for every single file. Instead, we can tar the file before feeding to netcat, and untar the file at the receiving end. As long as we do not compress the files, we can keep the CPU usage small. 

#Open two terminals, one for the source and another one for the target machine.

#On the target machine:
#Go to the directory, e.g., 
cd /data

#Run the following:
nc -l 9999| tar xvfp -

#On the source machine:
#Go to the directory, e.g.,
cd /data

#Pick a port number that is not being used, e.g., 9999
tar -cf - . | nc target_machine 9999

Unlike rsync, the process will start right the way, and the maximum speed is around 45 to 60MB/s in a gigabit network.

Conclusion

Candidates Top Speed (w/o compression) Top Speed (w/ compression) Resume Stability Instant Start?
scp 40MB/s 25MB/s No Low Instant
rsync 25MB/s 50MB/s Yes Medium Long Preparation
rsyncd 30MB/s 70MB/s Yes High Short Preparation
netcat 60MB/s (tar w/o -z) 40MB/s (tar w/ -z) No Very High Instant

–Derrick

Our sponsors:

[FreeBSD]Problem to Update cURL-7.31.0

Posted on by

When I tried to update the cURL to cURL 7.31.0 today for my FreeBSD, it stopped and gave the following error messages:

configure: using CFLAGS: -O2 -pipe -DLDAP_DEPRECATED -fno-strict-aliasing
configure: CFLAGS error: CFLAGS may only be used to specify C compiler flags, not macro definitions. Use CPPFLAGS for: -DLDAP_DEPRECATED
configure: error: Can not continue. Fix errors mentioned immediately above this line.
===>  Script "configure" failed unexpectedly.
Please report the problem to [email protected] [maintainer] and attach the
"/usr/ports/ftp/curl/work/curl-7.31.0/config.log" including the output of the
failure of your make command. Also, it might be a good idea to provide an
overview of all packages installed on your system (e.g. a /usr/sbin/pkg_info
-Ea).
*** [do-configure] Error code 1

Stop in /usr/ports/ftp/curl.
*** [build] Error code 1

Stop in /usr/ports/ftp/curl.

===>>> make failed for ftp/curl
===>>> Aborting update

===>>> Update for ftp/curl failed
===>>> Aborting update

===>>> Killing background jobs
Terminated

===>>> You can restart from the point of failure with this command line:
       portmaster  ftp/curl

===>>> Exiting

It is very simple the fix this problem. However, the prerequisite is to give up the support to LDAP. If you are not sure whether your cURL needs LDAP or not, you probably don’t need it.

cd /usr/ports/ftp/curl
sudo make config

Remove the LDAP related features, such as LDAP, LDAPS. Try to re-build cURL again.

sudo make clean
sudo make

If everything looks good (i.e., not complaining any more), you can resume the update process, e.g.,

sudo portmaster -Da

Now your FreeBSD should be happy.

–Derrick

Our sponsors:

[FreeBSD]Portsnap / gunzip: can’t stat: files/… .gz: No such file or directory

Posted on by

My FreeBSD got an unknown fever today. When I updated my ports using portsnap, it gave the following error message:

#sudo portsnap fetch
Looking up portsnap.FreeBSD.org mirrors... 6 mirrors found.
Fetching snapshot tag from your-org.portsnap.freebsd.org... done.
Fetching snapshot metadata... done.
Updating from Thu May 23 09:08:53 CDT 2013 to Thu May 23 09:25:25 CDT 2013.
Fetching 0 metadata patches. done.
Applying metadata patches... done.
Fetching 0 metadata files... done.
gunzip: can't stat: files/992a1325cdc9a00a3543aa38fdf58903cdf70eaee02b8bb8aebea5505ac7b3f8.gz: No such file or directory
Fetching 0 patches. done.
Applying patches... done.
Fetching 0 new ports or files... done.
Building new INDEX files... gunzip: can't stat: /var/db/portsnap/files/09f65f8a730283fd31d068a5927ed46d95e37540f89090c257d7809b75116293.gz: No such file or directory
gunzip: can't stat: /var/db/portsnap/files/e3d3219617c1ea87cdfac7c8df0a52d611b191be8a80fd97f511277dff4cce77.gz: No such file or directory
gunzip: can't stat: /var/db/portsnap/files/8c2576279258f0d1b8762df8fc1e0cb4bcfcd23b6b09cdb4e7d68886af35ed7d.gz: No such file or directory
done.

Apparently, something in /var/db/portsnap/ is broken. Many people will try to remove /var/db/portsnap/ and run the command again. Do not do it. It will make portsnap failed. Instead, do the following:

sudo cp -r /var/db/portsnap /var/db/portsnap_backup
sudo rm -Rf /var/db/portsnap/tag /var/db/portsnap/files/*
sudo portsnap fetch extract
sudo portsnap update

Now your portsnap should be happy.

–Derrick

Our sponsors:

[Solved]FreeBSD: Problem to Update glib20

Posted on by

I decided to install Java on my FreeBSD box today. It wasn’t a very good experience. Primary the system is not automated. It requires a lot of manual works.

Anyway, after waiting for couple hours, I found that the process was stuck on glib. The system could not build the /usr/ports/devel/glib20. Here is the error message: 

===>  Building for glib-2.34.3
gmake  all-recursive
gmake[1]: Entering directory `/usr/ports/devel/glib20/work/glib-2.34.3'
Making all in .
gmake[2]: Entering directory `/usr/ports/devel/glib20/work/glib-2.34.3'
gmake[2]: Nothing to be done for `all-am'.
gmake[2]: Leaving directory `/usr/ports/devel/glib20/work/glib-2.34.3'
Making all in m4macros
gmake[2]: Entering directory `/usr/ports/devel/glib20/work/glib-2.34.3/m4macros'
gmake[2]: Nothing to be done for `all'.
gmake[2]: Leaving directory `/usr/ports/devel/glib20/work/glib-2.34.3/m4macros'
Making all in glib
gmake[2]: Entering directory `/usr/ports/devel/glib20/work/glib-2.34.3/glib'
gmake  all-recursive
gmake[3]: Entering directory `/usr/ports/devel/glib20/work/glib-2.34.3/glib'
Making all in libcharset
gmake[4]: Entering directory `/usr/ports/devel/glib20/work/glib-2.34.3/glib/libcharset'
gmake[4]: Nothing to be done for `all'.
gmake[4]: Leaving directory `/usr/ports/devel/glib20/work/glib-2.34.3/glib/libcharset'
Making all in update-pcre
gmake[4]: Entering directory `/usr/ports/devel/glib20/work/glib-2.34.3/glib/update-pcre'
gmake[4]: Nothing to be done for `all'.
gmake[4]: Leaving directory `/usr/ports/devel/glib20/work/glib-2.34.3/glib/update-pcre'
Making all in .
gmake[4]: Entering directory `/usr/ports/devel/glib20/work/glib-2.34.3/glib'
  CC       gstrfuncs.lo
  CC       gthreadpool.lo
gstrfuncs.c:330: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'get_C_locale'
gstrfuncs.c: In function 'g_ascii_strtod':
gstrfuncs.c:700: warning: implicit declaration of function 'strtod_l'
gstrfuncs.c:700: warning: implicit declaration of function 'get_C_locale'
gstrfuncs.c: In function 'g_ascii_formatd':
gstrfuncs.c:902: error: 'locale_t' undeclared (first use in this function)
gstrfuncs.c:902: error: (Each undeclared identifier is reported only once
gstrfuncs.c:902: error: for each function it appears in.)
gstrfuncs.c:902: error: expected ';' before 'old_locale'
gstrfuncs.c:904: error: 'old_locale' undeclared (first use in this function)
gstrfuncs.c:904: warning: implicit declaration of function 'uselocale'
gstrfuncs.c: In function 'g_ascii_strtoull':
gstrfuncs.c:1148: warning: implicit declaration of function 'strtoull_l'
gstrfuncs.c: In function 'g_ascii_strtoll':
gstrfuncs.c:1195: warning: implicit declaration of function 'strtoll_l'
gmake[4]: *** [gstrfuncs.lo] Error 1
gmake[4]: *** Waiting for unfinished jobs....
gmake[4]: Leaving directory `/usr/ports/devel/glib20/work/glib-2.34.3/glib'
gmake[3]: *** [all-recursive] Error 1
gmake[3]: Leaving directory `/usr/ports/devel/glib20/work/glib-2.34.3/glib'
gmake[2]: *** [all] Error 2
gmake[2]: Leaving directory `/usr/ports/devel/glib20/work/glib-2.34.3/glib'
gmake[1]: *** [all-recursive] Error 1
gmake[1]: Leaving directory `/usr/ports/devel/glib20/work/glib-2.34.3'
gmake: *** [all] Error 2
*** Error code 1

Stop in /usr/ports/devel/glib20.
*** Error code 1

Stop in /usr/ports/devel/glib20.

This is not a popular problem. After I google a while, I found a very similar problem here. Basically, the author suggests that the problem may be caused by an uncleaned run of freebsd-update. In short, you need to run the program twice, i.e.,

sudo freebsd-update fetch install
sudo reboot
sudo freebsd-update install

Obviously, it didn’t work for me. So I decided to try my last solution: pkg_add

sudo pkg_add -r glib20

and I tried to resume the installation:

cd /usr/ports/java/jdk16
sudo make install

It worked!

Hope this little trick is helpful to you.

–Derrick

Our sponsors:

[FreeBSD]The net/samba35 port has been deleted: Obsoleted by net/samba36

Posted on by

FreeBSD is designed for users who got lots of spare time. Every once a while, it will give you some troubles (or “snacks” in programmer language), which help you to burn some of your spare time. If you cannot handle these troubles, then FreeBSD is not for you.

Although FreeBSD is a rock solid operating system, it is not smart enough to tell whether the owners (i.e., you and me) got plenty of spare time or not. Anyway, comparing to Windows Server, I think FreeBSD is considered an honor student already.

Okay, let’s go back to the error message. When I tried to update FreeBSD, I got the following error message: 

Building new INDEX files... done.
===>>> Starting check of installed ports for available updates

===>>> The net/samba35 port has been deleted: Obsoleted by net/samba36
===>>> Aborting update

Terminated

Again, this is an old-school error. The port-maintainer decided that the samba35 is too old, and he likes to replace it by samaba36. We all love the new stuffs, but we want the transition goes smooth too. Apparently, it seems that the port-maintainer totally forgot about it.

If you check the directory, you will notice that /usr/ports/net/samba35 is deleted. If you try to soft-link samba36 to samba35, portmaster will likely to complain. The only way to solve this problem is by removing samba35 first, and installing samaba36 afterward.

So here is how to solve this problem:

sudo pkg_info | grep samba35

You will see something like the following:

samba35-3.5.19      A free SMB and CIFS client and server for UNIX

Let’s delete it.

sudo pkg_delete -f samba35-3.5.19

FreeBSD may remind you to manually remove Samba’s configuration, DO NOT do it.

WARNING: If you will *NOT* use this package anymore, please remove the
  following directories manually:
  /usr/local/etc/samba
  /var/log/samba
  /var/db/samba

Verify that the package has been deleted. The package should be gone.

sudo pkg_info | grep samba35

Now try to install the package again:

cd /usr/ports/net/samba36
sudo make install clean

After the installation is completed, don’t forget to start the Samba server:

sudo /usr/local/etc/rc.d/samba start

That’s it!

–Derrick

Our sponsors:

How to Upgrade FreeBSD from 8 to 9

Posted on by

Upgrading FreeBSD is very easy. Thanks for the simple architecture design and the disconnection between the kernel and the applications (ports), the whole process takes around 15 to 45 minutes. The coolest thing is that the system down time is very short. In my case, the down time was only about 3 minutes.

This table will help you to estimate the whole process and system down time:

Action Down Time? Approximate Duration
Download the update files None 5 to 15 minutes
Installing the update None 5 to 15 minutes
Helping FreeBSD to set up the system configurations None 1 to 3 minutes
Reboot Yes 1 to 3 minutes
Installing the update None 1 to 2 minutes
Reboot (Optional) Yes 1 to 3 minutes

Notice that this tutorial only works for i386 or amd64. For other CPU architectures such as Sparc, you will either reinstall the whole thing or rebuild the whole world (i.e., make buildworld). One way or the other, that’s a very time-consuming process.

First, I am assuming that your FreeBSD is up-to-date in your current version. You can do it by running the following command:

First, make sure that your ports are up to date.

#Download the latest ports
sudo portsnap fetch update

#Upgrade the old ports:
sudo portmaster -Da

Next, we want to upgrade the FreeBSD. Currently, the latest version is 9.1:

sudo freebsd-update upgrade -r 9.1-RELEASE

FreeBSD will try to download files for you. Just say yes.

Looking up update.FreeBSD.org mirrors... 4 mirrors found.
Fetching metadata signature for 8.2-RELEASE from update5.FreeBSD.org... done.
Fetching metadata index... done.
Fetching 1 metadata files... done.
Inspecting system... done.

The following components of FreeBSD seem to be installed:
kernel/generic world/base world/manpages

The following components of FreeBSD do not seem to be installed:
src/base src/bin src/cddl src/contrib src/crypto src/etc src/games
src/gnu src/include src/krb5 src/lib src/libexec src/release src/rescue
src/sbin src/secure src/share src/sys src/tools src/ubin src/usbin
world/catpages world/dict world/doc world/games world/info world/lib32
world/proflibs

Does this look reasonable (y/n)? y

Next, it will download the patches. Depending on your internet connection speed, it may take 1 to 10 minutes:

...
Applying patches... done.
Fetching 1962 files... done.
Attempting to automatically merge changes in files... done.

Now, FreeBSD will try to check whether your configuration is compatible with the new version or not. It may need your help to merge the configuration files manually. Notice that it uses VI, a standard (and very old, not really user-friendly) editor in Unix. If you’ve never used vi before, here are some simple commands:

#By default, a document is opened in read-only mode. To switch to a wrote mode:
i

#To set the mode to read only
Press the ESC key

#Delete a character
x

#Delete the whole line
dd


#Save a file
:w

#To quite vi
:q

#To save and quite
:wq

After some file editing, FreeBSD will give you a summary of what files will be changed. Just hit either ESC or q to end the list.

...
/usr/share/man/man3/lwres_addr_parse.3.gz
/usr/share/man/man3/lwres_buffer.3.gz
/usr/share/man/man3/lwres_buffer_add.3.gz
/usr/share/man/man3/lwres_buffer_back.3.gz
/usr/share/man/man3/lwres_buffer_clear.3.gz
/usr/share/man/man3/lwres_buffer_first.3.gz
/usr/share/man/man3/lwres_buffer_forward.3.gz
/usr/share/man/man3/lwres_buffer_getmem.3.gz

At this point, FreeBSD finish the upgrade plan. (It hasn’t made any change to your system.) Now we need to give a green light to FreeBSD to start upgrade the system.

sudo freebsd-update install

...Please reboot and run "/usr/sbin/freebsd-update install" again to finish installing updates.

After the reboot, run the command again to update the userland components:

sudo freebsd-update install


Installing updates...
Completing this upgrade requires removing old shared object files.
Please rebuild all installed 3rd party software (e.g., programs
installed from the ports tree) and then run "/usr/sbin/freebsd-update install"
again to finish installing updates.

Reboot the computer again to finish the upgrade.

sudo reboot

Now, we need to upgrade all of the ports. Notice that this step is optional. You don’t need to do it unless you found any problem after the upgrade.

#Upgrade all the installed ports
sudo portmaster -fa

Reboot the computer again to finish the upgrade.

sudo reboot

Make sure that FreeBSD is running on the latest version:

# uname -a
FreeBSD 9.1-RELEASE FreeBSD 9.1-RELEASE #0 r243825: Tue Dec  4 09:23:10 UTC 2012     [email protected]:/usr/obj/usr/src/sys/GENERIC  amd64

Have fun with FreeBSD.

–Derrick

Our sponsors:

[FreeBSD]MySQL server could not start

Posted on by

Today, I noticed that the MySQL server on my FreeBSD server is down for unknown reason, and today is Thanksgiving.

Initially, I tried to restart the MySQL server using the following command, but it didn’t help to restart the MySQL:

sudo /usr/local/etc/rc.d/mysql-server restart

and it gives me the following error message:

mysql not running? (check /var/db/mysql/icesquare.com.pid).

That’s interesting. So I check the pid file and I found that it is not available. After scratching my head for a while, I noticed one thing:

df

Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/ad4s1d    7.7G    7.7G     -1M   100%    /var

Obviously, the /var directory is full. No wonder why the MySQL could not create any file, which explains why it stopped working.

Since the server is already up and running, it will be too late to increase the size of /var without reinstalling the entire system. The simplest way is to soft link the /var directory to somewhere else. Since a lot of system applications depends on /var, I don’t want to move the entire /var into a different places. So I decide to softlink /var/log into /usr/var/log.

The steps are easy:

sudo su
mkdir -p /usr/var/
mv /var/log /usr/var/
ln -s /usr/var/log /var/

To verify your work, it should look something like that:

ls /var

lrwxr-xr-x   1 root    wheel       13B Nov 22 11:48 log -> /usr/var/log/

df

Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/ad4s1d    7.7G    1.1G    6.0G    15%    /var

Now, if you try to start the MySQL server again, it may or may not work. If you experience any difficulties, try to reboot the server. After the server is reboot, MySQL should be up and running again.

Lesson learned: Always create single partition. 🙂

–Derrick

Our sponsors: