[FreeBSD]Portsnap / gunzip: can’t stat: files/… .gz: No such file or directory

My FreeBSD got an unknown fever today. When I updated my ports using portsnap, it gave the following error message:

#sudo portsnap fetch
Looking up portsnap.FreeBSD.org mirrors... 6 mirrors found.
Fetching snapshot tag from your-org.portsnap.freebsd.org... done.
Fetching snapshot metadata... done.
Updating from Thu May 23 09:08:53 CDT 2013 to Thu May 23 09:25:25 CDT 2013.
Fetching 0 metadata patches. done.
Applying metadata patches... done.
Fetching 0 metadata files... done.
gunzip: can't stat: files/992a1325cdc9a00a3543aa38fdf58903cdf70eaee02b8bb8aebea5505ac7b3f8.gz: No such file or directory
Fetching 0 patches. done.
Applying patches... done.
Fetching 0 new ports or files... done.
Building new INDEX files... gunzip: can't stat: /var/db/portsnap/files/09f65f8a730283fd31d068a5927ed46d95e37540f89090c257d7809b75116293.gz: No such file or directory
gunzip: can't stat: /var/db/portsnap/files/e3d3219617c1ea87cdfac7c8df0a52d611b191be8a80fd97f511277dff4cce77.gz: No such file or directory
gunzip: can't stat: /var/db/portsnap/files/8c2576279258f0d1b8762df8fc1e0cb4bcfcd23b6b09cdb4e7d68886af35ed7d.gz: No such file or directory
done.

Apparently, something in /var/db/portsnap/ is broken. Many people will try to remove /var/db/portsnap/ and run the command again. Do not do it. It will make portsnap failed. Instead, do the following:

sudo cp -r /var/db/portsnap /var/db/portsnap_backup
sudo rm -Rf /var/db/portsnap/tag /var/db/portsnap/files/*
sudo portsnap fetch extract
sudo portsnap update

Now your portsnap should be happy.

–Derrick

Our sponsors:

[Solved]FreeBSD: Problem to Update glib20

I decided to install Java on my FreeBSD box today. It wasn’t a very good experience. Primary the system is not automated. It requires a lot of manual works.

Anyway, after waiting for couple hours, I found that the process was stuck on glib. The system could not build the /usr/ports/devel/glib20. Here is the error message:

===>  Building for glib-2.34.3
gmake  all-recursive
gmake[1]: Entering directory `/usr/ports/devel/glib20/work/glib-2.34.3'
Making all in .
gmake[2]: Entering directory `/usr/ports/devel/glib20/work/glib-2.34.3'
gmake[2]: Nothing to be done for `all-am'.
gmake[2]: Leaving directory `/usr/ports/devel/glib20/work/glib-2.34.3'
Making all in m4macros
gmake[2]: Entering directory `/usr/ports/devel/glib20/work/glib-2.34.3/m4macros'
gmake[2]: Nothing to be done for `all'.
gmake[2]: Leaving directory `/usr/ports/devel/glib20/work/glib-2.34.3/m4macros'
Making all in glib
gmake[2]: Entering directory `/usr/ports/devel/glib20/work/glib-2.34.3/glib'
gmake  all-recursive
gmake[3]: Entering directory `/usr/ports/devel/glib20/work/glib-2.34.3/glib'
Making all in libcharset
gmake[4]: Entering directory `/usr/ports/devel/glib20/work/glib-2.34.3/glib/libcharset'
gmake[4]: Nothing to be done for `all'.
gmake[4]: Leaving directory `/usr/ports/devel/glib20/work/glib-2.34.3/glib/libcharset'
Making all in update-pcre
gmake[4]: Entering directory `/usr/ports/devel/glib20/work/glib-2.34.3/glib/update-pcre'
gmake[4]: Nothing to be done for `all'.
gmake[4]: Leaving directory `/usr/ports/devel/glib20/work/glib-2.34.3/glib/update-pcre'
Making all in .
gmake[4]: Entering directory `/usr/ports/devel/glib20/work/glib-2.34.3/glib'
  CC       gstrfuncs.lo
  CC       gthreadpool.lo
gstrfuncs.c:330: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'get_C_locale'
gstrfuncs.c: In function 'g_ascii_strtod':
gstrfuncs.c:700: warning: implicit declaration of function 'strtod_l'
gstrfuncs.c:700: warning: implicit declaration of function 'get_C_locale'
gstrfuncs.c: In function 'g_ascii_formatd':
gstrfuncs.c:902: error: 'locale_t' undeclared (first use in this function)
gstrfuncs.c:902: error: (Each undeclared identifier is reported only once
gstrfuncs.c:902: error: for each function it appears in.)
gstrfuncs.c:902: error: expected ';' before 'old_locale'
gstrfuncs.c:904: error: 'old_locale' undeclared (first use in this function)
gstrfuncs.c:904: warning: implicit declaration of function 'uselocale'
gstrfuncs.c: In function 'g_ascii_strtoull':
gstrfuncs.c:1148: warning: implicit declaration of function 'strtoull_l'
gstrfuncs.c: In function 'g_ascii_strtoll':
gstrfuncs.c:1195: warning: implicit declaration of function 'strtoll_l'
gmake[4]: *** [gstrfuncs.lo] Error 1
gmake[4]: *** Waiting for unfinished jobs....
gmake[4]: Leaving directory `/usr/ports/devel/glib20/work/glib-2.34.3/glib'
gmake[3]: *** [all-recursive] Error 1
gmake[3]: Leaving directory `/usr/ports/devel/glib20/work/glib-2.34.3/glib'
gmake[2]: *** [all] Error 2
gmake[2]: Leaving directory `/usr/ports/devel/glib20/work/glib-2.34.3/glib'
gmake[1]: *** [all-recursive] Error 1
gmake[1]: Leaving directory `/usr/ports/devel/glib20/work/glib-2.34.3'
gmake: *** [all] Error 2
*** Error code 1

Stop in /usr/ports/devel/glib20.
*** Error code 1

Stop in /usr/ports/devel/glib20.

This is not a popular problem. After I google a while, I found a very similar problem here. Basically, the author suggests that the problem may be caused by an uncleaned run of freebsd-update. In short, you need to run the program twice, i.e.,

sudo freebsd-update fetch install
sudo reboot
sudo freebsd-update install

Obviously, it didn’t work for me. So I decided to try my last solution: pkg_add

sudo pkg_add -r glib20

and I tried to resume the installation:

cd /usr/ports/java/jdk16
sudo make install

It worked!

Hope this little trick is helpful to you.

–Derrick

Our sponsors:

[FreeBSD/Linux]How To Remove ZFS Meta Data

I have many hard drives circulating among my servers for testing purpose. For example, I took a hard drive from one server and put it on a different server. After doing this for many times, I’ve noticed that the ZFS has put many header information / meta data left on my hard drive. While it does not do anything harmful to the normal ZFS operation, I think it is not a good idea to have some outdated information living on my hard drive.

Here is an example:

#sudo zpool import

  pool: storage
    id: 4394681882400895515
 state: UNAVAIL
status: The pool was last accessed by another system.
action: The pool cannot be imported due to damaged devices or data.
   see: http://www.sun.com/msg/ZFS-8000-EY
config:

        storage                   UNAVAIL  insufficient replicas
          raidz1-0                UNAVAIL  insufficient replicas
            12688516256739208392  UNAVAIL  cannot open
            ada3                  ONLINE
            4218969245912188584   UNAVAIL  cannot open
            1537006695366032450   UNAVAIL  cannot open
            8194123525800888894   UNAVAIL  cannot open
            13778624724471040977  UNAVAIL  cannot open

  pool: storage
    id: 12159013771499288095
 state: FAULTED
status: One or more devices contains corrupted data.
action: The pool cannot be imported due to damaged devices or data.
        The pool may be active on another system, but can be imported using
        the '-f' flag.
   see: http://www.sun.com/msg/ZFS-8000-5E
config:

        storage                FAULTED  corrupted data
          6113585248511400089  UNAVAIL  corrupted data

Although ZFS provides a way (zpool labelclear) to remove this information, it only works if the hard drive is still attached to the server. If the hard drive is missing, there is nothing you can do. For example, the following command will be failed:

#sudo zpool labelclear -f ada3

I googled for solutions and I found many idea. Unfortunately, none of them works.

Anyway, I came up a solution that is quick, easy and simple. Since ZFS stores the header information in the first and last sector of the hard drive, all I need to do is to wipe out the first and the last sector. That’s it.

How To Remove ZFS Meta Data – Linux

#replace /dev/sdXX with that actual ID of your hard drive
dd if=/dev/zero of=/dev/sdXX bs=512 count=10
dd if=/dev/zero of=/dev/sdXX bs=512 seek=$(( $(blockdev --getsz /dev/sdXX) - 4096 )) count=1M

That’s it for Linux. Below is the FreeBSD version.

How To Remove ZFS Meta Data – FreeBSD

First, you will need to identify which hard drive you want to clean up. The easiest way is to use dmesg

#dmesg


or 

#dmesg | grep ada | grep MB | grep -v 'MB/s'


which will return something like this:


ada0: 381554MB (781422768 512 byte sectors: 16H 63S/T 16383C)
ada1: 1430799MB (2930277168 512 byte sectors: 16H 63S/T 16383C)
ada2: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C)
ada3: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C)
ada4: 1430799MB (2930277168 512 byte sectors: 16H 63S/T 16383C)

Next, I need to know which one is the system hard drive, something I don’t want to touch.

#df

/dev/ada0p2    362G    3.6G    329G     1%    /
devfs          1.0k    1.0k      0B   100%    /dev

In this example, my goal is very clear. I need to wipe clear ada1, ada2, ada3 and ada4, and leave ada0 untouched.

Next, I need to clear the first sector:

#sudo dd if=/dev/zero of=/dev/ada1 count=1 bs=512k

repeat this for the other hard drives

Next, I need to clear the last sector. You can use the sector information from dmesg (if available), or you can use the following command to find the location of the last sector:

#sudo diskinfo -c /dev/ada1

which will return something like the following:

/dev/ada1
        512             # sectorsize
        1500301910016   # mediasize in bytes (1.4T)
        2930277168      # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        2907021         # Cylinders according to firmware.
        16              # Heads according to firmware.
        63              # Sectors according to firmware.
        S1Y6J1KS710613  # Disk ident.

I/O command overhead:
        time to read 10MB block      0.091942 sec       =    0.004 msec/sector
        time to read 20480 sectors   1.945038 sec       =    0.095 msec/sector
        calculated command overhead                     =    0.090 msec/sector

In this example, the total number of sector is 2930277168 (mediasize in sectors).

To keep things simple, I am going to wipe out the hard drive from 2930270000 to the end (Replace the last four digits of the sector size to zero).

#sudo dd if=/dev/zero of=/dev/ada1 oseek=293027000

Now, repeat the same thing for each hard drive. Keep in mind that the sector size of each hard drive may not be the same. So it is better to run the command and get the sector information first.

After running these commands, the ZFS meta information should be removed. You can verify your work by doing this:

#sudo zfs import


which should output nothing.

That’s it. Enjoy building a new ZFS!

–Derrick

Our sponsors: