[Solved]VLC Crashes when playing Real Media Movie

Every time when I open a real media movie using VLC (VideoLAN) player, and when I try to fast forward or drag the time to a certain position, it crashes. It is very annoying because VLC is one of very few movie players that support real media format, especially on non-Windows platform, such as OS X and Ubuntu Linux.

Initially, I thought VLC is going to fix this bug very soon. After doing some researches, I found that this bug was introduced in a production release back in 2010. Although some users filed bug reports, it seems that it was not addressed. At least this bug is still exist in the released version.

After doing more investigations, I found that this bug was already addressed in the nightly builds version. Nightly builds version is a beta version. It comes with latest features. However, these features can be non-stable and not well tested. Since VLC crashes very often when playing real media movie, I think it doesn’t hurt to give the nightly builds version a try. Surprisingly, the overall experience is very good, and the problem seems gone!

If you like to give it a try, please click the follow link to download the nightly builds version.

» Download VLC Nightly Builds (Windows 32-bit)
* Get the one that looks something like: vlc-1.3.0-git-xxxxxxxx-xxxx-win32.exe
» Download VLC Nightly Builds (Windows 64-bit)
* Get the one that looks something like: vlc-1.3.0-git-xxxxxxxx-xxxx-win64.exe
» Download VLC Nightly Builds (OS X Intel 10.6 or later 64-bits)
* Get the one that looks something like: 1.2-branch-intel64-xxxxxxxx-xxxx.zip
» Download VLC Nightly Builds (OS X Intel 10.6 or later 32-bits)
* Get the one that looks something like: 1.2-branch-intel-xxxxxxxx-xxxx.zip
» Download VLC Nightly Builds (OS X PowerPC 32-bit)
* Get the one that looks something like: vlc-git-powerpc-xxxxxxxx-xxxx.zip

Our sponsors:

How to keep the server running when the CPU fan is dead (and without buring the CPU)?

Few days ago, I found that the CPU fan of this server (i.e., icesquare.com) was having some troubles. It was spinning slowly and making some weird noise. After opening the computer case and had a brief investigation, I found that the fan needed to be replaced. Unlike the computer case fan, CPU fan works with CPU socket. Given that there are so many different sockets available, it is not likely that the local stores have each in stock. In the other words, I need to order the CPU fan online. At the mean time, I need to power down the server or it will kill the CPU (over heated).

There are several way to minimize the down time:

  1. Get a back up server, or
  2. Remove a working CPU fan from other server, or
  3. Build your own CPU fan

The first two solutions require powering down other servers, which is not a good idea to me. That’s why I picked the third one. In the last week, my DYI fan works pretty well as temporary CPU fan. So far the CPU temperature was very stable and without any problem.

Different CPU fan has different design, including the air flow direction. A typical CPU fan actually draws the cold air into the CPU, and push the hot air out of the CPU.

Notice that some CPU fans work in an opposite way. Instead of pushing the cold air, they pull out the hot air (kind of like a vacuum). The best way to determine the air flow is by checking the fan rotation direction. After that, simply verify it with your right hand:


Based on this factor, it is possible to add a working computer fan on top of the broken CPU fan, as long as the air flow direction remains unchanged. In my situation, I used a regular computer case fan with similar dimension. i.e.,

Of course, you will need to power off the defective CPU fan first. Here is the final setup. You can use any regular plastic wrapped metal cable wire to secure the fan.

Although the air flow direction remains unchanged, the power of the air flow will be reduced because part of the energy is used to rotate the defective CPU fan instead of pushing the air to the CPU. In short, the cooling performance will be reduced and the CPU temperature will get increased. Therefore, it is a good idea to add more computer case fans to make up the difference. In my situation, originally I had a rear fan in additional to my CPU fan. Therefore I added one additional fan (One on rear, and one on the side). Keep in mind that you need to keep the computer case closed. Opening the case will decrease the cooling performance.

This setup had run for about a week at room temperature. During this period, I didn’t notice any temperature rise. Both system and CPU temperatures were below the warning threshold.

By using this solution, it is possible to keep the server running and keep your CPU happy. 🙂

FYI, in terms of the cooling performance, it is more efficient to push the cool air into the system than pulling the hot air out from the system in most cases.

Enjoy.

–Derrick

Our sponsors:

[FreeBSD-update]Installing updates…install: ///usr/src/lib/libc/gen/libc_dlopen.c: No such file or directory

When I tried to upgrade my FreeBSD system today, I received a last minute Christmas gift from the FreeBSD team. 🙂

#sudo freebsd-update fetch
Looking up update.FreeBSD.org mirrors... 4 mirrors found.
Fetching metadata signature for 8.2-RELEASE from update4.FreeBSD.org... done.
Fetching metadata index... done.
Inspecting system... done.
Preparing to download files... done.

The following files will be added as part of updating to 8.2-RELEASE-p5:
/usr/src/lib/libc/gen/libc_dlopen.c

I got the following error:

#sudo freebsd-update install
Installing updates...install: ///usr/src/lib/libc/gen/libc_dlopen.c: No such file or directory
 done.

Initially, I thought the problem was a typo in the freebsd-update profile, so I gave this a try:

sudo cat /var/db/freebsd-update/* | grep libc_dlopen.c

Unfortunately, I couldn’t find anything. So I investigated the problem, and I found that it was caused by the missing source. In a nut shell, if you did not install the FreeBSD with the source (i.e., /usr/src is empty), this problem will show up.

To fix this problem, simply create a dummy directory:

sudo mkdir -p /usr/src/lib/libc/gen

and re-run everything once again, i.e.,

#sudo freebsd-update fetch
#sudo freebsd-update install
#sudo reboot

After the computer finishes rebooting, verify that the upgrade is done:

# sudo freebsd-update fetch
Looking up update.FreeBSD.org mirrors... 4 mirrors found.
Fetching metadata signature for 8.2-RELEASE from update2.FreeBSD.org... done.
Fetching metadata index... done.
Inspecting system... done.
Preparing to download files... done.

No updates needed to update system to 8.2-RELEASE-p5.

That’s it!

Enjoy this last minute Christmas gift from the FreeBSD team. Merry Christmas! 🙂

–Derrick

Our sponsors:

How to Delete System Mail / Empty System Mail box in FreeBSD / Linux?

When I checked my FreeBSD system mail box today, I found that there were 8000 system mails sitting my mail box:

mail
Mail version 8.1 6/6/93.  Type ? for help.
"/var/mail/root": 8182 messages 8182 new
>N  1 [email protected]  Fri Sep 30 03:02  44/1330  "x.com output"
 N  2 [email protected]  Fri Sep 30 03:02  72/2464  "x.com output"
 N  3 [email protected]  Sat Oct  1 03:01  35/1075  "x.com output"
...
Another 8000 lines go here

There are several ways you can do. Either you can go through them one by one and delete them one at a time, i.e.,

d
[Hit the ENTER Key]

Or specify which mails you like to delete by ID, e.g.,

d x-y
[Hit the ENTER Key]

where x and y are the start and end of the mail ID, respectively.

For example, if you want to empty your FreeBSD mail box, you can do the following:

d 1-8182
[Hit the ENTER Key]

where 8182 is the number of mails I have.

You can find the number of your emails when you open your mail box:

"/var/mail/root": 8182 messages 8182 new

Hit the enter key again to confirm that the mail box is empty:

At EOF

And leave the mail box.

q

Now, try to access your mail box again, and the system should tell you that you have no system mail.

# mail
No mail for root

That’s it.

How to stop generating the system mails

Whenever the system run a background job, it will store the output of the program and email to the system mail box for auditing purposes. However, you can disable this feature if you don’t care the output.

Normally, we run a script this way:

start.sh

To avoid any output, we do the following instead:

start.sh > /dev/null 2>&1

For example, if you run many cron jobs, you need to go over your cron job list and add this to every single script, i.e.,

sudo nano /etc/crontab
or
crontab -e

and add /dev/null 2>&1 to every command.

Hope it helps.

–Derrick

Our sponsors:

Can’t connect to local MySQL server through socket ‘/var/lib/mysql/mysql.sock’ (2)

When I tried to connect to the MySQL server through PHP, I got the following error: Can’t connect to local MySQL server through socket ‘/var/lib/mysql/mysql.sock’ (2)

To give you some background, here is my situation. My Fedora Linux is a 64-bit machine, and I have installed a 32-bit version of XAMPP (i.e., 32-bit PHP, 32-bit MySQL) for the web server. Also, I have some PHP scripts that will be run in background. These scripts are run using 64-bit PHP. Both 32-bit and 64-bit PHP need to access the MySQL database. Long story short. I got the MySQL error message when I tried to execute using the 64-bit PHP, while the 32-bit PHP has no complain.

When PHP talks to MySQL, it needs to talk to the MySQL representative, i.e., mysql.sock. If it cannot talk to the mysql.sock, then PHP will be unable to access the MySQL database.

The reason why my 64-bit PHP could not talk to MySQL because of a wrong location of mysql.sock. To solve this problem, simply do the following:

First, let’s find out where is the PHP.ini of the 64-bit PHP:

sudo find / -name "php.ini"

In Fedora Linux, it is:

/etc/php.ini

Then, we need to find out where is the mysql.sock:

sudo find / -name "mysql.sock"

In my case, it is:

/opt/lampp/var/mysql/mysql.sock

Now, let’s tell PHP the location of the mysql.sock:

sudo nano /etc/php.ini

And change the following line from:

mysql.default_socket =

to:

mysql.default_socket = /opt/lampp/var/mysql/mysql.sock

Now, try to run your script again. The problem should be gone.

Have fun with MySQL.

–Derrick

Our sponsors:

ZFS+USB: Building a Super Large Server Using USB Memory, CF Card and SD Card.

I got a lot of unused USB thumb drives, CF flash card and SD cards sitting in my drawer. The sizes range from 8MB to 8GB. Unlike few years ago, it is getting a lot easier to access to the Internet, so I no longer need to carry my data via memory device any more. Instead, I simply connect to the Internet and the data is with me. That’s why my USB thumb drivers / CF flash cards / SD cards have been sitting in my drawer for few years.

I got an idea one day. It would be a waste to let them sitting in my drawer (or waiting to be sent to landfill). Why not I use them to build a file server. At least I can test out whether the idea is doable or not. So here are the candidates:

  • USB Thumb drives

    1. 8GB x 2
    2. 1GB x 1
    3. 256MB x 2
  • CF Flash Card

    1. 2GB x 1
    2. 1GB x 1
    3. 512MB x 2
  • SD Card

    1. 4GB x 2

As you can see from my list, the size of each candidates varies from 256MB to 8GB. So it will be interesting to put them together and build a super large file server.

Most computer has multiple USB ports available. If you don’t have enough USB port, get a powered USB hub (i.e., the USB hub has it own power unit), it will be more efficient then getting the power from the computer. For the CF card, I use a SYBA SY-PCI48001 PCI to Compact Flash Adapter to connect my CF Flash cards to my computer. For the SD cards, I simply connect each of them to a Sandisk USB SD card reader.

Okay, let’s talk about the software. I am going to use ZFS to implement it, because it is quick and simple. First, connect all devices to your computer, and make sure that your operating systems can recognize all of them. In this tutorial, I am using FreeBSD as a tutorial. However, the idea should be the same in other ZFS ready system, such as Solaris.

Make sure that all USB devices are recognized by your operating system. In FreeBSD, the devices are registered as /dev/da* or /dev/ad*:

dmesg | egrep 'ad|da'

Now, you need to think about how to group your devices together. Do you simply want to build a pure USB ZFS pool, or a hybrid hard drive/USB pool. To keep thing simple, I will go with pure USB ZFS pool.

Suppose I am going to create a pure USB pool, which simply include every device in one single place:

zpool create myzpool /dev/ad0 /dev/ad1 /dev/ad2 /dev/da0 /dev/da1 /dev/da2

where the ad* and da* are the locations of my devices.

This will create a big pool. When you write some data to this pool, e.g.,

sudo dd if=/dev/random of=/myzpool/test_file count=10g bs=1M

The system will simply split the file into multiple chunks, and write all chunks to each USB devices at the same time.

Now let’s verify the pool information:

zpool iostat -v
               capacity     operations    bandwidth
pool         used  avail   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
myzpool
  ad12       969M   112K      0      0  1.15K  66.4K
  ad13      1.90G   112K      0      0  1.74K  66.4K
  ad14       480M    11K      0      0  3.08K  66.9K
  da0          1G  2.78G      0      0  5.55K  66.1K
  da1          1G  2.78G      0      0  5.55K  66.1K
  da2        240M    80K      0      0  2.41K  66.8K
  da3        240M   112K      0      0  1.87K  66.8K
  da4       7.50G    80K      0      0  4.35K  66.5K
  da5       7.50G    96K      0      0  2.98K  66.3K
  da6        972M   112K      0      0    278  39.9K
----------  -----  -----  -----  -----  -----  -----

As you can see, the system is split the data and write it to each devices. ZFS is very smart to adjust the number of split to optimize the performance.

Okay, what about the performance? Honestly you can’t expect too much from a pure-USB zpool, because the write speed is limited to 40MB/s, which is way too slow compared to the disk. The only advantage is that there is no moving parts, which significant decrease the failure rate, and the overall cost is cheap. Now, let’s make talk about the hybrid pool, a combination of USB and hard drive pool.

A hybrid ZFS pool is a combination of hard drives and USB drives. In my experiment, I put the USB devices as log and cache devices, while the hard drives are used as main storage. If you don’t know what is ZFS log or ZFS cache, you can think about a ZFS log devices is a buffer for writing the data, while a ZFS cache is for reading the data.

Ideally, you should use two identical devices (same size) for ZFS log (writing the data). For ZFS cache, it doesn’t matter.

First, let’s create our ZFS pool with the storage devices (i.e., hard drives) only.

zpool create myzpool raidz /dev/ad0 /dev/ad1 /dev/ad2

Next, we need to add the ZFS log. We are going to create a mirror, so that they have to be identical.

zpool add myzpool log mirror /dev/da0 /dev/da1

Finally, we add the ZFS cache.

zpool add myzpool cache /dev/da2 /dev/da3 /dev/da4

And let’s take a look to the whole picture:

zpool iostat -v
               capacity     operations    bandwidth
pool         used  avail   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
myzpool     5.83T  6.79T      9      2  1.03M   311K
  raidz1    5.83T  6.79T      9      1  1.03M   179K
    ad0         -      -      2      0   171K  29.9K
    ad1         -      -      2      0   171K  29.9K
    ad2         -      -      2      0   171K  29.9K
    ad3         -      -      2      0   171K  29.9K
    ad4         -      -      2      0   171K  29.9K
    ad5         -      -      2      0   171K  29.9K
    ad6         -      -      2      0   171K  29.9K
  da0        128K  3.78G      0      0      0  65.9K
  da1           -      -      0      0      0  65.9K
cache           -      -      -      -      -      -
  da2        961M     8M      0      0  1.14K  66.6K
  da3       1.89G     8M      0      0  1.73K  66.6K
  da4        472M     8M      0      0  3.06K  67.0K
  da5        232M     8M      0      0  2.40K  66.9K
  da6        232M     8M      0      0  1.86K  66.9K
  da7       7.50G     8M      0      0  4.33K  66.6K
  da8       7.50G     8M      0      0  2.96K  66.5K
  da9        964M     8M      0      0    276  39.6K
----------  -----  -----  -----  -----  -----  -----

With this combination, I get a pretty good performance (both read/write). When I copy the data from Windows to this ZFS pool using Samba, I can get a pretty high transfer speed (Over 100MB/s). Sometimes it get even close to 110MB/s. This result is very amazing given that my hard drives are the standard SATA drives (non-SSD) only.

The reliability of the USB devices / CF card / SD card sometimes questionable. That’s one of the reason why I don’t use them as the permanent storage media (using as Cache / log is okay). In this design, I use two SD cards (4GB x 2 = 8GB) as the ZFS log devices. Since they are set up as mirror, if one dies, the other one will kick in, which will minimize the data lost. For the cache devices, if one device is failed, I can remove it from the ZFS pool at any time. There will be no data lost so it will be okay.

I have run this super large server for few months already. There is about 200GB data I/O everyday, so far I am very happy with the overall performance. The most important thing is, those unused memory devices are now very happy as they don’t need to be sent to landfill.

Our sponsors:

[Solved]TFTP: Error code 2: Access violation

This post is an extension of my original post on TFTP: How to Setup TFTP. Although the post was written for Ubuntu, the same idea can be applied to all systems.

So, here is our problem. You set up a TFTP server. It starts fine. You upload a file, it gives no error. Now, you try to download a file, it throws you an error:

tftp> get myfile.jpg
Error code 2: Access violation

If you google the error message, Error code 2: Access violation on the web, you’ve probably noticed many solutions, some of them are very creative, such as running a touch command to create the file before downloading it etc. Before you try those solutions, ask yourself this question first: Do you want to touch 100 files before you download 100 files?

Anyway. I am not sure what cause the TFTP gives the error message: Error code 2: Access violation. However I never experienced this error in my system before with my configurations and settings. Therefore, I am going to share you my configurations for your reference.

My TFTP Configuration:

service tftp
{
   protocol = udp
   port = 69
   socket_type = dgram
   wait = yes
   user = nobody
   server = /usr/sbin/in.tftpd
   server_args = var/lib/tftpboot -s
   disable = no
}

There are couple things you need to pay attention. First, my TFTP root directory is /var/lib/tftpboot, and in my configuration file, I skip the first /. Second, I put a flag, -s in the server_args. That helps to keep TFTP from complain.


Next, the permission of the TFTP root directory plays an important role. Make sure that the permission and ownership are set correctly.

sudo chown -R nobody:nobody /var/lib/tftpboot
sudo chmo7 -R 777 /var/lib/tftpboot

Now, you can simply restart the system:

sudo service xinetd stop
sudo service xinetd start

and try to put a file and download it back:

tftp localhost
tftp> put myfile.jpg
Sent 56733279 bytes in 5.7 seconds

tftp> get myfile.jpg
Received 56733279 bytes in 5.7 seconds

That’s it.

Happy TFTP.

–Derrick

Our sponsors:

Natural Sort. MySQL. Sorting a 2D Array By Multiple Columns in PHP Naturally. Array_MultiSort with Natural Sort

MySQL has a biggest missing feature: It does not sort naturally. This has been an opened request since 2003, and MySQL team has no plan to fix this in a near future. I have looked for a solution to solve this problem for a while. Unfortunately, I found nothing useful. Here is a summary:

The + 0 Trick

Many developers suggest this solution:

SELECT names FROM your_table ORDER BY age + 0 ASC

This is not a good solution because it only works in some very specific situations. This query fails in most general cases.

Drupal Patch

Some developers suggest to use the patch from Drupal. Forget it, I could not even install it in my system.

My solution: Let PHP to handle the dirty work

So I decide to move the process to PHP. Before you say something about the performance issue, please keep reading my story first. I will talk about it in the later section.

MySQL is a relational database, it stores everything in a table. If we express that in an array, it will look something like this:

Array
(
    [1] => Array
        (
            [Name] => John
            [Identification] => 10
            [Address] => 100 Michigan Ave.
        )

    [2] => Array
        (
            [Name] => Peter
            [Identification] => 1000
            [Address] => 1 Michigan Ave.
        )

    [3] => Array
        (
            [Name] => Tom
            [Identification] => 2
            [Address] => 10 Michigan Ave.
        )

    [4] => Array
        (
            [Name] => Paul
            [Identification] => 1
            [Address] => 10 Michigan Ave.
        )

)

where the array key represents the record ID in the table.

Now, let say I want to sort the contents in the following orders:

  1. Sort by Identification: Ascending
  2. Then by Address: Descending
  3. Then by Name: Ascending

I specify the orders in an array:

$order = array('Identification' => 'ASC',
			   'Address' => 'DESC',
			   'Name' => 'ASC');

And include the following function in my code:

//array_multiSort with natural sort
function naturalSort2D(&$array){

   if (!function_exists(naturalSort2DCompare)){
	function naturalSort2DCompare($a, $b){
		global $order;

		foreach($order as $key => $value){
			
			if (!isset($a[$key])) continue;

			unset($compareResult);

                        //Case insensitive string comparisons using a "natural order" algorithm
			$compareResult = strnatcasecmp($a[$key], $b[$key]);
			
			if ($compareResult === 0) continue;
			
			$value = strtoupper(trim($value));
			
			if ($value === 'DESC'){
				$compareResult = $compareResult*-1;
			}
			
			return $compareResult;
		}
		
		return 0;
	}
   }
	
   //Maintain index association
   //Preserve array keys
   //Use usort if you don't care about key index
   uasort($array, 'naturalSort2DCompare');
	
   return true;
}

Now I sort the array:

naturalSort2D($array);

And I get my result like the following:

Array
(
    [4] => Array
        (
            [Name] => Paul
            [Identification] => 1
            [Address] => 10 Michigan Ave.
        )

    [3] => Array
        (
            [Name] => Tom
            [Identification] => 2
            [Address] => 10 Michigan Ave.
        )

    [1] => Array
        (
            [Name] => John
            [Identification] => 10
            [Address] => 100 Michigan Ave.
        )

    [2] => Array
        (
            [Name] => Peter
            [Identification] => 1000
            [Address] => 1 Michigan Ave.
        )

)

Performance

Before we talk about the performance issue, I like to talk about my database design first. In my application, the records are stored in a table. It is a multiple user system. Each user can access their own records only. Each user has about 3000 records, and we have 1000 users. Therefore we have about 3 millions of records in one table.

Here is a data structure:

#Information input by users
Name
Identification
Address

#Information input by system
ID
UserID

So the SQL quere is nothing more than something like:

Select * FROM `database`.`table` WHERE `UserID` = '1'

Previously, I let MySQL to handle the filter and sorting, i.e.,

Select * FROM `database`.`table` WHERE (`UserID` = '1') AND (`Address` LIKE `%Michigan%`) ORDER BY `Identification` ASC LIMIT 5, 10

After moving to PHP, I just let MySQL to handle the filter,

Select * FROM `database`.`table` WHERE (`UserID` = '1') AND (`Address` LIKE `%Michigan%`)

Notice that I will get all qualified records, could be few thousands. Then I let PHP to do the sorting:

$order = array('Identification' => 'ASC');
naturalSort2D($array);

Now I have a sorted array. Next I need to slice the records:

$array = array_slice($array, 5, 10);

Sounds like lots of work huh? Now let’s talk about the performance impact. Guess what, I see performance improvement for about 15%!

In the new approach, MySQL does lesser work. PHP does more work. Apparently, PHP works more effectively than MySQL. However, this may not true for all database model. Remember that in my situation, the system needs to handle at most few thousands records at a time? So I think PHP is doing better in this range. If your application needs to handle millions of records at a time, e.g., bookstore, then you better let MySQL to do the work.

By the way, the new approach allows to sort the dynamic generated content, which is a big plus in my situation. That is something I cannot do in MySQL approach.

Have fun.

–Derrick

Our sponsors:

[PHP]How to get the number of CPU cores in Fedora, Ubuntu, and FreeBSD

I am working on a PHP application which needs to be deploy to multiple places. One of the variables is the number of CPU core. Since each server has different number of CPU cores, I need to specify the number of CPU core in my application for each server. I think there should be a smarter way to do it.

PHP is a scripting language, it has limit support on accessing the hardware level information. In short, there is no library or function to do it. Fortunately, we can do it via shell command. In the other words, the following methods are not limited to PHP, it will work in any languages, as long as your language supports running the UNIX command and catch the return.

Getting the number of CPU Cores – Linux

(Tested on Fedora, Ubuntu Linux, should work on other Linuxs because they all use the same Linux kernel.)

cat /proc/cpuinfo | grep processor | wc -l

This will return something like this:

8

Getting the number of CPU Cores – FreeBSD

sysctl -a | grep 'hw.ncpu' | cut -d ':' -f2

which will return something like this (notice the extra space before the number):

8

Now, let’s put everything together. Run the command inside your application (Here I am using PHP for example):

//Linux
$cmd = "cat /proc/cpuinfo | grep processor | wc -l";

//FreeBSD
$cmd = "sysctl -a | grep 'hw.ncpu' | cut -d ':' -f2";

$cpuCoreNo = intval(trim(shell_exec($cmd)));

Of course, you can make the application to detect the system automatically:

$cmd = "uname";
$OS = strtolower(trim(shell_exec($cmd)));

switch($OS){
   case('linux'):
      $cmd = "cat /proc/cpuinfo | grep processor | wc -l";
      break;

   case('freebsd'):
      $cmd = "sysctl -a | grep 'hw.ncpu' | cut -d ':' -f2";
      break;

   default:
      unset($cmd);
}

if ($cmd != ''){
   $cpuCoreNo = intval(trim(shell_exec($cmd)));
}

That’s it! Happy PHPing.

–Derrick

Our sponsors: