One or more devices has experienced an error resulting in data corruption. Applications may be affected.<\/strong>. There can be million reasons to cause this error message showing up. Of course, 99% of them are caused by hardware failure, such as bad hard drives, broken cables, defective motherboard, or even bad memory. In this article, I am assuming that you already eliminated these possibilities, and have scratched your head for hours, and still have no clue which part went wrong. In fact, that’s what I did today.<\/p>\n\nLong story short. Here is how I experienced this error:\n<\/p>\n
\r\nFreeBSD: 8.2-> 9.0\r\nZFS: 4 -> 5\r\nZPool: 15 -> 28\r\n<\/pre>\nMy system was working fine (FreeBSD 8.2; ZFS ver. 4, Zpool ver. 15), everything seems perfect. After I upgraded my system to FreeBSD 9, I upgraded the ZFS and Zpool to ver.5 and ver. 28, respectively. Everything seemed working fine until I check the status:\n<\/p>\n
\r\nsudo zpool status -v\r\n<\/pre>\n\r\n#sudo zpool status -v\r\n pool: storage\r\n state: ONLINE\r\nstatus: One or more devices has experienced an error resulting in data\r\n corruption. Applications may be affected.\r\naction: Restore the file in question if possible. Otherwise restore the\r\n entire pool from backup.\r\n see: http:\/\/www.sun.com\/msg\/ZFS-8000-8A\r\nconfig:\r\n\r\n NAME STATE READ WRITE CKSUM\r\n storage ONLINE 0 0 0\r\n raidz1-0 ONLINE 0 0 0\r\n ada0 ONLINE 0 0 0\r\n ada2 ONLINE 0 0 0\r\n ada3 ONLINE 0 0 0\r\n ada4 ONLINE 0 0 0\r\n ada5 ONLINE 0 0 0\r\n ada6 ONLINE 0 0 0\r\n ada7 ONLINE 0 0 0\r\n\r\nerrors: Permanent errors have been detected in the following files:\r\n\r\n storage\/data:<0x0>\r\n<\/pre>\nThere are few things you need to pay attention:<\/p>\n
The pool seems working fine, otherwise you will see FAULTED instead of ONLINE:<\/p>\n
\r\nstate: ONLINE\r\n<\/pre>\n<\/p>\n
The system has no problem to read\/write the data. Doesn’t seem to be hardware problem:<\/p>\n
\r\nNAME STATE READ WRITE CKSUM\r\n storage ONLINE 0 0 0\r\n raidz1-0 ONLINE 0 0 0\r\n ada0 ONLINE 0 0 0\r\n ada2 ONLINE 0 0 0\r\n ada3 ONLINE 0 0 0\r\n ada4 ONLINE 0 0 0\r\n ada5 ONLINE 0 0 0\r\n ada6 ONLINE 0 0 0\r\n ada7 ONLINE 0 0 0\r\n<\/pre>\nThis error message may give you some clue what’s wrong. Notice that storage and data are the pool names.<\/p>\n
\r\nerrors: Permanent errors have been detected in the following files:\r\n\r\n storage\/data:<0x0>\r\n<\/pre>\n\nThe <0x0> represents the meta data of the pool. I think the problem may come from the upgrade process. Here are the steps how to solve this problem.\n<\/p>\n
Force Clearing the Error<\/h3>\n
You can try to clear the error by running:<\/p>\n
\r\nsudo zpool clear -F mypool\r\n<\/pre>\nIf it can clear the error, you are done. However, it is likely that it won’t work, and you need to move to the next step.<\/p>\n
Scrubbing the Pool<\/h3>\n
You can try to scrub the entire pool by running:<\/p>\n
\r\nsudo zpool scrub mypool\r\n<\/pre>\n<\/div>\n
This will make the system to inspect every single block and correct the error. Although this process is long (It took 5 hours to inspect my 10TB pool), there is a very high chance that the problem will be corrected. Don’t forget to clear error after scrubbing the pool.<\/p>\n
Making each devices online again<\/h3>\n
If the error still exists after scrubbing the entire pool (and clearing the error), you can try to force making each device go online:<\/p>\n
\r\nsudo zpool online mypool \/dev\/ada0 \/dev\/ada2 \/dev\/ad4 ...\r\n<\/pre>\nTry to reboot the computer<\/h3>\n
This is the last thing you can try. This will force the computer to mount the pool again. Hopefully it will clear the error and error status.<\/p>\n
Rebuild the pool<\/h3>\n
If none of the method works, the only solution left is to rebuild the pool.<\/p>\n
\r\n#Backup your data first\r\n#sudo zpool destroy mypool\r\n#sudo zpool create mypool ...\r\n<\/pre>\nGood luck.<\/p>\n
<\/p>\n","protected":false},"excerpt":{"rendered":"
When you check the ZFS status, you may find the following error message: One or more devices has experienced an error resulting in data corruption. Applications may be affected.. There Continue Reading →<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_hide_featured_image":false,"_jetpack_featured_image_duplicate_dismissed":0,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[94,7,144,6,69],"tags":[171,177],"class_list":["post-2562","post","type-post","status-publish","format-standard","hentry","category-computer-problems-computer-help-pc-support","category-freebsd","category-sever-problem","category-technology","category-troubleshooting-technology","tag-freebsd","tag-zfs"],"jetpack_publicize_connections":[],"jetpack_shortlink":"https:\/\/wp.me\/p1sqzr-Fk","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":1380,"url":"https:\/\/icesquare.com\/wordpress\/how-to-improve-zfs-performance\/","url_meta":{"origin":2562,"position":0},"title":"How to Improve ZFS performance","author":"Derrick","date":"August 24, 2014","format":false,"excerpt":"First Edition: March 14, 2010 Last Edited: January 10, 2026 \u00a0 In this tutorial, I will show you how to improve the performance of your ZFS using the affordable consumer-grade hardware (e.g., Gigabit network card, standard SATA non-SSD hard drives, consumer-grade motherboard etc.). Many people found a problem on their\u2026","rel":"","context":"In "Computer Problems"","block_context":{"text":"Computer Problems","link":"https:\/\/icesquare.com\/wordpress\/category\/technology\/computer-problems-computer-help-pc-support\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/icesquare.com\/wordpress\/wp-content\/uploads\/2011\/01\/zfs_logo.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":3458,"url":"https:\/\/icesquare.com\/wordpress\/how-to-stress-test-zfs-system\/","url_meta":{"origin":2562,"position":1},"title":"How to Stress Test ZFS System","author":"Derrick","date":"January 26, 2014","format":false,"excerpt":"If you have set up a ZFS system, you may want to stress test the system before putting it in a production environment. There are many different ways to stress test the system. The most common way is to fill the entire pool using dd. However, I think scrubbing the\u2026","rel":"","context":"In "Computer Problems"","block_context":{"text":"Computer Problems","link":"https:\/\/icesquare.com\/wordpress\/category\/technology\/computer-problems-computer-help-pc-support\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":4648,"url":"https:\/\/icesquare.com\/wordpress\/zfs-on-linux-what-to-do-when-resilver-takes-very-long\/","url_meta":{"origin":2562,"position":2},"title":"[ZFS On Linux] What to Do When Resilver Takes Very Long","author":"Derrick","date":"January 13, 2020","format":false,"excerpt":"If you check your Zpool health status and you notice an error like the following: sudo zpool status pool: myzpool state: DEGRADED status: One or more devices are faulted in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Replace the\u2026","rel":"","context":"In "Computer Problems"","block_context":{"text":"Computer Problems","link":"https:\/\/icesquare.com\/wordpress\/category\/technology\/computer-problems-computer-help-pc-support\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3914,"url":"https:\/\/icesquare.com\/wordpress\/zfs-linux-vs-freebsd\/","url_meta":{"origin":2562,"position":3},"title":"Running ZFS on Linux: Things you should know and be aware of","author":"Derrick","date":"February 29, 2016","format":false,"excerpt":"First Published: Feb 29, 2016 Last Revised: January 10, 2026 ZFS is a next-generation file system renowned for its reliability and advanced features. However, due to legal and licensing issues, it is not included by default in most Linux distributions. Fortunately, you can still install (ZFS on Linux) with just\u2026","rel":"","context":"In "Computer Problems"","block_context":{"text":"Computer Problems","link":"https:\/\/icesquare.com\/wordpress\/category\/technology\/computer-problems-computer-help-pc-support\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1214,"url":"https:\/\/icesquare.com\/wordpress\/building-a-super-large-and-reliable-file-server-with-mixed-size-of-harddisks\/","url_meta":{"origin":2562,"position":4},"title":"Building a Super Large and Reliable File Server with Mixed Size of Harddisks","author":"Derrick","date":"January 17, 2011","format":false,"excerpt":"In this article, I am going to show you how to build a super large, ultra reliable, expandable file server with mixed size of hard drives. Here is my criteria: Large Storage Capacity. At this moment, the largest hard drive is 2TB. I am looking for at least 10TB. Great\u2026","rel":"","context":"In "Computer Problems"","block_context":{"text":"Computer Problems","link":"https:\/\/icesquare.com\/wordpress\/category\/technology\/computer-problems-computer-help-pc-support\/"},"img":{"alt_text":"","src":"\/wordpress\/wp-content\/uploads\/2011\/01\/zfs_logo-150x150.jpg","width":350,"height":200},"classes":[]},{"id":3342,"url":"https:\/\/icesquare.com\/wordpress\/freebsd-zfs-faulted-kernel-ada0ahcich4000-lost-device\/","url_meta":{"origin":2562,"position":5},"title":"[FreeBSD] ZFS FAULTED \/ kernel: (ada0:ahcich4:0:0:0): lost device","author":"Derrick","date":"October 21, 2013","format":false,"excerpt":"I am so happy that I finally solved a problem today. This problem had existed in my server farms for few months already. In the past few months, I had absolutely no idea what was causing the issues. Here is my story: I built a file server using FreeBSD and\u2026","rel":"","context":"In "Computer Problems"","block_context":{"text":"Computer Problems","link":"https:\/\/icesquare.com\/wordpress\/category\/technology\/computer-problems-computer-help-pc-support\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/icesquare.com\/wordpress\/wp-json\/wp\/v2\/posts\/2562","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/icesquare.com\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/icesquare.com\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/icesquare.com\/wordpress\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/icesquare.com\/wordpress\/wp-json\/wp\/v2\/comments?post=2562"}],"version-history":[{"count":36,"href":"https:\/\/icesquare.com\/wordpress\/wp-json\/wp\/v2\/posts\/2562\/revisions"}],"predecessor-version":[{"id":2601,"href":"https:\/\/icesquare.com\/wordpress\/wp-json\/wp\/v2\/posts\/2562\/revisions\/2601"}],"wp:attachment":[{"href":"https:\/\/icesquare.com\/wordpress\/wp-json\/wp\/v2\/media?parent=2562"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/icesquare.com\/wordpress\/wp-json\/wp\/v2\/categories?post=2562"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/icesquare.com\/wordpress\/wp-json\/wp\/v2\/tags?post=2562"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
![\"\"](\"\/wordpress\/wp-content\/uploads\/2012\/01\/HomerStranglesBart2.gif\" "\\"Oh")
<\/p>\n