General NAS-Central Forums

Welcome to the NAS community
It is currently Wed Sep 20, 2017 12:52 pm

All times are UTC




Post new topic Reply to topic  [ 44 posts ]  Go to page Previous  1, 2, 3  Next
Author Message
PostPosted: Sat Jan 04, 2014 3:39 pm 
Offline

Joined: Thu Aug 09, 2012 11:15 am
Posts: 44
Well, the recovery has stopped suddenly without finishing (I guess), but now I can check the details of the device:

Quote:
mdadm --examine /dev/sdb2
/dev/sdb2:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x2
Array UUID : cba54300:251e4d89:9dbf101c:f04b7735
Name : ix2-200-D8S0EN:1 (local to host ix2-200-D8S0EN)
Creation Time : Fri Jun 8 03:34:59 2012
Raid Level : raid1
Raid Devices : 2

Avail Dev Size : 3865066976 (1843.01 GiB 1978.91 GB)
Array Size : 3865066976 (1843.01 GiB 1978.91 GB)
Super Offset : 3865067232 sectors
Recovery Offset : 970693888 sectors
State : clean
Device UUID : e33b3e1d:43d248f6:f10ae7a2:daeb92ac

Update Time : Sat Jan 4 16:14:48 2014
Checksum : 32dbaffe - correct
Events : 1280018


Array Slot : 2 (0, failed, 1)
Array State : uU 1 failed


But the Array Slot says 'failed'. Should I mark it as failed, remove it and re-add it againg to get the array to reconstruct the array?

The array detail is:
Quote:
mdadm --detail /dev/md1
/dev/md1:
Version : 01.00
Creation Time : Fri Jun 8 03:34:59 2012
Raid Level : raid1
Array Size : 1932533488 (1843.01 GiB 1978.91 GB)
Used Dev Size : 3865066976 (3686.02 GiB 3957.83 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 1
Persistence : Superblock is persistent

Update Time : Sat Jan 4 16:14:48 2014
State : clean, degraded
Active Devices : 1
Working Devices : 2
Failed Devices : 0
Spare Devices : 1

Name : ix2-200-D8S0EN:1 (local to host ix2-200-D8S0EN)
UUID : cba54300:251e4d89:9dbf101c:f04b7735
Events : 1280018

Number Major Minor RaidDevice State
0 8 2 0 active sync /dev/sda2
2 8 18 1 spare rebuilding /dev/sdb2


Well, after mark it as failed, it has started the rebuilding again:

Quote:
mdadm --manage /dev/md1 --fail /dev/sdb2
mdadm: set /dev/sdb2 faulty in /dev/md1
root@ix2-200-D8S0EN:/# cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid6] [raid5] [raid4]
md1 : active raid1 sdb2[2] sda2[0]
1932533488 blocks super 1.0 [2/1] [U_]
[>....................] recovery = 0.0% (873536/1932533488) finish=368.5min speed=87353K/sec

md0 : active raid1 sda1[0] sdb1[1]
20980800 blocks [2/2] [UU]


Top
 Profile  
 
PostPosted: Sat Jan 04, 2014 11:00 pm 
Offline

Joined: Thu Aug 09, 2012 11:15 am
Posts: 44
Na, the recovery finishes (not at 100%), then the device becomes 'spare' again.


Top
 Profile  
 
PostPosted: Sun Jan 05, 2014 8:37 am 
Offline

Joined: Mon Jun 16, 2008 10:45 am
Posts: 6039
Did you have a look in dmesg if still I/O errors are occurring? And did you look if smartctl is available?


Top
 Profile  
 
PostPosted: Sun Jan 05, 2014 2:11 pm 
Offline

Joined: Thu Aug 09, 2012 11:15 am
Posts: 44
dmesg doesn't say anythig:

Quote:
dmesg -c >/dev/null
root@ix2-200-D8S0EN:/# mdadm --manage /dev/md1 --add /dev/sdb2
mdadm: Cannot open /dev/sdb2: Device or resource busy
root@ix2-200-D8S0EN:/# dmesg
root@ix2-200-D8S0EN:/# dmesg


And yes, smartctl is available, but I don't know how to use it.


Top
 Profile  
 
PostPosted: Sun Jan 05, 2014 4:09 pm 
Offline

Joined: Mon Jun 16, 2008 10:45 am
Posts: 6039
dmesg shows the kernel log buffer. You can call it any time and see the kernel logs of the last minuttes/hours. 'dmesg -c' clears the buffer. In this case you could run
Code:
dmesg |  grep "I/O"
while the rebuild is running, to see if I/O errors are occurring.

For smartctl you'll have to use
Code:
smartctl --all /dev/sdb
or maybe
Code:
smartctl -d marvell --all /dev/sdb


Top
 Profile  
 
PostPosted: Sun Jan 05, 2014 5:03 pm 
Offline

Joined: Thu Aug 09, 2012 11:15 am
Posts: 44
with the marvell option it does something, but I don't understand anything

Quote:
smartctl -d marvell --all /dev/sdb
smartctl 5.40 2010-10-16 r3189 [arm-none-linux-gnueabi] (local build)
Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Device Model: ST2000DM001-9YN164
Serial Number: S1E08YN5
Firmware Version: CC4B
User Capacity: 2,000,398,934,016 bytes
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: 8
ATA Standard is: ATA-8-ACS revision 4
Local Time is: Sun Jan 5 18:01:28 2014 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
See vendor-specific Attribute list for marginal Attributes.

General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 584) seconds.
Offline data collection
capabilities: (0x73) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
No Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 1) minutes.
Extended self-test routine
recommended polling time: ( 225) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x3085) SCT Status supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 117 099 006 Pre-fail Always - 138070846
3 Spin_Up_Time 0x0003 094 094 000 Pre-fail Always - 0
4 Start_Stop_Count 0x0032 099 099 020 Old_age Always - 1265
5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail Always - 0
7 Seek_Error_Rate 0x000f 081 060 030 Pre-fail Always - 130340756
9 Power_On_Hours 0x0032 086 086 000 Old_age Always - 12339
10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0
12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 15
183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0
184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0
187 Reported_Uncorrect 0x0032 099 099 000 Old_age Always - 1
188 Command_Timeout 0x0032 100 099 000 Old_age Always - 4295032833
189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0
190 Airflow_Temperature_Cel 0x0022 058 042 045 Old_age Always In_the_past 42 (26 142 53 39)
191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0
192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 14
193 Load_Cycle_Count 0x0032 085 085 000 Old_age Always - 31793
194 Temperature_Celsius 0x0022 042 058 000 Old_age Always - 42 (0 20 0 0)
197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0
240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 192521909050365
241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 3344956179065
242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 264098957423622

SMART Error Log Version: 1
ATA Error Count: 3792 (device log contains only the most recent five errors)
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 3792 occurred at disk power-on lifetime: 12309 hours (512 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 06 ff ff ff 4f 00 16:26:25.281 READ FPDMA QUEUED
2f 00 01 10 00 00 00 00 16:26:25.095 READ LOG EXT
60 00 06 ff ff ff 4f 00 16:26:24.464 READ FPDMA QUEUED
2f 00 01 10 00 00 00 00 16:26:24.279 READ LOG EXT
60 00 06 ff ff ff 4f 00 16:26:23.647 READ FPDMA QUEUED

Error 3791 occurred at disk power-on lifetime: 12309 hours (512 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 06 ff ff ff 4f 00 16:26:25.281 READ FPDMA QUEUED
2f 00 01 10 00 00 00 00 16:26:25.095 READ LOG EXT
60 00 06 ff ff ff 4f 00 16:26:24.464 READ FPDMA QUEUED
2f 00 01 10 00 00 00 00 16:26:24.279 READ LOG EXT
60 00 06 ff ff ff 4f 00 16:26:23.647 READ FPDMA QUEUED

Error 3790 occurred at disk power-on lifetime: 12309 hours (512 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 06 ff ff ff 4f 00 16:26:24.464 READ FPDMA QUEUED
2f 00 01 10 00 00 00 00 16:26:24.279 READ LOG EXT
60 00 06 ff ff ff 4f 00 16:26:23.647 READ FPDMA QUEUED
2f 00 01 10 00 00 00 00 16:26:23.462 READ LOG EXT
60 00 06 ff ff ff 4f 00 16:26:22.856 READ FPDMA QUEUED

Error 3789 occurred at disk power-on lifetime: 12309 hours (512 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 06 ff ff ff 4f 00 16:26:24.464 READ FPDMA QUEUED
2f 00 01 10 00 00 00 00 16:26:24.279 READ LOG EXT
60 00 06 ff ff ff 4f 00 16:26:23.647 READ FPDMA QUEUED
2f 00 01 10 00 00 00 00 16:26:23.462 READ LOG EXT
60 00 06 ff ff ff 4f 00 16:26:22.856 READ FPDMA QUEUED

Error 3788 occurred at disk power-on lifetime: 12309 hours (512 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 06 ff ff ff 4f 00 16:26:23.647 READ FPDMA QUEUED
2f 00 01 10 00 00 00 00 16:26:23.462 READ LOG EXT
60 00 06 ff ff ff 4f 00 16:26:22.856 READ FPDMA QUEUED
2f 00 01 10 00 00 00 00 16:26:22.670 READ LOG EXT
60 00 06 ff ff ff 4f 00 16:26:22.039 READ FPDMA QUEUED

SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]


SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.


Top
 Profile  
 
PostPosted: Sun Jan 05, 2014 7:04 pm 
Offline

Joined: Mon Jun 16, 2008 10:45 am
Posts: 6039
Actually this doesn't look bad.
SMART wrote:
197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0
The number of sectors from which the firmware is aware they are wrong. They can only be exchange after a write operation.
SMART wrote:
5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail Always - 0
The number of sectors which are already replaced this way.
SMART wrote:
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0
Number of errors cause by bad cabling.
(in all this cases the zero at the end is the actual value.

SMART wrote:
SMART overall-health self-assessment test result: PASSED
The big total. The disk is healthy.

The bad news is that then I can't explain where the bad sectors are gone. You 'resetted' them by dd'ing zeros, but the replace count is still zero.
You didn't exchange the disks, did you? In that case we are looking to the wrong disk.


Top
 Profile  
 
PostPosted: Sun Jan 05, 2014 7:27 pm 
Offline

Joined: Thu Aug 09, 2012 11:15 am
Posts: 44
Quote:
The bad news is that then I can't explain where the bad sectors are gone. You 'resetted' them by dd'ing zeros, but the replace count is still zero.
You didn't exchange the disks, did you? In that case we are looking to the wrong disk.


No, I've never touch the disks. But I don't understand why do you say it's the wrong
disk, /dev/sdb2 was the disk that failed so /dev/sdb might be the problematic disk

Quote:
mdadm --detail /dev/md1
/dev/md1:
Version : 01.00
Creation Time : Fri Jun 8 03:34:59 2012
Raid Level : raid1
Array Size : 1932533488 (1843.01 GiB 1978.91 GB)
Used Dev Size : 3865066976 (3686.02 GiB 3957.83 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 1
Persistence : Superblock is persistent

Update Time : Sun Jan 5 15:10:02 2014
State : clean, degraded
Active Devices : 1
Working Devices : 2
Failed Devices : 0
Spare Devices : 1

Name : ix2-200-D8S0EN:1 (local to host ix2-200-D8S0EN)
UUID : cba54300:251e4d89:9dbf101c:f04b7735
Events : 1280050

Number Major Minor RaidDevice State
0 8 2 0 active sync /dev/sda2
2 8 18 1 spare rebuilding /dev/sdb2


Last mdadm --detail says that /dev/sdb2 is the 'spare' device, sda2 is active. Isn't there a way
to tell the array to rebuild or sync the spare device? or something else?

I'm starting to think that NAS devices are very fragile, and complex to manage. :roll:


Top
 Profile  
 
PostPosted: Mon Jan 06, 2014 7:07 pm 
Offline

Joined: Mon Jun 16, 2008 10:45 am
Posts: 6039
Quote:
But I don't understand why do you say it's the wrong disk, /dev/sdb2 was the disk that failed
I said that if you swapped the disks, then we are looking to the wrong one. Or at least I meant to say that.
Quote:
Last mdadm --detail says that /dev/sdb2 is the 'spare' device, sda2 is active. Isn't there a way to tell the array to rebuild or sync the spare device? or something else?
Not that I'm aware of. There are some tricks like growing the array to span 3 devices, but in that case the array will still be degraded, and so the red led will stay on.

Maybe you should start over, and do it proper:
Code:
mdadm --manage /dev/md1 --remove /dev/sdb2
mdadm --zero-superblock /dev/sdb2
mdadm --manage /dev/md1 --add /dev/sdb2


Quote:
I'm starting to think that NAS devices are very fragile
Your firmware is based on Debian, and Debian is supposed to be rock-stable. The GUI put on top of it can be fragile, but that has nothing to do with mdadm.
Quote:
and complex to manage
Well, the manual says: If the led is red, pull the defective disk, and exchange it by a new one. That's not complex. It gets complex if you want to do it another way.


Top
 Profile  
 
PostPosted: Mon Jan 06, 2014 8:47 pm 
Offline

Joined: Thu Aug 09, 2012 11:15 am
Posts: 44
Yes, I know that mdadm it isn't the problem, and that Debian is a very strong distro, I was only whining a little bit :)

Well, I'm trying to remove the device, but I can't because it's busy. So I've marked it as failed then removed it, but now it seems that it's mounted as readonly :?:

Quote:
root@ix2-200-D8S0EN:/# mdadm --manage /dev/md1 --remove /dev/sdb2
mdadm: hot remove failed for /dev/sdb2: Device or resource busy
root@ix2-200-D8S0EN:/# mdadm --manage /dev/md1 --fail /dev/sdb2
mdadm: set /dev/sdb2 faulty in /dev/md1
root@ix2-200-D8S0EN:/# mdadm --manage /dev/md1 --remove /dev/sdb2
mdadm: hot removed /dev/sdb2
root@ix2-200-D8S0EN:/# mdadm --zero-superblock /dev/sdb2
mdadm: Couldn't open /dev/sdb2 for write - not zeroing


Top
 Profile  
 
PostPosted: Tue Jan 07, 2014 8:18 am 
Offline

Joined: Mon Jun 16, 2008 10:45 am
Posts: 6039
Code:
dmesg -c
mdadm --zero-superblock /dev/sdb2
dmesg


Top
 Profile  
 
PostPosted: Tue Jan 07, 2014 8:22 am 
Offline

Joined: Thu Aug 09, 2012 11:15 am
Posts: 44
Mijzelf wrote:
Code:
dmesg -c
mdadm --zero-superblock /dev/sdb2
dmesg


Thank you, I'll do it this afternoon when I come back home.


Top
 Profile  
 
PostPosted: Tue Jan 07, 2014 7:15 pm 
Offline

Joined: Thu Aug 09, 2012 11:15 am
Posts: 44
Ummh, something is wrong with the device :S

Quote:
dmesg -c
ng I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x3d8770a0 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x4c40 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x403f6310 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x3d8770a0 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x4c40 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x403f6310 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x3d8770a0 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x4c40 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x403f6310 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x3d8770a0 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x4c40 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x403f6310 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x3d8770a0 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x4c40 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x403f6310 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x3d8770a0 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x4c40 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x403f6310 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x3d8770a0 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x39980840 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x4c40 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x403f6310 ("xfs_trans_read_buf") error 5 buf count 8192
xfs_imap_to_bp: xfs_trans_read_buf()returned an error 5 on dm-2. Returning error.
sd 0:0:0:0: rejecting I/O to offline device
I/O error in filesystem ("dm-2") meta-data dev dm-2 block 0x9ba8 ("xfs_trans_read_buf") error 5 buf count 4096
root@ix2-200-D8S0EN:/# mdadm --zero-superblock /dev/sdb2
mdadm: Couldn't open /dev/sdb2 for write - not zeroing
root@ix2-200-D8S0EN:/# dmesg


Top
 Profile  
 
PostPosted: Tue Jan 07, 2014 8:28 pm 
Offline

Joined: Mon Jun 16, 2008 10:45 am
Posts: 6039
I wonder if somehow /dev/sdb2 is still in the array. You can have a look with 'cat /proc/mdstat'. If it is, you can reboot, and then I guess a removed device can no longer be part of the array? :?

I'm not sure if those I/O errors are actually hardware errors. The error seems to originate in dm-2, which is a (virtual) raid array, and the error is that I/O to an offline device is rejected.

The partition is removed from the array, so it's offline. But somehow the filesystem seems still to be trying to read it. (Seeing the xfs errors). The filesystem doesn't know about the raidarray, so it seems there is still a layer which is redirecting requests to sdb2?


Top
 Profile  
 
PostPosted: Tue Jan 07, 2014 8:41 pm 
Offline

Joined: Thu Aug 09, 2012 11:15 am
Posts: 44
Well, I marked the device as failed then removed it as you told me two or three posts above. Then I tried to do the zero-superblock but it can't be done because it says it can't write on a readonly device. Should I add the device to the array again then try to do 'dmesg -c'?


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 44 posts ]  Go to page Previous  1, 2, 3  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 12 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group