LG N4B1: RAID5 Disk failure

LG Electronics
Post Reply
Micio
Posts: 5
Joined: Mon Jan 05, 2015 12:33 pm

LG N4B1: RAID5 Disk failure

Post by Micio » Mon Jan 05, 2015 6:38 pm

Hi all,
yesterday my nas (lg n4b1) has a power loss and at reboot the raid failed to start :shock: .
Hi so did the follwing and the raid started, but it seems that a partition is not working: it's /dev/sda3 (raid is /dev/md2)

Code: Select all

root@LG-NAS /etc # mdadm -As /dev/md2
mdadm: /dev/md2 assembled from 3 drives - not enough to start the array while not clean - consider --force.
root@LG-NAS /etc # mdadm -As --force /dev/md2
mdadm: /dev/md2 has been started with 3 drives (out of 4).
root@LG-NAS /etc # mdadm --detail /dev/md2
/dev/md2:
        Version : 00.90.03
  Creation Time : Sun Dec 19 01:00:20 2010
     Raid Level : raid5
     Array Size : 5853884928 (5582.70 GiB 5994.38 GB)
  Used Dev Size : 1951294976 (1860.90 GiB 1998.13 GB)
   Raid Devices : 4
  Total Devices : 3
Preferred Minor : 2
    Persistence : Superblock is persistent

    Update Time : Sun Jan  4 18:26:48 2015
          State : clean, degraded
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 64K

           UUID : 334b3196:2f5447a1:428a12e3:3a137d3f
         Events : 0.117147

    Number   Major   Minor   RaidDevice State
       0       0        0        0      removed
       1       8       19        1      active sync   /dev/sdb3
       2       8       35        2      active sync   /dev/sdc3
       3       8       51        3      active sync   /dev/sdd3
root@LG-NAS /etc # cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md1 : active raid1 sdd2[3] sdc2[2] sdb2[1] sda2[4](F)
      256960 blocks [4/3] [_UUU]

md2 : active raid5 sdb3[1] sdd3[3] sdc3[2]
      5853884928 blocks level 5, 64k chunk, algorithm 2 [4/3] [_UUU]

md0 : active raid1 sdd1[3] sdc1[2] sdb1[1] sda1[4](F)
      1959808 blocks [4/3] [_UUU]

unused devices: <none>
What should I do to fix things and avoid to loose 6TB of data?
I got 2 other disks of the same capacity but different model (seagate instead of WD).. Can I replace the failed disk with one of these?
I have also see that 2 other raid are running in my nas where is mounted the root filesystem, is it a problem?

Thanks!!

Grazie mille!

Mijzelf
Posts: 6198
Joined: Mon Jun 16, 2008 10:45 am

Re: LG N4B1: RAID5 Disk failure

Post by Mijzelf » Tue Jan 06, 2015 8:25 am

Seeing your listing (all raid arrays status _UUU) your first disk just died. Yes, you can exchange it with any Sata disk with at least the same size. (At sector level. On the disk is a 'big total' number, about 3906250000, and the new disk should have at least that size. Bigger is no problem)

After exchanging you maybe have to initiate a resync in the webinterface.
I have also see that 2 other raid are running in my nas where is mounted the root filesystem, is it a problem?
No.

Micio
Posts: 5
Joined: Mon Jan 05, 2015 12:33 pm

Re: LG N4B1: RAID5 Disk failure

Post by Micio » Tue Jan 06, 2015 2:31 pm

Should I format the new disk in some specific way before replacing?

Mijzelf
Posts: 6198
Joined: Mon Jun 16, 2008 10:45 am

Re: LG N4B1: RAID5 Disk failure

Post by Mijzelf » Tue Jan 06, 2015 3:49 pm

No. The contents is overwritten anyway. Maybe the nas will refuse to use the disk when it's not empty (for security reasons), in that case remove all partitions.

Micio
Posts: 5
Joined: Mon Jan 05, 2015 12:33 pm

Re: LG N4B1: RAID5 Disk failure

Post by Micio » Tue Jan 06, 2015 9:10 pm

Things get complicated.. I rebooted the nas once and this is what I got

Code: Select all

Jan  7 00:50:33 (none) kernel: md: md2: raid array is not clean -- starting background reconstruction
Jan  7 00:50:33 (none) kernel: raid5: device sda3 operational as raid disk 1
Jan  7 00:50:33 (none) kernel: raid5: device sdc3 operational as raid disk 3
Jan  7 00:50:33 (none) kernel: raid5: device sdb3 operational as raid disk 2
Jan  7 00:50:33 (none) kernel: raid5: cannot start dirty degraded array for md2
Jan  7 00:50:33 (none) kernel: RAID5 conf printout:
Jan  7 00:50:33 (none) kernel:  --- rd:4 wd:3
Jan  7 00:50:33 (none) kernel:  disk 1, o:1, dev:sda3
Jan  7 00:50:33 (none) kernel:  disk 2, o:1, dev:sdb3
Jan  7 00:50:33 (none) kernel:  disk 3, o:1, dev:sdc3
Jan  7 00:50:33 (none) kernel: raid5: failed to run raid set md2
Jan  7 00:50:33 (none) kernel: md: pers->run() failed ...
Jan  7 00:50:38 (none) crond[1684]: USER root pid 2272 cmd /etc/cron/cron.d/fancontrol

root@LG-NAS ~ # mdadm --examine /dev/sd[abcd]1 | grep Event
         Events : 0.116668    
         Events : 0.116669
mdadm: cannot open /dev/sdd1: No such device or address
         Events : 0.116669
mdadm --detail /dev/md2
root@LG-NAS ~ # mdadm --examine /dev/sd[abcd]1 | grep Event
root@LG-NAS ~ # mdadm --detail /dev/md2
/dev/md2:
        Version : 00.90.03
  Creation Time : Sun Dec 19 01:00:20 2010
     Raid Level : raid5
  Used Dev Size : 1951294976 (1860.90 GiB 1998.13 GB)
   Raid Devices : 4
  Total Devices : 3
Preferred Minor : 2
    Persistence : Superblock is persistent

    Update Time : Tue Jan  6 22:14:25 2015
          State : active, degraded, Not Started
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 64K

           UUID : 334b3196:2f5447a1:428a12e3:3a137d3f
         Events : 0.120064

    Number   Major   Minor   RaidDevice State
       0       0        0        0      removed
       1       8        3        1      active sync   /dev/sda3
       2       8       19        2      active sync   /dev/sdb3
       3       8       35        3      active sync   /dev/sdc3
The event count is not the same so the Raid cannot start..

Note that I didn't replaced any drive yet.

What should I do?

Mijzelf
Posts: 6198
Joined: Mon Jun 16, 2008 10:45 am

Re: LG N4B1: RAID5 Disk failure

Post by Mijzelf » Wed Jan 07, 2015 8:44 am

Number Major Minor RaidDevice State
0 0 0 0 removed
1 8 19 1 active sync /dev/sdb3
2 8 35 2 active sync /dev/sdc3
3 8 51 3 active sync /dev/sdd3
Number Major Minor RaidDevice State
0 0 0 0 removed
1 8 3 1 active sync /dev/sda3
2 8 19 2 active sync /dev/sdb3
3 8 35 3 active sync /dev/sdc3
Apparently your disk died after booting, last time. Now it's not found anymore, and so the disk names shifted. But that doesn't matter.

You can simply force the assambly:

Code: Select all

mdadm --assemble /dev/md2 /dev/sd[abc]3 --force --run
But I don't know if the firmware will recognize the raid array then. Worst case you'll have to add the new disk manually.

Micio
Posts: 5
Joined: Mon Jan 05, 2015 12:33 pm

Re: LG N4B1: RAID5 Disk failure

Post by Micio » Wed Jan 07, 2015 8:34 pm

It gives this

Code: Select all

root@LG-NAS ~ # mdadm --manage /dev/md2 -S
mdadm: stopped /dev/md2
root@LG-NAS ~ # mdadm --assemble /dev/md2 /dev/sd[abcd]3 --force --run
mdadm: cannot open device /dev/sdd3: No such device or address
mdadm: /dev/sdd3 has no superblock - assembly aborted
should I start to cry? :shock:

Mijzelf
Posts: 6198
Joined: Mon Jun 16, 2008 10:45 am

Re: LG N4B1: RAID5 Disk failure

Post by Mijzelf » Wed Jan 07, 2015 9:00 pm

Nah. First read better. That is hard with tears in your eyes.

Micio
Posts: 5
Joined: Mon Jan 05, 2015 12:33 pm

Re: LG N4B1: RAID5 Disk failure

Post by Micio » Wed Jan 07, 2015 9:14 pm

after another reboot I got this

Code: Select all

md2 : active raid5 sdc3[3] sdb3[2] sda3[4](F)
      5853884928 blocks level 5, 64k chunk, algorithm 2 [4/2] [__UU]
1 faulty...
However it says that raid is now running.. In truth when I try to read content it gives me I/O error

Mijzelf
Posts: 6198
Joined: Mon Jun 16, 2008 10:45 am

Re: LG N4B1: RAID5 Disk failure

Post by Mijzelf » Thu Jan 08, 2015 8:21 am

This is bad. So you had 1 drive dead, and a 2nd one shortly hereafter failing. I would suspect the NAS itself. It still *can* be the disks (4 disks from the same batch. Used in the same conditions, with the same load. One dies. The odds that the other disks will also fail shortly are bigger than you like. (One of the reasons I don't like raid) ).

You'd better take the disks out, connect them to a Linux PC, and see if that helps. Also, have you examined the failed disk in another device?

Your array is down now. If it indeed is the disk, you might be able to create a low level copy of the failing using dd_rescue, to save your data. If it's indeed the disk, I would replace all disks, by the way. 2 dead out of a batch of four...

Post Reply