mdadm / RAID mess

Post Reply
sammm00
Posts: 1
Joined: Mon Jan 18, 2016 1:39 am

mdadm / RAID mess

Post by sammm00 » Mon Jan 18, 2016 1:40 am

I'm in a bit of a mess with mdadm and my RAID array, I'd be very grateful if anyone had any ideas.

I tried to grow my RAID 6 array consisting of 5x 3TB drives with another 3TB drive, however after 12 hours the reshape didn't appear to be making any progress; reshape = 0.0% (0/2930265088). Unfortunately at some point I suffered power loss and upon powering on the array wouldn't start. After googling I've read that the first moments of a reshape are critical, however as the reshape seemed to be stuck I'm hoping there's a way to recover from this? I also read that you can define a back up file, unfortunately the guide I was following for the reshape didn't mention this, so I don't have one. All 5 are reporting clean and correct checksums.

At the moment I don't care about trying to grow the array, I'd just like to try and recover data if possible.

Here's the output of mdadm --examine for each of the old drives.

Code: Select all

root@nas:~# mdadm --examine /dev/sda
/dev/sda:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x4
     Array UUID : b4812ae8:804e182d:f813a5e1:e9ee6da4
           Name : nas:Storage  (local to host nas)
  Creation Time : Mon Aug  4 02:03:59 2014
     Raid Level : raid6
   Raid Devices : 6

 Avail Dev Size : 5860531120 (2794.52 GiB 3000.59 GB)
     Array Size : 11721060352 (11178.07 GiB 12002.37 GB)
  Used Dev Size : 5860530176 (2794.52 GiB 3000.59 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
   Unused Space : before=1968 sectors, after=944 sectors
          State : clean
    Device UUID : f8863d07:23d5d4af:dd1d2f40:e366d96e

  Reshape pos'n : 0
  Delta Devices : 1 (5->6)

    Update Time : Mon Jan 18 00:27:39 2016
       Checksum : bc4a9451 - correct
         Events : 551

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 0
   Array State : AAAAA. ('A' == active, '.' == missing, 'R' == replacing)
root@nas:~# mdadm --examine /dev/sdb
/dev/sdb:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x4
     Array UUID : b4812ae8:804e182d:f813a5e1:e9ee6da4
           Name : nas:Storage  (local to host nas)
  Creation Time : Mon Aug  4 02:03:59 2014
     Raid Level : raid6
   Raid Devices : 6

 Avail Dev Size : 5860531120 (2794.52 GiB 3000.59 GB)
     Array Size : 11721060352 (11178.07 GiB 12002.37 GB)
  Used Dev Size : 5860530176 (2794.52 GiB 3000.59 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
   Unused Space : before=1968 sectors, after=944 sectors
          State : clean
    Device UUID : f38040a7:8e6d340e:ac386daa:80f03141

  Reshape pos'n : 0
  Delta Devices : 1 (5->6)

    Update Time : Mon Jan 18 00:27:39 2016
       Checksum : f743cb24 - correct
         Events : 551

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 1
   Array State : AAAAA. ('A' == active, '.' == missing, 'R' == replacing)
root@nas:~# mdadm --examine /dev/sdc
/dev/sdc:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x4
     Array UUID : b4812ae8:804e182d:f813a5e1:e9ee6da4
           Name : nas:Storage  (local to host nas)
  Creation Time : Mon Aug  4 02:03:59 2014
     Raid Level : raid6
   Raid Devices : 6

 Avail Dev Size : 5860531120 (2794.52 GiB 3000.59 GB)
     Array Size : 11721060352 (11178.07 GiB 12002.37 GB)
  Used Dev Size : 5860530176 (2794.52 GiB 3000.59 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
   Unused Space : before=1968 sectors, after=944 sectors
          State : clean
    Device UUID : d829a9b5:ec6f0b86:06d406be:e508fc22

  Reshape pos'n : 0
  Delta Devices : 1 (5->6)

    Update Time : Mon Jan 18 00:27:39 2016
       Checksum : 72e72e79 - correct
         Events : 551

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 3
   Array State : AAAAA. ('A' == active, '.' == missing, 'R' == replacing)
root@nas:~# mdadm --examine /dev/sdd
/dev/sdd:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x4
     Array UUID : b4812ae8:804e182d:f813a5e1:e9ee6da4
           Name : nas:Storage  (local to host nas)
  Creation Time : Mon Aug  4 02:03:59 2014
     Raid Level : raid6
   Raid Devices : 6

 Avail Dev Size : 5860531120 (2794.52 GiB 3000.59 GB)
     Array Size : 11721060352 (11178.07 GiB 12002.37 GB)
  Used Dev Size : 5860530176 (2794.52 GiB 3000.59 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
   Unused Space : before=1968 sectors, after=944 sectors
          State : clean
    Device UUID : 0fbc03cd:4dfb176e:e5474a86:d7752374

  Reshape pos'n : 0
  Delta Devices : 1 (5->6)

    Update Time : Mon Jan 18 00:27:39 2016
       Checksum : 8bb92893 - correct
         Events : 551

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 4
   Array State : AAAAA. ('A' == active, '.' == missing, 'R' == replacing)
root@nas:~# mdadm --examine /dev/sde
/dev/sde:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x4
     Array UUID : b4812ae8:804e182d:f813a5e1:e9ee6da4
           Name : nas:Storage  (local to host nas)
  Creation Time : Mon Aug  4 02:03:59 2014
     Raid Level : raid6
   Raid Devices : 6

 Avail Dev Size : 5860531120 (2794.52 GiB 3000.59 GB)
     Array Size : 11721060352 (11178.07 GiB 12002.37 GB)
  Used Dev Size : 5860530176 (2794.52 GiB 3000.59 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
   Unused Space : before=1960 sectors, after=944 sectors
          State : clean
    Device UUID : e89405bf:0fa45530:4d4bd566:3af77917

  Reshape pos'n : 0
  Delta Devices : 1 (5->6)

    Update Time : Mon Jan 18 00:27:39 2016
  Bad Block Log : 512 entries available at offset 72 sectors
       Checksum : c3e22f41 - correct
         Events : 551

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 2
   Array State : AAAAA. ('A' == active, '.' == missing, 'R' == replacing)
Output of mdadm -D /dev/md127

Code: Select all

/dev/md127:
        Version : 1.2
     Raid Level : raid0
  Total Devices : 5
    Persistence : Superblock is persistent

          State : inactive

  Delta Devices : 1, (-1->0)
      New Level : raid6
     New Layout : left-symmetric
  New Chunksize : 512K

           Name : nas:Storage  (local to host nas)
           UUID : b4812ae8:804e182d:f813a5e1:e9ee6da4
         Events : 551

    Number   Major   Minor   RaidDevice

       -       8        0        -        /dev/sda
       -       8       16        -        /dev/sdb
       -       8       32        -        /dev/sdc
       -       8       48        -        /dev/sdd
       -       8       64        -        /dev/sde
When trying to force assemble:

Code: Select all

root@nas:~# mdadm --assemble --force /dev/md127 /dev/sda /dev/sdb /dev/sdc /dev/sdd /dev/sde
mdadm: Failed to restore critical section for reshape, sorry.
       Possibly you needed to specify the --backup-file
And with "--invalid-backup"

Code: Select all

root@nas:~# mdadm --assemble --force /dev/md127 /dev/sda /dev/sdb /dev/sdc /dev/sdd /dev/sde --invalid-backup
mdadm: failed to RUN_ARRAY /dev/md127: Invalid argument
root@nas:~# mdadm -D /dev/md127
/dev/md127:
        Version : 1.2
  Creation Time : Mon Aug  4 02:03:59 2014
     Raid Level : raid6
  Used Dev Size : -1
   Raid Devices : 6
  Total Devices : 5
    Persistence : Superblock is persistent

    Update Time : Mon Jan 18 00:27:39 2016
          State : active, degraded, Not Started
 Active Devices : 5
Working Devices : 5
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 512K

  Delta Devices : 1, (5->6)

           Name : nas:Storage  (local to host nas)
           UUID : b4812ae8:804e182d:f813a5e1:e9ee6da4
         Events : 551

    Number   Major   Minor   RaidDevice State
       0       8        0        0      active sync   /dev/sda
       1       8       16        1      active sync   /dev/sdb
       5       8       64        2      active sync   /dev/sde
       3       8       32        3      active sync   /dev/sdc
       4       8       48        4      active sync   /dev/sdd
      10       0        0       10      removed
Thanks very much.

Post Reply