NAS Volume Down after adding new HDD and updating Firmware

Mindspeed Comcerto 2000 based
Mijzelf
Posts: 6197
Joined: Mon Jun 16, 2008 10:45 am

Re: NAS Volume Down after adding new HDD and updating Firmwa

Post by Mijzelf » Thu Dec 08, 2016 2:06 pm

I should look better at the output I asked for.

Code: Select all

   9        2 16588825344 md2
253        0     102400 dm-0
253        1 16588718080 dm-1
You clearly have a second device, slightly (100MB) smaller than your raid array. So I guess you have an LVM container inside your raid array, and you have to mount/repair dm-1 instead of md2.

I don't know where the device node is located, you can find it by

Code: Select all

find /dev -name dm-1

equanox
Posts: 11
Joined: Sun Dec 04, 2016 3:08 pm

Re: NAS Volume Down after adding new HDD and updating Firmwa

Post by equanox » Fri Dec 09, 2016 3:40 pm

Hey mate,

I think something happened :D

Code: Select all


~ # mount -t ext4 /dev/dm-1 /tmp/mountpoint
mount: wrong fs type, bad option, bad superblock on /dev/mapper/vg_abd9690f-lv_c62bd447,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.
~ # e2fsck /dev/dm-1
e2fsck 1.42.12 (29-Aug-2014)

ext2fs_check_desc: Corrupt group descriptor: bad block for block bitmap
e2fsck: Group descriptors look bad... trying backup blocks...
/dev/dm-1: recovering journal
e2fsck: unable to set superblock flags on /dev/dm-1


/dev/dm-1: ***** FILE SYSTEM WAS MODIFIED *****

/dev/dm-1: ********** WARNING: Filesystem still has errors **********

~ #
~ # mount -t ext4 /dev/dm-1 /tmp/mountpoint
~ #

cat /proc/mounts

Code: Select all

~ # cat /proc/mounts
rootfs / rootfs rw 0 0
/proc /proc proc rw,relatime 0 0
/sys /sys sysfs rw,relatime 0 0
none /proc/bus/usb usbfs rw,relatime 0 0
devpts /dev/pts devpts rw,relatime,mode=600 0 0
ubi7:ubi_rootfs2 /firmware/mnt/nand ubifs ro,relatime 0 0
/dev/md0 /firmware/mnt/sysdisk ext4 ro,relatime,user_xattr,barrier=1,data=ordere                              d 0 0
/dev/loop0 /ram_bin ext2 ro,relatime,user_xattr,barrier=1 0 0
/dev/loop0 /usr ext2 ro,relatime,user_xattr,barrier=1 0 0
/dev/loop0 /lib/security ext2 ro,relatime,user_xattr,barrier=1 0 0
/dev/loop0 /lib/modules ext2 ro,relatime,user_xattr,barrier=1 0 0
/dev/loop0 /lib/locale ext2 ro,relatime,user_xattr,barrier=1 0 0
/dev/ram0 /tmp/tmpfs tmpfs rw,relatime,size=5120k 0 0
/dev/ram0 /usr/local/etc tmpfs rw,relatime,size=5120k 0 0
ubi3:ubi_config /etc/zyxel ubifs rw,relatime 0 0
configfs /sys/kernel/config configfs rw,relatime 0 0
/dev/mapper/vg_abd9690f-lv_c62bd447 /tmp/mountpoint ext4 rw,relatime,user_xattr,                              barrier=1,stripe=16,data=ordered 0 0

So I think its mounted now? What shall I do next?

Thank you in advance!



EDIT: Wow!!! I restarted the NAS and all my data is save and everything is fine. Really dude, thank you very much. I dunno how I can thank you - is it possible to donate for this board? Please tell me.

Have a great weekend!

Mijzelf
Posts: 6197
Joined: Mon Jun 16, 2008 10:45 am

Re: NAS Volume Down after adding new HDD and updating Firmwa

Post by Mijzelf » Sat Dec 10, 2016 9:52 am

equanox wrote:EDIT: Wow!!! I restarted the NAS and all my data is save and everything is fine.
The volume has the 'new' size? Great.

Do yourself a favor and make sure you have backups. Your array will fail, some day.
is it possible to donate for this board? Please tell me.
http://www.nas-central.org/wiki/Site_Support

equanox
Posts: 11
Joined: Sun Dec 04, 2016 3:08 pm

Re: NAS Volume Down after adding new HDD and updating Firmwa

Post by equanox » Fri Jan 20, 2017 4:34 pm

Hi dude,

I guess my problem isn't solved yet :-(

I dunno what is wrong, a few days everything worked fine but now I can't use my NAS anymore.

At the beginning I wasn't able to download files directly on the NAS - I always got an error from my browers or jDownloader.

But I was able to watch movies from my NAS or to copy/paste files from my NAS to my PC.

It was hard for me to reach the NAS via Browser to get to the GUI admin panel. I realized that the CPU was always at 100%. There were several processes causing these peaks.

You can see it in the screenshots attached.


After this I deactivated the myZyxcel Cloud and the Twonky Media Server because Google told me that this could be the reason for this CPU peaks.

Nothing helped - after a restart of the NAS I was able to use it 2min and then the CPU was at 100% and I wasn't able to download anything directly to my NAS.


I reset my NAS but nothing helped. At the moment I can't reach my files because the NAS is starting with all 4 HDDs in it and in one run all 4 LEDs blink one after each other for about 15 seconds. After this the LED of the first HDD is blinking unlimited time.

But the problem is the last 8 TB HDD I entererd in the NAS (like described in this thread). I guess the filesystem isn't completly right atm. I can start my NAS without any problems or CPU peaks when I don't put in the last 8TB HDD.

The very strange thing is, that the 8TB HDD gets extremly hot when put into the NAS. It gets so hot that I think the NAS will burn down when I dont release the HDD after 1 hour.


I don't know what to do - maybe you can help me again? :-S I already donated for your awesome board and service. Thanks in advance!

Code: Select all


BusyBox v1.19.4 (2016-10-05 15:30:26 CST) built-in shell (ash)
Enter 'help' for a list of built-in commands.

~ # cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md2 : active linear sda3[0] sdd3[3] sdc3[2] sdb3[1]
      16588825344 blocks super 1.2 64k rounding

md1 : active raid1 sdb2[5] sdd2[7] sdc2[6] sda2[4]
      1998784 blocks super 1.2 [4/4] [UUUU]

md0 : active raid1 sdd1[7] sdb1[5] sdc1[6] sda1[4]
      1997760 blocks super 1.2 [4/4] [UUUU]

unused devices: <none>
~ # cat /proc/partitions
major minor  #blocks  name

   7        0     154624 loop0
  31        0        256 mtdblock0
  31        1        512 mtdblock1
  31        2        256 mtdblock2
  31        3      10240 mtdblock3
  31        4      10240 mtdblock4
  31        5     112640 mtdblock5
  31        6      10240 mtdblock6
  31        7     112640 mtdblock7
  31        8       6144 mtdblock8
   8        0 2930266584 sda
   8        1    1998848 sda1
   8        2    1999872 sda2
   8        3 2926266368 sda3
   8       16 2930266584 sdb
   8       17    1998848 sdb1
   8       18    1999872 sdb2
   8       19 2926266368 sdb3
   8       32 2930266584 sdc
   8       33    1998848 sdc1
   8       34    1999872 sdc2
   8       35 2926266368 sdc3
   8       48 7814026584 sdd
   8       49    1998848 sdd1
   8       50    1999872 sdd2
   8       51 7810026496 sdd3
  31        9     102424 mtdblock9
   9        0    1997760 md0
   9        1    1998784 md1
  31       10       4464 mtdblock10
   9        2 16588825344 md2
 253        0     102400 dm-0
 253        1 16588718080 dm-1


~ # cat/proc/mounts
-sh: cat/proc/mounts: not found
~ # cat /proc/mounts
rootfs / rootfs rw 0 0
/proc /proc proc rw,relatime 0 0
/sys /sys sysfs rw,relatime 0 0
none /proc/bus/usb usbfs rw,relatime 0 0
devpts /dev/pts devpts rw,relatime,mode=600 0 0
ubi7:ubi_rootfs2 /firmware/mnt/nand ubifs ro,relatime 0 0
/dev/md0 /firmware/mnt/sysdisk ext4 ro,relatime,user_xattr,barrier=1,data=ordered 0 0
/dev/loop0 /ram_bin ext2 ro,relatime,user_xattr,barrier=1 0 0
/dev/loop0 /usr ext2 ro,relatime,user_xattr,barrier=1 0 0
/dev/loop0 /lib/security ext2 ro,relatime,user_xattr,barrier=1 0 0
/dev/loop0 /lib/modules ext2 ro,relatime,user_xattr,barrier=1 0 0
/dev/loop0 /lib/locale ext2 ro,relatime,user_xattr,barrier=1 0 0
/dev/ram0 /tmp/tmpfs tmpfs rw,relatime,size=5120k 0 0
/dev/ram0 /usr/local/etc tmpfs rw,relatime,size=5120k 0 0
ubi3:ubi_config /etc/zyxel ubifs rw,relatime 0 0
/dev/mapper/vg_abd9690f-lv_c62bd447 /i-data/c62bd447 ext4 rw,noatime,user_xattr,barrier=1,stripe=16,data=ordered,usrquota 0 0
/dev/mapper/vg_abd9690f-vg_info_area /mnt/vg_info_area/vg_abd9690f ext4 rw,relatime,user_xattr,barrier=1,stripe=16,data=ordered 0 0


You do not have the required permissions to view the files attached to this post.

Mijzelf
Posts: 6197
Joined: Mon Jun 16, 2008 10:45 am

Re: NAS Volume Down after adding new HDD and updating Firmwa

Post by Mijzelf » Sat Jan 21, 2017 12:29 pm

So without the 8TB disk everything is working fine? Did you ever see that the filesystem actually was 16TB in size? Or is only the raidarray/lv that size?

The disk shouldn't get hot. So if it does, I guess you have a hardware problem. Can you connect it to a PC and run diagnositic software from the disk vendor?

The active processes also point to a hardware failure. The process 'flush' is supposed to flush the (dirtied) disk cache to disk, and normally it doesn't use many cpu cycles. 'ksoftirq' is an irq offloader. Instead of handling irq's in the interrupt service routine, it's offloaded to a dedicated thread. This process is also not supposed to use many cpu cycles. Don't know what 'pfe_ctrl_timer' is. Google suggests the process is specific for the Mindspeed SoC on a NAS5xx. In which case it's also hardware related.

I think that running 'dmesg' with the 8TB disk plugged in will show you hardware errors.

equanox
Posts: 11
Joined: Sun Dec 04, 2016 3:08 pm

Re: NAS Volume Down after adding new HDD and updating Firmwa

Post by equanox » Sat Jan 21, 2017 5:07 pm

Mijzelf wrote:So without the 8TB disk everything is working fine? Did you ever see that the filesystem actually was 16TB in size? Or is only the raidarray/lv that size?

The disk shouldn't get hot. So if it does, I guess you have a hardware problem. Can you connect it to a PC and run diagnositic software from the disk vendor?

The active processes also point to a hardware failure. The process 'flush' is supposed to flush the (dirtied) disk cache to disk, and normally it doesn't use many cpu cycles. 'ksoftirq' is an irq offloader. Instead of handling irq's in the interrupt service routine, it's offloaded to a dedicated thread. This process is also not supposed to use many cpu cycles. Don't know what 'pfe_ctrl_timer' is. Google suggests the process is specific for the Mindspeed SoC on a NAS5xx. In which case it's also hardware related.

I think that running 'dmesg' with the 8TB disk plugged in will show you hardware errors.

That sounds bad :/

Okay - sorry I'm a big noob. What do I have to type in Telnet to dmesg my 8TB disk?

dmesg sd03 ? :P

Mijzelf
Posts: 6197
Joined: Mon Jun 16, 2008 10:45 am

Re: NAS Volume Down after adding new HDD and updating Firmwa

Post by Mijzelf » Sun Jan 22, 2017 12:32 pm

Just dmesg. That shows the kernel log buffer. Normally it only contains the bootlog, plus hardware changes. (Plugged in USB sticks, ethernet connection and disconnection, ...) Hardware failures are also logged.

Post Reply