Shares vanished

Oxnas 810 based
adriansienicki
Posts: 21
Joined: Sun May 17, 2015 11:00 am
Location: Internet
Contact:

Shares vanished

Post by adriansienicki » Mon May 18, 2015 2:06 am

Hi,

I wonder if anyone has ever come across similar issue. On attempt to access any of my network folders I get the following message:

Windows cannot access \\ip\folder
Check the spelling of the name. Otherwise, there might be a problem with your network. To try to identify and resolve network problems, click Diagnose.

See the picture below (message and shares in a browser):
Image


I checked my disks if they were OK under another system via FSproxy in read-only mode: both disks are fine.

I tried several times "scan" and "scan and repair" functions from browser but it gave no effect at all.
The output of "scan" but I can hardly get anything out of it:

Code: Select all

Scan Result
Phase 1 - find and verify superblock...
Phase 2 - using internal log
- scan filesystem freespace and inode maps...
agf_freeblks 4294967293 counted 13 in ag 1
sb_fdblocks 18134174 counted 18134190
- found root inode chunk
Phase 3 - for each AG...
- scan (but don't clear) agi unlinked lists...
- process known inodes and perform inode discovery...
- agno = 0
- agno = 1
- agno = 2
- agno = 3
- process newly discovered inodes...
Phase 4 - check for duplicate blocks...
- setting up duplicate extent list...
- check for inodes claiming duplicate blocks...
- agno = 0
- agno = 1
- agno = 2
- agno = 3
No modify flag set skipping phase 5
Phase 6 - check inode connectivity...
- traversing filesystem ...
- traversal finished ...
- moving disconnected inodes to lost+found ...
Phase 7 - verify link counts...
No modify flag set skipping filesystem flush and exiting.
I also tried to use fsck via telnet but the output was not exactly the thing I'd been expecting:
fsck applied to /dev/loop0:

Code: Select all

fsck from util-linux 2.20
e2fsck 1.41.14 (22-Dec-2010)
/dev/loop0 is mounted. e2fsck: Cannot continue, aborting
I also tried to reload my firmware but then I get the following message: Invalid file.
My configuration: NSA221, firmware: 4.41 (AFM.0).
Is it the latest firmware?

Any help would be greatly appreciated.

Mijzelf
Posts: 6251
Joined: Mon Jun 16, 2008 10:45 am

Re: Shares vanished

Post by Mijzelf » Mon May 18, 2015 12:33 pm

According to the question marks in the webinterface, there is something wrong with the shares.

Can you post the output of

Code: Select all

cat /proc/partitions
cat /proc/mdstat
mdadm --examine /dev/sd[ab]
cat /proc/mounts

adriansienicki
Posts: 21
Joined: Sun May 17, 2015 11:00 am
Location: Internet
Contact:

Re: Shares vanished

Post by adriansienicki » Tue May 19, 2015 11:15 pm

OK, so here's the output:

Code: Select all

sh-4.1# cat /proc/partitions
major minor  #blocks  name

   7     0     117760 loop0
   8     0 1953514584 sda
   8     1     514048 sda1
   8     2 1952997952 sda2
   8    16 1953514584 sdb
   8    17     514048 sdb1
   8    18 1952997952 sdb2
  31     0        128 mtdblock0
  31     1       1792 mtdblock1
  31     2       1664 mtdblock2
  31     3        448 mtdblock3
  31     4         48 mtdblock4
  31     5          8 mtdblock5
  31     6          8 mtdblock6
   9     0 1952997888 md0
   8    48    3956736 sdd
   8    49      19005 sdd1
   8    50    3935211 sdd2
   8    32     123904 sdc
   8    33     123799 sdc1
therefore sd[ab]2 are the right ones to examine cause sd[ab] won't produce any results:

Code: Select all

sh-4.1# cat /proc/mdstat
Personalities : [linear] [raid0] [raid1]
md0 : active raid1 sdb2[0] sda2[1]
      1952997888 blocks [2/2] [UU]

unused devices: <none>
- as expected no superblock sd[ab]

Code: Select all

sh-4.1# mdadm --examine /dev/sd[ab]
mdadm: No md superblock detected on /dev/sda.
mdadm: No md superblock detected on /dev/sdb.
so here are sdx2:

Code: Select all

sh-4.1# mdadm --examine /dev/sda2
/dev/sda2:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 5d659448:8431fbe5:11e34d9f:dcce2c98
  Creation Time : Tue Jan 17 14:17:57 2012
     Raid Level : raid1
  Used Dev Size : 1952997888 (1862.52 GiB 1999.87 GB)
     Array Size : 1952997888 (1862.52 GiB 1999.87 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 0

    Update Time : Tue May 19 22:43:14 2015
          State : clean
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 9254d302 - correct
         Events : 227776


      Number   Major   Minor   RaidDevice State
this     1       8        2        1      active sync   /dev/sda2

   0     0       8       18        0      active sync   /dev/sdb2
   1     1       8        2        1      active sync   /dev/sda2

sh-4.1# mdadm --examine /dev/sdb2
/dev/sdb2:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 5d659448:8431fbe5:11e34d9f:dcce2c98
  Creation Time : Tue Jan 17 14:17:57 2012
     Raid Level : raid1
  Used Dev Size : 1952997888 (1862.52 GiB 1999.87 GB)
     Array Size : 1952997888 (1862.52 GiB 1999.87 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 0

    Update Time : Tue May 19 22:43:14 2015
          State : clean
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 9254d310 - correct
         Events : 227776


      Number   Major   Minor   RaidDevice State
this     0       8       18        0      active sync   /dev/sdb2

   0     0       8       18        0      active sync   /dev/sdb2
   1     1       8        2        1      active sync   /dev/sda2
and mounts:

Code: Select all

sh-4.1# cat /proc/mounts
rootfs / rootfs rw 0 0
/dev/root / ext2 rw 0 0
/proc /proc proc rw 0 0
none /proc/bus/usb usbfs rw 0 0
none /sys sysfs rw 0 0
none /dev/pts devpts rw 0 0
/dev/mtdblock4 /zyxel/mnt/info jffs2 ro 0 0
/dev/sdc1 /zyxel/mnt/NAND ext3 rw,data=ordered 0 0
/dev/sda1 /zyxel/mnt/sysdisk ext2 ro 0 0
/dev/loop0 /ram_bin ext2 ro 0 0
/dev/loop0 /usr ext2 ro 0 0
/dev/loop0 /sbin ext2 ro 0 0
/dev/loop0 /bin ext2 ro 0 0
/dev/loop0 /lib ext2 ro 0 0
none /tmp tmpfs rw 0 0
none /var tmpfs rw 0 0
none /home tmpfs rw 0 0
none /mnt tmpfs rw 0 0
none /etc tmpfs rw 0 0
none /dev tmpfs rw 0 0
none /usr/local/etc tmpfs rw 0 0
none /usr/local/var tmpfs rw 0 0
none /usr/local/dmsf tmpfs rw 0 0
/dev/mtdblock3 /etc/zyxel jffs2 rw 0 0
none /i-data tmpfs rw 0 0
none /e-data tmpfs rw 0 0
/dev/md0 /i-data/5d659448 xfs rw,nouuid,usrquota 0 0
/dev/sdd1 /e-data/5558-E13A vfat rw,uid=99,fmask=0000,dmask=0000,codepage=cp437,iocharset=utf8,shortname=mixed 0 0
/dev/sdd2 /e-data/4a5548c6-b3f2-4100-9f08-d935df995268 ext2 rw,relatime 0 0
/dev/mtdblock3 /usr/local/apache/web_framework/data/config jffs2 rw 0 0

Mijzelf
Posts: 6251
Joined: Mon Jun 16, 2008 10:45 am

Re: Shares vanished

Post by Mijzelf » Wed May 20, 2015 7:37 am

Everything looks healthy. Array is assembled and mounted. I think your 'sysvol' symlink is damaged. viewtopic.php?f=249&t=16861&p=93451#p93451

In your case the mountpoint is /i-data/5d659448

adriansienicki
Posts: 21
Joined: Sun May 17, 2015 11:00 am
Location: Internet
Contact:

Re: Shares vanished

Post by adriansienicki » Thu May 21, 2015 3:56 am

Many thanks, I'm gonna try to recreate it in the evening.

adriansienicki
Posts: 21
Joined: Sun May 17, 2015 11:00 am
Location: Internet
Contact:

Re: Shares vanished

Post by adriansienicki » Fri May 22, 2015 6:15 am

a curiuos thing happened - now I'm unable to connect to to telnet. I get the following message: Could not open connection to the host, on port 23: Connect failed.
I got through posts on that and I found out that pushing reset button and logging within 3 minutes would do. Now I'm logging into telnet server but the admin credentials are wrong to login.
I couldn't check the solution with sysvol since my telnet seems to be off.

Mijzelf
Posts: 6251
Joined: Mon Jun 16, 2008 10:45 am

Re: Shares vanished

Post by Mijzelf » Fri May 22, 2015 3:43 pm

How did you get your telnet interface? AFAIK it's not supported in the 221, unless you are using the backdoor.

Anyway, if you PM me your mac address, I can tell you the password of user NsaRescueAngel, which has full rights.

adriansienicki
Posts: 21
Joined: Sun May 17, 2015 11:00 am
Location: Internet
Contact:

Re: Shares vanished

Post by adriansienicki » Sat May 23, 2015 3:46 pm

Unfortunatelly, rm and ln -s didn't produce any results
Output from viewtopic.php?f=249&t=16861&p=93451#p93451 - advised commands I pasted below:

Code: Select all

/ # cat /proc/partitions
major minor  #blocks  name

   7     0     117760 loop0
   8     0 1953514584 sda
   8     1     514048 sda1
   8     2 1952997952 sda2
   8    16 1953514584 sdb
   8    17     514048 sdb1
   8    18 1952997952 sdb2
  31     0        128 mtdblock0
  31     1       1792 mtdblock1
  31     2       1664 mtdblock2
  31     3        448 mtdblock3
  31     4         48 mtdblock4
  31     5          8 mtdblock5
  31     6          8 mtdblock6
   9     0 1952997888 md0
   8    32     123904 sdc
   8    33     123799 sdc1

/ # cat /proc/mdstat
Personalities : [linear] [raid0] [raid1]
md0 : active raid1 sdb2[0] sda2[1]
      1952997888 blocks [2/2] [UU]

unused devices: <none>
/ # cat /proc/mounts
rootfs / rootfs rw 0 0
/dev/root / ext2 rw 0 0
/proc /proc proc rw 0 0
none /proc/bus/usb usbfs rw 0 0
none /sys sysfs rw 0 0
none /dev/pts devpts rw 0 0
/dev/mtdblock4 /zyxel/mnt/info jffs2 ro 0 0
/dev/sdc1 /zyxel/mnt/NAND ext3 rw,data=ordered 0 0
/dev/sda1 /zyxel/mnt/sysdisk ext2 ro 0 0
/dev/loop0 /ram_bin ext2 ro 0 0
/dev/loop0 /usr ext2 ro 0 0
/dev/loop0 /sbin ext2 ro 0 0
/dev/loop0 /bin ext2 ro 0 0
/dev/loop0 /lib ext2 ro 0 0
none /tmp tmpfs rw 0 0
none /var tmpfs rw 0 0
none /home tmpfs rw 0 0
none /mnt tmpfs rw 0 0
none /etc tmpfs rw 0 0
none /dev tmpfs rw 0 0
none /usr/local/etc tmpfs rw 0 0
none /usr/local/var tmpfs rw 0 0
none /usr/local/dmsf tmpfs rw 0 0
/dev/mtdblock3 /etc/zyxel jffs2 rw 0 0
none /i-data tmpfs rw 0 0
none /e-data tmpfs rw 0 0
/dev/md0 /i-data/5d659448 xfs rw,nouuid,usrquota 0 0
/dev/mtdblock3 /usr/local/apache/web_framework/data/config jffs2 rw 0 0

/ # df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root                 7931      3786      4145  48% /
none                     14336      2728     11608  19% /dev/pts
/dev/mtdblock4              48        28        20  58% /zyxel/mnt/info
/dev/sdc1               119879     40402     73288  36% /zyxel/mnt/NAND
/dev/sda1               497829    490379      7450  99% /zyxel/mnt/sysdisk
/dev/loop0              114027     99120     14907  87% /ram_bin
/dev/loop0              114027     99120     14907  87% /usr
/dev/loop0              114027     99120     14907  87% /sbin
/dev/loop0              114027     99120     14907  87% /bin
/dev/loop0              114027     99120     14907  87% /lib
none                     14336      2728     11608  19% /tmp
none                     14336      2728     11608  19% /var
none                     14336      2728     11608  19% /home
none                     14336      2728     11608  19% /mnt
none                     14336      2728     11608  19% /etc
none                     14336      2728     11608  19% /dev
none                     14336      2728     11608  19% /usr/local/etc
none                     14336      2728     11608  19% /usr/local/var
none                     14336      2728     11608  19% /usr/local/dmsf
/dev/mtdblock3             448       264       184  59% /etc/zyxel
none                     14336      2728     11608  19% /i-data
none                     14336      2728     11608  19% /e-data
/dev/md0             1952866816 1880338392  72528424  96% /i-data/5d659448
/dev/mtdblock3             448       264       184  59% /usr/local/apache/web_framework/data/config

/ # ls -l /etc/zyxel/storage/sysvol
lrwxrwxrwx    1 root     root            16 May 23 15:47 /etc/zyxel/storage/sysvol -> /i-data/5d659448

/ # ls -l /etc/zyxel/storage/sysvol/
ls: /etc/zyxel/storage/sysvol/: Input/output error

adriansienicki
Posts: 21
Joined: Sun May 17, 2015 11:00 am
Location: Internet
Contact:

Re: Shares vanished

Post by adriansienicki » Sat May 23, 2015 4:40 pm

BTW:

on NsaRescueAngel on trying to ls /i-data I get the following

Code: Select all

/ # /i-data $ ls -l
-sh: /i-data: Permission denied
/ # cd /i-data
/i-data # ls
ls: ./5d659448: Input/output error
md0

Mijzelf
Posts: 6251
Joined: Mon Jun 16, 2008 10:45 am

Re: Shares vanished

Post by Mijzelf » Sat May 23, 2015 6:33 pm

That is bad. An input/output error is almost always a hardware error. Can you post the output of

Code: Select all

ls -l /i-data/5d659448
dmesg | tail -n 50
(The 'ls' is there to make sure the last 50 lines of the kernel buffer contain information about the I/O error.)

adriansienicki
Posts: 21
Joined: Sun May 17, 2015 11:00 am
Location: Internet
Contact:

Re: Shares vanished

Post by adriansienicki » Sat May 23, 2015 7:06 pm

here you are, you're probably right - it must be something with hardware.

Code: Select all

/ # ls -l /i-data/58659448
ls: /i-data/58659448: No such file or directory

/ # dmesg | tail -n 50
[<c0025184>] (dump_stack+0x0/0x1c) from [<bf03e70c>] (xfs_error_report+0x5c/0x68 [xfs])
[<bf03e6b0>] (xfs_error_report+0x0/0x68 [xfs]) from [<bf03e774>] (xfs_corruption_error+0x5c/0x68 [xfs])
 r5:bf078b04 r4:bf011230
[<bf03e718>] (xfs_corruption_error+0x0/0x68 [xfs]) from [<bf01107c>] (xfs_alloc_read_agf+0xe8/0x27c [xfs])
 r6:bf011230 r5:fffffffd r4:00000000
[<bf010f94>] (xfs_alloc_read_agf+0x0/0x27c [xfs]) from [<bf011230>] (xfs_alloc_pagf_init+0x20/0x48 [xfs])
[<bf011210>] (xfs_alloc_pagf_init+0x0/0x48 [xfs]) from [<bf024ab4>] (xfs_bmap_btalloc+0xa10/0xa68 [xfs])
 r5:cee5d3a0 r4:0000003c
[<bf0240a4>] (xfs_bmap_btalloc+0x0/0xa68 [xfs]) from [<bf024b44>] (xfs_bmap_alloc+0x38/0x48 [xfs])
[<bf024b0c>] (xfs_bmap_alloc+0x0/0x48 [xfs]) from [<bf028780>] (xfs_bmapi+0xc84/0x13fc [xfs])
[<bf027afc>] (xfs_bmapi+0x0/0x13fc [xfs]) from [<bf032220>] (xfs_da_grow_inode+0xe8/0x3a4 [xfs])
[<bf032138>] (xfs_da_grow_inode+0x0/0x3a4 [xfs]) from [<bf03a238>] (xfs_dir2_block_to_leaf+0x2c/0x1e4 [xfs])
[<bf03a20c>] (xfs_dir2_block_to_leaf+0x0/0x1e4 [xfs]) from [<bf036b98>] (xfs_dir2_block_addname+0x6dc/0xaa8 [xfs])
[<bf0364bc>] (xfs_dir2_block_addname+0x0/0xaa8 [xfs]) from [<bf035020>] (xfs_dir_createname+0xe4/0x144 [xfs])
[<bf034f3c>] (xfs_dir_createname+0x0/0x144 [xfs]) from [<bf067f8c>] (xfs_create+0x368/0x504 [xfs])
[<bf067c24>] (xfs_create+0x0/0x504 [xfs]) from [<bf074644>] (xfs_vn_mknod+0xf0/0x19c [xfs])
[<bf074554>] (xfs_vn_mknod+0x0/0x19c [xfs]) from [<bf074720>] (xfs_vn_create+0x14/0x18 [xfs])
[<bf07470c>] (xfs_vn_create+0x0/0x18 [xfs]) from [<c0091c2c>] (vfs_create+0xc0/0x134)
[<c0091b6c>] (vfs_create+0x0/0x134) from [<c00953e0>] (open_namei+0x648/0x744)
 r8:c4959a58 r7:c4959a58 r6:cb27bd00 r5:00028043 r4:00028042
[<c0094d98>] (open_namei+0x0/0x744) from [<c0087688>] (do_filp_open+0x2c/0x4c)
[<c008765c>] (do_filp_open+0x0/0x4c) from [<c0087854>] (filp_open+0x24/0x28)
 r5:cb27bde6 r4:00000057
[<c0087830>] (filp_open+0x0/0x28) from [<c008d214>] (do_coredump+0x2b0/0x824)
[<c008cf64>] (do_coredump+0x0/0x824) from [<c004c0ec>] (get_signal_to_deliver+0x330/0x444)
[<c004bdbc>] (get_signal_to_deliver+0x0/0x444) from [<c0029380>] (do_signal+0x5c/0x4ec)
[<c0029324>] (do_signal+0x0/0x4ec) from [<c0029840>] (do_notify_resume+0x30/0x34)
[<c0029810>] (do_notify_resume+0x0/0x34) from [<c002660c>] (work_pending+0x1c/0x20)
Filesystem "md0": XFS internal error xfs_trans_cancel at line 1163 of file fs/xfs/xfs_trans.c.  Caller 0xbf067dbc
[<c0025184>] (dump_stack+0x0/0x1c) from [<bf03e70c>] (xfs_error_report+0x5c/0x68 [xfs])
[<bf03e6b0>] (xfs_error_report+0x0/0x68 [xfs]) from [<bf05f610>] (xfs_trans_cancel+0x10c/0x134 [xfs])
 r5:00000000 r4:0045c100
[<bf05f504>] (xfs_trans_cancel+0x0/0x134 [xfs]) from [<bf067dbc>] (xfs_create+0x198/0x504 [xfs])
 r8:00000000 r7:cb27bbcc r6:00000075 r5:00000000 r4:00000000
[<bf067c24>] (xfs_create+0x0/0x504 [xfs]) from [<bf074644>] (xfs_vn_mknod+0xf0/0x19c [xfs])
[<bf074554>] (xfs_vn_mknod+0x0/0x19c [xfs]) from [<bf074720>] (xfs_vn_create+0x14/0x18 [xfs])
[<bf07470c>] (xfs_vn_create+0x0/0x18 [xfs]) from [<c0091c2c>] (vfs_create+0xc0/0x134)
[<c0091b6c>] (vfs_create+0x0/0x134) from [<c00953e0>] (open_namei+0x648/0x744)
 r8:c4959a58 r7:c4959a58 r6:cb27bd00 r5:00028043 r4:00028042
[<c0094d98>] (open_namei+0x0/0x744) from [<c0087688>] (do_filp_open+0x2c/0x4c)
[<c008765c>] (do_filp_open+0x0/0x4c) from [<c0087854>] (filp_open+0x24/0x28)
 r5:cb27bde6 r4:00000057
[<c0087830>] (filp_open+0x0/0x28) from [<c008d214>] (do_coredump+0x2b0/0x824)
[<c008cf64>] (do_coredump+0x0/0x824) from [<c004c0ec>] (get_signal_to_deliver+0x330/0x444)
[<c004bdbc>] (get_signal_to_deliver+0x0/0x444) from [<c0029380>] (do_signal+0x5c/0x4ec)
[<c0029324>] (do_signal+0x0/0x4ec) from [<c0029840>] (do_notify_resume+0x30/0x34)
[<c0029810>] (do_notify_resume+0x0/0x34) from [<c002660c>] (work_pending+0x1c/0x20)
xfs_force_shutdown(md0,0x8) called from line 1164 of file fs/xfs/xfs_trans.c.  Return address = 0xbf05f624
Filesystem "md0": Corruption of in-memory data detected.  Shutting down filesystem: md0
Please umount the filesystem, and rectify the problem(s)
it asks me to rectify the problems... how nice.

adriansienicki
Posts: 21
Joined: Sun May 17, 2015 11:00 am
Location: Internet
Contact:

Re: Shares vanished

Post by adriansienicki » Sat May 23, 2015 8:40 pm

and here's the xfs_db's output:

Code: Select all

/ # xfs_db -c "freespc -s" /dev/md0
xfs_db: /dev/md0 contains a mounted filesystem

fatal error -- couldn't initialize XFS library
/ #

Mijzelf
Posts: 6251
Joined: Mon Jun 16, 2008 10:45 am

Re: Shares vanished

Post by Mijzelf » Sun May 24, 2015 11:11 am

This is beyond my knowledge, I'm afraid. I don't see any clear sources of the problem. Only
Filesystem "md0": Corruption of in-memory data detected. Shutting down filesystem: md0
suggests that there is a memory problem, but maybe it can also be caused by a filesystem corruption or an unreadable sector. The filesystem corruption might be visible in the stack dump, but I don't see any sector error. Maybe it's not in the last 50 lines.

Anyway, I think it's a good idea to get the disks out and connect them to another Linux system, to find out if there is a disk problem or a hardware problem in the NAS. Or maybe it's enough to repair the filesystem, which is hard to do in the NAS. It should not be mounted, but you can't simply unmount it in the NAS. Futher I guess the NAS doesn't have enough memory to run xfs_repair on a 2TB filesystem.

adriansienicki
Posts: 21
Joined: Sun May 17, 2015 11:00 am
Location: Internet
Contact:

Re: Shares vanished

Post by adriansienicki » Sun May 24, 2015 11:33 am

Yes, you're probably right. It must have been caused by a hardware problem. My ubuntu and fedora - both supporting xfs - claim that there are no problems with my hdds. I was strongly advised against connecting my drives to external OS but I had to try. Well it paid off - at least I have now 100% certainty that there's something wrong with my NSA221.
I asked also Zyxel for an assistance, but my warranty is void and I don't expect much out of that.
Anyway, thanks a lot.

Mijzelf
Posts: 6251
Joined: Mon Jun 16, 2008 10:45 am

Re: Shares vanished

Post by Mijzelf » Sun May 24, 2015 12:22 pm

Your FFP stick failed either. Yet the firmware seems to run fine. Have you checked the power supply?

Post Reply