Medion Life P89636: MetaRepository: Pyload: unrar error

[Pollux]

Hi,
I've got the Medion Life P89636 aka NSA-212 with MetaRepository and the Application pyload installed.
Pyload runs fine, just the automatic extraction "ExtractArchive" doesn't work.
Here's the log from pyload:

Code: Select all

994	04.03.2017 20:16:05	INFO	ADDON ExtractArchive: Check package: test
995	04.03.2017 20:16:05	INFO	ADDON ExtractArchive: test.part1.rar | Extract to: /i-data/md0/public/test/
996	04.03.2017 20:16:06	ERROR	ADDON ExtractArchive: (13, 'Permission denied')
997	04.03.2017 20:16:06	ERROR	ADDON ExtractArchive: test.part1.rar | Archive error | unrar: /usr/lib/libstdc++.so.6: version `CXXABI_ARM_1.3.3' not found (required by unrar)
998	04.03.2017 20:16:06	ERROR	ADDON ExtractArchive: test.part1.rar | Extract failed 

I think there are two points,
- first 'Permission denied', but pyload is running with user 'admin' and the owner of the folder /i-data/md0/public is 'admin', group 'nobody', permissions '0777'. Pyload can create folders in the directory for the packages downloaded, but the packages couldn't be extracted from the plugin/addon 'ExtractArchive' in the same folder?!?

- second, as already described by Mijzelf here: http://zyxel.nas-central.org/wiki/Using ... _(NSA-212) the NSA-212 needs another version of '/usr/lib/libstdc++.so.6' for some packages.
Are they available for NSA-212?
If no, I have no clue how to fix this, a quick search brings me here:
https://rpmfind.net/linux/rpm2html/sear ... 2B%2B.so.6. I would pick an 'arm' package, but which exactly? And how to proceed?

If you need some more information, just give me a note.


Before installing pyload via MetaRepository I was using pyload via optware via ffp (fonz fun plug), that was running good.
Maybe it's some help, I followed these instructions (in German - but an online-translator might be useful ):
1. HDD-Installation of ffp (fonz fun plug): https://wolf-u.li/4523/hdd-installation ... -geraeten/
2. Installation of additional packages: https://wolf-u.li/4531/installation-zus ... _plug-0-7/
3. Installation of optware: https://wolf-u.li/3869/installation-von ... c-ch3mnas/
4. Installation of pyload: https://wolf-u.li/3921/installation-von ... f-dem-nas/

With this configuration it's downloading and auto-extracting, but you have to set the parameter in 'Extract Archive' for 'Move to trash instead delete' (from ON to OFF, because NSA-212 isn't moving things to trash) again after every restart of the NSA-212.
Issue is known: https://github.com/pyload/pyload/issues/1122 merged with https://github.com/pyload/pyload/issues/1157.
That's a little bit annoying and that's why I gave the MetaRepository a try.


Best regards,
Pollux

[Mijzelf]

You can use the libstdc++ extracted from NSA310 rootfs, as provided here. But of course there is no easy universal way to add it to your box, else I would have provided it with MR.

Normally a library file is searched for in /lib and /usr/lib, and maybe some other places dependent on the way the executable and /lib/ld-linux.so are build. As /lib is volatile, and /usr/lib is read-only, these directories are not suitable for installing extra libs needed by installable packages. For that reason they are installed in /usr/local/zy-pkgs/lib, and to tell /lib/ld-linux.so where to find them, an environment variable LD_LIBRARY_PATH=/usr/local/zy-pkgs/lib/ is set.
When this environment variable is set, ld-linux.so first searches the directory(ies) specified here, and then the default paths. So the naive way to install the libstdc++ libraries is to put them in /usr/local/zy-pkgs/lib/, as they will be used rather than the versions in /usr/lib/.

However

The firmware has set that variable globally, which means that the firmware binaries using that library also get the NSA310 ones. And unfortunately the incompatibility is 2 ways. So the naive solution will break other parts of the firmware.

If unrar is the only binary in pyload which needs libstdc++.so, you can solve the problem with a wrapper script. Locate unrar (probably in /usr/local/zy-pkgs/bin) and rename it to unrar.bin.
Then create an (executable) shell script unrar instead, containing:

Code: Select all

#!/bin/sh

LD_LIBRARY_PATH=/path/where/you/stored/libstdc++/ exec $0.bin "$@"

This way unrar gets it's private LD_LIBRARY_PATH, and other binaries are not affected.

If other parts of pyload also need this library, you'll have to edit the pyload startscript, to provide a private LD_LIBRARY_PATH to the daemon, on startup.

[Pollux]

Hi Mijzelf,
Thank you very much for your very quick reply.

Here's what I did: (I'm describing it as detailed as possible, maybe it's sounding a little bit like a instruction for a DAU - just to exclude EIFOK )
1. Downloaded file 'libstdc++.tar.gz' with Windows and extracted it with WinRAR to temporary folder, moved the extracted files with WinSCP to 'i-data/md0/admin/libstdc++/', gave the folder 'i-data/md0/admin/libstdc++/' the permission '0755' and the files in it also permission '0755'
2. In '/usr/local/zy-pkgs/bin': renamed 'unrar' to 'unrar.bin'
3. In '/usr/local/zy-pkgs/bin': created script 'unrar' containing:

Code: Select all

#!/bin/sh

LD_LIBRARY_PATH=/i-data/md0/admin/libstdc++/ exec $0.bin "$@"

4. In '/usr/local/zy-pkgs/bin': Gave 'unrar' Permision '0755'
5. Restarted the NSA-212 via SSH with 'reboot'

Pyload is still giving this error in the log:

Code: Select all

254	05.03.2017 22:17:33	INFO	Download finished: 000test.rar
255	05.03.2017 22:17:33	INFO	Package finished: 000test
256	05.03.2017 22:17:33	INFO	ADDON ExtractArchive: Check package: 000test
257	05.03.2017 22:17:34	INFO	ADDON ExtractArchive: 000test.rar | Extract to: /i-data/md0/public/000test/
258	05.03.2017 22:17:34	ERROR	ADDON ExtractArchive: (13, 'Permission denied')
259	05.03.2017 22:17:34	ERROR	ADDON ExtractArchive: 000test.rar | Archive error | unrar: /usr/lib/libstdc++.so.6: version `CXXABI_ARM_1.3.3' not found (required by unrar)
260	05.03.2017 22:17:34	ERROR	ADDON ExtractArchive: 000test.rar | Extract failed 

The erorr still states the path is '/usr/lib/'?!? Did I something wrong?
Is Pyload maybe using some 'daemonised' unrar from another source?!?


By the way, is it normal to have in the directory '/usr/local/zy-pkgs/bin' 'unrar' and 'unrar-5.4.5'? Is the one the standard and the other from the MR?

Code: Select all

Name        Size in B   Changed
unrar         308.604   08.05.2013 04:59:05
unrar-5.4.5   266.732   29.01.2017 15:09:35

I also renamed 'unrar-5.4.5' to 'unrar.bin' and tried, but the same error occurs in the log. It seems as it doesn't take the private LD_LIBRARY_PATH. Where's the pyload startscript located? Should I run pyload with this private LD_LIBRARY_PATH?


Best regards,
Pollux

[Mijzelf]

By the way, is it normal to have in the directory '/usr/local/zy-pkgs/bin' 'unrar' and 'unrar-5.4.5'?

You installed RandomTools, which also provides an unrar (It seems that the unrar provided with (the previous version of?) nzbget isn't suitable to extract recent rar files. So I provided the latest.).
The startscript of RandomTools generates a symlink /bin/unrar which points to unrar-5.4.5. If /bin is first in the search path, this unrar will be used.

If you created an unrar-5.4.5.bin and a unrar-5.4.5 script, this should also work, of course.

Is Pyload maybe using some 'daemonised' unrar from another source?!?

Have you tried to run unrar manually? '/usr/local/zy-pkgs/bin/unrar' should show it's manual. (And of course also just 'unrar', invoking the RT version.)

If you have RT, you also have ldd.

Code: Select all

LD_LIBRARY_PATH=/i-data/md0/admin/libstdc++/ ldd /usr/local/zy-pkgs/bin/unrar.bin

should show where it finds it's libraries.

Where's the pyload startscript located?

In /usr/locals/zy=pkgs/etc/init.d/patched/. Normally the startscripts are in /usr/locals/zy=pkgs/etc/init.d/, but because the Zyxel packages handle webinterface urls differently from the Medion one's, the MR startscript is put in between, to do some translation.

[Pollux]

Yes, I've installed MetaRepository (MR), MidnightCommander, MuninLite, RandomTools (RT), SMART, Tweaks, ffp and pyload.

Have you tried to run unrar manually?

If I'm starting the '/usr/local/zy-pkgs/bin/unrar' it shows UNRAR 5.40:

Code: Select all

# /usr/local/zy-pkgs/bin/unrar

UNRAR 5.40 freeware      Copyright (c) 1993-2016 Alexander Roshal

Usage:     unrar <command> -<switch 1> -<switch N> <archive> <files...>
               <@listfiles...> <path_to_extract\>

<Commands>
(...)

But if I'm just using 'unrar' I got the same error as in the pyload log.

Code: Select all

# unrar
unrar: /usr/lib/libstdc++.so.6: version `CXXABI_ARM_1.3.3' not found (required by unrar)

If I'm using '/bin/unrar' I'm getting the same error:

Code: Select all

# /bin/unrar
/bin/unrar: /usr/lib/libstdc++.so.6: version `CXXABI_ARM_1.3.3' not found (required by /bin/unrar)

The check of the link '/bin/unrar' shows to 'unrar-5.4.5'.

Code: Select all

/bin/unrar -> /usr/local/zy-pkgs/bin/unrar-5.4.5

Is 'unrar-5.4.5' broken?

Code: Select all

# /usr/local/zy-pkgs/bin/unrar-5.4.5
/usr/local/zy-pkgs/bin/unrar-5.4.5: /usr/lib/libstdc++.so.6: version `CXXABI_ARM_1.3.3' not found (required by /usr/local/zy-pkgs/bin/unrar-5.4.5)

That's the output from ldd:

Code: Select all

# LD_LIBRARY_PATH=/i-data/md0/admin/libstdc++/ ldd /usr/local/zy-pkgs/bin/unrar.bin
        libstdc++.so.6 => /i-data/md0/admin/libstdc++/libstdc++.so.6 (0x40028000)
        libm.so.6 => /lib/libm.so.6 (0x400e0000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x4018c000)
        libpthread.so.0 => /usr/lib/libpthread.so.0 (0x401a0000)
        libc.so.6 => /lib/libc.so.6 (0x401c0000)
        /lib/ld-linux.so.3 (0x40000000)

How should I proceed? Deinstall RT and reinstall it? What do you think?

[Pollux]

I added the full path for unrar.bin (copied 'unrar-5.4.5') to the wrapper-script:

Code: Select all

#!/bin/sh

LD_LIBRARY_PATH=/i-data/md0/admin/libstdc++/ exec /usr/local/zy-pkgs/bin/unrar.bin "$@"

Then it's working, I'm getting this by entering 'unrar':

Code: Select all

# unrar

UNRAR 5.40 freeware      Copyright (c) 1993-2016 Alexander Roshal

Usage:     unrar <command> -<switch 1> -<switch N> <archive> <files...>
               <@listfiles...> <path_to_extract\>

<Commands>
(...)

And if I change the pointing to the original 'unrar' (which I renamed to 'unrar.bck'):

Code: Select all

#!/bin/sh

LD_LIBRARY_PATH=/i-data/md0/admin/libstdc++/ exec /usr/local/zy-pkgs/bin/unrar.bck "$@"

It's also working and giving UNRAR 4.00:

Code: Select all

unrar

UNRAR 4.00 freeware      Copyright (c) 1993-2011 Alexander Roshal

Usage:     unrar <command> -<switch 1> -<switch N> <archive> <files...>
               <@listfiles...> <path_to_extract\>

<Commands>
(...)

[Mijzelf]

Is 'unrar-5.4.5' broken?

No, it just has the same problem as the stock unrar.

It's also working

So your problem is solved now?

[Pollux]

So your problem is solved now?

After a short test with a .txt in a .rar it seems to download an extract. The .txt is readable.
Yes, now unrar is working without error. Jippie!


But in the pyload log it still states a "permission denied" error from ExtractArchive.

Code: Select all

942	06.03.2017 21:57:20	INFO	Download finished: 000test.rar
943	06.03.2017 21:57:20	INFO	Package finished: 000test
944	06.03.2017 21:57:20	INFO	ADDON ExtractArchive: Check package: 000test
945	06.03.2017 21:57:20	INFO	ADDON ExtractArchive: 000test.rar | Extract to: /i-data/md0/public/000test/
946	06.03.2017 21:57:20	ERROR	ADDON ExtractArchive: (13, 'Permission denied')
947	06.03.2017 21:57:20	INFO	ADDON ExtractArchive: Deleting 1 files
948	06.03.2017 21:57:20	INFO	ADDON ExtractArchive: 000test.rar | Extracting finished 

It's extracting to the same folder in which the downloaded parts are saved. Do you have any clue why the error 'Permission denied' occurs?

[Mijzelf]

Do you have any clue why the error 'Permission denied' occurs?

No. Maybe add some logging:

Code: Select all

!#/bin/sh

date >>/tmp/unrar.log
whoami >>/tmp/unrar.log
echo "$0  $@" >>/tmp/unrar.log

LD_LIBRARY_PATH=...

Now you will know how exactly unrar is invoked, as which user, and maybe that will make it clear.

[Pollux]

Hi Mijzelf,

I've added the logging to the scripts and downloaded my '000test.txt' paket in '000test.rar'. Here's the log:

Code: Select all

Wed Mar  8 17:42:33 GMT 2017
admin
/bin/unrar  l -o+ -xsample -x*.nfo -x*.DS_Store -xindex.dat -xthumb.db -y -p- -v /i-data/md0/public/000test/000test.rar
Wed Mar  8 17:42:33 GMT 2017
admin
/bin/unrar  x -o+ -xsample -x*.nfo -x*.DS_Store -xindex.dat -xthumb.db -y -p- /i-data/md0/public/000test/000test.rar /i-data/md0/public/000test/
Wed Mar  8 17:42:33 GMT 2017
admin
/bin/unrar  vb -o+ -xsample -x*.nfo -x*.DS_Store -xindex.dat -xthumb.db -y -p- -v /i-data/md0/public/000test/000test.rar

What I can read out of this is, unrar is executed as user 'admin'.
Folder public has the permission '0777', group 'nobody [99]', owner 'admin [501]' and the pyload/ExtractArchive created folder '000test' has the same ones.
I think I can live with this error. Pyload is extracting and the extracted files are ok.
Thank you very much!

The only thing of which I have to be aware is an update of 'unrar' in the RandomTools from the MetaRepository, or?

Best regards,
Pollux

[Mijzelf]

Oh, did the extraction succeed? In that case the 'permission denied' can have a totally other source. For instance an attempt to move the rar file to a recyclebin, which doesn't exist.

The only thing of which I have to be aware is an update of 'unrar' in the RandomTools from the MetaRepository, or?

What do you mean?

[Pollux]

For instance an attempt to move the rar file to a recyclebin, which doesn't exist.

This option is set to 'off' for Extract Archive. Here are the parameters of the plugin ExtractArchive in pyload:

Code: Select all

Move to trash instead delete: 	off
Repair broken archives (RAR required): 	off
Use password file: 	on
Extract archives in archives: 	on
Try to extract broken archives: 	off
Create subfolder for each package: 	off
Extract files to folder: 	'empty'
Activated: 	on
Process priority: 	0
Don't extract the following files: 	sample,*.nfo,*.DS_Store,index.dat,thumb.db
Extract archives ending with extension: 	7z,bz2,bzip2,gz,gzip,lha,lzh,lzma,rar,tar,taz,tbz,tbz2,tgz,xar,xz,z,zip
Password file: 	unrar_passwords.txt
Extract with full paths: 	on
Run after all downloads was processed: 	off
Delete archive after extraction: 	on
Overwrite files: 	on

Do I have to make the running script / program of ExtractArchive (I think it's named 'ExtractArchive.py') writeable?!? So it could be overwritten?!?

What do you mean?

I've renamed the 'unrar-5.4.5' to 'unrar-5.4.5.bin' and I've the wrapper script named 'unrar-5.4.5' in the directory '/usr/local/zy-pkgs/bin/'. I think this is the directory in which the RandomTools are saved.
If there's an update of the RandomTools to e.g. 'unrar-8.8.8' then
- the old unrar-5.4.5 will be deleted, which is the wrapper script and
- the new link for 'unrar' will point from '/bin/' to '/usr/local/zy-pkgs/bin/unrar-8.8.8'. And this 'unrar-8.8.8' wouldn't work because it has no wrapper script.

Hopefully this isn't written too crazy.

[Mijzelf]

Hopefully this isn't written too crazy.

No. You are right. After updating (or re-installing) RT you'll have to re-apply the patch.