New package: Entware-ng

Marvell Kirkwood based
Post Reply
Mijzelf
Posts: 6226
Joined: Mon Jun 16, 2008 10:45 am

Re: New package: Entware-ng

Post by Mijzelf » Fri Jan 26, 2018 9:17 am

My analysis so far:
According to the Entware-ng archive the previous version of bash was 4.3.42, the current one is 4.4.12.
According to bash CHANGES the use of pselect in readline is introduced in 4.4-beta.

So the previous Entware-ng package of bash did not include a call to pselect, the current one might. (And does. a grep in the binaries proves that.)

The implementation of pselect() in glibc-2.23:

Code: Select all

int
__pselect (int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds,
           const struct timespec *timeout, const sigset_t *sigmask)
{
<snip>
  int result;

#ifndef CALL_PSELECT6
# define CALL_PSELECT6(nfds, readfds, writefds, exceptfds, timeout, data) \
  SYSCALL_CANCEL (pselect6, nfds, readfds, writefds, exceptfds, timeout, data)
#endif

  result = CALL_PSELECT6 (nfds, readfds, writefds, exceptfds, timeout,
                          &data);

# ifndef __ASSUME_PSELECT
  if (result == -1 && errno == ENOSYS)
    result = __generic_pselect (nfds, readfds, writefds, exceptfds, timeout,
                                sigmask);
# endif

  return result;
}
So glibc indeed has a fallback if pselect6 is not available. But __ASSUME_PSELECT shouldn't be set. So I have to find out how to unset that #define. (Or patch the code. But that will have to be done at several places, as the function __generic_pselect() has to be available too.
barmalej2 wrote:So, glibc recompilation with stubs using 2.6.15 or even 2.6.31.8 kernel headers will not help to run binaries, which uses at least pselect6,ppoll or epoll_pwait. The only way is to recompile affected binaries itself
I do not agree. Binaries seldom call kernel functions directly. They use libc functions, which can use emulations. (Which of course can have some drawbacks. The __generic_pselect() has a race condition, but hey, it's used for years without much trouble. I guess the need for pselect6 is intensified with the introduction of multi processor systems.)
I had a look at ppoll(), and it has a simular fallback. Unfortunately that is not true for epoll_pwait():

Code: Select all

#ifdef __NR_epoll_pwait
int epoll_pwait (int epfd, struct epoll_event *events,
                 int maxevents, int timeout,
                 const sigset_t *set)
{
  return SYSCALL_CANCEL (epoll_pwait, epfd, events, maxevents,
                         timeout, set, _NSIG / 8);
}

#else

int epoll_pwait (int epfd, struct epoll_event *events,
                 int maxevents, int timeout,
                 const sigset_t *set)
{
  __set_errno (ENOSYS);
  return -1;
}

#endif
I'd expect a call to epoll_wait() here.

barmalej2
Posts: 2395
Joined: Sun Apr 29, 2012 5:24 pm

Re: New package: Entware-ng

Post by barmalej2 » Fri Jan 26, 2018 11:47 am

Mijzelf wrote:But __ASSUME_PSELECT shouldn't be set. So I have to find out how to unset that #define.
This is exactly, what I proposed in previous post providing the relevant link . I can't imagine how you can run binary built with glibc __ASSUME_PSELECT enabled, unless you patch glibc to undefine __ASSUME_PSELECT to mach your kernel supported syscalls and the fallback machanism comes true only then. Again this shows that is important for glibc to match kernel features.
Mijzelf wrote:They use libc functions, which can use emulations.
Yes, but as you noticed not all syscals has fallback. Regarding glibc ppoll it is something new for me, because uClibc always fail, when ppoll is involved.

philippetev
Posts: 80
Joined: Tue Apr 17, 2012 7:45 am

Re: New package: Entware-ng

Post by philippetev » Fri Jan 26, 2018 11:49 am

Mijzelf wrote:My analysis so far:
According to the Entware-ng archive the previous version of bash was 4.3.42, the current one is 4.4.12.
According to bash CHANGES the use of pselect in readline is introduced in 4.4-beta.

So the previous Entware-ng package of bash did not include a call to pselect, the current one might. (And does. a grep in the binaries proves that.)

The implementation of pselect() in glibc-2.23:

Code: Select all

int
__pselect (int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds,
           const struct timespec *timeout, const sigset_t *sigmask)
{
<snip>
  int result;

#ifndef CALL_PSELECT6
# define CALL_PSELECT6(nfds, readfds, writefds, exceptfds, timeout, data) \
  SYSCALL_CANCEL (pselect6, nfds, readfds, writefds, exceptfds, timeout, data)
#endif

  result = CALL_PSELECT6 (nfds, readfds, writefds, exceptfds, timeout,
                          &data);

# ifndef __ASSUME_PSELECT
  if (result == -1 && errno == ENOSYS)
    result = __generic_pselect (nfds, readfds, writefds, exceptfds, timeout,
                                sigmask);
# endif

  return result;
}
So glibc indeed has a fallback if pselect6 is not available. But __ASSUME_PSELECT shouldn't be set. So I have to find out how to unset that #define. (Or patch the code. But that will have to be done at several places, as the function __generic_pselect() has to be available too.
barmalej2 wrote:So, glibc recompilation with stubs using 2.6.15 or even 2.6.31.8 kernel headers will not help to run binaries, which uses at least pselect6,ppoll or epoll_pwait. The only way is to recompile affected binaries itself
I do not agree. Binaries seldom call kernel functions directly. They use libc functions, which can use emulations. (Which of course can have some drawbacks. The __generic_pselect() has a race condition, but hey, it's used for years without much trouble. I guess the need for pselect6 is intensified with the introduction of multi processor systems.)
I had a look at ppoll(), and it has a simular fallback. Unfortunately that is not true for epoll_pwait():

Code: Select all

#ifdef __NR_epoll_pwait
int epoll_pwait (int epfd, struct epoll_event *events,
                 int maxevents, int timeout,
                 const sigset_t *set)
{
  return SYSCALL_CANCEL (epoll_pwait, epfd, events, maxevents,
                         timeout, set, _NSIG / 8);
}

#else

int epoll_pwait (int epfd, struct epoll_event *events,
                 int maxevents, int timeout,
                 const sigset_t *set)
{
  __set_errno (ENOSYS);
  return -1;
}

#endif
I'd expect a call to epoll_wait() here.
How about if you just build the latest version 2.26 (targeting the old kernel of course), which contain some changes, described by barmalej2 here?
ZyXEL NSA-310 FW 4.70(AFK.1) MetaRepository + Fonz Fun Plug 0.7zypkg004

Mijzelf
Posts: 6226
Joined: Mon Jun 16, 2008 10:45 am

Re: New package: Entware-ng

Post by Mijzelf » Sat Jan 27, 2018 10:32 am

barmalej2 wrote:Regarding glibc ppoll it is something new for me, because uClibc always fail, when ppoll is involved.
That is a mayor difference between glibc and uClibc. uClibc most prominent feature is it's size. To be so small some other features are sacrificed. As uClibc targets embedded systems, the need to apply stubs is not high. In most cases you build your toolchain against your kernel. So all features are fixed.

glibc tries to provide a stable API, binaries build for glibc version x, will also work on glibc version y. And it should work on all kernels. To work on all kernels, without ignoring the features of newer kernels, it needs stubs. These add to the size and complexity of the library, but glibc is more targeted on compatibility than on size.

About ASSUME_PTHREAD, it is by default defined

Code: Select all

/* pselect/ppoll were introduced just after 2.6.16-rc1.  On x86_64 and
   SH this appeared first in 2.6.19-rc1, on ia64 in 2.6.22-rc1.  */
#define __ASSUME_PSELECT        1
#define __ASSUME_PPOLL          1
and only undefined for some versions of the microblaze (whatever that is) or m68k kernel. So using unpatched 2.23 glibc sources it's not possible to build a glibc for arm with a pselect stub.
Of course that can be solved (if I can get the build process running. Entware-ng doesn't build at all at the moment for me.)

philippetev
Posts: 80
Joined: Tue Apr 17, 2012 7:45 am

Re: New package: Entware-ng

Post by philippetev » Sat Jan 27, 2018 7:48 pm

Meanwhile, I made a new bash package, configured with ac_cv_func_pselect=no to serve as a workaround for now. It's compiled against the current Entware-ng packages tree, so it should work for most people. Those, who are interested, the new ipk package is here.
Mijzelf wrote:That could be

Code: Select all

#!/bin/busybox sh

[ -t 1 ] && exec /opt/bin/bash "$@"

exec /bin/busybox sh "$@"
This checks if stdout is a console, and exec's bash if that's true.
One correction: in order all its settings file to be read (/etc/profile, /etc/bash_profile, ~/.profile, ~/.bash_profile, etc.) bash must be started with the --login option like this:

Code: Select all

[ -t 1 ] && exec /opt/bin/bash --login "$@"
otherwise none of the above (except maybe /etc/bashrc and ~/.bashrc, if exist) will be sourced.
ZyXEL NSA-310 FW 4.70(AFK.1) MetaRepository + Fonz Fun Plug 0.7zypkg004

barmalej2
Posts: 2395
Joined: Sun Apr 29, 2012 5:24 pm

Re: New package: Entware-ng

Post by barmalej2 » Sat Jan 27, 2018 10:25 pm

Mijzelf wrote:glibc tries to provide a stable API, binaries build for glibc version x, will also work on glibc version y. And it should work on all kernels.
Well, it sounds too good to be true. In theory everything is possible, but in practice you have non-working bash binary on NSA3**, because of changes in glibc versions. My little investigation regarding pselect() changes in glibc:
2009-11-19. Assume pselect6 and ppoll on ARM for kernels 2.6.32 and later
--- a/sysdeps/unix/sysv/linux/arm/kernel-features.h
+++ b/sysdeps/unix/sysv/linux/arm/kernel-features.h
@@ -59,6 +59,8 @@

#include_next <kernel-features.h>

-/* These syscalls are not implemented yet for ARM. */
-#undef __ASSUME_PSELECT
-#undef __ASSUME_PPOLL
+/* Support for pselect6, ppoll and epoll_pwait was added in 2.6.32. */
+#if __LINUX_KERNEL_VERSION < 0x020620
+# undef __ASSUME_PSELECT
+# undef __ASSUME_PPOLL
+#endif
2014-05-12. Clean up kernel version conditionals for pre-2.6.32 kernels.
This patch does some initial cleanup, following the move to 2.6.32
minimum kernel version, by removing __LINUX_KERNEL_VERSION
conditionals that are now always-true or always-false.
In the case of
__ASSUME_ARG_MAX_STACK_BASED, where the conditional used a kernel
version that was itself in a macro, the associated sysconf.c code is
also cleaned up and __ASSUME_ARG_MAX_STACK_BASED removed completely.
--------------snip------------
* sysdeps/unix/sysv/linux/arm/kernel-features.h
(__ASSUME_SIGFRAME_V2): Likewise.
)__ASSUME_EVENTFD2): Likewise.
(__ASSUME_SIGNALFD4): Likewise.
(__ASSUME_PSELECT): Do not undefine conditionally.
(__ASSUME_PPOLL): Likewise.
--------------snip------------
--- a/sysdeps/unix/sysv/linux/arm/kernel-features.h
+++ b/sysdeps/unix/sysv/linux/arm/kernel-features.h
-/* Support for pselect6, ppoll and epoll_pwait was added in 2.6.32. */
-#if __LINUX_KERNEL_VERSION < 0x020620
-# undef __ASSUME_PSELECT
-# undef __ASSUME_PPOLL
-#endif
--------------snip---------------
diff --git a/sysdeps/unix/sysv/linux/kernel-features.h b/sysdeps/unix/sysv/linux/kernel-features.h
+/* pselect/ppoll were introduced just after 2.6.16-rc1. On x86_64 and
+ SH this appeared first in 2.6.19-rc1, on ia64 in 2.6.22-rc1. */
+#define __ASSUME_PSELECT 1
+#define __ASSUME_PPOLL 1
So, I think, bash would run with all glibc versions between 2009-11-19 and 2014-05-12. Incompatible ones are since 2014-05-12.
Overall I agree, that glibc is less "headache" libc comparing with others libc implementations.
philippetev wrote:Meanwhile, I made a new bash package, configured with ac_cv_func_pselect=no to serve as a workaround for now
Sounds great.

Mijzelf
Posts: 6226
Joined: Mon Jun 16, 2008 10:45 am

Re: New package: Entware-ng

Post by Mijzelf » Sun Jan 28, 2018 8:35 pm

barmalej2 wrote:in practice you have non-working bash binary on NSA3**, because of changes in glibc versions.
Nope. glibc did not change, only the bash package did. The difference is that bash now uses pselect, which it didn't do before.
pselect never worked in my libc-2.4.24 package. Apparently it's not used much, or the binaries which did handled the ENOSYS correctly, and choose another method.

Meanwhile I managed to build a new libc, which has all available emulations switched on. Including pselect(). The stock bash runs now on my NSA325.
@philippetev : Can you test it? I send you a download link.

philippetev
Posts: 80
Joined: Tue Apr 17, 2012 7:45 am

Re: New package: Entware-ng

Post by philippetev » Sun Jan 28, 2018 8:39 pm

Mijzelf wrote:
barmalej2 wrote:@philippetev : Can you test it? I send you a download link.
Sure, go ahead!
ZyXEL NSA-310 FW 4.70(AFK.1) MetaRepository + Fonz Fun Plug 0.7zypkg004

Mijzelf
Posts: 6226
Joined: Mon Jun 16, 2008 10:45 am

Re: New package: Entware-ng

Post by Mijzelf » Tue Jan 30, 2018 8:38 am

Added the libc update to the repo. @dexhain: I guess this solves also your problems.

dexhain
Posts: 27
Joined: Tue Apr 02, 2013 11:50 am

Re: New package: Entware-ng

Post by dexhain » Tue Jan 30, 2018 10:32 am

Mijzelf wrote:Added the libc update to the repo. @dexhain: I guess this solves also your problems.
Think the version number in the Packages file needs a bump, it's still at 2.23-4. I was able to get the new version by using opkg's --force-reinstall though, and yes, bash works fine now (although this issue meant I've since given zsh and fish a proper try :)).

It doesn't seem to have fixed the python3 virtualenv issue I was having, but I have a couple of things to try on that first (complete re-install of the python3 packages, and possibly try a downgrade to the previous version)

Mijzelf
Posts: 6226
Joined: Mon Jun 16, 2008 10:45 am

Re: New package: Entware-ng

Post by Mijzelf » Tue Jan 30, 2018 11:58 am

dexhain wrote:Think the version number in the Packages file needs a bump, it's still at 2.23-4.
Don't know how that happened. I did update it. But maybe I renamed it the wrong way at the end. Anyway, it should be solved by now.

philippetev
Posts: 80
Joined: Tue Apr 17, 2012 7:45 am

Re: New package: Entware-ng

Post by philippetev » Tue Jan 30, 2018 3:48 pm

BTW, messing with the /bin/sh symlink (as suggested here) is a bad idea. The wrapper script actually works, but the NAS (at least mine NSA310) wasn't able to reboot or shutdown on several occasions. Changing the default shell in /etc/passwd would be better idea, it changes only the login shell for the user and none of the scripts will be affected, because all of them are set to use /bin/sh (#!/bin/sh) by default.
I've added the following line to /opt/etc/init.d/S99ProfileHook:
echo "[ -t 1 -a -f /opt/etc/profile ] && . /opt/etc/profile # Entware Hook" >>/etc/profile
sed -i 's/root:\/bin\/sh/root:\/opt\/bin\/bash/g' /etc/passwd
and I'm using it for already three or four days. No shutdown/reboot problems so far.
ZyXEL NSA-310 FW 4.70(AFK.1) MetaRepository + Fonz Fun Plug 0.7zypkg004

Mijzelf
Posts: 6226
Joined: Mon Jun 16, 2008 10:45 am

Re: New package: Entware-ng

Post by Mijzelf » Tue Jan 30, 2018 9:05 pm

Thanks for the idea. I guess my script causes problems because /bin/sh is also the she-bang for the shutdown scripts (and other, less visible scripts). That should be no problem, but maybe poor busybox can't handle that.

philippetev
Posts: 80
Joined: Tue Apr 17, 2012 7:45 am

Re: New package: Entware-ng

Post by philippetev » Wed Jan 31, 2018 3:59 pm

One more tip: if you want to use one and the same settings in bash and busybox, store all your shell settings in ~/.bash_profile and make a symlink to that file in the home folder, called ~/.profile. The first name is being sourced automatically by bash, the second one (the symlink) - by busybox.
ZyXEL NSA-310 FW 4.70(AFK.1) MetaRepository + Fonz Fun Plug 0.7zypkg004

philippetev
Posts: 80
Joined: Tue Apr 17, 2012 7:45 am

Re: New package: Entware-ng

Post by philippetev » Fri Feb 09, 2018 7:35 pm

Guys, do you know any way to start the processes as other user? Entware's rc.func-based init script subsystem doesn't seems to support starting the processes as other user than root and you know this is a potential security hole.
ZyXEL NSA-310 FW 4.70(AFK.1) MetaRepository + Fonz Fun Plug 0.7zypkg004

Post Reply