General NAS-Central Forums

Welcome to the NAS community
It is currently Sun Sep 24, 2017 11:03 pm

All times are UTC




Post new topic Reply to topic  [ 19 posts ]  Go to page Previous  1, 2
Author Message
PostPosted: Thu May 04, 2017 12:17 am 
Offline

Joined: Fri Jul 03, 2015 10:47 am
Posts: 4
Mijzelf wrote:
/etc is on a ramdrive, and so it doesn't survive a reboot. You'll have to put your mount in a script, and put that in /usr/local/zy-pkgs/etc/init.d/. It will be called on boot with a 'startup' argument.

Please enlight me, Mijzelf, where is such place in NAS542.
I find out that reboot-resistant is /etc/zyxel/ and packages are in /i-data/.system/zy-pkgs on NAS542. But no etc/init.d in no one of that dirs, and my effort to rename/copy/modify settings from one of packages (in order to set my own startup script) was not successful (probably I did not modify some configuration file where all packages are enumerated, so system do not know about existence of my fake-package). I also locate my package to /i-data/sysvol/.PKG/ and give appropriate 755 rights to /i-data/sysvol/.PKG/<packagename>/etc/init.d/<packagename> scripts with no effect :(

In rcS1 and rcS2 I also do not found reference to place, where I can put my own startup script. Can you please tell me such place location (on NAS542)?

They completely changed mechanism of booting, disks referencing and so on, isn't they?.. Nevertheless, your FFP stick still booting well. So, you know the way :-)

I never wrote my own regular package, because I simply do not found any materials, learning how to do so (for NAS542). Can you please take a link to some tutorial?
Sorry for asking on NAS542 in NSA320 section...


Last edited by voland on Thu May 04, 2017 8:02 pm, edited 3 times in total.

Top
 Profile  
 
PostPosted: Thu May 04, 2017 7:13 pm 
Offline

Joined: Mon Jun 16, 2008 10:45 am
Posts: 6039
voland wrote:
In rcS1 and rcS2 I also do not found reference to place, where I can put my own startup script. Can you please tell me such place location (on NAS542)?

The file /etc/init.d/zypkg_controller.sh does the job, and contains this gem:
Code:
   # - start to startup packages according to ${USRPKG_DEPS_START}
   while read zypkg; do
      bname=`basename ${zypkg}`
      zypkg=${ZYPKG_PKG_SRC_PATH}/${bname}
      Processed_Packages=${Processed_Packages}:${bname}
      write_log "- starting package \"${bname}\" ..."
      #ckeck ZYPKG_DEPS format
      CHKVERSION=`grep "/i-data/" ${zypkg}`
      if [ "$?" != "0" ]; then
         PKGVOLPATH=`grep ${bname} ${PKGSTATUSFILE} |grep "Installed-Rule" |awk -F":" '{print $2}' |sed 's/\/$//g' |sed 's/ //g'`
         zypkg=${PKGVOLPATH}/etc/init.d/${bname}
      fi
      if [ ! -x ${zypkg} ]; then
         write_log "---> Error: start-up program \"${zypkg}\" is not existed or not excutable"
      else
         ${zypkg} startup
         if [ "$?" == "0" ]; then
            write_log "---> start \"${bname}\" successfully."
         else
            write_log "---> start \"${bname}\" failed."
         fi
      fi
   done < ${USRPKG_DEPS_START}
So if you add a startscript to ${USEPKG_DEPS_START} and that startscript is inside ${ZYPKG_PKG_SRC_PATH}, and it is executable, than it will be called with a param 'startup' on boot.
${USRPKG_DEPS_START} is /i-data/.system/zy-pkgs/USRPKG_DEPS_START. That file doesn't exist, but you can create it. ${ZYPKG_PKG_SRC_PATH} is /i-data/.system/zy-pkgs/. The script has to be in this directory, but a symlink will also do.
If you also want to call your script on shutdown, use the file /i-data/.system/zy-pkgs/USRPKG_DEPS_SHUTDOWN . Your script will be called with 'shutdown'.

Quote:
I never wrote my own regular package, because I simply do not found any materials, learning how to do so (for NAS542). Can you please take a links on some tutorials?
I partly documented the case here. Actually it's no big deal. By just extracting some existing packages, it's rather easy to find how a package is build.


Top
 Profile  
 
PostPosted: Mon May 08, 2017 4:50 am 
Offline

Joined: Fri Jul 03, 2015 10:47 am
Posts: 4
Mijzelf wrote:
voland wrote:
In rcS1 and rcS2 I also do not found reference to place, where I can put my own startup script. Can you please tell me such place location (on NAS542)?
The file /etc/init.d/zypkg_controller.sh does the job. <...> So if you add a startscript to ${USEPKG_DEPS_START} and that startscript is inside ${ZYPKG_PKG_SRC_PATH}, and it is executable, than it will be called with a param 'startup' on boot.
${USRPKG_DEPS_START} is /i-data/.system/zy-pkgs/USRPKG_DEPS_START. That file doesn't exist, but you can create it. ${ZYPKG_PKG_SRC_PATH} is /i-data/.system/zy-pkgs/. The script has to be in this directory, but a symlink will also do.
If you also want to call your script on shutdown, use the file /i-data/.system/zy-pkgs/USRPKG_DEPS_SHUTDOWN . Your script will be called with 'shutdown'.
To be honest, I analyzed /etc/init.d/zypkg_controller.sh before, and (experimentally) found that script is not fully functional, at least part you mentioned above. After seen inside typos like "#ckeck ZYPKG_DEPS format" in place no dealing at all with ${ZYPKG_DEPS} (despite it is commented out) and realizing I cannot modifying this file permanently, I leave it. But your advice and respect to your recognizable work push me to analyze it more thoroughly. So, there are results:
  • Function write_log() in /etc/init.d/zypkg_controller.sh have (for speeding up reason?) commented out code for writing message to logfile, leaving only part for write message to stdout:
    Code:
    write_log()
    {
       echo $1
       #echo "`date "+%Y-%m-%d %H:%M:%S"` [booting] $1" >> ${ZYPKG_LOG_FILE}
    }
    This prevents from logging booting process to file.
    Furthermore, just "[booting]" is weak identification of message source; I expecting script name there.
  • If you put only one line without <Enter> inside USRPKG_DEPS_START/USRPKG_DEPS_SHUTDOWN, it will not work at all - script just do not enter appropriate "while do" loop. And this is expectable. Just for remembering.
  • If you leave WinSCP (for those who work under Windows) at "Auto" file transfer mode, files without extension (like USRPKG_DEPS_START) will be treated as binary, so your <Enter> will be saved as <CR><LF>, and then ${zypkg} and ${bname} will contain <CR>, what brings error as you can see there, literally, with newline:
    Code:
    Error: start-up program "/etc/init.d/<my-script-name>
    " is not existed or not excutable
    So please be prudent and turn on "Text" file transfer mode or use PuTTY/native Linux environment for modifying config files.
  • Function write_log() from calling zypkg_controller.sh, nor any defined there variables, cannot be used in my own script, listed in USRPKG_DEPS_START. But, for example, ${ECHO} is defined in /init and through /linuxrc somehow passed to /etc/init.d/rcS (without "export ECHO" nor "set -a"). This effectively prevents from system integration of my script.
    If such (obvious) Linux script calling behavior is used there for security reason (e.g. for preventing from modifying of system variables) or for return code receiving reason (BTW preventing also from unintentional finishing calling script by "exit 0" command in subscript), ZyXEL developers could just source it in subscript, such as
    Code:
    (. ${zypkg} startup)
    instead of just calling
    Code:
    ${zypkg} startup
    In way zypkg_controller.sh is actually written, I must to code my own write_log() function with specifying exactly path of system log file, or write to my own log file, or send messages somewhere in vanishing stdout :(
  • During analyzing of fragment (at line 216 and below)
    Code:
    [ "${IsStarted}" == "0" ] && continue
    # start to run start commands for packages
    echo ${zypkg} | grep "^#" >/dev/null
    if [ "$?" != "0" ]; then
       bname=`basename ${zypkg}`
       Processed_Packages=${Processed_Packages}:${bname}
       write_log "- starting package \"${bname}\" ..."
       # ckeck ZYPKG_DEPS format
       CHKVERSION=`grep "/i-data/" ${zypkg}`
    I noticed absence of redefining ${zypkg} with full path as
    Code:
    zypkg=${ZYPKG_PKG_SRC_PATH}/${bname}
    after defining "bname". Without this line executing `grep "/i-data/" ${zypkg}` might end up with error "No such file or directory".
    And YES, grep failed with EVERY package. And YES, appropriate message is put to stderr every time.
    Do anybody test this script, hey, ZyXEL guys?... :shock:
  • During analyzing of fragment in next section (at line 256 and below)
    Code:
    CHKVERSION=`grep "/i-data/" ${zypkg}`
    if [ "$?" != "0" ]; then
    I am confusedly thought "WTH, for just run my script, located in "${ZYPKG_PKG_SRC_PATH}/${bname}", as they demand all after all, ZyXEL devs push me to mention string "/i-data/" (without any meaningful functionality) in my code?! Why? How? For what reason? Do I must complain some rules? "
    Indeed, absence in my script line like
    Code:
    # Hello ZyXEL devs, thanks for using /i-data/ in my NAS!
    will lead mentioned above "if" to be evaluated positively and so lead to attempt to parse an special file (see below) for (incorrectly written, as you can see below) attempt to find an especial path to my script (see below).
    I am able to understand speculating like "If script author will want locate his script in some non-standard place, he for sure will mention that special place in script description, and because /i-data/* is certain place to permanent storage, there with high probability will "/i-data/" string mentioned; in opposite case (script located in standard place, defined in ${ZYPKG_PKG_SRC_PATH}) author have almost no reason to state exact script location; therefore we could use this string for ensuring non-standard script placing by author, and therefore do appropriate next steps for locating it path". In this case, however, code must have "==" instead of "!=" !
    Well, but opposite way of thinking?!... Incomprehensible for me. Explainable only by developer mistake IMHO.
    Anyway, to rely on some free text inside of script... this is at least irresponsible.
    So, if you want your own startup/shutdown script (located in standard /i-data/.system/zy-pkgs/) would be loaded successfully, please do not forget to place "/i-data/" text somewhere inside, everywhere, in ever manner, no matter how :D
  • During analyzing of fragment (at line 256 and below)
    Code:
    CHKVERSION=`grep "/i-data/" ${zypkg}`
    if [ "$?" != "0" ]; then
       PKGVOLPATH=`grep ${bname} ${PKGSTATUSFILE} |grep "Installed-Rule" |awk -F":" '{print $2}' |sed 's/\/$//g' |sed 's/ //g'`
       zypkg=${PKGVOLPATH}/etc/init.d/${bname}
    fi
    I am again confusedly thought "Well, they tried to parse ${PKGSTATUSFILE} (which is /etc/zyxel/pkgconf/status) for case of non-standard script placement. They want to find "Installed-Rule" line (where path to script is defined by script author), situated after package name in ${PKGSTATUSFILE}. But...
    1. That file is intended for packages, not for individual scripts! Or may be they suggests "There is "/i-data/" text in code => This is the package"?!
    2. WHOLE ${PKGSTATUSFILE} is walked through straight in the next part of zypkg_controller.sh, searching ALL "Installed-Rule" strings. Why to search there now, for non-package scripts?
    3. And worthe of all: executing `grep ${bname} ${PKGSTATUSFILE}` will return only ONE LINE, in which, for 120%, they WILL NOT find "Installed-Rule". Will NOT!!

    Well, it just enough to add "-A <number>" to appropriate grep for working as (probably) intended; and in our case <num> should be 7 IMHO. So, code must look like
    Code:
    PKGVOLPATH=`grep -A 7 ${bname} ${PKGSTATUSFILE} |grep "Installed-Rule" |awk -F":" '{print $2}' |sed 's/\/$//g' |sed 's/ //g'`
    And, unfortunately, such code was copied from a few lines above in zypkg_controller.sh, from part of starting all packages according to ${ZYPKG_DEPS}, where it functioning incorrectly too :(
  • During analyzing of fragment in next section (at line 274)
    Code:
    cat /etc/zyxel/pkg_conf/status | grep Installed-Rule | awk '{print $2}' |  while read zypkg; do
    I noticed that file /etc/zyxel/pkg_conf/status is mentioned directly instead of referencing on ${PKGSTATUSFILE} variable :?
  • The same story with /i-data/.system/zy-pkgs/ZYPKG_DEPS two lines below previous case.
  • During analyzing of fragment (at line 274 and below)
    Code:
    cat /etc/zyxel/pkg_conf/status | grep Installed-Rule | awk '{print $2}' |  while read zypkg; do
       PKGName=`echo ${zypkg} | awk -F "/" '{print $5}'`
       cat /i-data/.system/zy-pkgs/ZYPKG_DEPS | grep $PKGName > /dev/null 2>&1
       if [ "$?" != "0" ]; then
          echo ${Processed_Packages} | grep ${PKGName} > /dev/null 2>&1
          if [ "$?" != "0" ]; then
             EXEINIFILE=${zypkg}/etc/init.d/$PKGName
             if [ ! -x ${EXEINIFILE} ]; then
    I have found line
    Code:
    EXEINIFILE=${zypkg}/etc/init.d/$PKGName
    might be
    Code:
    EXEINIFILE=${zypkg}etc/init.d/$PKGName
    Actually no big deal, it is functioning as is and BTW preventing from theoretically possible mistake in package path definition (by accidental abandoning of trailing "/") in /etc/zyxel/pkg_conf/status.
  • All I told above is valid also for '"stop" )' section of zypkg_controller.sh, with the exception of 3rd part (at line 395 and below), where coders was so desperately slapdash, that forgot even replace "start-up" and "start" with "shutdown"... :oops: Thanks God, only in logging messages...
  • In one place we have
    Code:
    write_log "- Failed: <...>"
    in other place
    Code:
    write_log "---> Error: <...>"

    In one place we have
    Code:
    write_log "Starting <...>"
    in other place
    Code:
    write_log "- starting <...>"
    in third place
    Code:
    write_log "---> start <...>"
    In one place log statements in /i-data/.system/zy-pkgs/tmp/zypkg.log have correct timestamp, and in other statements time is not adjusted with current timezone.

    No code culture, no rules, no common patterns... ANARCHY rules there :(

Well...
We have what we have.

Questions:
  • How is possible to modify files in /etc/* or /ram_bin/* in permanent way (i.e. for surviving reboot)?
  • How can I see a messages, sent by scripts to stdout during startup? /var/log is almost empty. Is there such a tool like dmesg?

Mijzelf wrote:
Quote:
I never wrote my own regular package, because I simply do not found any materials, learning how to do so (for NAS542). Can you please take a links on some tutorials?
I partly documented the case here. Actually it's no big deal. By just extracting some existing packages, it's rather easy to find how a package is build.
Om-nom-nom, thank you for detailed tutorial for different firmwares, good job! Partly I discovered mechanism by, as you say, analyzing other packages directory structure and file content and behavior, but still some details remain unknown to me, so big thank you, Mijzelf!

P.S. If you eventually decide to enrich your tutorial with my researches above, I will have no objections.
P.P.S. Are you an ZyXEL developer, Mijzelf? Can you appeal to ZyXEL for check and correct their scripts?


Top
 Profile  
 
PostPosted: Mon May 08, 2017 9:45 am 
Offline

Joined: Mon Jun 16, 2008 10:45 am
Posts: 6039
Wow, In-depth analysis!

You are right, the script is not very coherent, and maybe that's my fault. This part (the reading of USRPKG_DEPS_START) is new for firmware 5, and it's by default not in use. (That file doesn't exist).
I *think* ZyXEL planned to implement the possibility to install 3th party packages, by simply uploading them in the webinterface, and this is a part of the implementation, which is not mature.
That feature never made it, and maybe that is because I already ported MetaRepository to firmware 5 before they had time to finish it. And now we are left with a strange semi-manufactured scriptlet.

Anyway, you asked for a way to run your own scripts, and this is a way. OK, the script has to contain the string '/i-data/'. In most cases that string will already be there, as /i-data/ is the only sane place to put your binaries and other scripts. I guess that's the reason I never stumbled upon this 'feature'.
(To be honest, I only used this for starting my 'optware' package. When MetaRepository for fw 5.10 was ready, I had more convenient ways to get my scripts run).
The lines I quoted are not in use by the regular packages, nor by the 3th party packages installed through MetaRepository (which are 'regular', from the NAS' point of view). Their startscript /i-data/sysvol/.PKG/<package>/etc/init.d/<package> is called another way.

Quote:
inside typos like "#ckeck ZYPKG_DEPS format"
Bear with them. The scripts are written by Taiwanese engineers. English is /not/ their native language.

Quote:
How is possible to modify files in /etc/* or /ram_bin/* in permanent way (i.e. for surviving reboot)?
For /etc/ you can't, unless you are building your own firmware. /etc/ is inside the initramfs, which is embedded in the kernel.
For /ram_bin/ there is a possibility, but you should use it with care. /ram_bin is the mountpoint of an ext2 containing file sysdisk.img, which is mounted read-only. Further the containing partition of sysdisk.img (sda1 on fw 4, md0 on fw 5) is mounted read-only.
The file sysdisk.img itself is extracted from flash. The compressed file is part of a firmware update.
On boot the checksum of sysdisk.img is compared to a stored one, and if it doesn't fit, a fresh copy of sysdisk.img is extracted. This mechanism is used for firmware updates. So if you change anything in /ram_bin/*, it will be reverted on boot.
There is a way out. If you read /etc/init.d/rcS (containing among others the sysdisk.img extraction code), you'll see that the existence of a file /firmware/mnt/sysdisk/mount.sda1.rw.flag will skip the checksum test, and the partition md0 will be mounted rw (on /firmware/mnt/sysdisk/).
So, to be able to edit /ram_bin/*, and let it survive a reboot, you'll have to
Code:
mount -o remount,rw /firmware/mnt/sysdisk/
touch /firmware/mnt/sysdisk/mount.sda1.rw.flag
reboot
After reboot you maybe have to remount /ram_bin/ rw. On fw 4.x that wasn't necessary, can't remember the status for fw 5. At least I don't see the mount.sda1.rw.flag flag handled in the mounting of sysdisk.img.

You'll have to remove the mount.sda1.rw.flag before updating the firmware. Theoretically it's possible that a new initramfs won't be compatible with an old sysdisk.img file, leaving you with a non-booting box. (In that case you can still use the 'Univeral stick' to get early telnet access, but why accept that inconvenience?)

Quote:
How can I see a messages, sent by scripts to stdout during startup? /var/log is almost empty. Is there such a tool like dmesg?
Not that I'm aware of. AFAIK the stdout messages are only send to, well, stdout. You can see them, but only by looking to console, which is on the serial port.

Quote:
If you eventually decide to enrich your tutorial with my researches above, I will have no objections.
Well, thanks. I don't think I will use it, as this mechanism is not really a part of the package system, and the package system is complex enough without sidely related stuff.

Quote:
P.P.S. Are you an ZyXEL developer, Mijzelf?
Seeing your analysis of that script, that's almost an insult. No, I have nothing to do with ZyXEL. My
activities are just a private hobby.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 19 posts ]  Go to page Previous  1, 2

All times are UTC


Who is online

Users browsing this forum: Bing [Bot] and 17 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group