Scribe scribe - syslog-ng and logrotate installer

[Swistheater]

This is already built into amtm disk-check...

well earlier today i noticed the amtm said the router is using its default time. is this a forced feature or is it just stating that was the choice chosen by the router?

Note: it did this when it was waiting to do disk check.

it said that it had waited too long and the router was using default time.

 

[Swistheater]

This is already built into amtm disk-check...

okay i just look at the code of the script and it appears that it is the router and not the script. sooo i have another route to persue..

 

[Swistheater]

a simple solution is if the router decides to use the router default,,,

you can add a simple

logger -t "$(basename $0)" "waiting for Network..."
until ping -nq -c3 1.1.1.1; do
   sleep 1
done
service restart_ntpd

heres results of adding those two lines
being added to the disk check script....

ay  5 01:06:13 kernel: sd 0:0:0:0: [sda] Assuming drive cache: write through
May  5 01:06:13 kernel: usbcore: registered new interface driver usblp
May  5 01:06:13 kernel:  sda: sda1 sda2 sda3
May  5 01:06:13 kernel: xhci_hcd 0000:00:0c.0: WARN: Stalled endpoint
May  5 01:06:13 kernel: sd 0:0:0:0: [sda] Assuming drive cache: write through
May  5 01:06:13 kernel: sd 0:0:0:0: [sda] Attached SCSI removable disk
May  5 01:06:13 custom_script: Running /jffs/scripts/service-event (args: stop ntpd)
May  5 01:06:13 ntpd: Stopped ntpd
May  5 01:06:15 custom_script: Running /jffs/scripts/pre-mount (args: /dev/sda2 ext4)
May  5 01:06:29 YazFi: YazFi v3.2.2 starting up
May  5 01:06:31 YazFi: No YazFi guests are enabled in the configuration file!
May  5 01:06:46 amtm disk-check: NTP timeout (30s) reached, date is router default
May  5 01:06:46 pre-mount: waiting for Network...
May  5 01:06:48 rc_service: service 16464:notify_rc restart_ntpd
May  5 01:06:48 custom_script: Running /jffs/scripts/service-event (args: restart ntpd)
May  5 01:06:49 amtm disk-check: Probing 'ext4' on device /dev/sda2
May  5 01:06:49 amtm disk-check: Running disk check v2.4, with command 'e2fsck -p' on /dev/sda2
May  5 01:06:49 ntpd: Started ntpd
May  5 01:06:50 amtm disk-check: Disk check done on /dev/sda2
May  5 01:06:50 syslog: USB ext4 fs at /dev/sda2 mounted on /tmp/mnt/MY_SCRIPTS
May  5 01:06:50 usb: USB ext4 fs at /dev/sda2 mounted on /tmp/mnt/MY_SCRIPTS.
May  5 01:06:50 kernel: EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: user_xattr
May  5 01:06:50 custom_script: Running /jffs/scripts/post-mount (args: /tmp/mnt/MY_SCRIPTS)
May  5 01:06:50 rc_service: hotplug 7718:notify_rc restart_nasapps
May  5 01:06:50 custom_script: Running /jffs/scripts/service-event (args: restart nasapps)
May  5 01:06:50 avahi-daemon[334]: Got SIGTERM, quitting.
May  5 01:06:50 avahi-daemon[334]: Leaving mDNS multicast group on interface br0.IPv4 with address 192.168.1.1.
May  5 01:06:50 avahi-daemon[334]: Leaving mDNS multicast group on interface lo.IPv4 with address 127.0.1.1.
May  5 01:06:51 avahi-daemon[334]: avahi-daemon 0.7 exiting.
May  5 01:06:51 iTunes: daemon is stopped
May  5 01:06:51 FTP_Server: daemon is stopped
May  5 01:06:51 wsdd2[505]: Terminated received.
May  5 01:06:51 wsdd2[505]: terminating.
May  5 01:06:51 avahi-daemon[16741]: Found user 'nobody' (UID 65534) and group 'nobody' (GID 65534).
May  5 01:06:51 avahi-daemon[16741]: Successfully dropped root privileges.
May  5 01:06:51 avahi-daemon[16741]: avahi-daemon 0.7 starting up.
May  5 01:06:51 avahi-daemon[16741]: WARNING: No NSS support for mDNS detected, consider installing nss-mdns!
May  5 01:06:51 avahi-daemon[16741]: No service file found in /tmp/avahi/services.
May  5 01:06:51 avahi-daemon[16741]: Loading new alias name RT-AC5300.
May  5 01:06:51 avahi-daemon[16741]: Joining mDNS multicast group on interface br0.IPv4 with address 192.168.1.1.
May  5 01:06:51 avahi-daemon[16741]: New relevant interface br0.IPv4 for mDNS.
May  5 01:06:51 avahi-daemon[16741]: Joining mDNS multicast group on interface lo.IPv4 with address 127.0.1.1.
May  5 01:06:51 avahi-daemon[16741]: New relevant interface lo.IPv4 for mDNS.
May  5 01:06:51 avahi-daemon[16741]: Network interface enumeration completed.
May  5 01:06:51 avahi-daemon[16741]: Registering new address record for 192.168.1.1 on br0.IPv4.
May  5 01:06:51 avahi-daemon[16741]: Registering new address record for 127.0.1.1 on lo.IPv4.
May  5 01:06:51 avahi-daemon[16741]: Registering new address record for 127.0.0.1 on lo.IPv4.
Jun 21 13:14:16 avahi-daemon[16741]: Server startup complete. Host name is RT-AC5300-1650.local. Local service cookie is 1864243659.
Jun 21 13:14:16 avahi-daemon[16741]: Alias name "RT-AC5300" successfully established.
Jun 21 13:14:16 ntpd: Initial clock set

this resulted in the clock actually being set correctly.
The only problem with baking this into the code is it places it right in the middle of the disk check script in this section of the script

if [ "$ntptimer" -ge "$ntptimeout" ]; then
    text="NTP timeout (${ntptimeout}s) reached, date is router default"
    printf "\\n$text\\n" >> $CHKLOG
    logger -t "$TAG" "$text"
    logger -t "$(basename $0)" "waiting for Network..."
    until ping -nq -c3 1.1.1.1; do
    sleep 1
    done
    service restart_ntpd
elif [ "$ntptimer" -gt "0" ]; then
    text="Waited ${ntptimer}s for NTP to sync date"
    printf "\\n$(date) $text\\n" >> $CHKLOG
    logger -t "$TAG" "$text"
else
    printf "\\n" >> $CHKLOG
fi

edited...
this only gives added value to people who are running disk check and experiencing this issue.
it gives no added value to those who are not running disk check, so some people may not benefit who are also suffering from this issue.

 

[elorimer]

A little off topic, we've gone.

 

[Swistheater]

to regress I am done if someone wants to make a new thread because of this issue they are more than welcome to. I have found my solution though

 

[thelonelycoder]

heres results of adding those two lines
being added to the disk check script....

Know that the maximum time a user script can hold up the firmware is 120 secs. That includes the job it'll do.
So for pre-mount this is
120 seconds
- NTP timer (30 seconds max for amtm disk-check)
- time to run the disk check(s), usually a few seconds
--------
~80 seconds left for your extra timer

Make sure you set a timeout accordingly, just as amtm does.

 

[Swistheater]

Know that the maximum time a user script can hold up the firmware is 120 secs. That includes the job it'll do.
So for pre-mount this is
120 seconds
- NTP timer (30 seconds max for amtm disk-check)
- time to run the disk check(s), usually a few seconds
--------
~80 seconds left for your extra timer

Make sure you set a timeout accordingly, just as amtm does.

your script successfully ran the diskchecks on time.

note i edited my earlier post with some revelations i had myself.

 

[SomeWhereOverTheRainBow]

why do you place @cmkelley

cru a logrotate "5 0 * * * /opt/sbin/logrotate /opt/etc/logrotate.conf >> /opt/tmp/logrotate.daily 2>&1" # added by scribe

inside post-mount as opposed to services-start?

 

[cmkelley]

why do you place @cmkelley

cru a logrotate "5 0 * * * /opt/sbin/logrotate /opt/etc/logrotate.conf >> /opt/tmp/logrotate.daily 2>&1" # added by scribe

inside post-mount as opposed to services-start?

So that the command doesn't get added to cron unless the USB drive is mounted.

 

[SomeWhereOverTheRainBow]

So that the command doesn't get added to cron unless the USB drive is mounted.

but doesn't it get basically spammed for each disk that gets mounted then? or do i have it mixed up with pre-mount? I know that may seem trivial, but truth be told I am not sure if it is or not.

 

[cmkelley]

but doesn't it get basically spammed for each disk that gets mounted then? or do i have it mixed up with pre-mount? I know that may seem trivial, but truth be told I am not sure if it is or not.

Yes, post-mount is run for every USB disk that is mounted, but not for the internal ram disks. Since most people have only one USB mount, this isn't an issue.

If you have more than one USB mount point, it's easy enough to enclose it in an if-then statement. The script writers mafia have an informal agreement to only add one line per script file, so we don't check for mount point. That said, re-running 'cru a <unique_id> <"min hour day month week command">' over and over doesn't cause any problems, it simply replaces the previous <unique_id> cron job with the new one. This is to allow you to modify cron jobs without having to delete them first.

But if I did install the cron job from services-start, then I would preferably have to check that Entware was installed so that the cron job wasn't trying to run a non-existent command. Because people sometimes boot their routers without their USB disk installed, the correct place (IMHO) to set up the cron job is in post-mount; if the USB disk isn't installed, it never tries to install the cron job.

 

[SomeWhereOverTheRainBow]

Yes, post-mount is run for every USB disk that is mounted, but not for the internal ram disks. Since most people have only one USB mount, this isn't an issue.

If you have more than one USB mount point, it's easy enough to enclose it in an if-then statement. The script writers mafia have an informal agreement to only add one line per script file, so we don't check for mount point. That said, re-running 'cru a <unique_id> <"min hour day month week command">' over and over doesn't cause any problems, it simply replaces the previous <unique_id> cron job with the new one. This is to allow you to modify cron jobs without having to delete them first.

But if I did install the cron job from services-start, then I would preferably have to check that Entware was installed so that the cron job wasn't trying to run a non-existent command. Because people sometimes boot their routers without their USB disk installed, the correct place (IMHO) to set up the cron job is in post-mount; if the USB disk isn't installed, it never tries to install the cron job.

Couldn't it be run from within entware and only start when the program starts in entware?. Or written as a "one line " if statement.

 

[cmkelley]

Couldn't it be run from within entware and only start when the program starts in entware?. Or written as a "one line " if statement.

I think you misunderstand what Entware is. There's no way to run anything "within Entware". Entware is just the name for a collection of packages installable through the opkg utility, which is similar to other linux package managers such as dpkg, apt, pacman, yum, etc.

logrotate only runs when it's called, it's not a service like syslog-ng, so there's no SXXlogrotate script in /opt/etc/init.d/. If the cron job isn't added somehow, logrotate never runs. cru isn't a service either, it's a program for managing the crontab for the router, a simpler version of the *nix crontab command. Running the cru command over and over doesn't put multiple copies of cru in memory, it doesn't put multiple copies of logrotate in memory, and it doesn't put multiple entries in the crontab with the same name. There is literally no discernible difference between running a cru command once, and running that same cru command over and over and over.

Certainly there are other options ... as you say, a one line if; then; fi statement; or I could have started it using the script I hook to in the syslog-ng script; or I could have added a small script file in /jffs/scripts just for starting it as thelonelycoder has done for diversion (which itself is called by a line added to post-mount btw); and I can think of a couple other methods as well. It's a matter of choice, and no matter which choice I made, somebody would think they have a better way.

 

[SomeWhereOverTheRainBow]

I think you misunderstand what Entware is. There's no way to run anything "within Entware". Entware is just the name for a collection of packages installable through the opkg utility, which is similar to other linux package managers such as dpkg, apt, pacman, yum, etc.

logrotate only runs when it's called, it's not a service like syslog-ng, so there's no SXXlogrotate script in /opt/etc/init.d/. If the cron job isn't added somehow, logrotate never runs. cru isn't a service either, it's a program for managing the crontab for the router, a simpler version of the *nix crontab command. Running the cru command over and over doesn't put multiple copies of cru in memory, it doesn't put multiple copies of logrotate in memory, and it doesn't put multiple entries in the crontab with the same name. There is literally no discernible difference between running a cru command once, and running that same cru command over and over and over.

Certainly there are other options ... as you say, a one line if; then; fi statement; or I could have started it using the script I hook to in the syslog-ng script; or I could have added a small script file in /jffs/scripts just for starting it as thelonelycoder has done for diversion (which itself is called by a line added to post-mount btw); and I can think of a couple other methods as well. It's a matter of choice, and no matter which choice I made, somebody would think they have a better way.

You are fine in your methods, I am just wondering if cru could cause collision when trying to schedule cron jobs from multiple locations I.e. Lets say some are being scheduled from services-start and some are being scheduled from firewall-start (which gets run several times) and some get scheduled from post-mount (which may get runned several times depending on users specs)

 

[Jack Yaz]

You are fine in your methods, I am just wondering if cru could cause collision when trying to schedule cron jobs from multiple locations I.e. Lets say some are being scheduled from services-start and some are being scheduled from firewall-start (which gets run several times) and some get scheduled from post-mount (which may get runned several times depending on users specs)

No - cron jobs have a name specified when adding a job, so unless you are adding jobs with the same name as those that get added by scripts, then there should be no issue whatsoever

 

[SomeWhereOverTheRainBow]

I was just asking because I was missing the log rotate cron job schedule, it would only be scheduled if I added a small sleep to post mount.

 

[cmkelley]

I was just asking because I was missing the log rotate cron job schedule, it would only be scheduled if I added a small sleep to post mount.

It would have been helpful to say that at the start ...

That's super odd, I've not heard of anyone else having that problem, I wouldn't even know where to start with that one. Is there an error anywhere in the logs if you don't have the sleep statement?

 

[SomeWhereOverTheRainBow]

It would have been helpful to say that at the start ...

That's super odd, I've not heard of anyone else having that problem, I wouldn't even know where to start with that one. Is there an error anywhere in the logs if you don't have the sleep statement?

No errors it seems like it will add to the cron tab for my first 2 drives after they mount, but it will disappear when trying to add it a third time once the third drive mounts. What is odd is I do not have this behavior with diversion though. And firewall start and services start should be running around the time that third one starts. It is just a mess it seems.

 

[Jack Yaz]

No errors it seems like it will add to the cron tab for my first 2 drives after they mount, but it will disappear when trying to add it a third time once the third drive mounts. What is odd is I do not have this behavior with diversion though. And firewall start and services start should be running around the time that third one starts. It is just a mess it seems.

No problems here and I have 4 partitions that get mounted and upwards of 10 script-added cron jobs

 

[akb]

Can i please get assistance on how I can add the following log to be seen with uiscribe. It is already seperated from syslog. "/opt/var/log/monit.log" I have not changed anything from source, but I am having a hard time understanding what changes I need to make to see monit.log show up in uiscribe