System Log saved every day/hour on external disk

[Iceberg]

Good Morning,
I apologize if there have been previously answers to my request.

I have an Asus RT-N66U with installed latest official firmware but it lacks some features that now I need, so after some searches I arrived here and discovered Merlin Firmware

I installed it and it's almost perfect for my needs, it extended range and strength of wireless signal compared to official firmware and have a lot of additional features.

For my actual needs It's only missing the possibility to save periodically the System LOG.

In this firmware like in the official it can be saved only manually, or maybe i am hot able to find the way to save it.

Is there a way to save it on and external disk like I found on Tools> Traffic Monitoring?

In System LOG there is the association IP-MAC ADDRESS with the time of connection and this is what I am searching.

Traffic monitoring stats are very useful but are lacjing the association to a specific mac address and the time of connection.

Thank you in advance for your help.

 

[Martineau]

Good Morning,
I apologize if there have been previously answers to my request.

I have an Asus RT-N66U with installed latest official firmware but it lacks some features that now I need, so after some searches I arrived here and discovered Merlin Firmware

I installed it and it's almost perfect for my needs, it extended range and strength of wireless signal compared to official firmware and have a lot of additional features.

For my actual needs It's only missing the possibility to save periodically the System LOG.

In this firmware like in the official it can be saved only manually, or maybe i am hot able to find the way to save it.

Is there a way to save it on and external disk like I found on Tools> Traffic Monitoring?

In System LOG there is the association IP-MAC ADDRESS with the time of connection and this is what I am searching.

Traffic monitoring stats are very useful but are lacjing the association to a specific mac address and the time of connection.

Thank you in advance for your help.

Search for 'syslog move'

Member JoeGreat posted an example script, but here is my modifed version:

#!/bin/sh

# As a minimum, call from post-mount to ensure SDCARD/USB device is available..
# Currently called from init-start (NOTE: init-start contains sleep 150;/jffs/scripts/syslog-move.sh which should have allowed post-mount to have run!)

logger -t "($(basename $0))" $$ SYSLOG Housekeeping Running...... `date` [$@] 

SOURCE=/tmp/syslog.log                       # original source of the syslog on /tmp
SYSLOG=/tmp/mnt/RT-N66U/Syslog/syslog.log    # destination of the syslog on USB disk
NASLOG="10.88.8.195"                         # Destination of Remote SYSLOG server NAS DS-110J

NOW=$(date +"%Y%m%d-%H%M%S")    # current date and time

mv $SYSLOG $SYSLOG-$NOW         # rename the previous log with timestamp
cp $SOURCE $SYSLOG-$NOW-BOOT    # copy current /tmp syslog to the new location
rm $SOURCE                      # delete the /tmp syslog

# Start SYSLOG on the USB disk with infinite size i.e. no GDG creation
killall syslogd
syslogd -O $SYSLOG -s 0

logger -t "($(basename $0))" $$ SYSLOG Housekeeping Complete...... `date` for $SYSLOG-$NOW 

# Allow RT-N66U GUI to see the Syslog!!!!! otherwise it shows it empty!!!
ln -s $SYSLOG $SOURCE           # create a symbolic link from the orginal syslog to the copied one

if [ "$?" -ne 0 ]; then    # check for error

   echo "Error in Syslog move! Script: $0"    # display the error message
   exit $?
else
   echo "Syslog move OK. Script: $0"    # display OK message
   exit 0
fi

Regards,

 

[Iceberg]

Thank you Martineau

So, there is a way to do what I need using a script, this is a good news.

I'm reading the threads related to 'syslog move'.

It seems not easy as I have absolutely no knowledge of scripts, how to create ad use them. I will start from 0.

 

[PeterPasm]

Thank you Martineau

So, there is a way to do what I need using a script, this is a good news.

I'm reading the threads related to 'syslog move'.

It seems not easy as I have absolutely no knowledge of scripts, how to create ad use them. I will start from 0.

Since I'm also new to the forum and I know that things can get quite confusing if you are "alone" to deal with problems what others may consider easy, so I wish you good luck, and don't give up

This might come handy perhaps: https://github.com/RMerl/asuswrt-merlin/wiki/User-scripts.

 

[Martineau]

Thank you Martineau

So, there is a way to do what I need using a script, this is a good news.

I'm reading the threads related to 'syslog move'.

It seems not easy as I have absolutely no knowledge of scripts, how to create ad use them. I will start from 0.

Start with Rmerlin's guide to User scripts and enable JFFS on the router

https://github.com/RMerl/asuswrt-merlin/wiki/User-scripts

Rather than TELNET/SSH into the RT-N66U and use the native 'vi' (or entware 'nano') editor, I create my scripts under Windows7/8 then use WinSCP to simply drag 'n' drop the file onto the RT-N66U;set permissions; and execute/test the scripts.

Regards,

 

[Iceberg]

An update of my "progress" in enabling the Syslog move to my USB drive.

At the moment I:

- Enabled JFFS partition
- Was Able to access the router using WIN SCP
- I can see the JFFS
- Was able to drag and drop files on this partition

At this point I tried to slightly modify the example scripts just changing the location of syslog to my usb drive.

I made this using Notepad++

I saved the scripts with .sh extension.

Copied them in JFFS folder.

From this point im locked.

Scripts are not working.

When I try to execute them from WINSCP I receive a message of error code 127 and then a message that says script not found.

I know that there is for sure a DUMB error but at the moment I can not say where is the error or errors.

 

[Martineau]

An update of my "progress" in enabling the Syslog move to my USB drive.

At the moment I:

- Enabled JFFS partition
- Was Able to access the router using WIN SCP
- I can see the JFFS
- Was able to drag and drop files on this partition

At this point I tried to slightly modify the example scripts just changing the location of syslog to my usb drive.

I made this using Notepad++

I saved the scripts with .sh extension.

Copied them in JFFS folder.

From this point im locked.

Scripts are not working.

When I try to execute them from WINSCP I receive a message of error code 127 and then a message that says script not found.

I know that there is for sure a DUMB error but at the moment I can not say where is the error or errors.

WinSCP isn't a complete IDE, but if you use WinSCP to create/edit the files then this should not occur.

However, you probably used Notepad++ in Windows CRLF mode.

You must edit files in Unix LF mode under Notepad++

e.g.

 

[Iceberg]

Now I was able to move syslog to USB HD with a script but it saved syslog only one time running it directly from WINSCP.

I tried a reboot but did not worked again.

Updated syslog is still in tmp/ folder.

You are right Martineau!!

I didn't pay attention to save in Unix LX mode, now at least it's working.

It's not yet what i'm trying to obtain but it's a working start point.

 

[Martineau]

Now I was able to move syslog to USB HD with a script but it saved syslog only one time running it directly from WINSCP.

I tried a reboot but did not worked again.

Updated syslog is still in tmp/ folder.

Computers are really dumb, they only do what you tell them to do, rather than what you expect them to do...

So will need to call your working script from one of RMerlins supplied scripts...

https://github.com/RMerl/asuswrt-merlin/wiki/User-scripts


I personally use init-start

#!/bin/sh

HARDWARE_MODEL=$(nvram get productid)
MYROUTER=$(nvram get computer_name)
BUILDNO=$(nvram get buildno)

/usr/bin/logger -t "($(basename $0))" $$ "Martineau $MYROUTER BOOT in progress... [$@]" 

# NOTE: Can't use Flash drive /tmp/mnt/RT-N66U/ 'cos it hasn't been mounted yet :-(
# Can't use Flash drive /tmp/mnt/RT-N66U/ 'cos it hasn't been mounted yet :-(
echo `date` > /tmp/BOOTINPROGRESS

# If we ensure that the mount point exists at this stage of the boot, it can eliminate the Hotplug boot error?
#   hotplug[427]: USB /dev/sdxn (ext4) failed to mount at the first try! 
#mkdir -p /tmp/mnt/RT-N66U

#Should be sufficient to cover physical boot process?
sleep 150


logger -t "($(basename $0))" $$ LEDs ON scheduled 06:00
/usr/sbin/cru a LEDsON "0 6 * * * /jffs/scripts/ledson.sh"

logger -t "($(basename $0))" $$ LEDs OFF scheduled 18:00
/usr/sbin/cru a LEDsOFF "0 18 * * * /jffs/scripts/ledsoff.sh"

# This script appears to be executed too early? so manually run it here?
/jffs/scripts/pptpd.postconf

[COLOR="Red"][B]# Move Syslog to USB Flash drive
/jffs/scripts/syslog-move.sh[/B][/COLOR]

rm /tmp/BOOTINPROGRESS


/usr/bin/logger -t "($(basename $0))" $$ "Martineau $MYROUTER BOOT Completed Firmware build $BUILDNO [$@]"

 

[Iceberg]

Thank you for your help Martineau.

I have edited scripts and at the moment I was able to move syslog.log from tmp/ to my usb HDD but not only once, i have it working on the HDD

It was also available in real time on Advanced Settings>System LOG>General LOG

Now the next and "final" step is to have it working after a reboot.

I'm working on init-start, using a modified version of the one you have posted, but for sure I have made some errors because it's not working on boot.

I will try until i have a working script.

 

[Iceberg]

I have a very simple init-start.sh script that works if I start it directly from WINSCP.

This is the final result from Martineau init-start.

" #!/bin/sh

# Move Syslog to USB Flash drive
/jffs/scripts/syslog-move.sh

rm /tmp/BOOTINPROGRESS "

If I execute this, it starts syslog-move.sh and I have it working on USB drive.

Now, i'm for sure missing a part.

How do I make init-start.sh executable on reboot?

 

[Martineau]

I have a very simple init-start.sh script that works if I start it directly from WINSCP.

This is the final result from Martineau init-start.

" #!/bin/sh

# Move Syslog to USB Flash drive
/jffs/scripts/syslog-move.sh

rm /tmp/BOOTINPROGRESS "

If I execute this, it starts syslog-move.sh and I have it working on USB drive.

Now, i'm for sure missing a part.

How do I make init-start.sh executable on reboot?

RMerlins auto-exec scripts do not have the .sh suffix!!!

 

[Iceberg]

RMerlins auto-exec scripts do not have the .sh suffix!!!

I learned a new thing.

So I have to save it without .sh suffix, now I will try and see if it works.

 

[Martineau]

I learned a new thing.

So I have to save it without .sh suffix, now I will try and see if it works.

As posted previously....RMerlin's guide to User scripts describes all of the auto-exec script names:

https://github.com/RMerl/asuswrt-mer...i/User-scripts

Regards,

 

[Iceberg]

As posted previously....RMerlin's guide to User scripts describes all of the auto-exec script names:

https://github.com/RMerl/asuswrt-mer...i/User-scripts

Regards,

I read it the first time you posted it but I keep saving scripts with .sh suffix.

Now it works!!

I tried to reboot and also simulated a power loss, now every time the scripts are working and saving syslog.log in HDD folder.

A very big Thank You.

 

[Iceberg]

I post again in this thread because after a few days I found that there is a last problem to solve.

The problem is syslog.log size.

I just found that when it reaches 256KB syslog stops lo log in USB disk and start to log again in tmp folder on router.

Even a restart don't work because init-start script don't work in this particular situation.

I have to start manually init-start to have again syslog.log on usb disk.

I read across the forum and found that there is a limited size for syslog.log file that is ok if it's stored in ram but in a USB disk I have no storage limitations.

How do I set and higher size limit for syslog.log file or better, it's possible to have no size limit?

Thank you for your help

 

[Martineau]

I post again in this thread because after a few days I found that there is a last problem to solve.

The problem is syslog.log size.

I just found that when it reaches 256KB syslog stops lo log in USB disk and start to log again in tmp folder on router.

Even a restart don't work because init-start script don't work in this particular situation.

I have to start manually init-start to have again syslog.log on usb disk.

I read across the forum and found that there is a limited size for syslog.log file that is ok if it's stored in ram but in a USB disk I have no storage limitations.

How do I set and higher size limit for syslog.log file or better, it's possible to have no size limit?

Thank you for your help

As I have been bouncing between RT-N66U and RT-AC56U with new scripts I don't believe I have experienced this ( a reboot every other day doesn't let the syslog grow on the USB drive! )

My original syslog move script specifies unlimited size...

syslogd -O $SYSLOG -s 0

based on the syslogd help:

admin@RT-AC56U:/tmp/home/root# syslogd --help
BusyBox v1.20.2 (2014-01-16 23:46:31 EST) multi-call binary.

Usage: syslogd [OPTIONS]

System logging utility
(this version of syslogd ignores /etc/syslog.conf)

        -n              Run in foreground
        -O FILE         Log to FILE (default:/var/log/messages)
        -l N            Log only messages more urgent than prio N (1-8)
        -S              Smaller output
        [COLOR="Red"]-s SIZE         Max size (KB) before rotation (default:200KB, 0=off)[/COLOR]
        -b N            N rotated logs to keep (default:1, max=99, 0=purge)
        -R HOST[:PORT]  Log to IP or hostname on PORT (default PORT=514/UDP)
        -L              Log locally and via network (default is network only if -R)

Regards,

 

[Iceberg]

Thank for your help you Martineau.

I tried to edit your syslog move script but after a lot of attempts I was not able to have it working properly so I had to use a simpler version I found on the forum.

"#!/bin/sh
SOURCE=/tmp/syslog.log # original source of the syslog
SYSLOG=/tmp/mnt/160GB/SYSLOG/syslog.log # destination of the syslog - need to by adjusted
NOW=$(date +"%Y%m%d-%H%M%S") # current date and time
mv $SYSLOG $SYSLOG-$NOW # rename the previous log with timestamp
cp $SOURCE $SYSLOG # copy current syslog to the new location
rm $SOURCE # delete the syslog
ln -s $SYSLOG $SOURCE # create a symbolic link from the original syslog to the copied one
if [ "$?" -ne 0 ] # check for error
then
echo "Error in Syslog move! Script: $0" # display the error message
exit $?
else
echo "Syslog move OK. Script: $0" # display OK message
exit 0
fi "

Now I have to figure out where to insert the unlimited size in the above script.

"syslogd -O $SYSLOG -s 0"

Another change I noticed after the discovery of 256KB size limit is that before reaching the limit, every reboot a new logfile was created and the old one was saved with date and time of last write.

Now every reboot, and I tried also to simulate a power loss, it does not create a new one but it start writing again in the old syslog.log file.

Informations are not deleted so it's not a problem but I don't understand the reasons of this.

I think also that the solution of a daily reboot could (i think there is need of another script for automatic reboot) solve the problem (if a new file starts to be recreated again because now it's not happening) but only with an higher size limit because I reached the 256KB size in only about 16 hours after I started logging firewall packets.

 

[DreamCypher]

Another script for USB syslog

I thought I would share a similar set of scripts that I wrote to perform this function.

My initial thought was to use the built-in log rotation in syslogd, but the busybox version included in asuswrt seems to have neutered that functionality by limiting the rotation to 1 log file, rather than the 99 indicated in the busybox documentation.

Building on what others have posted, I came up with the following, which moves the syslog.log file to the USB drive (keeping the entries that were created before the move) and sets a limit of 4096k per log. (I tested this with up to 20480k per log without problems, but decided to keep it at 4MB due to a note in the source code that files larger than that could be problematic.). It also enables the log rotation withing syslogd when it becomes full.

Then I wrote another small script to rotate the logs further, so I could keep 10 old log files... this script uses cron to run once an hour to check for the syslog.log-1 file that is created when the log is rotated. (I realize that logrotate is a better choice, but I haven't installed entware yet.)

I created syslog-move.sh and syslog-rotate.sh in /jffs/scripts

syslog-move.sh:

#!/bin/sh
#
logger RedirectingSyslogd to USB

# Syslog.log source file
SOURCE=/tmp/syslog.log

# Syslog.log target file
SYSLOG=/tmp/mnt/USBPATH/router/logs/syslog.log

#Stop existing syslogd
killall syslogd

# Add lines from tmp syslog to USB syslog and create symlink

if [ ! -L $SOURCE ]; then
cat $SOURCE >> $SYSLOG
# delete tmp syslog file
rm $SOURCE
# create symbolic link
ln -s $SYSLOG $SOURCE
fi

#restart syslogd with 4 MB file limit and log rotation enabled
syslogd -m 0 -S -O $SYSLOG -s 4096 -l 7 -b 1

#add cron job to rotate logs

if [ -s /jffs/scripts/syslog-rotate.sh ]; then
cru a syslog-rotate "0 * * * * /jffs/scripts/syslog-rotate.sh"
fi

syslog-rotate.sh:

#! /bin/sh

#set SYSLOG Directory
SYSLOGDIR=/tmp/mnt/USBPATH/router/logs/
cd $SYSLOGDIR


# Rotate Logs if syslog.log-1 exists
if [ -s syslog.log-1 ]; then 

# SET i to one less than the number of logs to keep.  (i.e. "8" means 9 old logs will be kept)
for i in $(awk 'BEGIN { for ( i=8; i>0; i-- ) { print i; } }');
do if [ -f syslog.$i.log ]; then mv syslog.$i.log syslog.$(($i+1)).log; fi;
done
mv syslog.log-1 syslog.1.log
fi

then I created a services-start file in /jffs/scripts/ so it would run on reboot

services-start:

#!/bin/sh
#services-start script (runs after all services start at boot)

/jffs/scripts/syslog-move.sh

You would need to replace /tmp/mnt/USBPATH/router/logs/ with your own directory in the scripts above.

The script could easily be modified for larger log files, or to keep more copies. It is worth noting that this method also appends to the existing syslog when the router reboots... no need to rotate the log on each reboot.

It would be nice if the syslogd in asuswrt supported additional log rotation... maybe that's something that could be fixed in a future release... it seems that busybox is supposed to support up to 99 logs being rotated.

 

[sammyano]

I thought I would share a similar set of scripts that I wrote to perform this function.

My initial thought was to use the built-in log rotation in syslogd, but the busybox version included in asuswrt seems to have neutered that functionality by limiting the rotation to 1 log file, rather than the 99 indicated in the busybox documentation.

Building on what others have posted, I came up with the following, which moves the syslog.log file to the USB drive (keeping the entries that were created before the move) and sets a limit of 4096k per log. (I tested this with up to 20480k per log without problems, but decided to keep it at 4MB due to a note in the source code that files larger than that could be problematic.). It also enables the log rotation withing syslogd when it becomes full.

Then I wrote another small script to rotate the logs further, so I could keep 10 old log files... this script uses cron to run once an hour to check for the syslog.log-1 file that is created when the log is rotated. (I realize that logrotate is a better choice, but I haven't installed entware yet.)

I created syslog-move.sh and syslog-rotate.sh in /jffs/scripts

syslog-move.sh:

#!/bin/sh
#
logger RedirectingSyslogd to USB

# Syslog.log source file
SOURCE=/tmp/syslog.log

# Syslog.log target file
SYSLOG=/tmp/mnt/USBPATH/router/logs/syslog.log

#Stop existing syslogd
killall syslogd

# Add lines from tmp syslog to USB syslog and create symlink

if [ ! -L $SOURCE ]; then
cat $SOURCE >> $SYSLOG
# delete tmp syslog file
rm $SOURCE
# create symbolic link
ln -s $SYSLOG $SOURCE
fi

#restart syslogd with 4 MB file limit and log rotation enabled
syslogd -m 0 -S -O $SYSLOG -s 4096 -l 7 -b 1

#add cron job to rotate logs

if [ -s /jffs/scripts/syslog-rotate.sh ]; then
cru a syslog-rotate "0 * * * * /jffs/scripts/syslog-rotate.sh"
fi

syslog-rotate.sh:

#! /bin/sh

#set SYSLOG Directory
SYSLOGDIR=/tmp/mnt/USBPATH/router/logs/
cd $SYSLOGDIR


# Rotate Logs if syslog.log-1 exists
if [ -s syslog.log-1 ]; then 

# SET i to one less than the number of logs to keep.  (i.e. "8" means 9 old logs will be kept)
for i in $(awk 'BEGIN { for ( i=8; i>0; i-- ) { print i; } }');
do if [ -f syslog.$i.log ]; then mv syslog.$i.log syslog.$(($i+1)).log; fi;
done
mv syslog.log-1 syslog.1.log
fi

then I created a services-start file in /jffs/scripts/ so it would run on reboot

services-start:

#!/bin/sh
#services-start script (runs after all services start at boot)

/jffs/scripts/syslog-move.sh

You would need to replace /tmp/mnt/USBPATH/router/logs/ with your own directory in the scripts above.

The script could easily be modified for larger log files, or to keep more copies. It is worth noting that this method also appends to the existing syslog when the router reboots... no need to rotate the log on each reboot.

It would be nice if the syslogd in asuswrt supported additional log rotation... maybe that's something that could be fixed in a future release... it seems that busybox is supposed to support up to 99 logs being rotated.

thanks for your post - I used your script but syslog.log still listed in /tmp, I thought the script was supposed to delete it [# delete tmp syslog file
rm $SOURCE]?