It's a little more than that. This one line is there to remove a line from the system log that Skynet thinks gets logged by chron to syslog.log. It uses a destructive sed command on syslog.log, instead of copying to a temp file, doing sed there, and copying back the file (all of which sed is doing itself), which I think could follow the symlink (not sure, haven't tried it).
# system-specific logs may be also be configured here.
/opt/var/log/messages {
size 2M
rotate 4
postrotate
killall -HUP syslog-ng
endscript
}
# opkg install syslog-ng logrotate# put any customization files in this directory
@include "/opt/etc/syslog-ng.d/"
log {
source(src);
source(net);
source(kernel);
destination(messages);
};# Skynet creates a lot of log messages, put them elsewhere
# Skynet setup now allows scraping a specified file
# this is the file to have Skynet scrape
destination d_skynet {
file("/opt/var/log/skynet-0.log");
};
filter f_blocked {
match("BLOCKED -" value("MESSAGE"));
};
# value("PROGRAM") matches to logger -t PROGRAM
filter f_skynet {
match("Skynet: " value("MSGHDR"));
};
# this logs f_blocked to d_skynet and nowhere else
log {
source(src);
source(kernel);
filter(f_blocked);
destination(d_skynet);
flags(final);
};
# this logs f_skynet to d_skynet but allows it to pass to messages
log {
source(src);
source(kernel);
filter(f_skynet);
destination(d_skynet);
};
#eof# syslog-ng -s# /jffs/scripts/firewall/opt/var/log/skynet-0.log#!/bin/sh
# separate killing syslogd and linking syslog.log;
# symlink might get broken even if syslogd not running
kill_syslogd (){
# kill any/all running syslogd
if [ ! "X$(pidof syslogd)" = "X" ]; then
killall syslogd
fi
# webGUI System Log = /tmp/syslog.log
if [ ! -L "/tmp/syslog.log" ]; then
cat /tmp/syslog.log >> /opt/var/log/messages
rm /tmp/syslog.log /tmp/syslog.log-1
ln -s /opt/var/log/messages /tmp/syslog.log
fi
# make /jffs/syslog.log and log-1 directories if not already
# prevents system log rotater from writing to jffs
if [ ! -d "/jffs/syslog.log" ]; then
rm /jffs/syslog.log
mkdir /jffs/syslog.log
fi
if [ ! -d "/jffs/syslog.log-1" ]; then
rm /jffs/syslog.log-1
mkdir /jffs/syslog.log-1
fi
}
ENABLED=yes
PROCS=syslog-ng
ARGS=""
PREARGS=""
PRECMD="kill_syslogd"
DESC=$PROCS
PATH=/opt/sbin:/opt/bin:/opt/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
. /opt/etc/init.d/rc.funcmissingok
notifempty/opt/var/log/messages {
minsize 1024k
postrotate
/usr/bin/killall -HUP syslog-ng
endscript
}cru a logrotate "5 0 * * * /opt/sbin/logrotate /opt/etc/logrotate.conf #rotate logs daily @ 00:05#"# /opt/etc/init.d/S01syslog-ng start
# cru a logrotate "5 0 * * * /opt/sbin/logrotate /opt/etc/logrotate.conf #rotate logs daily @ 00:05#"There are 18 lines of the dcd crash message.
Awesome job!!
But don't they each start with a time stamp? I mean, each one is a separate message, not one message with 18 new line characters? [Release] Asuswrt-Merlin 384.9 is now available
Feb 9 11:34:19 kernel: dcd[24458]: unhandled level 3 translation fault (11) at 0x00000000, esr 0x92000007
Feb 9 11:34:19 kernel: pgd = ffffffc012777000
Feb 9 11:34:19 kernel: [00000000] *pgd=000000000a3f5003, *pud=000000000a3f5003, *pmd=000000000b63e003, *pte=0000000000000000
Feb 9 11:34:19 kernel: CPU: 1 PID: 24458 Comm: dcd Tainted: P O 4.1.27 #2
Feb 9 11:34:19 kernel: Hardware name: Broadcom-v8A (DT)
Feb 9 11:34:19 kernel: task: ffffffc01c085540 ti: ffffffc00b678000 task.ti: ffffffc00b678000
Feb 9 11:34:19 kernel: PC is at 0xf6e1bf44
Feb 9 11:34:19 kernel: LR is at 0x1dc74
Feb 9 11:34:19 kernel: pc : [<00000000f6e1bf44>] lr : [<000000000001dc74>] pstate: 600e0010
Feb 9 11:34:19 kernel: sp : 00000000ffa6a0b8
Feb 9 11:34:19 kernel: x12: 000000000009ff10
Feb 9 11:34:19 kernel: x11: 00000000f60ff024 x10: 00000000000a02b4
Feb 9 11:34:19 kernel: x9 : 00000000f60ff670 x8 : 00000000000a076c
Feb 9 11:34:19 kernel: x7 : 00000000f60ff6a8 x6 : 00000000000a0766
Feb 9 11:34:19 kernel: x5 : 0000000000000000 x4 : 00000000f60ff654
Feb 9 11:34:19 kernel: x3 : 0000000000000000 x2 : 00000000ffa6a094I'm not sure what you mean by this? I've only moved the @include line up above the already existing log line in the base file. If you don't move the @include line, everything will go to the main messages log, because the file is read from the top down, so the logging to messages occurs before filtering out the stuff you don't want to be there. The flags(final) callout in the included file effectively discards messages matching the filter after writing them.
Probably. It's designed for edge cases that may actually never exist.
I don't think they're separate messages. I _think_ the kernel is spitting it out all at once as a single multi-line message, and syslog-ng (same as syslogd) is splitting the multi-line message into individual lines and giving them all the same timestamp. I could be wrong about this, but I don't know how to test it other than try to set up a mulit-line filter.
What I mean is the messages logging should be in the subdirectory too, so all of the logging is in that file. I have had an update to syslog-ng overwrite the syslog-ng.conf file.
You'd still have to edit the syslog-ng.conf file every time you updated syslog-ng. Here is Entware's default syslog-ng.conf file:
@version: 3.17
options {
chain_hostnames(no);
create_dirs(yes);
flush_lines(0);
keep_hostname(yes);
log_fifo_size(256);
log_msg_size(1024);
stats_freq(0);
flush_lines(0);
use_fqdn(no);
};
source src {
internal();
unix-dgram("/dev/log");
};
source net {
udp(ip(0.0.0.0) port(514));
};
source kernel {
file("/proc/kmsg" program_override("kernel"));
};
destination messages {
file("/opt/var/log/messages");
};
log {
source(src);
source(net);
source(kernel);
destination(messages);
};
# put any customization files in this directory
@include "/opt/etc/syslog-ng.d/"chain_hostnames(no);
flush_lines(0);
stats_freq(43200);
I've had a go at creating a HOWTO for setting up syslog-ng. It's written around moving the iptables logs off into their own log, and includes solutions for keeping the symlinks working. I'd like to have someone who doesn't have syslog-ng set up on their router (or someone with a spare router they can set up to avoid familial discord) give it a go.
*** HOWTO: Use syslog-ng to relocate iptables logs generated by Skynet ***
Purpose: use syslog-ng to replace syslogd, and relocate skynet logs.
It’s assumed the following are installed (otherwise you wouldn’t need this in the first place!):
It’s also assumed that you are fairly comfortable with the command line, and that you know how to edit text files on the router. The editors ‘nano’ and ‘vi’ are part of the base firmware, a full ‘vim’ and I’m sure other editors are available in Entware.
- Asuswrt-Merlin
- jffs scripts enabled in firmware
- Entware (preferably installed using amtm, see SNB forums)
- Skynet 6.7.0 or higher (preferably installed using amtm, see SNB forums)
First, install syslog-ng and logrotate* from Entware:
* = logrotate is technically not required, but a very good idea. There are reports that if the system log gets too big, the webGUI will have issues trying to read it if you click on the System Log tab.Code:# opkg install syslog-ng logrotate
Personally, I take the unix “do one thing only, and do it well” to the extreme. Although I think most people just have one huge syslog-ng.conf file with everything in it, I like the idea of separate files for seapare log actions. In order for this all this to work properly using this approach, the /opt/etc/syslog-ng.conf file needs a minor tweak to move the @include line to before the log line (make sure to move it, not just copy it, there should be only 1 @include line):
The reason for this is that if the @include line is after the log line, everything will still be logged in messages, and the whole point of this is to keep excessive messages out of the system log!Code:# put any customization files in this directory @include "/opt/etc/syslog-ng.d/" log { source(src); source(net); source(kernel); destination(messages); };
Next, create a file in /opt/etc/syslog-ng.d/ for the filter. I called it “skynet” but you can call it whatever you want, the name doesn't matter. Note that it shouldn’t be executable. Fire up your favorite editor and paste the below in:
The above strips all the BLOCKED messages from iptables, putting them in a file where Skynet can use them for statistics, but still allows the normal skynet messages to go into the system logs (normally /opt/var/log/messages with syslog-ng). All of this could just as easily be copied into the /opt/etc/syslog-ng, just make sure it’s copied immediately before the log line. To ensure the configuration is correct, use:Code:# Skynet creates a lot of log messages, put them elsewhere # Skynet setup now allows scraping a specified file # this is the file to have Skynet scrape destination d_skynet { file("/opt/var/log/skynet-0.log"); }; filter f_blocked { match("BLOCKED -" value("MESSAGE")); }; # value("PROGRAM") matches to logger -t PROGRAM filter f_skynet { match("Skynet: " value("MSGHDR")); }; # this logs f_blocked to d_skynet and nowhere else log { source(src); source(kernel); filter(f_blocked); destination(d_skynet); flags(final); }; # this logs f_skynet to d_skynet but allows it to pass to messages log { source(src); source(kernel); filter(f_skynet); destination(d_skynet); }; #eof
If that runs without errors, then syslog-ng believes all the scripts are correct. Now, fire up the Skynet configuration script:Code:# syslog-ng -s
From the menu, select option 11 (Settings), then option 10 (Syslog Location) from the sub-menu, then option 1 (syslog.log) to change the location of syslog. Enter:Code:# /jffs/scripts/firewall
It is vital that this file match the file in the “destination” section of the skynet configuration file. Otherwise Skynet won’t be able to gather statistics. No need to change the syslog-1 file location, it won’t exist with syslog-ng running and syslogd stopped.Code:/opt/var/log/skynet-0.log
Now we need to make sure syslogd is stopped at every boot, and syslog-ng is started. In the /opt/etc/init.d directory there should be a file called S01syslog-ng. It’s probably easiest to replace the entire thing with the following:
Almost there! All that remains is to set up logrotate. I suggest adding the following 2 lines to your /opt/etc/logrotate.conf file:Code:#!/bin/sh # separate killing syslogd and linking syslog.log; # symlink might get broken even if syslogd not running kill_syslogd (){ # kill any/all running syslogd if [ ! "X$(pidof syslogd)" = "X" ]; then killall syslogd fi # webGUI System Log = /tmp/syslog.log if [ ! -L "/tmp/syslog.log" ]; then cat /tmp/syslog.log >> /opt/var/log/messages rm /tmp/syslog.log /tmp/syslog.log-1 ln -s /opt/var/log/messages /tmp/syslog.log fi # make /jffs/syslog.log and log-1 directories if not already # prevents system log rotater from writing to jffs if [ ! -d "/jffs/syslog.log" ]; then rm /jffs/syslog.log mkdir /jffs/syslog.log fi if [ ! -d "/jffs/syslog.log-1" ]; then rm /jffs/syslog.log-1 mkdir /jffs/syslog.log-1 fi } ENABLED=yes PROCS=syslog-ng ARGS="" PREARGS="" PRECMD="kill_syslogd" DESC=$PROCS PATH=/opt/sbin:/opt/bin:/opt/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin . /opt/etc/init.d/rc.func
logroate also supports externally-sourced configuration files. The below could just as easily be pasted at the end of /opt/etc/logrotate.conf, but I choose to put it in /opt/etc/logrotate.d/syslog-ngCode:missingok notifempty
The logrotate configuration options aren’t complex and you can change settings to your liking. If you choose to enable compression of rotated logs, you need to add “delaycompress” to your logrotate.conf as well so the first rotated log won’t be compressed. This is not for readability, this prevents problems due to the way that some programs access their logs.Code:/opt/var/log/messages { minsize 1024k postrotate /usr/bin/killall -HUP syslog-ng endscript }
Lastly the following line to the end of the post-mount script:
Now, if you're feeling lucky, you should be able to just reboot the router and all will be working. If you want to get it going without a reboot, enter the following at the command prompt:Code:cru a logrotate "5 0 * * * /opt/sbin/logrotate /opt/etc/logrotate.conf #rotate logs daily @ 00:05#"
Code:# /opt/etc/init.d/S01syslog-ng start # cru a logrotate "5 0 * * * /opt/sbin/logrotate /opt/etc/logrotate.conf #rotate logs daily @ 00:05#"
I also have another syslog-ng file to move the WLCEVENTD messages off to their own file.
And I'm coming to the conclusion I'm wrong. :-( I now think it is spitting it out one line at time, which means AFAICT, there's no "easy" way to filter them. I tried using the indent-multi-line function but either I'm using it wrong (possible, but it passes the syntax check) or they're individual lines. Back to the drawing board.
UPDATE:
filter f_skynet {
match("Ranges Banned" value("MESSAGE"));
};
That would indeed. Diversions Dnsmasq logs have these file names: dnsmasq.log to dnsmasq.log2, or up to dnsmasq.log4 if alternate blocking list is enabled.
All the indent-multiline does is add a tab character to the new line character, I think. Of course it will pass the syntax check, as that won't tell you why nothing matches it.
destination openvpn1 { file("/opt/var/log/openvpn1"); };
destination openvpn2 { file("/opt/var/log/openvpn2"); };
filter f_openvpn1 { facility(daemon) and program("ovpn-server1"); };
filter f_openvpn2 { facility(daemon) and program("ovpn-server2"); };
log { source(src); filter(f_openvpn1); destination(openvpn1); };
log { source(src); filter(f_openvpn2); destination(openvpn2); };
Welcome To SNBForums
SNBForums is a community for anyone who wants to learn about or discuss the latest in wireless routers, network storage and the ins and outs of building and maintaining a small network.
If you'd like to post a question, simply register and have at it!
While you're at it, please check out SmallNetBuilder for product reviews and our famous Router Charts, Ranker and plenty more!
