Scribe scribe - syslog-ng and logrotate installer

[Butterfly Bones]

Ok, I have this all up and all logs get redirected, except for Skynet. It is running and I see the stats change in the green line when I start it, using the debug option to watch as events are blocked shows nothing, no entries in the syslog.

I see no entries in the GUI syslog after syslog-ng starts of anything. Also the router GUI is very slow to respond to any clicks or page changes. All the SSH actions in a terminal are normal, as are amtm, Skynet, Stubby, etc.

I have no idea where to search. I used the skynet file in the entware/share/examples since none was found in the syslog-ng.d directory after install.

If someone has troubleshooting tips, that would be great.

 

[cmkelley]

Ok, I have this all up and all logs get redirected, except for Skynet. It is running and I see the stats change in the green line when I start it, using the debug option to watch as events are blocked shows nothing, no entries in the syslog.

I see no entries in the GUI syslog after syslog-ng starts of anything. Also the router GUI is very slow to respond to any clicks or page changes. All the SSH actions in a terminal are normal, as are amtm, Skynet, Stubby, etc.

I have no idea where to search. I used the skynet file in the entware/share/examples since none was found in the syslog-ng.d directory after install.

If someone has troubleshooting tips, that would be great.

Is there an /opt/var/log/skynet-0.log file? That's where skynet stuff should be going. The skynet file /share/examples is exactly the one I'm using.

I've never had an issue with the router GUI slowing down.

 

[cmkelley]

Ok, I have this all up and all logs get redirected, except for Skynet. It is running and I see the stats change in the green line when I start it, using the debug option to watch as events are blocked shows nothing, no entries in the syslog.

I see no entries in the GUI syslog after syslog-ng starts of anything. Also the router GUI is very slow to respond to any clicks or page changes. All the SSH actions in a terminal are normal, as are amtm, Skynet, Stubby, etc.

I have no idea where to search. I used the skynet file in the entware/share/examples since none was found in the syslog-ng.d directory after install.

If someone has troubleshooting tips, that would be great.

Stand by, I forgot I had changed it from what I was using. Staying up too late. Lemme troubleshoot a bit.

 

[Butterfly Bones]

Is there an /opt/var/log/skynet-0.log file? That's where skynet stuff should be going. The skynet file /share/examples is exactly the one I'm using.

I've never had an issue with the router GUI slowing down.

Correct there is no /opt/var/log/skynet-0.log I think the GUI was me making changes to the files and then running "scribe restart" too many times. I have installed / uninstalled four more times since then and nada. I have no idea. I check in Skynet for file location, and it is default syslog.log. After I uninstall and reboot it all works with lots of junk in the syslog, and all the skynet logging shows as well as running the "[1] --> Show Debug Entries As They Appear" which I know worked in my attempts a year ago until the symlink was broken.

Is your sig outdated or are you really still on 384.9? I'm on 384.10 final and I doubt there is a difference, just grasping at straws here.

 

[cmkelley]

Ok, I have this all up and all logs get redirected, except for Skynet. It is running and I see the stats change in the green line when I start it, using the debug option to watch as events are blocked shows nothing, no entries in the syslog.

I see no entries in the GUI syslog after syslog-ng starts of anything. Also the router GUI is very slow to respond to any clicks or page changes. All the SSH actions in a terminal are normal, as are amtm, Skynet, Stubby, etc.

I have no idea where to search. I used the skynet file in the entware/share/examples since none was found in the syslog-ng.d directory after install.

If someone has troubleshooting tips, that would be great.

I outsmarted myself. My bad. I tried to simplify it and broke it. It passed syntax, but it didn't do what I intended. New version on GitHub, but if you want to cut and paste:

# this MUST BE the file configured in Skynet as the syslog.log location
# DO NOT use /tmp/syslog.log or /opt/var/log/messages here!
destination d_skynet {
   file("/opt/var/log/skynet-0.log");
};

# logs everything from Skynet to /opt/var/log/skynet-0.log
filter f_skynet {
   program("Skynet") or
   message("BLOCKED -") or
   message("DROP IN=");
};

# final flag stops processing of messages matching the f_skynet filter
log {
   source(src);
   source(kernel);
   filter(f_skynet);
   destination(d_skynet);
   flags(final);
};

#eof

Sorry, clearly I need to be more careful. :-(

 

[cmkelley]

Correct there is no /opt/var/log/skynet-0.log I think the GUI was me making changes to the files and then running "scribe restart" too many times. I have installed / uninstalled four more times since then and nada. I have no idea. I check in Skynet for file location, and it is default syslog.log. After I uninstall and reboot it all works with lots of junk in the syslog, and all the skynet logging shows as well as running the "[1] --> Show Debug Entries As They Appear" which I know worked in my attempts a year ago until the symlink was broken.

Is your sig outdated or are you really still on 384.9? I'm on 384.10 final and I doubt there is a difference, just grasping at straws here.

No, I'm still on .9; just gonna wait for .11 .10_1 apparently from the pre-beta test builds, since RMerlin said it would be "soon".

 

[Butterfly Bones]

I outsmarted myself. My bad. I tried to simplify it and broke it. It passed syntax, but it didn't do what I intended. New version on GitHub, but if you want to cut and paste:

# this MUST BE the file configured in Skynet as the syslog.log location
# DO NOT use /tmp/syslog.log or /opt/var/log/messages here!
destination d_skynet {
   file("/opt/var/log/skynet-0.log");
};

# logs everything from Skynet to /opt/var/log/skynet-0.log
filter f_skynet {
   program("Skynet") or
   message("BLOCKED -") or
   message("DROP IN=");
};

# final flag stops processing of messages matching the f_skynet filter
log {
   source(src);
   source(kernel);
   filter(f_skynet);
   destination(d_skynet);
   flags(final);
};

#eof

Sorry, clearly I need to be more careful. :-(

No worry, most of us make big mistakes, we just don't share and let others see our mishaps.

I appreciate you doing this and a few bumps along the way are well worth it to me.

Let's see if this goes............

 

[cmkelley]

No worry, most of us make big mistakes, we just don't share and let others see our mishaps.

I appreciate you doing this and a few bumps along the way are well worth it to me.

Let's see if this goes............

I've gotten way past caring who sees my screw-ups, I just feel bad when my screw-ups cause problems for other people.

 

[Butterfly Bones]

I've gotten way past caring who sees my screw-ups, I just feel bad when my screw-ups cause problems for other people.

Hey, no worries, if I wanted things easy I would not be here running all these scripts and trying beta versions of things. I expect things to break, I'm good at troubleshooting most everything except scripting.

So now it works! Yee Haw!
All logs are going into /opt/var/log as expected. So nothing shows in the GUI syslog now and that instead is the "messages" log? That is where I see the dropbear messages of log in and out using ssh.

And not a bid deal now with the /opt/var/log/skynet showing entries, but the debug watch thing in Skynet does not show, is that correct or is there something else I don't understand? Again, not really important since all the stats work in that part of the Skynet menu, just curious.

 

[cmkelley]

Hey, no worries, if I wanted things easy I would not be here running all these scripts and trying beta versions of things. I expect things to break, I'm good at troubleshooting most everything except scripting.

So now it works! Yee Haw!
All logs are going into /opt/var/log as expected. So nothing shows in the GUI syslog now and that instead is the "messages" log? That is where I see the dropbear messages of log in and out using ssh.

And not a bid deal now with the /opt/var/log/skynet showing entries, but the debug watch thing in Skynet does not show, is that correct or is there something else I don't understand? Again, not really important since all the stats work in that part of the Skynet menu, just curious.

You should still be getting the dropbear stuff in the GUI syslog. It sounds like /opt/etc/init.d/S01syslog-ng isn't calling the rc.func.syslog-ng script for some reason, so the links and such aren't being made correctly. Is /tmp is syslog.log symlinked to /opt/var/log/messages? That symlink allows the GUI syslog to see the messages file.

Hrmmm. Skynet debug watch seems to be working for me. If you go into your usb device, in the skynet directory lives the skynet.cfg fiile. If you cat that, there should be 2 lines in there that say:

syslogloc="/opt/var/log/skynet-0.log"
syslog1loc="/tmp/syslog.log-1"

if syslogloc is still pointing at /tmp/syslog.log, then the script failed to correctly call skynet to fix the log file location.

If the symlink isn't there and sysloglog is pointing to the wrong thing, then your installation is broken, you should try removing and re-installing, but back up anything in /opt/etc/syslog-ng.d and /opt/share/syslog-ng/examples that you've modified beforehand because those will be wiped when removing syslog-ng.

 

[elorimer]

If syslogd restarts, it wipes the symlink, creates a new /tmp/syslog.log and usually puts in what had been in syslog.log up to the point syslog-ng started, perhaps from memory. I found that it could take the GUI a long time to load that, on the order of five seconds or so; perhaps it is a refresh cycle.

 

[Xentrk]

v0.5_3
2019-03-31

NOTE: scribe should currently be considered ALPHA quality code. I think it works, I've done limited testing on my test router, and I've ironed out what I think are the obvious bugs.

scribe is a script that installs syslog-ng and logrotate from Entware. The scribe script really just acts as an interface to syslog-ng and to a lesser extent, logrotate. scribe is only invoked by the user, not automatically by any other script during startup, shutdown, etc.

Congratulations on the release. syslog-ng is not the easiest to understand and I've made several failed passes at understanding how everything works. I have it working at a minimal capacity with log rotate feature fully working. At one point, I tried to parse out OpenVPN events to their own log file. But not sure I want to do that now as I need to see how other features interact with the updown and vpnrouting functionality for some scripts I am working on.

 

[cmkelley]

Apr  1 16:43:00 (VPN_Failover.sh): 31536 VPN Client Monitor: Checking VPN Client 1 connection status....

Not sure about that part. The logged message looks odd to me, because there is no hostname.

If I follow the way syslog-ng parses this, the date stamp and had there been a hostname, the hostname is the HEADER part of the message.
All the rest is MSG part, divided into the name of the program and the message text, divided by the ":" Here, the name of the program looks like (VPN_Failover.sh). I believe program() returns the program name, so I think you may need "(VPN_Failover.sh)", and maybe some escape characters. But I think the message part of the filter would work. match() is deprecated now.

This discussion slipped by me ...

I don't think VPN_Failover.sh is the program name, I think that script is using the logger command, so I don't know what the "program" would be. If you grab expandlog from /opt/share/syslog-ng/examples and put that into syslog.d (and run `scribe restart`) if you can trigger the vpn failover then the /opt/var/log/expanded-syslog.log file will have a detailed breakout of the log. That should give you some ideas of how to filter the log entry properly.

 

[tomsk]

A single line is added to /opt/etc/init.d/S01syslog-ng that calls a helper script to kill the system syslogd and make the necessary changes to the filesystem to keep syslogd from causing problems should something cause it to restart.

In my early forays into syslog-ng this was managed as a function called by the pre-command ... how do you ensure the helper script is called before syslog-ng tries to start?

 

[Martineau]

This discussion slipped by me ...

I don't think VPN_Failover.sh is the program name, I think that script is using the logger command, so I don't know what the "program" would be. If you grab expandlog from /opt/share/syslog-ng/examples and put that into syslog.d (and run `scribe restart`) if you can trigger the vpn failover then the /opt/var/log/expanded-syslog.log file will have a detailed breakout of the log. That should give you some ideas of how to filter the log entry properly.

Already answered how to trap my script messages in post #18

 

[cmkelley]

In my early forays into syslog-ng this was managed as a function called by the pre-command ... how do you ensure the helper script is called before syslog-ng tries to start?

I insert a line into S01syslog-ng above the line that calls rc.func that calls rc.func.syslog-ng (which I copy to init.d) which adds the function, sets the precmd value, and exports TZ. Trying to stick with the "add only one line" mantra of the SNB script-writers.

sed -i '/rc.func.syslog-ng/d; \~/opt/etc/init.d/rc.func~i . /opt/etc/init.d/rc.func.syslog-ng # added by scribe\n' $init_d/S01syslog-ng

 

[cmkelley]

Already answered how to trap my script messages in post #18

Agree, but as noted, `match` is deprecated, I don't know how much longer until it is removed. My pedantry insists that I not use deprecated functions.

 

[Martineau]

Agree, but as noted, `match` is deprecated, I don't know how much longer until it is removed. My pedantry insists that I not use deprecated functions.

Ahh OK

Guess I'll have to install scribe and migrate all of my existing rules.

EDIT: Expandlog returns

================================================
 PROGRAM=(VPN_Failover.sh)#
 PID=#
 BSDTAG=4B#
 TAG=0c#
 TAGS=.source.src#
 FACILITY=user#
 FACILITY_NUM=1#
 LEVEL=warning#
 LEVEL_NUM=4#
 PRI=12#
 PRIORITY=warning#
 FULLHOST=RT-AC86U#
 FULLHOST_FROM=RT-AC86U#
 HOST=RT-AC86U#
 HOST_FROM=RT-AC86U#
 LOGHOST=RT-AC86U.#
 MSGHDR=(VPN_Failover.sh): #
 MSGID=#
 MSGONLY=32075 **VPN Client Monitor: Switching VPN Client 1 to VPN Client 2 (Reason: VPN Client 1 STATE=0;Disconnected)#
 MSG=32075 **VPN Client Monitor: Switching VPN Client 1 to VPN Client 2 (Reason: VPN Client 1 STATE=0;Disconnected)#
 MESSAGE=32075 **VPN Client Monitor: Switching VPN Client 1 to VPN Client 2 (Reason: VPN Client 1 STATE=0;Disconnected)#
 SOURCE=src#

 

[skeal]

This really looks like it's coming along niceley. I'm a log watcher of sorts and would really like a solution to the Skynet logging to syslog. This appears to be it. I'll wait to jump in though, as I'm not well health wise right now. I've wanted this solution since Skynet came out...LOL Thank you for this outstanding effort.

 

[cmkelley]

Ahh OK

Guess I'll have to install scribe and migrate all of my existing rules.

Heh, don't do that on my account. I know I'm a bit over-zealous at times ... just be aware that if you update syslog-ng in the future and stuff starts breaking, then the `match` function is a likely suspect.

EDIT: Expandlog returns

================================================
 PROGRAM=(VPN_Failover.sh)#
 PID=#
 BSDTAG=4B#
 TAG=0c#
 TAGS=.source.src#
 FACILITY=user#
 FACILITY_NUM=1#
 LEVEL=warning#
 LEVEL_NUM=4#
 PRI=12#
 PRIORITY=warning#
 FULLHOST=RT-AC86U#
 FULLHOST_FROM=RT-AC86U#
 HOST=RT-AC86U#
 HOST_FROM=RT-AC86U#
 LOGHOST=RT-AC86U.#
 MSGHDR=(VPN_Failover.sh): #
 MSGID=#
 MSGONLY=32075 **VPN Client Monitor: Switching VPN Client 1 to VPN Client 2 (Reason: VPN Client 1 STATE=0;Disconnected)#
 MSG=32075 **VPN Client Monitor: Switching VPN Client 1 to VPN Client 2 (Reason: VPN Client 1 STATE=0;Disconnected)#
 MESSAGE=32075 **VPN Client Monitor: Switching VPN Client 1 to VPN Client 2 (Reason: VPN Client 1 STATE=0;Disconnected)#
 SOURCE=src#

I don't remember the () usually being there around the program name, but I haven't used this script for a while, and I'm not at home to look at the log. I only use that script periodically for debugging since it makes large logs. Are they there for other entries? If not, `program` might have to match on `(VPN_Failover.sh)`.