Scribe scribe - syslog-ng and logrotate installer

[New2This]

WOW, just checked mine, unbound .log its up to 27mb ( mines been logging since feb/3 )

Where do I go to turn it off?

 

[cmkelley]

got this when opening uiScribe today don't know if it relates to scribe or UiScribe

[2020-02-05T05:37:32.353122] WARNING: Configuration file format is too old, syslog-ng is running in compatibility mode. Please update it to use the syslog-ng 3.25 format at your time of convenience. To upgrade the configuration, please review the warnings about incompatible changes printed by syslog-ng, and once completed change the @version header at the top of the configuration file; config-version='3.23'

Yeah, when syslog-ng updates versions, it automatically throws that warning if the version number in the configuration file is less than the current version. I have a line in scribe that checks the version number in syslog-ng.conf against the program version anytime scribe's status is requested, but of course that doesn't get executed until scribe's status is requested.

I presume opkg is smart enough to restart the syslog-ng service when syslog-ng is upgrades, so I could add a version consistency check to that I suppose, although it seems to be virtually always a nuisance message rather than actually alerting of an actual problem. It would be nice if it was smart enough to only warn when there is an actual incompatibility in the configuration instead of the version number just being different.

 

[L&LD]

WOW, just checked mine, unbound .log its up to 27mb ( mines been logging since feb/3 )

Where do I go to turn it off?

In unbound_manager, select 'i' and re-install with the options you want.

At the very last step, do not say 'y' to re-use the previous unbound.conf file. Note that any customizations will be lost and will need to be redone.

 

[dave14305]

WOW, just checked mine, unbound .log its up to 27mb ( mines been logging since feb/3 )

Where do I go to turn it off?

there should be a lx option to Disable Logging in unbound_manager.

 

[L&LD]

Trying to get the unbound logging and rotating that Scribe can do in the latest Unbound v2.04, I ran into another problem.

My dnsmasq.log file is currently 135MB and getting larger!

Can I just delete this file? Or am I asking for more problems if I do?

What is controlling this file and making it grow so big? Have I accidentally enabled a switch and need to turn it off again?

 

[cmkelley]

Trying to get the unbound logging and rotating that Scribe can do in the latest Unbound v2.04, I ran into another problem.

My dnsmasq.log file is currently 135MB and getting larger!

Can I just delete this file? Or am I asking for more problems if I do?

What is controlling this file and making it grow so big? Have I accidentally enabled a switch and need to turn it off again?

Normal. Not shown in uiScribe because it doesn't go through the syslog service. Whether dsnmasq itself, or skynet, or something else writes it there, I don't know, but it's not written with syslog-ng.

 

[elorimer]

My dnsmasq.log file is currently 135MB and getting larger!

Can I just delete this file? Or am I asking for more problems if I do?

NOOOOO! Handled by Diversion.

 

[L&LD]

NOOOOO! Handled by Diversion.

Now I know! Too late, already deleted it.

 

[elorimer]

Now I know! Too late, already deleted it.

You are going to be chastised most severely by @thelonelycoder.

 

[L&LD]

I did a reboot and the file recreated itself. Just 161KB right now.

 

[elorimer]

For fun, I un-commented out the net source and configured my FreeNAS servers (the main one and the backup) to copy their logging to my router, with appropriate log configurations. That more or less sets up automagically their appearance on the uiScribe page and also routes them to loggly, but also identifying their source as separate from the router. Kind of cool.

The main one is very active so the maxsize or daily rotation will be important.

 

[cmkelley]

For fun, I un-commented out the net source and configured my FreeNAS servers (the main one and the backup) to copy their logging to my router, with appropriate log configurations. That more or less sets up automagically their appearance on the uiScribe page and also routes them to loggly, but also identifying their source as separate from the router. Kind of cool.

The main one is very active so the maxsize or daily rotation will be important.

I have to ask the obvious question - why are you putting your server log files on your router? Isn't the logical thing to do to route log files from all clients (including the router) to file server where (presumably) they will be backed up so they might be able to help you in case of a serious problem?

 

[elorimer]

I have to ask the obvious question - why are you putting your server log files on your router? Isn't the logical thing to do to route log files from all clients (including the router) to file server where (presumably) they will be backed up so they might be able to help you in case of a serious problem?

Fair point. The FreeNAS servers run syslog-ng and more or less trivially expect to maintain their own logs and copy them to some central location. From my (for fun) experiment, scribe and uiScribe more or less trivially can handle combining these logs in a place where I regularly look and easily display them, and also send them on to loggly if I want to look there. I could invert the direction and centralize them on a FreeNAS server but that is less trivially done.

And by "trivial" I mean to complement you and @Jack Yaz on the user-friendliness.

 

[cmkelley]

In response to the recent issues uncovered with upgrading syslog-ng, and a couple things I've picked at, scribe 2.4.2 is up.

Notable changes:

• logrotate A00global file now sets a global log file maximum size at 4 megs.
  • In this process I discovered comments in logrotate files need to go on their own line
• changed the wlceventd rotation from monthly to (the default) weekly. changed logs kept from 1 to 4 so same time frame should be saved.
• tried to automate updating syslog-ng.conf version number as much as possible. Now when it detects a difference between syslog-ng's version and the version in the conf file, it stops syslog-ng, fixes the conf file, and re-starts syslog-ng. This should help with issues some people experienced where syslog-ng stopped loging after being upgraded with entware. It's still going to require running scribe status or similar manually with each syslog-ng upgrade as far as I can tell.
• added installing / running uiScribe to the scribe utilities menu.
• removed un-needed "destination d_null { file("/dev/null"); };" from the blankmsg filter. Not specifying a file has the same result, and doesn't generate a "null" section on uiScribe.
• a couple minor cosmetic changes.

Unrelated, the "EOF occurred while reading control socket." message with non-HND platforms seems to have been resolved with the 3.25 version of syslog-ng. At least it is on my AC3200. Let me know if anyone is still getting it with syslog-ng 3.25. Of course, I updated firmware and syslog-ng at the same time, so YMMV.

As usual, it's late here, and I have to go to work in the morning. So if I broke anything you may have to wait a bit for a fix. I did some testing on my AC3200, and I do development on my AC86U, so hopefully I found the big problems.

 

[joe scian]

In response to the recent issues uncovered with upgrading syslog-ng, and a couple things I've picked at, scribe 2.4.2 is up.

Notable changes:

• logrotate A00global file now sets a global log file maximum size at 4 megs.
  • In this process I discovered comments in logrotate files need to go on their own line
• changed the wlceventd rotation from monthly to (the default) weekly. changed logs kept from 1 to 4 so same time frame should be saved.
• tried to automate updating syslog-ng.conf version number as much as possible. Now when it detects a difference between syslog-ng's version and the version in the conf file, it stops syslog-ng, fixes the conf file, and re-starts syslog-ng. This should help with issues some people experienced where syslog-ng stopped loging after being upgraded with entware. It's still going to require running scribe status or similar manually with each syslog-ng upgrade as far as I can tell.
• added installing / running uiScribe to the scribe utilities menu.
• removed un-needed "destination d_null { file("/dev/null"); };" from the blankmsg filter. Not specifying a file has the same result, and doesn't generate a "null" section on uiScribe.
• a couple minor cosmetic changes.

Unrelated, the "EOF occurred while reading control socket." message with non-HND platforms seems to have been resolved with the 3.25 version of syslog-ng. At least it is on my AC3200. Let me know if anyone is still getting it with syslog-ng 3.25. Of course, I updated firmware and syslog-ng at the same time, so YMMV.

As usual, it's late here, and I have to go to work in the morning. So if I broke anything you may have to wait a bit for a fix. I did some testing on my AC3200, and I do development on my AC86U, so hopefully I found the big problems.

updated to latest - no issues here

 

[L&LD]

Just updated too. Running smooth.

 

[Jack Yaz]

I did a reboot and the file recreated itself. Just 161KB right now.

Your Diversion stats will now be pointless until everything resets on Sunday (or whenever yoir weekly schedule is set in Diversion)

 

[XIII]

I just installed Scribe for the first time yesterday... (and I did not read all 55 pages first...)

1. Does the UI part (which I did not install yet) support the new 384.15 addons?
2. Did somebody already write a filter for NextDNS?

 

[cmkelley]

I just installed Scribe for the first time yesterday... (and I did not read all 55 pages first...)

1. Does the UI part (which I did not install yet) support the new 384.15 addons?
2. Did somebody already write a filter for NextDNS?

1: Yes, Jack updated uiScribe to support the new addon API. No changes were necessary, the new addons are for creating new tabs in the UI, uiScribe modifies the existing system log tab.
2: This I don't know.

 

[XIII]

1: Yes, Jack updated uiScribe to support the new addon API.

So this ("modifies the webui System Log page") is outdated?

Please select an option: ui

 Would you like to install uiScribe v1.2.0, a script by Jack Yaz
 that modifies the webui System Log page to show the various logs
 generated by syslog-ng in individual drop-down windows [y|n]?

(it's what kept me from installing uiScribe)