Scribe scribe - syslog-ng and logrotate installer

[elorimer]

If you look at /opt/etc/logrotate.conf, it should indicate that the log files will be rotated weekly.

If you want to do something else, you need a file--call it anything-- in /opt/etc/logrotate.d that might look like this:

/opt/var/log/openvpn.log {
    minsize 1024k
    daily
    rotate 9
    postrotate
    /usr/bin/killall -HUP syslog-ng
    endscript
}

This instructs logrotate to rotate that log daily, as long as it is at least 1meg, and to keep 9 days of logs.

 

[cmkelley]

I got a rather large log file that I would like to rotate so it does not choke http when viewing syslog.

The log file is /opt/var/openvpn.log and is currently 22MBs. How would I go about keeping this file around 4 MBs? I know it would have to do with a filter, but not versed enough to create one.

Any suggestions would be appreciated!

If you look at /opt/etc/logrotate.conf, it should indicate that the log files will be rotated weekly.

If you want to do something else, you need a file--call it anything-- in /opt/etc/logrotate.d that might look like this:

/opt/var/log/openvpn.log {
    minsize 1024k
    daily
    rotate 9
    postrotate
    /usr/bin/killall -HUP syslog-ng
    endscript
}

This instructs logrotate to rotate that log daily, as long as it is at least 1meg, and to keep 9 days of logs.

Actually, there is an openvpn logrotate file in /opt/share/logrotate/examples/ ... but I was thinking of casual openvpn users (like me) when I created the file, so it defaults to monthly rotation! No wonder you have a 22MB file ... oops. Currently, the file looks like

/opt/var/log/openvpn.log {
    monthly
    rotate 9
    postrotate
        /usr/bin/killall -HUP syslog-ng
    endscript
}

I think I should change it to look more like @elorimer's example instead. Makes much more sense when people may be using openvpn regularly.

 

[elorimer]

Or, change the skynet code to send that if syslog-ng is installed. Probably too much to ask @Adamm .

@Adamm has taken me up on this suggestion, and I wonder if someone else would also be game to try it.

Add a new line 999 to the firewall script to read

    [ -n "$(pidof syslog-ng)" ] && /usr/bin/killall -HUP syslog-ng

Then, delete the time-reap(2) parameter from the defnition in /opt/etc/syslog-ng.d/skynet file.

The idea is that if syslog-ng is running, the hourly skynet summary will send the hangup signal to syslog-ng, causing it to close and reopen its files. In the meantime, syslog-ng will leave its log files open for 60 seconds, maybe the full hour.

I'm running this on a 56U at 384.6 and an 87U on 384.13, and so far it seems to be working.

If you try it, also set skynet auto-update to off; unless it is incorporated into the main script, updates will undo the skynet part but not the syslog-ng part.

 

[Goobi]

If you look at /opt/etc/logrotate.conf, it should indicate that the log files will be rotated weekly.

If you want to do something else, you need a file--call it anything-- in /opt/etc/logrotate.d that might look like this:

/opt/var/log/openvpn.log {
    minsize 1024k
    daily
    rotate 9
    postrotate
    /usr/bin/killall -HUP syslog-ng
    endscript
}

This instructs logrotate to rotate that log daily, as long as it is at least 1meg, and to keep 9 days of logs.

I created a file called /opt/etc/logrotate.d/openvpn with the following;

opt/etc/logrotate.d/openvpn
/opt/var/log/openvpn.log {
    minsize 1024k
    daily
    rotate 9
    postrotate
    /usr/bin/killall -HUP syslog-ng
    endscript
}

I ran lr to rotate the log but the log file remains at over 24MB in size.

The logrotate log shows it was processed:

logrotate: "/opt/var/log/openvpn.log" 2019-11-25-0:0:0

Any pointers into what I may have overlooked that is causing the log not to get rotated?

Thanks again!

 

[Butterfly Bones]

I created a file called /opt/etc/logrotate.d/openvpn with the following;

opt/etc/logrotate.d/openvpn
/opt/var/log/openvpn.log {
    minsize 1024k
    daily
    rotate 9
    postrotate
    /usr/bin/killall -HUP syslog-ng
    endscript
}

I ran lr to rotate the log but the log file remains at over 24MB in size.

The logrotate log shows it was processed:

logrotate: "/opt/var/log/openvpn.log" 2019-11-25-0:0:0

Any pointers into what I may have overlooked that is causing the log not to get rotated?

Thanks again!

What did you set the file permissions to?

It needs to be 600 or -rw-------

 

[elorimer]

opt/etc/logrotate.d/openvpn
/opt/var/log/openvpn.log {
minsize 1024k
daily
rotate 9
postrotate
/usr/bin/killall -HUP syslog-ng
endscript
}

That first line shouldn't be part of the file--is it?

 

[cmkelley]

I created a file called /opt/etc/logrotate.d/openvpn with the following;

opt/etc/logrotate.d/openvpn
/opt/var/log/openvpn.log {
    minsize 1024k
    daily
    rotate 9
    postrotate
    /usr/bin/killall -HUP syslog-ng
    endscript
}

I ran lr to rotate the log but the log file remains at over 24MB in size.

The logrotate log shows it was processed:

logrotate: "/opt/var/log/openvpn.log" 2019-11-25-0:0:0

Any pointers into what I may have overlooked that is causing the log not to get rotated?

Thanks again!

What's the output of

logrotate -d /opt/etc/logrotate.conf

 

[Goobi]

Looks like the permissions was the culprit thanks much for the assist!

 

[Goobi]

I am noticing that although the openvpn.log file was rotated, it is still writing to the original now renamed log file instead of the new file which is empty. Would I have to restart openvpn whenever the log file is rotated?

 

[elorimer]

I am noticing that although the openvpn.log file was rotated, it is still writing to the original now renamed log file instead of the new file which is empty. Would I have to restart openvpn whenever the log file is rotated?

That's the purpose of the SIGHUP line, so that must not be getting executed.

What text editor did you use to create the logrotate.d/openvpn file? Perhaps you have DOS line endings rather than UNIX line endings?

dos2unix /opt/etc/logrotate.d/openvpn

 

[Goobi]

That's the purpose of the SIGHUP line, so that must not be getting executed.

What text editor did you use to create the logrotate.d/openvpn file? Perhaps you have DOS line endings rather than UNIX line endings?

dos2unix /opt/etc/logrotate.d/openvpn

I copied an existing file and made modifications using vi.

Should the sighup not be killing the openvpn client instead is syslog-ng?

/opt/var/log/openvpn.log {
    minsize 1024k
    daily
    rotate 9
    copytruncate
    compress
    postrotate
        /usr/bin/killall -HUP syslog-ng
    endscript
}

 

[elorimer]

Should the sighup not be killing the openvpn client instead is syslog-ng?

No, the only thing going on here is logging. Openvpn is sending log messages to the system logging function. Syslog-ng picks them up, and sorts messages into their respective files. When logrotate runs, it renames the existing openvpn.log file and starts a new one with the openvpn.log name. Meantime, syslog-ng is still writing to the old (renamed) locations. So the hangup signal needs to go to syslog-ng to tell it to close that file and open the new location.

But, you should try deleting that copytruncate line. I think that does the opposite--not sure--but so far that isn't in use here.

 

[cmkelley]

I copied an existing file and made modifications using vi.

Should the sighup not be killing the openvpn client instead is syslog-ng?

/opt/var/log/openvpn.log {
    minsize 1024k
    daily
    rotate 9
    copytruncate
    compress
    postrotate
        /usr/bin/killall -HUP syslog-ng
    endscript
}

No, the only thing going on here is logging. Openvpn is sending log messages to the system logging function. Syslog-ng picks them up, and sorts messages into their respective files. When logrotate runs, it renames the existing openvpn.log file and starts a new one with the openvpn.log name. Meantime, syslog-ng is still writing to the old (renamed) locations. So the hangup signal needs to go to syslog-ng to tell it to close that file and open the new location.

But, you should try deleting that copytruncate line. I think that does the opposite--not sure--but so far that isn't in use here.

Yes, definitely delete copytruncate; you shouldn't need that with syslog-ng (or rsyslog). A bit of googling reveals a number of people having issues related to the use of copytruncate.

@elorimer is correct, the file is written to by syslog-ng, not OpenVPN; OpenVPN is just writing to the system logging facility. Because logroate rotates by renaming files, the file handle that syslog-ng has still points to the renamed file. Sending SIGHUP to syslog-ng tells it to close all its files and re-open them, which creates new log files for any rotated logs. If you sent a SIGHUP to OpenVPN, it would terminate all VPN connections in use at the time ... the system logging facility handles the actual writing of the logs for exactly this reason.

 

[elorimer]

Also, you might delete the compress command here. I don't think it figures into your problem, but there already is a compress/delaycompress command in A00global. Makes me wonder what file you used to edit in the first place--I don't think it is from any scribe example--and what other things you have in the logrotate.d directory.

 

[cmkelley]

scribe v2.3_0 is up.

Notable changes:

1. When updating filters, files in /opt/etc/syslog-ng.d and /opt/etc/logrotate.d are checked against files in /opt/share/syslog-ng/examples and /opt/share/logrotate/examples to see if there is an updated version in the examples directory. You are offered the ability to view the diff before accepting or rejecting the revised file. If you reject the update, it will be offered again the next time you update filters.
2. As part of the above, the skynet syslog-ng file is no longer automatically updated when you update filters - it is treated like any other file and you will be asked before installation
3. A new option, ld, has been added to the utilities menu(su) to generate and view debugging info for logrotate. This information is also now included in the debug file generated by the d option in the utilities menu.

 

[elorimer]

1. The d option didn't show in the menu, but worked fine.
2. I updated the filters. It gave me the option to review a difference in /opt/etc/logrotate.d/logrotate which I think was just a space or tab character. Pretty nifty. It didn't give me the option to review my skynet config but overwrote it: in my case my original config did not have the time-reap(2) parameter.

 

[Butterfly Bones]

1. The d option didn't show in the menu, but worked fine.

If you meant to type "ld" it is under the "su" submenu, ala AMTM and Diversion.

 

[cmkelley]

1. The d option didn't show in the menu, but worked fine.
2. I updated the filters. It gave me the option to review a difference in /opt/etc/logrotate.d/logrotate which I think was just a space or tab character. Pretty nifty. It didn't give me the option to review my skynet config but overwrote it: in my case my original config did not have the time-reap(2) parameter.

I can't understand how that could happen. And I specifically tested that on my test router. It would have overwritten the one in /opt/share/syslog-ng/examples, but shouldn't have overwritten the one in /opt/etc/syslog-ng.d ... you don't have the one in syslog-ng.d symlinked to the other do you?

If you meant to type "ld" it is under the "su" submenu, ala AMTM and Diversion.

"d" (create debug file) is also in the "su" submenu.

EDIT: Actually, any command works at either menu. It simplifies the coding. Only certain ones are suppressed from view on the main menu (and from working) if scribe is not installed and/or syslog-ng is not running.

 

[elorimer]

If you meant to type "ld" it is under the "su" submenu, ala AMTM and Diversion.

Better to keep my mouth shut and be thought a fool than open it and remove all doubt. Yes, right there in black and white.

 

[elorimer]

It would have overwritten the one in /opt/share/syslog-ng/examples, but shouldn't have overwritten the one in /opt/etc/syslog-ng.d ... you don't have the one in syslog-ng.d symlinked to the other do you?

No, but I just now ran uf and it worked as you intended.