Scribe scribe - syslog-ng and logrotate installer
- Thread starter cmkelley
- Start date
cmkelley
Very Senior Member
Does it do that through the new API (like I thought) or is it still the same method as before?
I have a hard enough time remembering how my script works, I need to learn to keep my mouth shut about other people's scripts ...
Jack Yaz
Part of the Furniture
no problem. We're all here to learn! Since uiscribe is just overriding a page and doesn't need a new tab it's dead simple. The complex stuff is when overriding unused pages and needing to insert a tab. the new api makes adding a tab much simpler, and the user mount points removes the need for me to override obscure unused pages
Butterfly Bones
Very Senior Member
This is dumb silly really stupid question I need to clarify, sorry I'm on some serious pain killers and my mind is fuzzy, and will be for some time.
The new afpd filter, the - are removed and the + are added?
Code:
Update available for /opt/etc/logrotate.d/afpd.
(a)ccept, (r)eject, or (v)iew diff for this file? v
--- /opt/etc/logrotate.d/afpd
+++ /opt/share/logrotate/examples/afpd
@@ -1,6 +1,4 @@
/opt/var/log/afpd.log {
-minsize 1024k
- daily
rotate 2
postrotate
/usr/bin/killall -HUP syslog-ngThis line totally looses me.
Code:
@@ -1,6 +1,4 @@cmkelley
Very Senior Member
Until you're used to them diffs can be a bit difficult to understand, even without pain meds.This is dumb silly really stupid question I need to clarify, sorry I'm on some serious pain killers and my mind is fuzzy, and will be for some time.
Not quite. Minsize means it has to be at least that big to rotate, regardless of the period specified. So as it was before, the file had to be at least 1M, AND it had to have been at least a day since the log was last rotated. In other words, both conditions had to be satisfied to rotate the log. Yes, on our routers, we only run logrotate once a day at 00:05, but on other systems, logrotate might run every hour, or even more often.
In /opt/etc/logrotate.conf, there is a global default of weekly log rotation, so removing daily just changes it to weekly. With maxsize set to 4M, and the period to weekly, the log will be rotated either when it reaches 4M, OR when it's been a week since the last log was rotated, whichever comes first.
The original file could run for months without rotating the log if the log file filled very slowly. The revised file will rotate every week, but can rotate sooner if the log file fills rapidly.
So:
- Time period (daily, weekly, monthly, etc) sets the normal log rotation time.
- Minsize requires that the log be at least some size before being rotated, even if it is past the time period. But if it passes minsize before the time period is up, the log won't be rotated until the time period passes.
- Maxsize causes the log to be rotated when it reaches that size, even if the time period is not up yet. In this case if the time period is up before maxisize is reached, the log file will still be rotated.
Rotating logs based only on size is possible, but I don't make use of it in the files I supply with scribe.
The first number pair (1,6) is the starting line number(1) and number of lines shown that are part of the first file(6). The second pair is the starting line number(1) and the number of lines shown that are part of the second file(4). The pairs are preceded by - and + as a reminder of which file the pair refers to. Obviously, this isn't important in a files as short as these, but if I'm looking a diff between, say, one version of scribe and another, and I see a difference I want to investigate, the first number of each pair tells me where to find that section in each file.This line totally looses me.
ThanksCode:@@ -1,6 +1,4 @@
Back making trouble.
I have in the past in this thread described my method (here: scribe - syslog-ng and logrotate installer) of pouring the startup log messages back into syslog-ng to be processed and put into messages. That adds the host tag. I still have the May 5 time stamp for those messages that go to tmp/syslog.log before the time sync takes place.
I realized I could change the messages destination in syslog-ng.conf like so:
This changes the timestamp of the log statements going to the messages file from the system date to the date that syslog-ng received the message. As a result, all the log statements in messages are in order starting with the time I rebooted the router, starting with all the startup messages with a time stamp of when syslog-ng started.
I also added
to my post-mount script, with the result that my messages file now starts in order from the last time I rebooted the router. This means, though, that I can't recover error messages if something went wrong and I had to reboot.
I have in the past in this thread described my method (here: scribe - syslog-ng and logrotate installer) of pouring the startup log messages back into syslog-ng to be processed and put into messages. That adds the host tag. I still have the May 5 time stamp for those messages that go to tmp/syslog.log before the time sync takes place.
I realized I could change the messages destination in syslog-ng.conf like so:
Code:
destination messages {
file("/opt/var/log/messages" template("${R_DATE} ${HOST} ${MSGHDR}${MESSAGE}\n") );
};I also added
Code:
rm -f /tmp/mnt/USBNAME/entware/var/log/messages
Last edited:
XIII
Very Senior Member
This filter should work, right?
Code:
# log all NextDNS logs to nextdns.log
destination d_nextdns {
file("/opt/var/log/nextdns.log");
};
filter f_nextdns {
program("nextdns");
};
log {
source(src);
filter(f_nextdns);
destination(d_nextdns);
flags(final);
};
#eofcmkelley
Very Senior Member
From the scribe main menu, use "uf" to update the filters. One of the filters it should ask you about is /opt/etc/logrotate.d/A00global. Updating A00global will set the maximum log size to 4M. That should trim them down at the next logrotate run. If you've done this and are STILL having problems, post or PM me the results of
Code:
ls -lart /opt/var/log
ssechaud
Occasional Visitor
Yeah I think I have been running with the default 4MB. I was trying to lower it to 1MB to see if that helped but wasn't sure how to do it properly or if it would resolve the issue. Here is the ls output:
Code:
username@DSL-AC68U-0628:/tmp/home/root# ls -lart /opt/var/log
-rw-r----- 1 nobody root 640839 Feb 12 05:18 dnsmasq.log1
-rw-r----- 1 nobody root 3120859 Feb 12 05:20 dnsmasq.log2
drwxr-xr-x 2 username root 4096 Feb 12 06:43 .
drwxr-xr-x 7 username root 4096 Feb 12 17:44 ..
-rw------- 1 username root 382640 Feb 12 20:32 messages
-rw-r----- 1 nobody root 473747 Feb 12 20:34 dnsmasq.log
-rw------- 1 username root 3722 Feb 12 20:35 logrotate.log
-rw------- 1 username root 48219 Feb 12 20:35 syslog-ng.log
-rw------- 1 username root 10353438 Feb 12 20:35 skynet-0.logThe skynet log is too big and probably isn't being collapsed hourly through skynet. I think therein lies your problem.
cmkelley
Very Senior Member
The only large one is skynet-0.log, and it isn't touched by logrotate anyways. Skynet should clear it hourly. If you're getting over 10 gigs in 35 minutes though, that's HUGE, so I suspect for some reason the Skynet scrub isn't happening. It's just past the hour now, how big is skynet-0.log now?
ssechaud
Occasional Visitor
I just reset the logs on Skynet, and set the log file location using this command:
Not sure if that will fix it, I don't see a command to set the log to collapse hourly in the Skynet command.
Code:
sh /jffs/scripts/firewall settings syslog /opt/var/log/skynet-0.logcmkelley
Very Senior Member
What's the output of
Code:
cru -lssechaud
Occasional Visitor
Code:
username@DSL-AC68U-0628:/tmp/home/root# cru l
00 2 * * Thu sh /opt/share/diversion/file/update-bl.div reset #Diversion_UpdateBL#
20 5 * * * sh /opt/share/diversion/file/rotate-logs.div #Diversion_RotateLogs#
20 17 * * * diversion count_ads count #Diversion_CountAds#
5 0 * * * /opt/sbin/logrotate /opt/etc/logrotate.conf >> /opt/tmp/logrotate.daily 2>&1 #logrotate#
25 5 * * * sh /jffs/scripts/firewall banmalware #Skynet_banmalware#
25 1 * * Mon sh /jffs/scripts/firewall update #Skynet_autoupdate#
0 * * * * sh /jffs/scripts/firewall save #Skynet_save#
40 */12 * * * sh /jffs/scripts/firewall debug genstats #Skynet_genstats#
*/5 * * * * /jffs/scripts/connmon generate daily #connmon_daily#
2 * * * * /jffs/scripts/connmon generate weekly #connmon_weekly#
3 */3 * * * /jffs/scripts/connmon generate monthly #connmon_monthly#
12,42 * * * * /jffs/scripts/spdmerlin generate #spdMerlin#The firewall save job should scrub the log, so I wonder why it isn't running. On the other hand I don't have that genstats entry.
Butterfly Bones
Very Senior Member
That is the new webgui stats generation to get a nice graphs page under the Firewall tab.
https://github.com/Adamm00/IPSet_ASUS
Code:
( sh /jffs/scripts/firewall settings webui enable|disable ) Enable/Disable WebUIcmkelley
Very Senior Member
Yeah, at this point I think it's either (a) you have an issue with your Skynet installation, or (b) you're really getting ~20 gigs per hour of Skynet logs which is massive, and I'd think your internet would barely be functional at that rate. Judging from what I have now, I get about 20 megs/hour of Skynet logs - 3 orders of magnitude less!
ssechaud
Occasional Visitor
I think it was, after setting the log file location it seems I am no longer having an issue with viewing in the browser. I will keep an eye on it.
5stringdeath
Regular Contributor
I keep getting this error in syslog.ng-log:
Not sure how to fix it, or if its even a scribe issue, but I see it in uiScribe in the WebUI which lately has been locking up and becoming non-responsive until I restart scribe.
I've tried two complete uninstalls, reboots, etc.
RT-AC3100 (armv7l) FW-384.15
Code:
Feb 13 08:47:01 AC3100 syslog-ng[10549]: The current log file has a mismatching size/inode information, restarting from the beginning; state='affile_sd_curpos(/var/lib/logrotate.status)', stored_inode='7925593', cur_file_inode='7937689', stored_size='207', cur_file_size='207', raw_stream_pos='0'Not sure how to fix it, or if its even a scribe issue, but I see it in uiScribe in the WebUI which lately has been locking up and becoming non-responsive until I restart scribe.
I've tried two complete uninstalls, reboots, etc.
RT-AC3100 (armv7l) FW-384.15
Similar threads
Similar threads
Similar threads
-
Scribe Strange Issue with Scribe / Syslog-NG after Reboot
- Started by ScottW
- Replies: 7
-
Scribe Scribe nvram log_path - what should it be?
- Started by vibroverbus
- Replies: 4
-
Scribe Issues with scribe.... Errors all over the place after restarting scribe!
- Started by SomeWhereOverTheRainBow
- Replies: 8
-
Scribe [SOLVED] syslog-ng intermittent failure to shutdown which messes all logging
- Started by Tarek Yag
- Replies: 15
-
AdGuardHome There are tons of dnsproxy error messages occuring on syslog.
- Started by jklim
- Replies: 2
-
Updated scMerlin to 2.5.5 - syslog stops at 'syslogd exiting'
- Started by thecheapseats
- Replies: 9