Scribe scribe - syslog-ng and logrotate installer

[unsynaps]

I might be dumb but I was looking at the examples.

Should 'ascd' be 'acsd'?

 

[SomeWhereOverTheRainBow]

scribe v2.3_0 is up.

Notable changes:

1. When updating filters, files in /opt/etc/syslog-ng.d and /opt/etc/logrotate.d are checked against files in /opt/share/syslog-ng/examples and /opt/share/logrotate/examples to see if there is an updated version in the examples directory. You are offered the ability to view the diff before accepting or rejecting the revised file. If you reject the update, it will be offered again the next time you update filters.
2. As part of the above, the skynet syslog-ng file is no longer automatically updated when you update filters - it is treated like any other file and you will be asked before installation
3. A new option, ld, has been added to the utilities menu(su) to generate and view debugging info for logrotate. This information is also now included in the debug file generated by the d option in the utilities menu.

You forgot to mention this new feature....

 

[Martineau]

You forgot to mention this new feature....

View attachment 20051

'/jffs/scripts/uiScribe.d/Main_LogStatus_Content.asp' needs to be updated

document.getElementById('dstzone').innerHTML = "<#195#>";

to probably

document.getElementById('dstzone').innerHTML = "<#211#>";

 

[elorimer]

On my 56U it comes out as "We suggest you to try a mix of letters, numbers, and symbols. Do you want to continue?"

211 is "According to South Korea government instruction, password must contain at least 8 characters including 1 alphabet letter, 1 special character, 1 numeric character."

172 is the time zone message.

 

[dave14305]

'/jffs/scripts/uiScribe.d/Main_LogStatus_Content.asp' needs to be updated

document.getElementById('dstzone').innerHTML = "<#195#>";

to probably

document.getElementById('dstzone').innerHTML = "<#211#>";

That’s for @Jack Yaz to fix.

 

[Martineau]

On my 56U it comes out as "We suggest you to try a mix of letters, numbers, and symbols. Do you want to continue?"

So use the number reported by:

echo $(($(grep -n "savings time is enabled" /www/EN.dict | cut -d":" -f1)-1))

 

[SomeWhereOverTheRainBow]

I thought scribe was going to push updates overnight. Man I was excited.

 

[cmkelley]

I thought scribe was going to push updates overnight. Man I was excited.

I don't touch webGUI pages, let alone push updates. That's so far past my abilities I can't even see it from here. Sorry you got your hopes up.

As has been noted, the enhanced system log page is actually courtesy of @Jack Yaz's uiScribe, he has to fix that.

 

[cmkelley]

I might be dumb but I was looking at the examples.

Should 'ascd' be 'acsd'?

It would appear I got it all backwards, yes. I'm surprised nobody else noticed ... I don't have that issue so I couldn't test the script.

Not at home, celebrating stuff yourself until you can't move and argue over politics and football day, but I'll try to fix it soon.

 

[Butterfly Bones]

It would appear I got it all backwards, yes. I'm surprised nobody else noticed ... I don't have that issue so I couldn't test the script.

Not at home, celebrating stuff yourself until you can't move and argue over politics and football day, but I'll try to fix it soon.

I do not use automatic channel selection either, but here is the word direct from RMerlin.
https://www.snbforums.com/threads/b...ta-is-now-available.56325/page-26#post-486725

 

[elorimer]

So use the number reported by:

echo $(($(grep -n "savings time is enabled" /www/EN.dict | cut -d":" -f1)-1))

Yay!!

 

[Jack Yaz]

I don't touch webGUI pages, let alone push updates. That's so far past my abilities I can't even see it from here. Sorry you got your hopes up.

As has been noted, the enhanced system log page is actually courtesy of @Jack Yaz's uiScribe, he has to fix that.

Fixed

 

[RMerlin]

On my 56U it comes out as "We suggest you to try a mix of letters, numbers, and symbols. Do you want to continue?"

211 is "According to South Korea government instruction, password must contain at least 8 characters including 1 alphabet letter, 1 special character, 1 numeric character."

172 is the time zone message.

This can vary between models. Just replace that with an actual English string instead of a tag.

 

[Martineau]

This can vary between models. Just replace that with an actual English string instead of a tag.

Surely a bit harsh on non-English speakers who feel more comfortable with the GUI in their native tongue?

I'm pretty sure there is a one-to-one correlation between the regional dictionary files for a specific msg-number on each model:

e.g.

./Dictionary_MSG.sh 211

      212: * Daylight savings time is enabled in this time zone.
 
./Dictionary_MSG.sh 211 fr

      212: * L'heure d'été est activée pour ce fuseau horaire.
 
 ./Dictionary_MSG.sh 211 de

      212: * Sommerzeit ist in dieser Zeitzone aktiviert.
 
./Dictionary_MSG.sh 211 kr

      212: * 일광 시간 절약제는 이 표준 시간대에서 활성화합니다.
 
./Dictionary_MSG.sh 211 fi

      212: * Kesäaika on käytössä tällä aikavyöhykkeellä.
 
./Dictionary_MSG.sh 211 es

      212: * El horario de verano está habilitado en esta zona horaria.
 
etc.

So during the install routine (or preferably during every boot sequence) it is trivial to patch the .asp with the correct msg-number for the current model.

MSGNUM=$(($(grep -n "savings time is enabled" /www/EN.dict | cut -d":" -f1)-1))
sed -ibak "/dstzone.*innerHTML/s/#.*#/#$MSGNUM#/" /jffs/scripts/uiScribe.d/Main_LogStatus_Content.asp

service restart_httpd

 

[cmkelley]

v2.3_1 is up ...

• s/ascd/acsd/g (h/t @unsynaps) - you may safely (should?) delete /opt/share/logrotate/examples/ascd and /opt/share/syslog-ng/examples/ascd
• Moved (c) 'show combined syslog-ng config' from main menu to utilities menu, to be consistent with showing the logrotate debug info, also in the utilities menu
• Added info / files on remote logging
  • Added /opt/share/syslog-ng/examples/A00remote to send all messages to remote log
  • Added /opt/share/syslog-ng/examples/README_REMOTE with brief discussion of remote logging options
  • Updated /opt/share/syslog-ng/examples/syslog-ng.conf-scribe to expand comments to state that enabling destination(log_server) in its log section will only send otherwise unfiltered messages to the log server (i.e. only the messages that would end up in /opt/var/log/messages). This is only an update to the comments and your /opt/etc/syslog-ng.conf file will be untouched - I currently do not have a mechanism for automatic updating of that file other than updating the version number when a new version of syslog-ng is released
• Updated /opt/share/syslog-ng/example/README.1st to reflect the above changes

 

[elorimer]

You will note that Skynet v6.9.2 is revised to send a hangup to syslog-ng after it completes its hourly log purge. This means the time-reap(2) option is no longer necessary and can be deleted. This is working fine for me, but if you see a problem shout.

It also works fine if you leave the time-reap in, but it means that syslog-ng will close the skynet-0 log file 2 seconds after writing to it, all the time. If you take it out, syslog-ng leaves the file open a max of 60 seconds for additional writes, the way it handles the others. Not a big diff in this application.

 

[Butterfly Bones]

You will note that Skynet v6.9.2 is revised to send a hangup to syslog-ng after it completes its hourly log purge. This means the time-reap(2) option is no longer necessary and can be deleted. This is working fine for me, but if you see a problem shout.

It also works fine if you leave the time-reap in, but it means that syslog-ng will close the skynet-0 log file 2 seconds after writing to it, all the time. If you take it out, syslog-ng leaves the file open a max of 60 seconds for additional writes, the way it handles the others. Not a big diff in this application.

Are you seeing these entries in your /opt/var/log/syslog-ng.log?

Dec  4 01:00:00 RT-AC86U-4608 syslog-ng[20386]: Configuration reload request received, reloading configuration;
Dec  4 01:00:00 RT-AC86U-4608 syslog-ng[20386]: Configuration reload finished;
Dec  4 02:00:00 RT-AC86U-4608 syslog-ng[20386]: Configuration reload request received, reloading configuration;
Dec  4 02:00:00 RT-AC86U-4608 syslog-ng[20386]: Configuration reload finished;
Dec  4 02:25:00 RT-AC86U-4608 syslog-ng[20386]: Configuration reload request received, reloading configuration;
Dec  4 02:25:00 RT-AC86U-4608 syslog-ng[20386]: Configuration reload finished;
Dec  4 03:00:00 RT-AC86U-4608 syslog-ng[20386]: Configuration reload request received, reloading configuration;
Dec  4 03:00:00 RT-AC86U-4608 syslog-ng[20386]: Configuration reload finished;
Dec  4 04:00:00 RT-AC86U-4608 syslog-ng[20386]: Configuration reload request received, reloading configuration;
Dec  4 04:00:00 RT-AC86U-4608 syslog-ng[20386]: Configuration reload finished;
Dec  4 05:00:00 RT-AC86U-4608 syslog-ng[20386]: Configuration reload request received, reloading configuration;
Dec  4 05:00:00 RT-AC86U-4608 syslog-ng[20386]: Configuration reload finished;
Dec  4 06:00:01 RT-AC86U-4608 syslog-ng[20386]: Configuration reload request received, reloading configuration;
Dec  4 06:00:01 RT-AC86U-4608 syslog-ng[20386]: Configuration reload finished;

The new version of Skynet is likely the reason with the changes. I saw it begin on 1 Dec., the day that I began testing the new Skynet feature to support dnscrypt-proxy. I was not aware that the changes in how Skynet sends a hangup to syslog-ng, only what you posted a few days ago about the changes that could be made.

I left the time-reap(2) in my skynet filter file until yesterday, then removed it after your post; the hourly lines stay the same. I don't think these entries in the syslog-ng.log are a problem, just something new with the changes in Skynet logging. Others may or may not need to be aware.

 

[elorimer]

e you seeing these entries in your /opt/var/log/syslog-ng.log?

Yes, I think those are triggered by the hangup signal. They were kicked off daily by logrotate as well. Others too, I think.

I'm thinking of filtering them out and discarding them.

I don't think we have performance questions going on here; I think this is more about following an approach where the incredibly useful combo of these scripts can be maintained going forward with the least burden on our authors while the external elements are in continuous motion. @cmkelley certainly has had to deal with adapting to changes in syslog-ng--a powerful tool scaling to big installs--in a soho router environment

 

[cmkelley]

You will note that Skynet v6.9.2 is revised to send a hangup to syslog-ng after it completes its hourly log purge. This means the time-reap(2) option is no longer necessary and can be deleted. This is working fine for me, but if you see a problem shout.

It also works fine if you leave the time-reap in, but it means that syslog-ng will close the skynet-0 log file 2 seconds after writing to it, all the time. If you take it out, syslog-ng leaves the file open a max of 60 seconds for additional writes, the way it handles the others. Not a big diff in this application.

No, that is the old behaviour. With 3.23 and up, the file remains open indefinitely (well, until the HUP signal), the default time_reap for for destinations without a macro (which is most of the filters I supply with scribe) is 0, not 60 - it was 60 for everything previously.

Yes, I think those are triggered by the hangup signal. They were kicked off daily by logrotate as well. Others too, I think.

Correct, sending SIGHUP to syslog-ng causes it to close all files and reload its configuration.

I'm thinking of filtering them out and discarding them.

I don't think we have performance questions going on here; I think this is more about following an approach where the incredibly useful combo of these scripts can be maintained going forward with the least burden on our authors while the external elements are in continuous motion. @cmkelley certainly has had to deal with adapting to changes in syslog-ng--a powerful tool scaling to big installs--in a soho router environment

Funny thing that, I'd think these constant changes to existing functionality would drive sysadmins batty. Certainly they wouldn't upgrade for every point release, but when they did decide to update, they'd stand a good chance of being bit by one of the "oh by the way" changes. Or, maybe anyone like that uses the paid, supported version.

 

[Butterfly Bones]

This is slightly OT. I know others send data via Scribe to Loggly for analysis. Somewhere between 2300 and 2400 hours on 10 Dec. no data from my AC86U appears on Loggly. Scribe is running fine locally and I have confirmed all log filtering is functional.

All checks I have done via the Net is that Loggly is functional. I temporarily stopped Scribe via the menu and then ran

syslog-ng -Fevd

in a terminal. All checks are successful, and I see a connection to Loggly

[2019-12-11T08:50:52.391776] Syslog connection established; fd='14', server='AF_INET(52.34.108.226:514)', local='AF_INET(0.0.0.0:0)'

but no data is streamed apparently. Usually I can see the Skynet Blocked Inbound messages as they are sent.

Is Loggly working for those who use it? Thank you.