Scribe scribe 3.x_y - syslog-ng and logrotate installer

[Ripshod]

For me it took months for this error to show up. If you are willing to ride it out that long, then maybe the issue could be resolved.

Long term test? I'm up for that challenge.

 

[SomeWhereOverTheRainBow]

Long term test? I'm up for that challenge.

Okay, so this issue with scribe completely borked skynet. skynet would not start at all, syslog-ng kept getting "killed" whenever I restarted skynet to try to get it working. I have also ran into issues where scribe (syslog-ng) prevented adguardhome from starting as well. I am wondering if syslog-ng is downloading a new default config or if something is happening behind the scenes with it that is not being accounted for by scribe.

 

[Ripshod]

skynet-0.log doesn't get rotated according to scribe's logrotate.log

 

[SomeWhereOverTheRainBow]

skynet-0.log doesn't get rotated according to scribe's logrotate.log

I think that is because the file is managed by skynet. it is truncated. --- I may be wrong, but this is my understanding behind that.

 

[SomeWhereOverTheRainBow]

@Ripshod , the actual error is coming from syslog-ng.conf (or lack there of), the real question is, what is causing this breakage. Could the issue be caused by the USB not mounting before scribe starts?

The error message encountered suggests a syntax error in syslog-ng configuration file. The parser is expecting a version number as the first line in the configuration file, and it seems to encounter a floating-point number instead.

here is the default config:

https://raw.githubusercontent.com/cynicastic/scribe/master/syslog-ng.share/syslog-ng.conf-scribe

Here is a possible cause of the problem:

opkg list syslog-ng
syslog-ng - 4.2.0-1 - syslog-ng reads and logs messages to the system console, log
 files, other machines and/or users as specified by its
 configuration file.

the default config version only specifies 4.1.

Error parsing config, syntax error, unexpected LL_FLOAT, expecting end of file in /opt/etc/syslog-ng.conf:1:1-1:4:
1-----> 4.2
1-----> ^^^
2       4.2
3       4.2
4       4.2
5       4.2
6       4.2


syslog-ng documentation: https://www.syslog-ng.com/technical-documents/list/syslog-ng-open-source-edition
contact:
        GitHub Project: https://github.com/syslog-ng/syslog-ng
        Chat with the Developers: https://gitter.im/syslog-ng/syslog-ng
        Mailing List: https://lists.balabit.hu/mailman/listinfo/syslog-ng
[2023-12-07T21:23:44.029665] ERROR: configuration files without a version number have become unsupported in syslog-ng 3.13, please specify a version number using @version as the first line in the configuration file;

 

[SomeWhereOverTheRainBow]

Well, that is my best deep dive. Everyone is welcome to pick at it. Do what ever it takes to figure it out.

 

[elorimer]

I responded to the post in the other thread.

 

[SomeWhereOverTheRainBow]

I responded to the post in the other thread.

Thank you for your detailed response.

 

[ankhazam]

Just adding my 2 bits... Creating a custom filter for hostapd clears (redirects) a lot of messages into the new file... System Messages become readable once again!

 

[cmkelley]

I'm back (sort of).

I apologize for my extended absense. What was originally supposed to be a 4 to 5 month remodel stretched to 9 months. Partly our fault ("while you're here, why don't you ...") and partly our city's fault (2+ months for a straightforward permit). Because of the network setup in the AirB&B we were staying at, I had to kinda kludge my network together in a way that was not at all condusive to even playing with scribe. We only moved back into the house April 3rd and we are still in boxes all over the house; we don't even have blinds on the windows (yes, it was a pretty major remodel). We survived the remodel, according to my wife's therapist, remodels have a nasty habit of turning into divorces(!) But we've got well over 10,000 miles on our divorce bike (tandem) so I guess we have lots of practice with such stuff. On the plus side, it's like moving into a whole new house, and things that we have really wanted to do to the house for the last 25 years are finally done (e.g. my wife has her bay window). The only way we're moving out of this house ever again is feet first.

I've just got my basic network here at home set back up (not sure where all my Raspberry Pis got to in all this), somewhere along the line one of our server backup drives failed (but not any of our primary server drives, thankfully), and I'm still trying to fix that mess. It will still be a while before I can get fully back into it, but in the near term I'll try to review problems people have had over the past year and start of list of things to look into. And then try to remember how to code again, lol.

Again, sorry for the extended absense. Looks like One Identity at least hasn't outright broken anything with syslog-ng in the last year, so I guess that's a plus.

 

[elorimer]

The only thing going on I'm aware of is the code that changes the version in the .conf file didn't handle well the change from version 3.xx to version 4.xx and overwrites all the config lines with 4.2. See post 185.

 

[matthew_eli]

For installing it on GT-BE98 or in any router with 3006.102.1 or later, I've removed the following check:

[ "$( ver_num "$fwvers" )" -lt "$( ver_num "$fwreqd" )"

from the part of the code checking the FW version:

if [ ! -x "/opt/bin/opkg" ] || [ "$fwname" != "$merlin" ] || [ "$( ver_num "$fwvers" )" -lt "$( ver_num "$fwreqd" )" ]

For sure there is a more elegant way, but I gues involves checking the major release first (3006 instead of 3004)

EDIT: excepting this, everithying seems working smoothly!

EDIT 2: I created a PR for this: https://github.com/cynicastic/scribe/pull/46; I ask you to test it before merging, but I can confirm it works in my environment

 

[swejuggalo]

Do I need to do something to get wlceventd log working?

"Log file will display here. If you are seeing this message,it means the log file cannot be loaded.
Please check your USB to check the /opt/var/log directory exists."

I don't really see that the folder is missing. Had WiFi on for a while and havn't seen anything happening here. No major thing, since I don't intend to use WiFi on this device, but still.
Is this a Scribe or uiScribe issue? Everything else is visible.

 

[elorimer]

This usually happens to me when one of the log files is too big. Check the files in that directory and see if any are over 1MB. If you find any, figure out why logrotate isn't rotating it. (Except dnsmasq and sysnet--they do their own).

The other time it happens can usually be fixed by refreshing the browser or toggling the "auto refresh" button on/off. It can take several seconds for a large log file to load.

 

[swejuggalo]

This usually happens to me when one of the log files is too big. Check the files in that directory and see if any are over 1MB. If you find any, figure out why logrotate isn't rotating it. (Except dnsmasq and sysnet--they do their own).

The other time it happens can usually be fixed by refreshing the browser or toggling the "auto refresh" button on/off. It can take several seconds for a large log file to load.

Ran a manual rotate just to see what happened. It triggered warnings and failures, stopped taking care of Skynet and that became empty too...
Back to normal but still nothing in wlceventd.log
In /opt/var/log that log is simply missing.
Oh wait.... Works!

Let's see if that happens again.
What if a file goes big? Can I control the rotate interval? Or it must be daily?

 

[elorimer]

On logrotate: scribe edits post-mount to create a cron job so logrotate runs daily. 4 old logs are kept, with 2-4 being compressed.

/opt/etc/logrotate.conf sets a default of weekly rotation and then includes /opt/etc/logrotate.d

/opt/etc/logrotate.d/A00global loads, and sets a maximum file size of 4M.

So scribe is set up so that every log is rotated at least weekly, but if on the daily run, it is over 4M, it is rotated immediately.

Other files in /opt/etc/logrotate.d alter that behavior, and there should be one for each destination log you have. For example, messages is rotated daily, but only if it is at least 1M in size, and 9 are kept. You need one for each destination because you need to restart syslog-ng to refresh itself, otherwise it will keep the destination pointed to the old log file, which no longer exists after rotation.

As to wlceventd.log, make sure that you have the destination defined.

 

[swejuggalo]

On logrotate: scribe edits post-mount to create a cron job so logrotate runs daily. 4 old logs are kept, with 2-4 being compressed.

/opt/etc/logrotate.conf sets a default of weekly rotation and then includes /opt/etc/logrotate.d

/opt/etc/logrotate.d/A00global loads, and sets a maximum file size of 4M.

So scribe is set up so that every log is rotated at least weekly, but if on the daily run, it is over 4M, it is rotated immediately.

Other files in /opt/etc/logrotate.d alter that behavior, and there should be one for each destination log you have. For example, messages is rotated daily, but only if it is at least 1M in size, and 9 are kept. You need one for each destination because you need to restart syslog-ng to refresh itself, otherwise it will keep the destination pointed to the old log file, which no longer exists after rotation.

As to wlceventd.log, make sure that you have the destination defined.

Something made wlceventd.log work after the reinstall. So now I know that it works on 388.8.
Normally, under most scenarios it should be up and working with defaults, I guess.

If Merlin ends up on the BE88U one day, who knows.
But for now I'm happy letting the AX88U managing Skynet/Diversion with the BE88U worrying about the wired and wireless clients, even if it would mean staying on stock firmware forever at BE88U.

 

[visortgw]

For installing it on GT-BE98 or in any router with 3006.102.1 or later, I've removed the following check:

[ "$( ver_num "$fwvers" )" -lt "$( ver_num "$fwreqd" )"

from the part of the code checking the FW version:

if [ ! -x "/opt/bin/opkg" ] || [ "$fwname" != "$merlin" ] || [ "$( ver_num "$fwvers" )" -lt "$( ver_num "$fwreqd" )" ]

For sure there is a more elegant way, but I gues involves checking the major release first (3006 instead of 3004)

EDIT: excepting this, everithying seems working smoothly!

EDIT 2: I created a PR for this: https://github.com/cynicastic/scribe/pull/46; I ask you to test it before merging, but I can confirm it works in my environment

Works as intended after removing check as you indicated! Thanks for chasing this down.

 

[matthew_eli]

Works as intended after removing check as you indicated! Thanks for chasing this down.

If you want, you can also try the PR I edited on GitHub, waiting to be merged. I tested it in my environment, and seems to work as intended. It does check the major release before the minor fw version and if it's ok, it installs the script

 

[elorimer]

@Martinski

I made the changes from your #47 pull request in my installed scribe script and I think there may be a problem. I think syslog-ng expects the version to be 4.7, not 4.7.1, so it fails to start. If you correct the conf file to 4.7, scribe corrects it to 4.7.1 and there you go again.

If you disable the check, change the version to 4.7, syslog-ng starts.

EDIT: fixed