Scribe scribe - syslog-ng and logrotate installer

Xentrk · May 19, 2019

cmkelley said:
You need to whitelist "codeload.github.com" in Diversion.

Adding this function to the installer may help. May need to check with the Diversion and Skynet authors if a process list function needs to happen for the entry to take effect.

Code:

# Create shared-Scribe-whitelist file if one does not exist
# to prevent codeload.github.com from being blocked by AB-Solution and Skynet

whitelist_github_domain () {
    if [ ! -s "/jffs/shared-Scribe-whitelist" ];then
    printf "codeload.github.com\n" > /jffs/shared-Scribe-whitelist
fi
}

Xentrk · May 19, 2019

Going thru the files in the installer. /opt/tmp/logrotate.daily:

Code:

error: /opt/etc/logrotate.conf:31 duplicate log entry for /opt/var/log/messages

Xentrk · May 19, 2019

I got it to finally work buy uninstalling scribe followed by a reboot.. Then, running installer script again by copy/pasting the snip on the README.md page. Must have been a conflict with the current installatoin or config files for logrotate and syslog-ng. I have two messages about the duplicate entry

Code:

logrotate: error: /opt/etc/logrotate.conf:31 duplicate log entry for /opt/var/log/messages
logrotate: error: /opt/etc/logrotate.conf:31 duplicate log entry for /opt/var/log/messages

Jack Yaz · May 19, 2019

Xentrk said:
Adding this function to the installer may help. May need to check with the Diversion and Skynet authors if a process list function needs to happen for the entry to take effect.

Code:

# Create shared-Scribe-whitelist file if one does not exist # to prevent codeload.github.com from being blocked by AB-Solution and Skynet whitelist_github_domain () { if [ ! -s "/jffs/shared-Scribe-whitelist" ];then printf "codeload.github.com\n" > /jffs/shared-Scribe-whitelist fi }

it's already whitelisted in the latest Diversion
https://www.snbforums.com/threads/diversion-the-router-ad-blocker.48538/page-143#post-490907

cmkelley · May 19, 2019

Twiglets said:
I added a '-k' flag to the curl line (in the 'scribe' script) to ignore the verification part, BUT it had lots of problems running the script !!! (lots of dirs/files not found !!!???)

I ended up using git clone to copy the files/.zip down and manually set up the directories etc in /jffs/scripts/... and /opt/tmp/[scribe]|[scribe-master] .....

I ran the 'scribe' script multiple times and fixed the missing dirs/files until it was happy !!!

I was eventually able to run the 'scribe install' and 'scribe restart' to confirm all was working (after copying the 'right' files into /opt/tmp/[scribe]|[scribe-master]/[logrotate.share]|[syslog-ng.share])

Big Hint:
Make sure all the dirs/files are chmod'ed to '0755' otherwise the script is designed to ignore them.
I think this may possibly have been my problem !!!

As noted, whitelisting "codeload.github.com" in Diversion (or updating to Diversion 4.1.0) is needed to fix it. As to the issues you encountered after adding the '-k' flag, I'm at a loss. I've uninstalled and reinstalled multiple times on my test router, both new installations and overwriting existing, and I've never encountered any problems like that. The Entware installers for syslog-ng and logrotate create the directories with the appropriate permissions, the only caveat is anything added to /opt/etc/logrotate.d or /opt/etc/syslog-ng.d must not be writable by anyone other than the the user. So 655, 644, 600 all work for files, they don't need to be executable. Those are restrictions placed by logrotate and syslog-ng, not scribe.

Your setup may or may not be proper because of how you've done it, there's not way for me to tell for sure. If you have problems, I recommend uninstalling scribe, ensuring Diversion is 4.1.0 or higher, and re-installing scribe. You might do that anyways just be sure.

Butterfly Bones · May 19, 2019

glehel said:
can someone help you how to set up loggly?

As @elorimer stated, I've posted the filter file and there was a long exchange between @joe scian and me with many quite few posts (start here and read down).

If you have specific questions, post here and someone will help, of course.

Xentrk · May 19, 2019

@cmkelley

Looks like the installation script did not make /jffs/scripts/service-event executable.

Code:

custom_script: Found service-event, but script is not set executable!

I made it executable using chmod 755 service-event command.

cmkelley · May 19, 2019

Xentrk said:
@cmkelley

Looks like the installation script did not make /jffs/scripts/service-event executable.

Code:

custom_script: Found service-event, but script is not set executable!

I made it executable using chmod 755 service-event command.

Ooops. Forgot not everyone has at least one of Jack Yaz's scripts already installed. My bad.

cmkelley · May 19, 2019

Xentrk said:
@cmkelley

Looks like the installation script did not make /jffs/scripts/service-event executable.

Code:

custom_script: Found service-event, but script is not set executable!

I made it executable using chmod 755 service-event command.

Also you should add

Code:

#!/bin/sh

as the first line of /jffs/scripts/service-event

cmkelley · May 19, 2019

scribe v1.2_0 is up

Tests for Skynet before installing and asks if you want to abort to install Skynet
Refuse to install (new installation only) if Diversion is < v4.1.0
Recognize a new installation so above & checks for Merlin / Entware / Skynet only run on new installations
Create service-event correctly if doesn't already exist and ensures it is executable; thanks to Xentrk for finding that bug
Won't try to kill klogd & syslogd if service command is "stop"
Some logic changes (e.g. true/false instead of yes/no) because I haven't yet learned that better is the enemy of good enough
Some interface changes; see above for why

Just needs a "scribe update" to update.

L&LD · May 20, 2019

cmkelley said:
scribe v1.2_0 is up

Tests for Skynet before installing and asks if you want to abort to install Skynet

Refuse to install (new installation only) if Diversion is < v4.1.0

Recognize a new installation so above & checks for Merlin / Entware / Skynet only run on new installations

Create service-event correctly if doesn't already exist and ensures it is executable; thanks to Xentrk for finding that bug

Won't try to kill klogd & syslogd if service command is "stop"

Some logic changes (e.g. true/false instead of yes/no) because I haven't yet learned that better is the enemy of good enough

Some interface changes; see above for why

Just needs a "scribe update" to update.

Easy, no hassle update on my RT-AC86U.

El Mariachi · May 20, 2019

Yep clean update on my RT-AC87U also.

Twiglets · May 20, 2019

cmkelley said:
As noted, whitelisting "codeload.github.com" in Diversion (or updating to Diversion 4.1.0) is needed to fix it. As to the issues you encountered after adding the '-k' flag, I'm at a loss. I've uninstalled and reinstalled multiple times on my test router, both new installations and overwriting existing, and I've never encountered any problems like that. The Entware installers for syslog-ng and logrotate create the directories with the appropriate permissions, the only caveat is anything added to /opt/etc/logrotate.d or /opt/etc/syslog-ng.d must not be writable by anyone other than the the user. So 655, 644, 600 all work for files, they don't need to be executable. Those are restrictions placed by logrotate and syslog-ng, not scribe.

Your setup may or may not be proper because of how you've done it, there's not way for me to tell for sure. If you have problems, I recommend uninstalling scribe, ensuring Diversion is 4.1.0 or higher, and re-installing scribe. You might do that anyways just be sure.

Thanks for the reply.

Don't worry it is more likely to be something I did !!!
[I tend to learn by jumping in with two feet and breaking things

]

It is working fine now and 'all is happy in the world'

(Logs are being created in the correct places and log rotates are working.)

Many thanks for all the hard work as I would never have got it working myself

P.S.
For peace of mind I did the install again after the whitelist fix and all works 100%

So it was me !!!

Xentrk · May 20, 2019

I just did an uninstall followed by a new install. Worked good except I still see the warning messages:

Code:

==> /opt/tmp/logrotate.daily <==
error: /opt/etc/logrotate.conf:31 duplicate log entry for /opt/var/log/messages
error: /opt/etc/logrotate.conf:31 duplicate log entry for /opt/var/log/messages

elorimer · May 20, 2019

After scribe update to 1.2_0 scribe status still reports an update is available.

Edit: Nevermind. User error, probably because this is white on white using Juice on a Chromebook. Thanks @Xentrk for setting me straight.

Xentrk · May 20, 2019

elorimer said:
After scribe update to 1.2_0 scribe status still reports an update is available.

I did a "scribe status" and it reports it is up to date.

Code:

                            _
                         _ ( )
       ___    ___  _ __ (_)| |_      __
     /',__) /'___)( '__)| || '_`\  /'__`\
     \__, \( (___ | |   | || |_) )(  ___/
     (____/`\____)(_)   (_)(_,__/'`\____)
     syslog-ng and logrotate installation
     v1.2_0 (master)  Coded by cynicastic


 checking configuration for necessary scribe hooks ...

          checking S01syslog-ng ... present.

         checking service-event ... present.

            checking post-mount ... present.

    checking logrotate cron job ... present.

 checking syslog-ng daemon ...

         syslog-ng syntax check ... okay!

      checking syslog-ng daemon ... alive.

          scribe installed version: v1.2_0 (master)
             scribe GitHub version: v1.2_0 (master)
                    scribe is up to date!

cmkelley · May 20, 2019

Xentrk said:
I just did an uninstall followed by a new install. Worked good except I still see the warning messages:

Code:

==> /opt/tmp/logrotate.daily <== error: /opt/etc/logrotate.conf:31 duplicate log entry for /opt/var/log/messages error: /opt/etc/logrotate.conf:31 duplicate log entry for /opt/var/log/messages

Hrmmm. I wonder if the logrotate.daily file is just left over from a previous issue/problem? I probably forgot to delete that file with the uninstall routine. Can you check the date stamp on that file? Does it match your most recent installation?

Also, what timezone are you in? I'm UTC-7 (PDT). Not that it matters to scribe, just so I can figure out if I need to break away in the next 6 hours or then next 12.

Xentrk · May 20, 2019

cmkelley said:
Hrmmm. I wonder if the logrotate.daily file is just left over from a previous issue/problem? I probably forgot to delete that file with the uninstall routine. Can you check the date stamp on that file? Does it match your most recent installation?

Also, what timezone are you in? I'm UTC-7 (PDT). Not that it matters to scribe, just so I can figure out if I need to break away in the next 6 hours or then next 12.

The file is from March 23 2019. It does not match the /opt/etc/logrotate.d timestamp of May 20. The installer may need to check for an existing file if it does not do so already. I do that on my scripts. If the installer encounters an existing file, it will rename the file by adding a timestamp to the end of the existing file name before installing the new file.

I am 14 hours ahead of you.
Current Offset: UTC/GMT +7 hours
Difference: 14 hours ahead of Los Angeles
None of this is that urgent that you need to schedule around my timezone though. I have to prepare for work now. Have a good day.

cmkelley · May 20, 2019

Xentrk said:
The file is from March 23 2019. It does not match the /opt/etc/logrotate.d timestamp of May 20. The installer may need to check for an existing file if it does not do so already. I do that on my scripts. If the installer encounters an existing file, it will rename the file by adding a timestamp to the end of the existing file name before installing the new file.

I am 14 hours ahead of you.
Current Offset: UTC/GMT +7 hours
Difference: 14 hours ahead of Los Angeles
None of this is that urgent that you need to schedule around my timezone though. I have to prepare for work now. Have a good day.

Okay, one more thing to add to 1.2_1.

Xentrk · May 20, 2019

cmkelley said:
Okay, one more thing to add to 1.2_1.

Feel free to clone the code in https://github.com/Xentrk/Asuswrt-Merlin-Linux-Shell-Scripts/blob/master/dhcpstaticlist.sh

Look at the functions Save_DHCP_Staticlist and Make_Backukp

Thread starter	Title	Forum	Replies	Date
S	Scribe Strange Issue with Scribe / Syslog-NG after Reboot	Asuswrt-Merlin AddOns	7	Dec 30, 2023
V	Scribe Scribe nvram log_path - what should it be?	Asuswrt-Merlin AddOns	4	Feb 17, 2024
	Scribe Issues with scribe.... Errors all over the place after restarting scribe!	Asuswrt-Merlin AddOns	8	Dec 7, 2023
	Scribe [SOLVED] syslog-ng intermittent failure to shutdown which messes all logging	Asuswrt-Merlin AddOns	15	Oct 21, 2024
J	AdGuardHome There are tons of dnsproxy error messages occuring on syslog.	Asuswrt-Merlin AddOns	2	Jul 26, 2024
T	Updated scMerlin to 2.5.5 - syslog stops at 'syslogd exiting'	Asuswrt-Merlin AddOns	9	Jun 25, 2024

Scribe scribe - syslog-ng and logrotate installer

Part of the Furniture

Part of the Furniture

Part of the Furniture

Part of the Furniture

Very Senior Member

Very Senior Member

Part of the Furniture

Very Senior Member

Very Senior Member

Very Senior Member

Part of the Furniture

Occasional Visitor

Senior Member

Part of the Furniture

Part of the Furniture

Part of the Furniture

Very Senior Member

Part of the Furniture

Very Senior Member

Part of the Furniture

Similar threads

Similar threads

Support SNBForums w/ Amazon

Sign Up For SNBForums Daily Digest