Scribe A filter to drop kernel dcd tainted errors :

[bibikalka]

My AC86U router produces endless kernel dcd errors due to AiProtection features which pretty much pollute the system log and make it impossible to look at in a fairly efficient manner. See such a dcd error message below - it's huge for each error instance. So I put together a new quick & dirty Scribe filter in /opt/etc/syslog-ng.d/ called bcm_err. It tries to grab bits & pieces of the dcd error output, and trash it. Since Asus is not going to fix the issue - there is no point in looking at these messages. I also added branches to trash a few other types of annoying kernel messages.

Ideally, it'd be nice to have a log bucket in Scribe to send any filtered stuff there that does not fit any pre-existing categories that have their own log files, and I did not want to pollute the existing ones.

bcm_err filter

# don't copy error messages from kernel

filter f_bcm_err {
    ( program("kernel") and
    message("0000") ) or
    ( program("kernel") and
    message("^x") ) or
    ( program("kernel") and
    message("is at") ) or
    ( program("kernel") and
    message("pgd") ) or
     ( program("kernel") and
    message("Hardware") ) or
   ( program("kernel") and
    message("potentially") ) or
  ( program("kernel") and
    message("Tainted") ) or
    ( program("kernel") and
    message("^dig") ) or
    ( program("kernel") and
    message("send_redir_page") ) or
    ( program("kernel") and
    message("task") );
};

log {
    source(src);
    filter(f_bcm_err);
    flags(final);
};

#eof

Endless dcd error output

Sep 22 00:44:35 RT-AC86U-9988 kernel: dcd[4644]: unhandled level 3 translation fault (11) at 0x00000000, esr 0x92000007
Sep 22 00:44:35 RT-AC86U-9988 kernel: pgd = ffffffc0014d7000
Sep 22 00:44:35 RT-AC86U-9988 kernel: [00000000] *pgd=0000000001646003, *pud=0000000001646003, *pmd=0000000006683003, *pte=0000000000000000
Sep 22 00:44:35 RT-AC86U-9988 kernel: CPU: 1 PID: 4644 Comm: dcd Tainted: P           O    4.1.27 #2
Sep 22 00:44:35 RT-AC86U-9988 kernel: Hardware name: Broadcom-v8A (DT)
Sep 22 00:44:35 RT-AC86U-9988 kernel: task: ffffffc012047580 ti: ffffffc001f90000 task.ti: ffffffc001f90000
Sep 22 00:44:35 RT-AC86U-9988 kernel: PC is at 0xf6ff0f44
Sep 22 00:44:35 RT-AC86U-9988 kernel: LR is at 0x1dce0
Sep 22 00:44:35 RT-AC86U-9988 kernel: pc : [<00000000f6ff0f44>] lr : [<000000000001dce0>] pstate: 600b0010
Sep 22 00:44:35 RT-AC86U-9988 kernel: sp : 00000000ffcbd858
Sep 22 00:44:35 RT-AC86U-9988 kernel: x12: 00000000000a2050
Sep 22 00:44:35 RT-AC86U-9988 kernel: x11: 00000000f62ff024 x10: 00000000000a23c4
Sep 22 00:44:35 RT-AC86U-9988 kernel: x9 : 00000000f62ff734 x8 : 00000000000a287c
Sep 22 00:44:35 RT-AC86U-9988 kernel: x7 : 00000000f62ff76c x6 : 00000000000a2876
Sep 22 00:44:35 RT-AC86U-9988 kernel: x5 : 0000000000000000 x4 : 00000000f62ff718
Sep 22 00:44:35 RT-AC86U-9988 kernel: x3 : 0000000000000000 x2 : 00000000ffcbd834
Sep 22 00:44:35 RT-AC86U-9988 kernel: x1 : 000000000007d72e x0 : 0000000000000000
Sep 22 00:44:35 RT-AC86U-9988 kernel: potentially unexpected fatal signal 11.
Sep 22 00:44:35 RT-AC86U-9988 kernel: CPU: 1 PID: 4644 Comm: dcd Tainted: P           O    4.1.27 #2
Sep 22 00:44:35 RT-AC86U-9988 kernel: Hardware name: Broadcom-v8A (DT)
Sep 22 00:44:35 RT-AC86U-9988 kernel: task: ffffffc012047580 ti: ffffffc001f90000 task.ti: ffffffc001f90000
Sep 22 00:44:35 RT-AC86U-9988 kernel: PC is at 0xf6ff0f44
Sep 22 00:44:35 RT-AC86U-9988 kernel: LR is at 0x1dce0
Sep 22 00:44:35 RT-AC86U-9988 kernel: pc : [<00000000f6ff0f44>] lr : [<000000000001dce0>] pstate: 600b0010
Sep 22 00:44:35 RT-AC86U-9988 kernel: sp : 00000000ffcbd858
Sep 22 00:44:35 RT-AC86U-9988 kernel: x12: 00000000000a2050
Sep 22 00:44:35 RT-AC86U-9988 kernel: x11: 00000000f62ff024 x10: 00000000000a23c4
Sep 22 00:44:35 RT-AC86U-9988 kernel: x9 : 00000000f62ff734 x8 : 00000000000a287c
Sep 22 00:44:35 RT-AC86U-9988 kernel: x7 : 00000000f62ff76c x6 : 00000000000a2876
Sep 22 00:44:35 RT-AC86U-9988 kernel: x5 : 0000000000000000 x4 : 00000000f62ff718
Sep 22 00:44:35 RT-AC86U-9988 kernel: x3 : 0000000000000000 x2 : 00000000ffcbd834
Sep 22 00:44:35 RT-AC86U-9988 kernel: x1 : 000000000007d72e x0 : 0000000000000000

 

[elorimer]

If you want to save these messages in a bucket, why not define a new file destination and send them there? Once you see if you are capturing what you meant to, you can comment the destination out to just trash them. If there really are a lot make sure you also have a log-rotate definition.

You might also rewrite your filter into two filters, one for the program and the other the "or" part for messages. As you have it your are testing each log message 10 times for the kernel part.

And, you might name it z_bcm_err so you only run this filter after everything else has been extracted already.

 

[bibikalka]

If you want to save these messages in a bucket, why not define a new file destination and send them there? Once you see if you are capturing what you meant to, you can comment the destination out to just trash them. If there really are a lot make sure you also have a log-rotate definition.

You might also rewrite your filter into two filters, one for the program and the other the "or" part for messages. As you have it your are testing each log message 10 times for the kernel part.

And, you might name it z_bcm_err so you only run this filter after everything else has been extracted already.

Excellent suggestions! I am new to this, and did not think efficiency at all.

It would still probably be nice to have scribe support an official trash log bucket by default - anything anybody wants to filter out would go there with little effort on the user part (just the filter file).

 

[elorimer]

@cmkelley has provided a number of samples but I don't see an easy way to have a trash log bucket. Syslog-ng isn't set up like that. But it isn't hard to do that.

Define a file destination as "trash.log". Then, in each logging statement where you are trashing messages that meet a filter, add that as a normal file destination. In each logging statement where you are sending messages to a separate file, but you want to perhaps see them in context, add that destination as a second destination. In syslog-ng.conf, in the log statement that goes to "messages", add that too. Now you have trash.log as the original raw feed.

Or, you can go the other way, and define as alphabetically first a logging statement that has all your sources, no filter, trash.log as a destination, and no final flag. Included it or not by naming the file with a leading ".", or commenting out all the statements.

Even if you are collecting messages from a bunch of routers, efficiency isn't all that important for syslog-ng here. Routers just don't generate a lot of logging statements to matter for an app designed to process thousands of messages a second.

 

[bibikalka]

...
You might also rewrite your filter into two filters, one for the program and the other the "or" part for messages. As you have it your are testing each log message 10 times for the kernel part.
...

Tweaked the filter for just 1 kernel test given that 12.2 still has many of the kernel errors:

filter z_bcm_err {
    ( program("kernel") and
    ( message("^x[0-9]") or
     message("^PC") or
     message("^LR") or
     message("^pc") or
     message("^sp") or
     message("^task") or
     message("^Hardware")) );
};

log {
    source(src);
    filter(z_bcm_err);
    flags(final);
};

#eof