Understanding the Log

[Nas.CloudBusinessPortal]

Iguessing, but my kernael does not seem to be noormal.


Sep 14 03:58:39 kernel: eth0 (Int switch port: 6) (Logical Port: 6) (phyId: 13) Link DOWN.
Sep 14 03:58:42 kernel: ^[[0;33;41mFCACHE ERROR: fc_config_mcast_group: Mcast: client has already JOINed the mcast group (duplicate JOIN)^[[0m
Sep 14 03:58:42 kernel: Mcast: new client info: group_addr<239.255.255.251> src_addr<0.0.0.0> is_ssm<0> tx_dev_p<ffffffc03458b000> wlinfo<0000> client_type<0> MAC<9c:8e:cd:00:14:85> rtp_seq_chk<0>
Sep 14 03:58:42 kernel: rtp_seq_chk<0> mcast_excl_udp_port<0>
Sep 14 03:58:42 kernel: Mcast: new client: matches with: mcast_group<0> client_id<1>
Sep 14 03:58:42 kernel: ^[[0;33;41m[ERROR mcast] bcm_mcast_clientinfo_node_add,843: blog_activate failure^[[0m
Sep 14 03:58:42 kernel: ^[[0;33;41m[ERROR mcast] bcm_mcast_fc_flow_add,924: Error adding client info node^[[0m
Sep 14 03:58:42 kernel: is_ssm 0 grp 239.255.255.251 src 0.0.0.0
Sep 14 03:58:42 kernel: mcast_exclude_udp_port 0x0 rtp seq check 0
Sep 14 03:58:42 kernel: ***Client List***
Sep 14 03:58:42 kernel: to_accel_dev eth5 wl info 0x0 clientmac 9c:8e:cd:1d:e0:ba
Sep 14 03:58:42 kernel: ***Rx Info list***
Sep 14 03:58:42 kernel: rxinfo blog_p 0xffffffc0331d5540
Sep 14 03:58:42 kernel: ^[[0;33;41m[ERROR mcast] bcm_mcast_blogrule_vlan_process,319: Unable to create flow, continue^[[0m
Sep 14 03:58:45 kernel: eth0 (Int switch port: 6) (Logical Port: 6) (phyId: 13) Link Up at 2500 mbps full duplex
Sep 14 03:58:47 WAN(0)_Connection: WAN(0) link up.
Sep 14 03:58:47 rc_service: wanduck 1475:notify_rc restart_wan_if 0
Sep 14 03:58:47 custom_script: Running /jffs/scripts/service-event (args: restart wan_if)
Sep 14 03:58:47 lldpd[2322]: removal request for address of 174.119.140.206%22, but no knowledge of it
Sep 14 03:58:47 dnsmasq[32056]: read /etc/hosts - 24 names
Sep 14 03:58:47 dnsmasq[32056]: read /jffs/addons/YazDHCP.d/.hostnames - 338 names
Sep 14 03:58:47 kernel: potentially unexpected fatal signal 6.
Sep 14 03:58:47 kernel: CPU: 2 PID: 32056 Comm: dnsmasq Tainted: P O 4.19.183 #1
Sep 14 03:58:47 kernel: Hardware name: RTAX88U_PRO (DT)
Sep 14 03:58:47 kernel: pstate: 00070010 (nzcv q A32 LE aif)

 

[RMerlin]

YazDHCP is causing dnsmasq to crash.

 

[Nas.CloudBusinessPortal]

thanks for the quick reply...

i try later to look into this.... a have some health issues first.

 

[Martinski]

YazDHCP is causing dnsmasq to crash.

I have yet to see clear evidence that conclusively shows causality between YazDHCP & the dnsmasq crashes. So far, all I’ve seen in the logs provided on various posts is temporal correlations.

If you have more data that shows YazDHCP as the root cause, would you please provide it or point me to where it can be found?

Without additional data points, concluding that event "A" causes event "B" simply because the logs show "B" happening immediately after "A" is a logical error (technically, it's a "post hoc ergo propter hoc" logical fallacy). IOW, the chronological order of the events is necessary but not sufficient evidence to establish causality.

Now, to be clear, I’m not saying that it's impossible for YazDHCP to be the culprit, only that it’s not conclusive based on the data presented up to this point. So if anyone has more data points showing YazDHCP as the cause of the crashes, I can certainly take a look & perhaps figure out where the problem is in the script.

BTW, here's a couple of posts from @JGrana who was experiencing similar crashes with dnsmasq but found a workaround. This may or may not be relevant to the particular case on this thread, but it's worth exploring, IMO.

Release - Asuswrt-Merlin 388.2 is now available for select models

I don't see that one on the supported list for Merlin FW. See here: https://www.asuswrt-merlin.net/about So it only refers to current routers with wifi 6 that support AsusWrt and does not refer to all Asus routers with wifi 6. My mistake of interpretation. All the best

www.snbforums.com

Release - Asuswrt-Merlin 388.2 is now available for select models

And other addons you have. Dnsmasq logging is on and working no problem here but that's all I have. So it seems there are other things causing your dnsmasq crashes. There's at least three of us that have this problem. Maybe it's a specific hardware model? I don't know... Regarding your...

www.snbforums.com

Just my 2 cents.

 

[SomeWhereOverTheRainBow]

thanks for the quick reply...

i try later to look into this.... a have some health issues first.

Do you have more logs to share from syslog? Perhaps more before and after logs.

 

[RMerlin]

I have yet to see clear evidence that conclusively shows causality between YazDHCP & the dnsmasq crashes.

dnsmasq crashed within seconds of YazDHCP loading. Possibly something it added to the host file (maybe just the insane number of entries itself is a problem - I know this was an issue with uclibc limitations on older router models where there were limitations to what could be added to the hosts file). That would definitely be the first thing to test since it's something that's not part of the firmware.

 

[Martinski]

dnsmasq crashed within seconds of YazDHCP loading. Possibly something it added to the host file (maybe just the insane number of entries itself is a problem - I know this was an issue with uclibc limitations on older router models where there were limitations to what could be added to the hosts file). That would definitely be the first thing to test since it's something that's not part of the firmware.

YazDHCP is not an add-on that runs constantly in the background. It runs only when the user calls it via CLI or loads the WebGUI. After 3 files are created (.hostnames, .optionslist & .staticlist), YazDHCP adds the following entries to the "/jffs/configs/dnsmasq.conf.add" file, restarts dnsmasq and then exits, so it's no longer running or loading anything by the time dnsmasq crashes.

addn-hosts=/jffs/addons/YazDHCP.d/.hostnames # YazDHCP_hostnames
dhcp-hostsfile=/jffs/addons/YazDHCP.d/.staticlist # YazDHCP_staticlist
dhcp-optsfile=/jffs/addons/YazDHCP.d/.optionslist # YazDHCP_optionslist

Now, it's possible that something in the "/jffs/addons/YazDHCP.d/.hostnames" file might be triggering a bug in the current dnsmasq version which leads to the crash (e.g. Special chars being used in some Hostnames? Number of entries exceeding a maximum limit?). However, the debug notes posted by @JGrana pointed out that the crashes that he was experiencing still occurred even after disabling YazDHCP & commenting out the call to the ".hostnames" file. At least in his case, it became clear that the crash was not caused by YazDHCP or the ".hostnames" file created by YazDHCP. This also established that the 2 events shown in the logs were sequentially related but not causally related.

IMO, the debugging steps that @JGrana took to troubleshoot the crashes he was seeing in his router is what other users experiencing similar dnsmasq crashes should do.

 

[SomeWhereOverTheRainBow]

YazDHCP is not an add-on that runs constantly in the background. It runs only when the user calls it via CLI or loads the WebGUI. After 3 files are created (.hostnames, .optionslist & .staticlist), YazDHCP adds the following entries to the "/jffs/configs/dnsmasq.conf.add" file, restarts dnsmasq and then exits, so it's no longer running or loading anything by the time dnsmasq crashes.

addn-hosts=/jffs/addons/YazDHCP.d/.hostnames # YazDHCP_hostnames
dhcp-hostsfile=/jffs/addons/YazDHCP.d/.staticlist # YazDHCP_staticlist
dhcp-optsfile=/jffs/addons/YazDHCP.d/.optionslist # YazDHCP_optionslist

Now, it's possible that something in the "/jffs/addons/YazDHCP.d/.hostnames" file might be triggering a bug in the current dnsmasq version which leads to the crash (e.g. Special chars being used in some Hostnames? Number of entries exceeding a maximum limit?). However, the debug notes posted by @JGrana pointed out that the crashes that he was experiencing still occurred even after disabling YazDHCP & commenting out the call to the ".hostnames" file. At least in his case, it became clear that the crash was not caused by YazDHCP or the ".hostnames" file created by YazDHCP. This also established that the 2 events shown in the logs were sequentially related but not causally related.

IMO, the debugging steps that @JGrana took to troubleshoot the crashes he was seeing in his router is what other users experiencing similar dnsmasq crashes should do.

so essentially, from the logs, all we actually know is that dnsmasq crashed? We don't officially know the cause. We have a general guess at the cause. And we don't know much else about the users setup except that this happen on the RT-AX88U Pro (indicated in the log).

I guess we are to assume the user is running the latest Merlin firmware. It is important to note, that if the user's firmware version is older than 388.2_2 then they could be experiencing the DNSMASQ crash issue.

The oldest firmware version available for the RT-AX88U-Pro is

which lacked the patch for dnsmasq.

(This is all hypothetically assuming the user is running 388.2 firmware with no "reliable" DNS upstream configured on their WAN page?)

 

[Nas.CloudBusinessPortal]

I have yet to see clear evidence that conclusively shows causality between YazDHCP & the dnsmasq crashes. So far, all I’ve seen in the logs provided on various posts is temporal correlations.

If you have more data that shows YazDHCP as the root cause, would you please provide it or point me to where it can be found?

Without additional data points, concluding that event "A" causes event "B" simply because the logs show "B" happening immediately after "A" is a logical error (technically, it's a "post hoc ergo propter hoc" logical fallacy). IOW, the chronological order of the events is necessary but not sufficient evidence to establish causality.

Now, to be clear, I’m not saying that it's impossible for YazDHCP to be the culprit, only that it’s not conclusive based on the data presented up to this point. So if anyone has more data points showing YazDHCP as the cause of the crashes, I can certainly take a look & perhaps figure out where the problem is in the script.

BTW, here's a couple of posts from @JGrana who was experiencing similar crashes with dnsmasq but found a workaround. This may or may not be relevant to the particular case on this thread, but it's worth exploring, IMO.

Release - Asuswrt-Merlin 388.2 is now available for select models

I don't see that one on the supported list for Merlin FW. See here: https://www.asuswrt-merlin.net/about So it only refers to current routers with wifi 6 that support AsusWrt and does not refer to all Asus routers with wifi 6. My mistake of interpretation. All the best

www.snbforums.com

Release - Asuswrt-Merlin 388.2 is now available for select models

And other addons you have. Dnsmasq logging is on and working no problem here but that's all I have. So it seems there are other things causing your dnsmasq crashes. There's at least three of us that have this problem. Maybe it's a specific hardware model? I don't know... Regarding your...

www.snbforums.com

Just my 2 cents.

Tell me what more i can send you. Ill be happy to assist.

 

[Nas.CloudBusinessPortal]

so essentially, from the logs, all we actually know is that dnsmasq crashed? We don't officially know the cause. We have a general guess at the cause. And we don't know much else about the users setup except that this happen on the RT-AX88U Pro (indicated in the log).

I guess we are to assume the user is running the latest Merlin firmware. It is important to note, that if the user's firmware version is older than 388.2_2 then they could be experiencing the DNSMASQ crash issue.

View attachment 53063

The oldest firmware version available for the RT-AX88U-Pro is

View attachment 53064

which lacked the patch for dnsmasq.

(This is all hypothetically assuming the user is running 388.2 firmware with no "reliable" DNS upstream configured on their WAN page?)

Firmware version 388.4
My dns servers are 1.1.1.1 and 1.0.0.1 pr spcifically 8.8.8.8 on manual seting of some clients.

 

[SomeWhereOverTheRainBow]

Firmware version 388.4
My dns servers are 1.1.1.1 and 1.0.0.1 pr spcifically 8.8.8.8 on manual seting of some clients.

What are the dns servers listed on the wan settings page of your router? Also, my post was just to illustrate other possibilities. It sounds like there is another issue causing the problem. It also shows that you cannot assume from logs alone, that any one possibility is the actual problem.

For example,

Some users in the links provided by @Martinski suggest checking the content of

addn-hosts=/jffs/addons/YazDHCP.d/.hostnames # YazDHCP_hostnames
dhcp-hostsfile=/jffs/addons/YazDHCP.d/.staticlist # YazDHCP_staticlist
dhcp-optsfile=/jffs/addons/YazDHCP.d/.optionslist # YazDHCP_optionslist

For errors in Mac or IP address formating, or invalid characters in hostnames.

Also, as @Martinski has pointed out, this is assuming YazDHCP is the culprit. The only way to test this is to comment out those lines for hostnames, staticlist, and optionslist & restart dnsmasq.

It is also possible that the calculations for how many static client reservations you can have on your device is wrong.

/jffs/addons/YazDHCP.d/.hostnames - 338 names

 

[JGrana]

As @Martinski mentioned, I had dnsmasq fatal unexpected when I moved from an AX88U to an AX88U Pro (the old AX88U had a bad radio..). YazDHCP always installed.

After lots of debugging, I found that an earlier release of dnsmasq, 2.85 worked fine. The newer release of dnsmasq was 2.89.

Ever since I reverted to v 2.85, no dnsmaq crashes since. This is through a few beta cycles and firmware update. For now, when I see that dnsmasq is updated in the firmware, I will test again - but keep 2.85 ready ;-)

I was fortunate that my older AX88U still booted fine and I could copy out the 2.85 version of dnsmasq from it.

BTW, I have 80 hostnames. Not excessive and well within what I would think dnsmasq supports for static assignments.

 

[SomeWhereOverTheRainBow]

As @Martinski mentioned, I had dnsmasq fatal unexpected when I moved from an AX88U to an AX88U Pro (the old AX88U had a bad radio..). YazDHCP always installed.

After lots of debugging, I found that an earlier release of dnsmasq, 2.85 worked fine. The newer release of dnsmasq was 2.89.

Ever since I reverted to v 2.85, no dnsmaq crashes since. This is through a few beta cycles and firmware update. For now, when I see that dnsmasq is updated in the firmware, I will test again - but keep 2.85 ready ;-)

I was fortunate that my older AX88U still booted fine and I could copy out the 2.85 version of dnsmasq from it.

BTW, I have 80 hostnames. Not excessive and well within what I would think dnsmasq supports for static assignments.

on RT-AX88U PRO, DHCP with YazDHCP claims to only support 250 clients, what does yours say?

I just recall in @Nas.CloudBusinessPortal logs, there was this message /jffs/addons/YazDHCP.d/.hostnames - 338 names. I don't know if this number correlates to the DHCP scope limit. If it does, then clearly @Nas.CloudBusinessPortal might be in breach of the maximum number of supported clients considering he also is running a RT-AX88U Pro. Either that, or the files could be corrupt.

BTW, I am running the YazDHCP development branch v1.0.6. It is quite possible that some of @Martinski fixes may have patched something broken in v1.0.5 or older. What I would do is manually copy over everything to a notepad or take screenshots, then switch to the YazDHCP development branch.

 

[Nas.CloudBusinessPortal]

What are the dns servers listed on the wan settings page of your router? Also, my post was just to illustrate other possibilities. It sounds like there is another issue causing the problem. It also shows that you cannot assume from logs alone, that any one possibility is the actual problem.

For example,

Some users in the links provided by @Martinski suggest checking the content of

addn-hosts=/jffs/addons/YazDHCP.d/.hostnames # YazDHCP_hostnames
dhcp-hostsfile=/jffs/addons/YazDHCP.d/.staticlist # YazDHCP_staticlist
dhcp-optsfile=/jffs/addons/YazDHCP.d/.optionslist # YazDHCP_optionslist

For errors in Mac or IP address formating, or invalid characters in hostnames.

Also, as @Martinski has pointed out, this is assuming YazDHCP is the culprit. The only way to test this is to comment out those lines for hostnames, staticlist, and optionslist & restart dnsmasq.

It is also possible that the calculations for how many static client reservations you can have on your device is wrong.

/jffs/addons/YazDHCP.d/.hostnames - 338 names

I don’t have 338 static IP addresses. I might have 145 to 150.

 

[dave14305]

Sep 14 03:58:47 dnsmasq[32056]: read /jffs/addons/YazDHCP.d/.hostnames - 338 names

I don’t have 338 static IP addresses. I might have 145 to 150.

One of you is lying. Check the file for corruption/duplication.

cat /jffs/addons/YazDHCP.d/.hostnames
cat /jffs/addons/YazDHCP.d/.staticlist

I think the problem is more likely to be in the staticlist file since that would be expected next in the dnsmasq messages.

 

[Nas.CloudBusinessPortal]

One of you is lying. Check the file for corruption/duplication.

cat /jffs/addons/YazDHCP.d/.hostnames
cat /jffs/addons/YazDHCP.d/.staticlist

I think the problem is more likely to be in the staticlist file since that would be expected next in the dnsmasq messages.

Is there any way to put the script in a debug mode to make additional log entries? I don’t mind if it’s over explicit.

 

[dave14305]

Is there any way to put the script in a debug mode to make additional log entries? I don’t mind if it’s over explicit.

Ultimately, you need to look at your own config being manipulated by the script. Reviewing the resulting files mentioned above is a good first step.

No one wants to share their personal data with the internet, but eventually someone has to do it for the greater good.

 

[Nas.CloudBusinessPortal]

Ultimately, you need to look at your own config being manipulated by the script. Reviewing the resulting files mentioned above is a good first step.

No one wants to share their personal data with the internet, but eventually someone has to do it for the greater good.

Well, right now my health went for a term to the worst, I am unable to physically spend lengthy periods of time in front of the computer. The doctor doesn’t want me spending any time in front of it.

So it looks like it’s gonna be one of those things that just sits there until Something happens

 

[dave14305]

So it looks like it’s gonna be one of those things that just sits there until Something happens

No worries, take care of yourself. We’ll badger @JGrana into providing some data that’s known to crash dnsmasq 2.89.

 

[JGrana]

Badger!?! badger?!? - for you @dave14305 with some Rochester NY roots, it would be my pleasure.

Fair warning. I am heading OOT this Tuesday - for 3 weeks. I will try some addition debugging early tomorrow.
When the network can be reboot a few times ;-)