FlexQoS FlexQoS issues with 388.4 HND5.04 models

[dave14305]

However, that router running 388.4 beta 2 with FlexQoS shown absolutely no reboot issues.

Are the rules the same between the 2 devices? If so, we can start to assume it depends on the actual traffic/packets being processed that triggers the problem.

 

[Yiannis]

Are the rules the same between the 2 devices? If so, we can start to assume it depends on the actual traffic/packets being processed that triggers the problem.

Yes rules are the same, I simply install FlexQoS and set which traffic category gets priority over the other from the menu. No special FlexQoS rules. But I have a new observation to share. Yesterday, my family returned to home, after 4-5 weeks. At that home I have the RT-AX86U Pro running in 388.4 beta 2, the one which stayed up and running for 5 days. Well, it seems the router on version 388.4 beta 2 was stable because there was no actual users at home (excluding some IoT devices). As soon as the users arrived, the router started rebooting every few minutes.

I am sorry for misdirecting the discussion. You are correct saying the reboot issue seems to trigger by the traffic.

 

[TurnstileBlues]

Yes rules are the same, I simply install FlexQoS and set which traffic category gets priority over the other from the menu. No special FlexQoS rules. But I have a new observation to share. Yesterday, my family returned to home, after 4-5 weeks. At that home I have the RT-AX86U Pro running in 388.4 beta 2, the one which stayed up and running for 5 days. Well, it seems the router on version 388.4 beta 2 was stable because there was no actual users at home (excluding some IoT devices). As soon as the users arrived, the router started rebooting every few minutes.

I am sorry for misdirecting the discussion. You are correct saying the reboot issue seems to trigger by the traffic.

This appears to be the same for me, too.
Everything functions as expected, until I start putting pressure on the router.

 

[nzwayne]

My AXE16000 (family uses AXE11000) with 388.4 installed having random reboots. In the debug work to resolve...first culprit was spdMerlin (documented in another thread -> https://www.snbforums.com/threads/spdmerlin-causes-router-to-reboot.84944/page-5#post-859671) but after removing spdMerlin reboots now only every 2-3 days (still an ouch).
Then after removing FlexQoS (waaa...) and using "Traditional QoS" there were no further reboots. The syslog showing reboot when FlexQoS active...
AXE16000 with FlexQoS active, reboot around 00:57
Sep 8 00:54:30 GT-AXE16000-1 bsd: bsd: Sending act Frame to 00:d4:9e:21:3b:cb with transition target eth10 ssid c8:7f:54:48:df:44
Sep 8 00:54:30 GT-AXE16000-1 bsd: bsd: BSS Transit Response: ifname=eth8, event=156, token=47, status=7, mac=00:00:00:00:00:00
Sep 8 00:54:30 GT-AXE16000-1 bsd: bsd: BSS Transit Response: STA reject
Sep 8 00:54:30 GT-AXE16000-1 bsd: bsd: Skip STA:00:d4:9e:21:3b:cb reject BSSID
Sep 8 00:56:48 GT-AXE16000-1 acsd: acs_build_candidates(704): eth8: number of valid chanspec is 0
Sep 8 00:59:30 kernel: klogd: exiting
Sep 8 00:59:30 syslogd exiting

Read in this thread that heavy router traffic may invoke the crash/reboot, but this router is only lightly loaded with traffic with Laptop & iPhone. I can swap out the AXE11000 for a (spare) AXE16000 for some load activity, if necessary.

To help @dave14305 & others in what's occurred here with 388.4...my plan...any (good/reasonable) suggestions will be taken on board for action!
1. Enable FlexQoS once again to prove the AXE16000 reboots are FlexQoS related.
2. From append #95 in this thread enable this logging... "This iptables rule would log such packets in syslog. Ideally, people experiencing reboots may see the messages in their syslog prior to the reboot, if this hypothesis has any value. Otherwise, it's just going to make log noise."
iptables -t mangle -I POSTROUTING -m mark --mark 0x1/0x7 -j LOG --log-prefix "[FlexQoS PRE Debug] " --log-tcp-options --log-ip-options
iptables -t mangle -A POSTROUTING -m mark --mark 0x1/0x7 -j LOG --log-prefix "[FlexQoS POST Debug] " --log-tcp-options --log-ip-options
3. If still reboots then from FlexQoS GUI change fq_codel to ASUS to see what occurs.
4. If still reboots then enable "Traditional QoS" as should prevent reboots from my data of today.

BTW: By current setup is... and I'm heavy user of Cisco's Webex during the work day.

 

[dave14305]

4. If still reboots then enable "Traditional QoS" as should prevent reboots from my data of today.

I would suggest running plain Adaptive QoS instead of Traditional QoS. If you have to resort to Traditional QoS, you might as well run CAKE instead.

Any hints of what causes the reboots will be welcome.

 

[chris.at]

You should really give CAKE a try. I was a big fan of FlexQoS since the first day (different author, different name) but switched to CAKE with replacing my AC68U with an AX86U after some days with FlexQoS just because I was curious how it would work out. And I have to say it's great. I also work from home with family and kids, streaming and gaming - absolutely no hickups with CAKE. I use it with the following /jffs/config/cake-qos.conf.add file:

ULPRIOQUEUE='besteffort'
DLPRIOQUEUE='besteffort'
ULOPTIONS='dual-srchost nat wash no-ack-filter rtt 100ms'
DLOPTIONS='dual-dsthost nat wash ingress rtt 100ms'

Yes, most is default here but so I could easily edit it without thinking about the options in the first days (and then left it this way). I think UL-/DLOPTIONS could be removed completely because of defaults, but UL-/DLPRIOQUEUE has other defaults.

Don't get me wrong, I appreciate the work of dave14305 (and his predecessors), but if/when there is no solution for the reboots and other problems, try CAKE instead. It also does a great job without the need (or the possibility, depends on your point of view) of granular customizing for single streams.

 

[MXThaurus]

You should really give CAKE a try.

I'm a streamer and gamer and after many tries, if target is to stabilize clients connection or gain better ping/bufferbloat on a specific device, cake will not do the trick for all. Still better stay with Adaptive QoS but FlexQoS on 388.4_4 (AX86U Pro) is stable for me and i'm very happy with it. I've only added on IPTables the right class for each IP was on a wrong category. I tried to used cake but in a long streaming/gaming sessions i was seeing ping spykes that with a correct configuration of FlexQoS has been completely removed. If i had any problem about reeboting, i would have been sticked to Adaptive QoS.

 

[Kingp1n]

You should really give CAKE a try. I was a big fan of FlexQoS since the first day (different author, different name) but switched to CAKE with replacing my AC68U with an AX86U after some days with FlexQoS just because I was curious how it would work out. And I have to say it's great. I also work from home with family and kids, streaming and gaming - absolutely no hickups with CAKE. I use it with the following /jffs/config/cake-qos.conf.add file:

ULPRIOQUEUE='besteffort'
DLPRIOQUEUE='besteffort'
ULOPTIONS='dual-srchost nat wash no-ack-filter rtt 100ms'
DLOPTIONS='dual-dsthost nat wash ingress rtt 100ms'

Yes, most is default here but so I could easily edit it without thinking about the options in the first days (and then left it this way). I think UL-/DLOPTIONS could be removed completely because of defaults, but UL-/DLPRIOQUEUE has other defaults.

Don't get me wrong, I appreciate the work of dave14305 (and his predecessors), but if/when there is no solution for the reboots and other problems, try CAKE instead. It also does a great job without the need (or the possibility, depends on your point of view) of granular customizing for single streams.

The only problem I see with using CAKEQOS is for those individuals that have speeds over 250+ Mbps since CAKE limits these speeds. However for me it seems I have better results with FlexQoS enabled.

 

[Kingp1n]

I'm a streamer and gamer and after many tries, if target is to stabilize clients connection or gain better ping/bufferbloat on a specific device, cake will not do the trick for all. Still better stay with Adaptive QoS but FlexQoS on 388.4_4 (AX86U Pro) is stable for me and i'm very happy with it. I've only added on IPTables the right class for each IP was on a wrong category. I tried to used cake but in a long streaming/gaming sessions i was seeing ping spykes that with a correct configuration of FlexQoS has been completely removed. If i had any problem about reeboting, i would have been sticked to Adaptive QoS.

I agree with you on this. I had similar issues when gaming while using CAKE. You mind sharing your FlexQoS setup (sharing screenshots) if you don't mind of course.

Hopefully the reboot issues can be fixed for the newer models but it all may be closed source.

I also wonder if folks with the reboot issues are using scripts that are no longer maintained with the newer firmwares i.e. x3mrouting, spdMerlin possibly scribe etc...a lot has changed from 3.0.04 to 3.0.0.6 as well.

I believe AMTM will need to be updated and input an asterisk for scripts that may cause issues for newer models (HND5.04) and/or using 3.0.0.6 fw.

 

[MXThaurus]

I agree with you on this. I had similar issues when gaming while using CAKE. You mind sharing your FlexQoS setup (sharing screenshots) if you don't mind of course.

Happy to share. As i am now on FTTH i removed FlexQoS just to test connection without any tweak, anyway i can share previous screenshots when i was on a FWA 30/3 where FlexQoS let me game with many other devices on LAN without any pyng spykes, it was a game changer for me. My Bandwidth table it's pretty odd, but it worked smoothly with my previous WAN/LAN configuration as i tracked with traffic classification any rate in realtime and set Bandwidth table consequentially to achieve better performances possible on gaming device.

 

[nzwayne]

Curious if anyone ever sees any packets that meet this requirement above. This iptables rule would log such packets in syslog. Ideally, people experiencing reboots may see the messages in their syslog prior to the reboot, if this hypothesis has any value. Otherwise, it's just going to make log noise.

iptables -t mangle -I POSTROUTING -m mark --mark 0x1/0x7 -j LOG --log-prefix "[FlexQoS PRE Debug] " --log-tcp-options --log-ip-options
iptables -t mangle -A POSTROUTING -m mark --mark 0x1/0x7 -j LOG --log-prefix "[FlexQoS POST Debug] " --log-tcp-options --log-ip-options

If it doesn't seem to precede any reboots, you can remove the logging rule by running:

iptables -t mangle -D POSTROUTING -m mark --mark 0x1/0x7 -j LOG --log-prefix "[FlexQoS PRE Debug] " --log-tcp-options --log-ip-options
iptables -t mangle -D POSTROUTING -m mark --mark 0x1/0x7 -j LOG --log-prefix "[FlexQoS POST Debug] " --log-tcp-options --log-ip-options

Had a random reboot at 07:25am on the AXE16000 w/388.4 & FlexQoS active with fq-codel. Had the logging active as you documented here. What was in the syslog for that period....seems very lacking (to me) on any data for the event.
Sep 13 07:22:40 GT-AXE16000-1 acsd: acs_build_candidates(704): eth8: number of valid chanspec is 0
Sep 13 07:23:03 GT-AXE16000-1 bsd: bsd: Sending act Frame to 0e:1b:bf:fe:1c:a3 with transition target eth10 ssid c8:7f:54:48:df:44
Sep 13 07:23:03 GT-AXE16000-1 bsd: bsd: BSS Transit Response: ifname=eth8, event=156, token=75, status=6, mac=34:10:c8:7f:54:48
Sep 13 07:23:03 GT-AXE16000-1 bsd: bsd: BSS Transit Response: STA reject
Sep 13 07:23:03 GT-AXE16000-1 bsd: bsd: Skip STA:0e:1b:bf:fe:1c:a3 reject BSSID
Sep 13 07:26:15 kernel: klogd: exiting
Sep 13 07:26:15 syslogd exiting
Sep 13 07:26:16 GT-AXE16000-1 vnstatd[4304]: vnStat daemon 2.10 started. (pid:4304 uid:0 gid:0 64-bit)
Sep 13 07:26:16 GT-AXE16000-1 vnstatd[4304]: Monitoring (1): eth0 (1000 Mbit)
Sep 13 07:26:16 GT-AXE16000-1 CanaKW: Started vnstatd from /jffs/scripts/post-mount.
Sep 13 07:26:16 GT-AXE16000-1 S61unbound: start Unbound DNS server /opt/etc/init.d/S61unbound
Sep 13 07:26:16 GT-AXE16000-1 rc_service: service 4330:notify_rc restart_dnsmasq
etc...

 

[dave14305]

What was in the syslog for that period....seems very lacking (to me) on any data for the event.

The log doesn’t look like any reboot happened, unless it’s sitting in another scribe log somewhere.

 

[nzwayne]

The log doesn’t look like any reboot happened, unless it’s sitting in another scribe log somewhere.

I've attached the two logs of data for 7:25am event. I manually rebooted at 11:38am to log data for a known reboot event. BTW: Using AMTM "Router Date Keeper".

 

[dave14305]

I've attached the two logs of data for 7:25am event. I manually rebooted at 11:38am to log data for a known reboot event. BTW: Using AMTM "Router Date Keeper".

I think you are losing the post-reboot boot log because Scribe is probably wiping it out when syslog-ng starts and the existing syslog.log file gets linked to the messages file. Not much use for the Scribe logs in this case.

 

[SomeWhereOverTheRainBow]

I think you are losing the post-reboot boot log because Scribe is probably wiping it out when syslog-ng starts and the existing syslog.log file gets linked to the messages file. Not much use for the Scribe logs in this case.

I hope you are able to find the smoking gun at somepoint.

 

[dave14305]

I hope you are able to find the smoking gun at somepoint.

Can’t even find the smoke, let alone the gun.

 

[Yiannis]

hi @dave14305 would you think a custom FlexQoS build with more logging enabled could be built especially for those people who face the issue?
it could maybe help in tracking down the problem.

 

[SomeWhereOverTheRainBow]

hi @dave14305 would you think a custom FlexQoS build with more logging enabled could be built especially for those people who face the issue?
it could maybe help in tracking down the problem.

It is not a bad idea to add logging, however one issue Dave has already pointed out is for users who are running things like scribe, it is missing a good chunk of the early boot up logs. When people post logs, it is more helpful to have the full unaltered/unfiltered logs.

 

[Yiannis]

Sure, I am happy to just install FlexQoS and nothing else in order to narrow down the issues

 

[dave14305]

Sure, I am happy to just install FlexQoS and nothing else in order to narrow down the issues

I’m waiting for someone to be able to post a complete syslog covering the reboot time and subsequent boot sequence. Surprisingly, this has been hard to come by.

Any extra iptables logging would be more likely to overload the router than to provide any clues.