Traffic spike fix from 384.14_2 was reverted in 386.1

[Morac]

I reported this in the 386.1 release thread. Sorry for the duplicate post, but I wanted to make sure it wasn’t lost in the fray.

I‘m seeing the “traffic spike” problem with the Traffic Monitor since upgrading from 384.19 to 386.1 on my RT-AC88U. This problem first appeared in 384.14_0 and was fixed in 384.14_2. I checked the source code and as far as I can tell that fix was lost in the upgrade to 386.1

Here’s where it was fixed In rstats.c:

rstats: revert some of the 384_81044 changes in an attempt to resolve… · RMerl/asuswrt-merlin.ng@6c4196a

… the traffic spikes

github.com

Here’s the current code:

asuswrt-merlin.ng/release/src/router/rstats/rstats.c at 71a0ae8f47dae468ec8ff20a01de4ddf46e3bcc5 · RMerl/asuswrt-merlin.ng

Third party firmware for Asus routers (newer codebase) - RMerl/asuswrt-merlin.ng

github.com

Note that all the changes have been reverted. Can this please be fixed again?

Thanks.

 

[RMerlin]

Asus will have to fix it upstream. I can no longer keep using my older code, there are too many recent changes that cannot be backported on top of it, so 386.1 had to switch to using their codebase for rstats.c.

 

[Morac]

I don’t want to belittle how much work you put into this, but the patch was only 6 lines of code changed in one file and all those lines are still there currently. It seems like it would be trivial to patch it.

Can you not just patch this one file?

If not can you at least let ASUS know about the fix since it seems like something they could include very easily (being only 6 lines).

 

[L&LD]

Or you can just let Asus know too. RMerlin already answered.

 

[dev_null]

FWIW I can confirm that adding vnstat via entware will track very consistently with the Traffic Monitor/Traffic Analyzer (and no Trend agreement required!) - I pursued this specifically because of the 17GB error on my AC66U_B1. (Interestingly the 17GB issue doesn't seem to be present on the AX86U).

I've been working with @JackYaz on a UI for the add-ons page for vnstat/vnstati... I just need to find enough time to fully document and re-validate my steps.

If you do add vnstat, be sure to change the location of the db files to a persistent location (check the conf file). At least on my first attempt the default configuration put the db files in a volatile location, so they were lost on reboot.

I try to get my steps and UI stuff posted this week.

 

[Morac]

I have no way of giving code patches to ASUS. Even if I did, they wouldn’t accept it.

My ISP charges me overages if I use more than a certain amount of data a month and the traffic monitor has been a way to keep track of that and keep the ISP honest.

Without that 6 line patch, the monitor is useless.

 

[Morac]

FWIW I can confirm that adding vnstat via entware will track very consistently with the Traffic Monitor/Traffic Analyzer (and no Trend agreement required!) - I pursued this specifically because of the 17GB error on my AC66U_B1. (Interestingly the 17GB issue doesn't seem to be present on the AX86U).

I've been working with @JackYaz on a UI for the add-ons page for vnstat/vnstati... I just need to find enough time to fully document and re-validate my steps.

If you do add vnstat, be sure to change the location of the db files to a persistent location (check the conf file). At least on my first attempt the default configuration put the db files in a volatile location, so they were lost on reboot.

I try to get my steps and UI stuff posted this week.

Thanks this sounds interesting, though I don’t have entware installed at all and would rather not install a lot of extra software on my router, so I’m hoping the existing Traffic Monitor can be fixed as for me it works perfectly, even without agreeing to the Trend agreement.

 

[L&LD]

But you do. Just link to the first post you made in this thread when you contact them.

Maybe, even @ASUSWRT_2020 can help out?

 

[Morac]

But you do. Just link to the first post you made in this thread when you contact them.

Maybe, even @ASUSWRT_2020 can help out?

Yes, I can point them to the code, but that code would have to get to the coders. A customer support agent isn’t going to pass along code. I’m not even sure they’d take this as a bug report.

I’m not sure where the original buggy code even comes from. The patch implies it was buggy code in the 384_81044 GPL. That came out over a year ago. That it’s still in the 386 GPL code either means it was never reported to ASUS or they didn’t bother fixing it (despite being a 6 line change).

 

[L&LD]

@Morac, sure, it starts by reporting it to CS. But if you're persistent enough, they will move you to higher and higher channels.

It seems like you have all the information needed to pass this along to them. It may take time, but this does seem fixable by a 'single person vs. Asus', to me.

 

[Morac]

@L&LD Last time I tried contacting ASUS they never even responded so I didn’t get a case number. I emailed them about this, but I don’t see the response being any different. There doesn’t seem to be any other way of contacting them without paying for support.

 

[RMerlin]

Can you not just patch this one file?

Downgrading 64-bit variables to 32-bit when Asus uses 64-bit elsewhere in the code will introduce other issues.

 

[Morac]

Downgrading 64-bit variables to 32-bit when Asus uses 64-bit elsewhere in the code will introduce other issues.

That’s unfortunate. I emailed ASUS, but am not expecting much.

Could the variables be left as 64-bit, but just truncate the upper 32-bit (or lower depending on the endian of the router CPU)? That or just throw out very large values. The spike values I’m seeing are impossible for commercial routers, let alone home routers. I’m talking over 17,000,000 Terrabytes in under a second.

Does rstats.c just compile into the rstats executable? I wonder if I could just take the old rstats executable and “replace” the built-in one with it since as mentioned, that one worked fine.

That or maybe create a tool to sanitize the save file. I wonder if ASUS tried that because when I checked just now the huge reception value from Feb 1 has been replaced with a normal value. Feb 2 now has the huge reception value. Jan 31 still has the huge transmit value.

 

[Jeffrey Young]

@dev_null , for nvStat, is the database file a straight forward SQL database file (looking to open with SQL lite)?

Watching this thread as rstat was the primary reason that I switched to Merlin. So far on 386.1 (AC86U), I have not noticed any weird yet on rstat behavior.

I use an excel VBS script right now to extract data from the rstat file for monthly records (help evaluate if my ISP package is worth keeping). However, if rstat starts to behave badly, I may look at vnStat as an alternative (OK, I might look at it anyway as a cool and fun thing to keep me occupied in this Covid 19 isolation).

Thanks

 

[dev_null]

@dev_null , for nvStat, is the database file a straight forward SQL database file (looking to open with SQL lite)?

Watching this thread as rstat was the primary reason that I switched to Merlin. So far on 386.1 (AC86U), I have not noticed any weird yet on rstat behavior.

I use an excel VBS script right now to extract data from the rstat file for monthly records (help evaluate if my ISP package is worth keeping). However, if rstat starts to behave badly, I may look at vnStat as an alternative (OK, I might look at it anyway as a cool and fun thing to keep me occupied in this Covid 19 isolation).

Thanks

According to this, yes, there is an export command for nvstat with which "you can dump the data in a text format delimited with semi-colon, which you can import to Excel or other db." I haven't tried it.

vnstat --dumpdb

 

[Wistuplu]

Dear Morac, dear forum,

Not that I expect a change, just to inform those that might experience this oddity.

FYI, I also experience funky figures with ghost spikes in the Traffic Monitor (the Real-time flavour) with 386.1. The scale of the graph is in tune with the ghost spikes (so, undesirably huge), which makes the real traffic nearly invisible. I get for instance the following figures for the wireless 5GHz. This is very unlike what 384.18 would typically report. (I can't say if 384.18 was completely immune from the problem, or was just affected rarely).

Current	Average	Maximum	Total
2.26 KB/s	21695.22 KB/s	2147483.45 KB/s	8,193.31 MB
3.40 KB/s	10858.12 KB/s	2147460.08 KB/s	4,100.63

AC86U dirty upgraded from 384.18 to 386.1. See table above. Also experienced after reset. See image below.

As the code sample above seems to indicate, it seems unrelated to consent with Trend Micro. On my router, the problem was present after the dirty upgrade with an "old" consent, and also after withdrawing the consent, and also after reaccepting. I mention this as Trend Micro was mentioned by @Morac in https://www.snbforums.com/threads/asuswrt-merlin-386-1-is-now-available.69783/page-18#post-658169 . It made me very excited to test with different Trend Micro acceptance status, but nope, no difference for me.

Take care
Best regards

PS: @Morac, My ISP does offer an interface where one can monitor the usage as "perceived"/registered by the ISP. Maybe yours does as well.

 

[Morac]

PS: @Morac, My ISP does offer an interface where one can monitor the usage as "perceived"/registered by the ISP. Maybe yours does as well.

My ISP also has a meter, but there have been issues in the past for some people where it is wrong. Having the measurement on my router itself (which is usually fairly close) allows me to double check them.

 

[Wistuplu]

FYI,
Issue reproduced on latest official Asus firmware, Firmware Version: 3.0.0.4.386_41634.
Reported to Asus in Belgium as E21020015895.

 

[dev_null]

Here's a peek at what the vnstat UI looks like. PM me for more info - I'm finishing up a bitbucket repository detailing how to install it on Merlin-based routers.

Spoiler

(Each section can be collapsed/hidden)

 

[Wistuplu]

The case E21020015895 is still ongoing with Asus. By the way, the problem also appears in Asus Beta 9.0.0.4.386_41994.