[Bug] CPU loading is 100% in core 1

[vanic]

Hi all guys,

I am Vanic, I need your help for feedback issue about "CPU usage on Core 1 to go up to nearly 100% while Core 2 remains under 2-3% usage. This causes massive slowdown in the performance of the router."
I know it happened in RT-AC68U / RT-AC87U, but I don't have enough information to trace what happened, if you encounter this issue, please contact with me.
1. conversation
2. mail to router_feedback@asus.com

If possible, please let me do remote debug, thanks all.

Lists : (updated in 2015/12/21)
1. Movistar : RT-AC87U 3.0.0.4_378_9383
2. Twawan : RT-AC68U 3.0.0.4_378_9313
3. SNB with Movistar Three VLANS setting : RT-AC87U 3.0.0.4_378_9383
4. Movistar user

We will simulate the triple vlan on our side to check what happened. (2015/12/30) ... proceeding

Thanks,
Vanic

 

[VANT]

What fw version ?
i use 9177, 9313, 9366 and latest and not saw this issue

 

[Piter]

I use FW 9383 and now also not saw this issue.

 

[cierzo]

Model: RT-AC87U
Firmware: 378.9383.

It happens to me as soon as I activate QoS (Adaptive)
It seems as if it breaks CTF and starts working without hardware acceleration.

Hope it helps.

 

[VANT]

i have enabled adaptive qos, but not see problem.
Can You tell us what you configure qos ?

 

[cierzo]

Those kind of things happen when I trust on my mind.... maybe this is not the bug you are looking for but I think it's a bug.

I'm testing right now:

- Qos Enabled. Adaptive Doesn't matter if downlad speed configured with 0, 300 or 330 Mb nor priority mode (preconfigured or custom);
When speedtesting my line I reach about 180 Mbps and 100% cpu1

- QoS disabled.
speed test reach 300 Mbps ... I would swear that cpu1 was about 60% but it's 100%

Anyway. This was not happening in previous releases

In Switch control I can see CTF enabled with both configurations (I've never seen CTF+FA)

 

[bradbort]

I just tested it. Adaptive Q0S on. Dowloading via RCN speedtest. 140 M download speed. Core one went to about 50%. Core 2 went to about 17%. When no downloading going on, both cores either 0 or low single digits.

 

[RMerlin]

While the CPU load is high, connect over SSH, and post the output of the following command:

top

(press Ctrl-C to exit the tool, as otherwise it keeps updating every second)

 

[vanic]

Hi @cierzo ,
Do you enable traffic analyzer? Does this issue always happen? If it still happened, could you help me to do remote debug? And you can follow Merlin's suggestion to find what process make CPU loading high.

Thanks,
Vanic

 

[cierzo]

Traffic analyzer disabled.
Apps analysis disabled.


Don't know if it's relevant but I'm using Movistar Triple VLAN config,
Tomorrow I'll with manual config (without TV) and Merlin 378.56_2 (this was working fine) and 380_57 (first try) with custom scripts in order to see tv. I'll post CPU usage result.

Top output:

With QoS disabled (about 290 Mbps download reached)

Mem: 83764K used, 171964K free, 0K shrd, 1496K buff, 24972K cached
CPU:  0.2% usr 10.2% sys  0.0% nic 26.7% idle  0.0% io  0.0% irq 62.6% sirq
Load average: 0.69 0.41 0.28 4/76 20499
  PID  PPID USER     STAT   VSZ %MEM CPU %CPU COMMAND
    3     2 admin    RW       0  0.0   0 46.6 [ksoftirqd/0]
  269     2 admin    RW       0  0.0   1 21.7 [kworker/1:1]
  268     2 admin    SW       0  0.0   0  2.4 [kworker/0:1]
  449     1 admin    R      652  0.2   1  0.7 tftpd
  472     1 admin    S     7604  2.9   1  0.6 httpd
  477     1 admin    S     6560  2.5   1  0.4 watchdog
  478     1 admin    S     6548  2.5   1  0.2 watchdog02
20498 18157 admin    R     1500  0.5   1  0.1 top
18156   456 admin    S     1072  0.4   1  0.1 dropbear -p 22 -a
14140   477 admin    S    21740  8.4   1  0.0 networkmap

With QoS enabled (about 160 Mbps download reached)

Mem: 106236K used, 149492K free, 0K shrd, 3192K buff, 29052K cached
CPU:  0.2% usr  9.2% sys  0.0% nic 34.6% idle  0.0% io  0.0% irq 55.7% sirq
Load average: 0.62 0.35 0.25 5/95 20337
  PID  PPID USER     STAT   VSZ %MEM CPU %CPU COMMAND
    3     2 admin    RW       0  0.0   0 26.2 [ksoftirqd/0]
  268     2 admin    RW       0  0.0   0 23.1 [kworker/0:1]
  269     2 admin    RW       0  0.0   1 14.3 [kworker/1:1]
  449     1 admin    R      652  0.2   1  0.6 tftpd
18156   456 admin    S     1072  0.4   1  0.5 dropbear -p 22 -a
  478     1 admin    S     6548  2.5   1  0.1 watchdog02
20294 18157 admin    R     1500  0.5   1  0.1 top

@vanic
Tell me what you need

 

[RMerlin]

High sirq usage is normal if NAT acceleration is disabled - it's caused by the network traffic itself.

 

[cierzo]

That's the reason that makes me think of a bug.

NAT Accel is configured as auto and the following message is present:
CTF(Cut Through Forwarding) is enabled.

 

[vanic]

That's the reason that makes me think of a bug.

NAT Accel is configured as auto and the following message is present:
CTF(Cut Through Forwarding) is enabled.

1. QoS : It means Traditional qos? If yes, as Merlin said.
2. CTF auto : when enable some function, we will disable CTF automatically, but GUI show nothing.
3. I receive three the same cases already.
- Two from Movistar Three VLANS
- One from Taiwan

I am proceeding to define what happened.

Thanks,
Vanic

 

[vanic]

Hi @cierzo ,
Could you send your setting file for me and capture picture about command "ps"? I will double check on my side. Please mail to router_feedback@asus.com with title [For Vanic].

Thanks,
Vanic

 

[trujulu]

Same here, I'm from Spain, and I have already sent the email for you.

(NAT is activated and no Qos)

CPU 2 has no load

 

[cierzo]

CPU has no load

CPU load is all you can see right of CPU label except "idle" one.
So you have a 61.3% CPU load with a 58.2 used for software irqs

@vanic Sorry for the delay but I'm not able to test or send anything right now. Flu is killing me. Hope to be able to do it in a couple of days

 

[trujulu]

CPU load is all you can see right of CPU label except "idle" one.
So you have a 61.3% CPU load with a 58.2 used for software irqs

@vanic Sorry for the delay but I'm not able to test or send anything right now. Flu is killing me. Hope to be able to do it in a couple of days

Sorry, I meant that the CPU 2 has no load.

 

[vanic]

@trujulu :
Sorry for late reply, I am in 4-days vacation, I will contact with you in 12/28~30.

Thanks,
Vanic

 

[cierzo]

@vanic

Config file and test report sent

 

[trujulu]

@vanic :

My issues with internet speed continues in 3.0.0.4.378.9460 and by other way I continue receiving multicast flood from another ports in the new version even after "the fix".

Finally, I am able to leave telnet/ssh open for a remote debug if needed