100% CPU Utilization by Miniupnpd and Netstat

[garycnew]

Today, after rebooting my Asus router, miniupnpd and netstat began consuming 100% cpu.

4401     1 admin    R      888  0.3   0 48.4 miniupnpd -f /etc/upnp/config
6910  6909 admin    R     1436  0.5   1 46.0 netstat -nl

Anyone know what might cause this issue?

Thanks.

 

[dave14305]

At least for netstat, what is the parent process ID 6909?

 

[garycnew]

At least for netstat, what is the parent process ID 6909?

Yes... netstat parent process ID is 6909.

6909 admin     1416 S    sh -c netstat -nl
6910 admin     1436 R    netstat -nl

 

[dave14305]

Yes... netstat parent process ID is 6909.

6909 admin     1416 S    sh -c netstat -nl
6910 admin     1436 R    netstat -nl

And what is 6909's parent? Keep following it until you find the evil source.

 

[garycnew]

It appears the parent process of netstat is dcd:

# pstree -aps 6910
preinit,1
  └─dcd,2828 -i 3600 -p 43200 -b -d /tmp/bwdpi/
      └─sh,6909 -c netstat -nl
          └─netstat,6910 -nl

# pstree -aps 4401
preinit,1
  └─miniupnpd,4401 -f /etc/upnp/config

In an earlier post, ColinTaylor referenced the dcd process to the Trend Micro Engine.

 

[dave14305]

If you run netstat -nl manually, does it complete? I would just kill 6910 and see if it comes back.

 

[garycnew]

If I disable Traffic Analyzer > Statistic, it does kill the dcd and netstat processes and frees 1-core and 50% of the total available cpu. However, the miniupnpd process continues consuming 1-core and 50% of the total available cpu.

Do you know where Traffic Analyzer - Statistic keeps its data? I'm wondering if it has run out of space?

P.S. netstat -nl does manually complete without issue.

 

[Tech9]

What router and firmware? DCD was crashing non-stop not long ago and causing 100% core load on some models.

 

[garycnew]

What router and firmware? DCD was crashing non-stop not long ago and causing 100% core load on some models.

Hey @Tech9

It's the same Asus model router that I've always referenced on this forum. Do you remember which it is... Without cheating? Yes... This is a test. :-D

Was it ever determined why DCD was crashing non-stop?

Thanks, again. Always a pleasure.


Gary

 

[Tech9]

In this case you have to be on firmware above 386.12_2 where DCD crashing was fixed. It was TrendMicro issue upstream. Asus implemented some workaround and updated Asuswrt, the fix came to Asuswrt-Merlin shortly after. Don't update to 386.14 though, has issues.

I remember everything.

 

[garycnew]

In this case you have to be on firmware above 386.12_2 where DCD crashing was fixed. It was TrendMicro issue upstream. Asus implemented some workaround and updated Asuswrt, the fix came to Asuswrt-Merlin shortly after. Don't update to 386.14 though, has issues.

@Tech9

It's strange that the TrendMicro issue would only affect my router, now. Traffic Analysis > Static still works, but causes 100% cpu utilization.

I'm still trying to understand how miniupnpd fits into this puzzle.

You should know better to worry about me upgrading my firmware. ;-D

Much Appreciated.


Gary

 

[Tech9]

I don't know the cause of your specific issue. Your routers are non-standard configuration. I only target test specific things people complain about on a clean system. This DCD crashing was present right off the bat the moment any feature using Trend Micro engine was enabled. It was causing Core 1 100% load perhaps due to constantly restarting the daemon. DCD is Data Collection Daemon, the payment method for the free services they provide.

 

[garycnew]

It was causing Core 1 100% load perhaps due to constantly restarting the daemon.

That sounds like the same symptoms. I need to try and figure out the miniupnpd correlation.

BTW... I'm up to about 11 of these Asus routers in my "non-standard" configuration, now. :-D

 

[Tech9]

Few months ago I received a large number new condition RT-AC1900P units for recycling and they still stay in one corner of my warehouse. Something stops me from destroying them, perhaps nostalgia and respect to the best and most successful router Asus have created. If I had someone like you around would make him happy tomorrow. Managed to donate a few, kept one for testing, but the rest will perhaps die on next space clean up.

 

[garycnew]

Few months ago I received a large number new condition RT-AC1900P units for recycling and they still stay in one corner of my warehouse. Something stops me from destroying them, perhaps nostalgia and respect to the best and most successful router Asus have created. If I had someone like you around would make him happy tomorrow. Managed to donate a few, kept one for testing, but the rest will perhaps die on next space clean up.

How similar is the RT-AC1900P to the RT-AC66U B1? How many do you have available? I'm always looking to add more nodes to the bot farm. Better to give them a second life; than, to recycle them.

 

[Tech9]

It's an RT-AC68U variant just like RT-AC66U B1 with upgraded BCM4709C0 1.4GHz CPU, the same 256MB RAM, the same BCM4360 radios, 256MB flash for some unknown reason (128MB on original RT-AC68U), some different heatsinks inside, upgraded flat style antennas like in RT-AC86U, runs the same RT-AC68U Asuswrt and Asuswrt-Merlin firmware. It was BestBuy exclusive model for North America. I know you are far...

 

[garycnew]

I know you are far...

I'm in Utah. Aren't you in Canada? How many do you have? Are they still in their original boxes? Depending on the quantity... I'm wondering about the cost for bulk shipping. Thanks, again.

 

[Tech9]

We have to move this conversation off-line.

 

[garycnew]

@ColinTaylor Any idea why Miniupnpd would consume all resources on 1 CPU core at the same time DCD is consuming all resources on the other CPU core?

Disabling Traffic Analyzer > Statistic frees the consumed resources on 1 CPU core and service stop_upnp frees the consumed resources on the other CPU core.

Is Miniupnpd, also, related to the TrendMicro issue @Tech9 was referencing? Appreciate your insight.

 

[ColinTaylor]

I'm not aware of any direct link between miniupnpd and dcd. I think the most likely solution to dcd problems is to install the latest firmware and/or not create custom network interfaces.

The only idea I can think of regarding miniupnpd is to strace it and see if there's something obvious that it's looping on.