WRT1900AC Spontaneous Reboots, lockups

[htismaqe]

Odd. According to this, your CPU is 100% idle.

I've pulled it a few times now. I only got one time where it didn't say CPU was 100% idle.

top -bn1
Mem: 100972K used, 150012K free, 0K shrd, 3948K buff, 24296K cached
CPU:  0.0% usr  0.0% sys  0.0% nic 95.6% idle  0.0% io  0.0% irq  4.3% sirq
Load average: 1.19 1.28 1.31 1/114 14567

 

[chadster766]

Anyone have longer that 4 days uptime in there sysinfo.cgi?

It would really be a big help to know because I'm passing this information on to the Linksys case regarding the reboot issue.

 

[htismaqe]

FYI, it appears like memory usage for devidentd is increasing over time. It started at 7.7 and is now at 12.8. I haven't yet seen it go down from one refresh to the next.

top -bn1
Mem: 103580K used, 147404K free, 0K shrd, 3948K buff, 24432K cached
CPU:  0.0% usr  4.5% sys  0.0% nic 95.4% idle  0.0% io  0.0% irq  0.0% sirq
Load average: 1.41 1.35 1.33 1/115 4695
  PID  PPID USER     STAT   VSZ %MEM CPU %CPU COMMAND
 4695  4605 root     R     2764  1.1   0  4.5 top -bn1 
  913     1 root     S    95540 37.9   1  0.0 /sbin/syseventd 
20168     1 root     S    32348 12.8   1  0.0 devidentd -r /tmp/devregex.json -p

 

[htismaqe]

Anyone have longer that 4 days uptime in there sysinfo.cgi?

It would really be a big help to know because I'm passing this information on to the Linksys case regarding the reboot issue.

I'm actually just short of 5 days.

UpTime:
 19:21:15 up 4 days, 22:41, load average: 1.41, 1.34, 1.32

 

[sfx2000]

FYI, it appears like memory usage for devidentd is increasing over time. It started at 7.7 and is now at 12.8. I haven't yet seen it go down from one refresh to the next.

top -bn1
Mem: 103580K used, 147404K free, 0K shrd, 3948K buff, 24432K cached
CPU:  0.0% usr  4.5% sys  0.0% nic 95.4% idle  0.0% io  0.0% irq  0.0% sirq
Load average: 1.41 1.35 1.33 1/115 4695
  PID  PPID USER     STAT   VSZ %MEM CPU %CPU COMMAND
 4695  4605 root     R     2764  1.1   0  4.5 top -bn1 
  913     1 root     S    95540 37.9   1  0.0 /sbin/syseventd 
20168     1 root     S    32348 12.8   1  0.0 devidentd -r /tmp/devregex.json -p

Interesting - this daemon (devidentd) is not part of the GPL drops,

What this daemon does is local network device discovery, which probably ties back into the Network Map widget in the GUI, as noted by the devregex.json argument..

The exciting part here - we have a daemon with a possible memory leak that has root privileges - this is definitely something not good... combine this with the fact that the Network Device Widget is also tracking Guest Accounts on an OpenWiFi opportunity - this is ringing alarms here for me...

this needs to be fixed...

 

[RMerlin]

Interesting - this daemon (devidentd) is not part of the GPL drops,

They don't include any of their proprietary daemons/tools in their GPL drop as far as I see. They do the bare minimum to follow the GPL, which is provide only their patches that are applied on top of GPL code.

 

[htismaqe]

Definitely looks like a memory leak. I'm waiting for someone from Sustaining Engineering to call me, she apparently only works Sunday through Thursday. I will see if I can explain it to her. Checked just now and it's up to 17.8.

top -bn1
Mem: 116768K used, 134216K free, 0K shrd, 3948K buff, 24444K cached
CPU:  0.0% usr  4.1% sys  0.0% nic 95.8% idle  0.0% io  0.0% irq  0.0% sirq
Load average: 1.44 1.32 1.28 1/114 5103
  PID  PPID USER     STAT   VSZ %MEM CPU %CPU COMMAND
 5103  5013 root     R     2764  1.1   1  4.1 top -bn1 
  913     1 root     S    95540 37.9   0  0.0 /sbin/syseventd 
[B]20168     1 root     S    45096 17.8   1  0.0 devidentd -r /tmp/devregex.json -p[/B]
31519     1 root     S    19796  7.8   1  0.0 /sbin/wl_link_status_monitor 
27567     1 root     S <  19736  7.8   0  0.0 /usr/bin/guardian 
 2699     1 root     S    13020  5.1   0  0.0 /usr/sbin/linkmgr 
28044     1 root     S     8072  3.2   0  0.0 /usr/bin/redirector

 

[htismaqe]

combine this with the fact that the Network Device Widget is also tracking Guest Accounts on an OpenWiFi opportunity

I'm not understanding the significance...

 

[sfx2000]

I'm not understanding the significance...

GuestNetwork on the Linksys - it's a open wifi connection, no WPA/WPA2, it's wide open - and the captive portal is served up with root privileges... So is this daemon... so now two edges to lift up to get root.

Not going into details here - but fuzz the drivers/apps, it's only a while before the script kiddies find out how to automate it...

....

yep, linksys - they made a couple of mistakes here...

got root?

sfx

 

[htismaqe]

GuestNetwork on the Linksys - it's a open wifi connection, no WPA/WPA2, it's wide open - and the captive portal is served up with root privileges... So is this daemon... so now two edges to lift up to get root.

Not going into details here - but fuzz the drivers/apps, it's only a while before the script kiddies find out how to automate it...

....

yep, linksys - they made a couple of mistakes here...

got root?

sfx

Ah ok. You're talking about a security issue.

For the record, it's more that the captive portal is served up via a process running as root rather than the fact the portal has no encryption right? Because just about all public wifi out there is unsecured - that's the point.

For me it's really not an issue. I'm very isolated (closest neighbor is over a mile away) so I'm not really at any risk. However, I would like to report this to Linksys for posterity's sake.

In looking at the top output, it looks like 99% of all the processes are running under root. Would you have a better recommendation, maybe create a separate user permission for the web UI stuff and have it all run under that?

Right now, I have 3 things I want to discuss with them:

1. My original problem of rebooting while configuring via the GUI
2. The fact the syseventd is consuming almost 40% memory
3. that devidentd appears to have a memory leak

I would like to add this security as a 4th, so if you could give me any pointers, I'd really appreciate it. I know just enough about Linux to be dangerous.

 

[htismaqe]

Also just FYI in regards to the "it's only a while before the script kiddies find out how to automate it"...

Based on my experience with the EA6900, I think it's probably safe to assume that the Guest network behavior is identical in all Linksys Smart Wifi routers, meaning that this security hole has at least been open for script kiddies to exploit since late 2012 when the EA6500 was released. I have no experience with the generation of routers before that running Cisco Cloud Connect but given that the CCC look and feel appears to be almost exactly like Smart Wifi, it's entirely possible they all use a similar mechanism too.

Do we know if there are any widely-circulated vulnerabilities for Linksys guest network access?

 

[chadster766]

Is anyone still having reboots or lockups?

 

[htismaqe]

Is anyone still having reboots or lockups?

I never had any outside of the GUI. But I guarantee you if I went in there right now and changed something it would reboot.

 

[chadster766]

I never had any outside of the GUI. But I guarantee you if I went in there right now and changed something it would reboot.

Yes but that's by design.

I really need confirmation that reboots are still occurring outside the UI.

 

[htismaqe]

Yes but that's by design.

I really need confirmation that reboots are still occurring outside the UI.

By design? Absolutely not.

It's not rebooting AFTER changing a setting. It's rebooting in the middle of changing a setting.

It's not a case of restarting the 2.4Ghz radio after changing the mode or channel. It's a case of the router completely restarting when editing a DHCP assignment before committing any changes on the LAN interface, for example.

 

[RogerSC]

Is anyone still having reboots or lockups?

Don't know, most likely if I switched back to using the WRT1900AC, I'd still have reboots. Nothing's been done that I know of to fix that. I'm currently using the R7000, very solid.

Did I miss a firmware update? My firmware version is 1.1.7.160582. Why would I stop having reboots without changing firmware?

Just wondering why you're asking the question?

 

[chadster766]

By design? Absolutely not.

It's not rebooting AFTER changing a setting. It's rebooting in the middle of changing a setting.

It's not a case of restarting the 2.4Ghz radio after changing the mode or channel. It's a case of the router completely restarting when editing a DHCP assignment before committing any changes on the LAN interface, for example.

So you are saying that your sysinfo.cgi uptime resets in the cases you mention?

 

[chadster766]

Don't know, most likely if I switched back to using the WRT1900AC, I'd still have reboots. Nothing's been done that I know of to fix that. I'm currently using the R7000, very solid.

Did I miss a firmware update? My firmware version is 1.1.7.160582. Why would I stop having reboots without changing firmware?

Just wondering why you're asking the question?

The reason I'm asking for confirmation is because Linksys is investigating the issue of random reboots occurring at my request and others in our team. In this and other posts users were posting that this was occurring regularly.

Since I posted in this post and other forum threads of the WRT1900AC resetting the LAN\WLAN on configuration changes was by design. I'm finding that complaints of random reboots has all but disappeared.

Instead they seem to have shifted to more 2.4ghz wireless disconnect issues.

I could be missing how the situation has change but I hope not.

 

[htismaqe]

So you are saying that your sysinfo.cgi uptime resets in the cases you mention?

Yep. It's a complete reboot of the router when it happens. It goes through the entire boot up process as if it had been power cycled. Uptime on sysinfo.cgi resets to zero.

I see "expected" behavior all the time. When I change a 2.4Ghz setting for example, the router restarts the interface which causes all clients to disconnect and reconnect to pickup the new settings. This happens with my Netgear router too. It's a necessary and expected step to reset the interface that was changed. Uptime on sysinfo.cgi does NOT reset.

However, some times (not always) changing that same setting causes the router to completely reboot as described above.

And keep in mind, it's not always settings that require that kind of reset either. I've had it reboot while changing a device name in Network Map, for example. I've also had it reboot while editing (not saving) DHCP reservations, so it was rebooting prior to even committing a change.

I haven't accessed the GUI at all since I talked to Linksys on 6/1. While on the phone with them, I had 2 reboots accessing the GUI with Safari on OSX and 1 reboot using IE9 on Windows XP. That's 3 reboots in less than 10 minutes. In looking at my current uptime, the answer seems pretty obvious. No reboots at all since those 3.

UpTime:
 19:09:16 up 5 days, 22:29, load average: 1.39, 1.31, 1.31

By the way, perhaps there isn't a memory leak in devidentd after all...

top -bn1
Mem: 96708K used, 154276K free, 0K shrd, 3948K buff, 24456K cached
CPU:  4.3% usr  0.0% sys  0.0% nic 95.6% idle  0.0% io  0.0% irq  0.0% sirq
Load average: 1.45 1.32 1.32 1/115 11810
  PID  PPID USER     STAT   VSZ %MEM CPU %CPU COMMAND
11810 11720 root     R     2764  1.1   1  4.3 top -bn1 
  913     1 root     S    95540 37.9   1  0.0 /sbin/syseventd 
11192     1 root     S    25132  9.9   1  0.0 devidentd -r /tmp/devregex.json -p
31519     1 root     S    19796  7.8   0  0.0 /sbin/wl_link_status_monitor 
27567     1 root     S <  19736  7.8   0  0.0 /usr/bin/guardian 
 2699     1 root     S    13020  5.1   1  0.0 /usr/sbin/linkmgr

 

[htismaqe]

Just an Fyi but it's pretty easy to tell what is happening by using a continuous ping targeting the LAN interface of the router.

When an interface resets, it takes about 5 seconds before traffic starts passing again. During that 5 seconds, you get "request timed out" messages.

When the router reboots, it takes a good 30 seconds before traffic starts passing again and during the middle part of that 30 second window, you'll see several "host is down" error messages.