WRT1900AC Spontaneous Reboots, lockups

[RogerSC]

The reason I'm asking for confirmation is because Linksys is investigating the issue of random reboots occurring at my request and others in our team. In this and other posts users were posting that this was occurring regularly.

Since I posted in this post and other forum threads of the WRT1900AC resetting the LAN\WLAN on configuration changes was by design. I'm finding that complaints of random reboots has all but disappeared.

Instead they seem to have shifted to more 2.4ghz wireless disconnect issues.

I could be missing how the situation has change but I hope not.

Okay, my reboots happened when the router was just sitting there, and were not 2.4GHz. band drops. Guess I'll have to get it out again and see what happens, then. I haven't changed any settings on it for some time, but I do bring up the status page occasionally to see what the uptime is. I did disable IPv6 for testing purposes, that's about it.

I assume that just looking at the status isn't what's supposed to be triggering reboots then, you actually have to change a setting in the GUI?

That's an odd design by the way. Most routers just restart the affected piece if you change a setting, not do a full reboot when settings are changed. Like if you change the wireless settings, they will restart the radios with the new parameters. Not do a full reboot. If you do a full reboot, that would take out the wired connections as well as the wireless, that doesn't make much sense to me.

A full reboot, although it shouldn't be necessary for just changing a setting most times, will certainly cause the new settings to be picked up. I guess you need to clear the decks on this router before changing any settings, make sure nothing important is happening, warn all the users, since a router reboot should be expected. Odd.

 

[htismaqe]

That's an odd design by the way. Most routers just restart the affected piece if you change a setting, not do a full reboot when settings are changed. Like if you change the wireless settings, they will restart the radios with the new parameters. Not do a full reboot. If you do a full reboot, that would take out the wired connections as well as the wireless, that doesn't make much sense to me.

Most routers INCLUDES the WRT1900AC.

Just like any other router, when I change a 5Ghz setting, for example, 6 or 7 times out of 10, it resets the 5Ghz radio and I'm back online in 5 seconds.

It's those other 3 times that are the issue. The router completely reboots. The reboots are not by design.

 

[RogerSC]

Most routers INCLUDES the WRT1900AC.

Just like any other router, when I change a 5Ghz setting, for example, 6 or 7 times out of 10, it resets the 5Ghz radio and I'm back online in 5 seconds.

It's those other 3 times that are the issue. The router completely reboots. The reboots are not by design.

I see, yes, I have to agree, I can't recall when I've seen a router reboot as the result of changing settings. You may be advised to reboot after making particular isolated changes so that the changes can take effect, but the router itself doesn't do it, since that would cause connections and current activity to be lost. To hear that this is by design is very surprising, actually pretty de-evolutionary *smile*.

And I do understand that you've been talking about reboots that are happening while you're in the GUI, but not when you've pressed "Apply". My comments are more general, addressing what sounds like a rather odd design decision.

I'm back on the WRT1900AC again with IPv6 enabled, will just let it run, no GUI changes will be made. In fact, all I'm going to look at is the status page. If that causes a reboot, that really is broken.

 

[chadster766]

I see, yes, I have to agree, I can't recall when I've seen a router reboot as the result of changing settings. You may be advised to reboot after making particular isolated changes so that the changes can take effect, but the router itself doesn't do it, since that would cause connections and current activity to be lost. To hear that this is by design is very surprising, actually pretty de-evolutionary *smile*.

And I do understand that you've been talking about reboots that are happening while you're in the GUI, but not when you've pressed "Apply". My comments are more general, addressing what sounds like a rather odd design decision.

I'm back on the WRT1900AC again with IPv6 enabled, will just let it run, no GUI changes will be made. In fact, all I'm going to look at is the status page. If that causes a reboot, that really is broken.

Thanks RogerSC I'm looking forward to your testing results

 

[sfx2000]

Right now, I have 3 things I want to discuss with them:

1. My original problem of rebooting while configuring via the GUI
2. The fact the syseventd is consuming almost 40% memory
3. that devidentd appears to have a memory leak

I would like to add this security as a 4th, so if you could give me any pointers, I'd really appreciate it. I know just enough about Linux to be dangerous.

I would just ask them why all of these tasks are running as "root" and not as a privileged user with specific permissions - yes some tasks have to launch as root, but normally they fork off and shutdown the primary.

If this was a router only, this would be a concern - but since this is also a NAS, where folks think they have a trusted volume, Linksys needs to be more careful about security in their designs.

As far as breaking the Linksys Guest Access portal - it's not about crashing out the portal, it's about crashing out lhttpd, and there are exploits there - and it's running as root as well...

Security has to be designed in - sadly enough, Linux has it, but Linksys (and others BTW) have implemented without it, and trying to patch it back in will be painful indeed.

sfx

 

[sfx2000]

Definitely looks like a memory leak. I'm waiting for someone from Sustaining Engineering to call me, she apparently only works Sunday through Thursday. I will see if I can explain it to her. Checked just now and it's up to 17.8.

India? Or perhaps Israel?

Today is tomorrow there... and weekends can be different.

In any event, it's cool to see what a couple of old-school telecom guys can do to get some attention

sfx

 

[htismaqe]

And I do understand that you've been talking about reboots that are happening while you're in the GUI, but not when you've pressed "Apply". My comments are more general, addressing what sounds like a rather odd design decision.

That's precisely what I'm saying. The reboots are not "by design". The router is crashing.

If it were by design, it wouldn't reboot after hitting "Save" or "Edit" on a DHCP reservation in the DHCP sub-screen. It would reboot after hitting "Apply" in the main LAN screen.

Furthermore, like I said, if it were by design, every time you changed a wireless setting, the router would reboot. It doesn't. The vast majority of the time, it resets the radio like any other router would. In fact, I've changed a setting, watched it reboot, and then changed the same setting again just to see and it doesn't reboot, it restarts the wireless interface as expected.

I'm back on the WRT1900AC again with IPv6 enabled, will just let it run, no GUI changes will be made. In fact, all I'm going to look at is the status page. If that causes a reboot, that really is broken.

I use the sysinfo.cgi several times a day and no reboots so far.

 

[htismaqe]

India? Or perhaps Israel?

Today is tomorrow there... and weekends can be different.

In any event, it's cool to see what a couple of old-school telecom guys can do to get some attention

sfx

From her voicemail, I'm guessing she's actually in the Philippines.

 

[sfx2000]

From her voicemail, I'm guessing she's actually in the Philippines.

Well, for what it is worth, once this project is kicked over to the sustaining engineering group - bugs fixed perhaps, but not major design changes...

oh well...

 

[htismaqe]

Well, for what it is worth, once this project is kicked over to the sustaining engineering group - bugs fixed perhaps, but not major design changes...

oh well...

Right. Sustaining Engineering or Lifecycle Engineering = minor improvements and bug fixes. Iterative, not transformative.

 

[htismaqe]

Looks like devidentd has returned to normal.

top -bn1
Mem: 89468K used, 161516K free, 0K shrd, 3948K buff, 24436K cached
CPU:  0.0% usr  0.0% sys  0.0% nic 95.8% idle  0.0% io  0.0% irq  4.1% sirq
Load average: 1.31 1.26 1.23 1/120 17393
  PID  PPID USER     STAT   VSZ %MEM CPU %CPU COMMAND
17393 17303 root     R     2764  1.1   1  4.1 top -bn1 
  913     1 root     S    95540 37.9   1  0.0 /sbin/syseventd 
31519     1 root     S    19796  7.8   0  0.0 /sbin/wl_link_status_monitor 
27567     1 root     S <  19736  7.8   0  0.0 /usr/bin/guardian 
16056     1 root     S    17848  7.0   0  0.0 devidentd -r /tmp/devregex.json -p

Things still look good in terms of stability.

UpTime:
 12:34:04 up 7 days, 15:54, load average: 1.31, 1.26, 1.23

 

[RogerSC]

Okay, got to 2 days of continuous uptime, so I'm switching back to the R7000 now. I don't like the fact that I can't go into the web admin GUI, or feel that I can't even have the web GUI running. I'm surprised because I recall having reboots when I was doing other things, outside of the GUI. On the other hand, I usually leave the GUI for the router I'm using up for monitoring, so just having it up may have some effect with this one. Very little setting changing goes on once I've configured a router, and even less with this router, where there''s virtually nothing to change.

page generated on Mon Jun  9 19:59:46 UTC 2014

UpTime:
 20:14:45 up 2 days, 39 min, load average: 1.33, 1.26, 1.24

I'll go back to the WRT1900AC when there's new firmware available for it. Or if there's something else to test...

 

[htismaqe]

Passed the 10 day mark.

UpTime:
 01:01:52 up 10 days,  4:21, load average: 1.29, 1.28, 1.30

Also, I've yet to hear back from Sustained Engineering. She left me a VM that she was available Sunday through Thursday, even though my best time to work on it is Saturday morning. I've sent her a couple of emails since the weekend and haven't heard a thing.

 

[chadster766]

Well I finally finished making a router stress testing platform.

I tried to recreate the "Kernel Panic" reboot that I captured from the serial console.

I wrote a script that loops through commands below 100 times:

1. Connect adapter1 to the WRT Wireless
2. Run iPerf throughput test
3. Disconnect adapter1
4. Connect adapter2 to the WRT Wireless
5. Run iPerf throughput test
6. Disconnect adapter2

I ran this twice consecutively. Once with adapter's set static ip's and once DHCP release and renews.

Approx 20gig throughput not a single failure

Any suggestions on addition testing? I'm thinking put the router's 2.4 ghz in mixed mode and have the adapters change protocols each time they connect.

What are your thoughts on whether or not this is a hardware interrupt conflict in the router's hardware config?

How can hardware interrupts be changed in U-Boot?

In an effort to help solve this issue I improved my Router Stress Tester.

It now does the below on each loop:

1. Automated router UI login
2. Changes and saves router’s wireless settings by rotating through the available wireless protocols for both 2.4Ghz and 5Ghz
3. Connects a DWA-552 wireless adapter to the 2.4Ghz for tests
4. Connects a DWA-180 wireless adapter to the 5Ghz for tests
5. Uses UPNP to delete and open ports for xBox simulation
6. Run a iPerf throughput test on both the 2.4Ghz and 5Ghz

5 GHZ Wireless "A Only" protocol with default 20Mhz channel width:

1. Clients have to repeatedly try to connect.
2. When they do connect the 5GHZ indicator light flashes on for a second and then stays off. The clients stay connected.
3. DWA-180 adapter with Linux driver would not connect.

My Conclusion:

• Bugs in the 5Ghz wireless driver.
• Reboots occur reliably when the 5Ghz wireless driver "A Only" setting has been used.
• Lockups occur reliably when the 5Ghz wireless driver "Mixed Mode" setting is used.

It would be great if a forum user that is having this issue could confirm this by changing there 5ghz to "N Only" and let us know if that solves the issue.

 

[htismaqe]

As you know, I do not have reboots outside of the GUI. However, it appears we may now know why.

My 5Ghz band is set to N-only and has been since I first got it. I've never had it set to anything else.

 

[chadster766]

As you know, I do not have reboots outside of the GUI. However, it appears we may now know why.

My 5Ghz band is set to N-only and has been since I first got it. I've never had it set to anything else.

I will advise Linksys engineering!

 

[RogerSC]

As you know, I do not have reboots outside of the GUI. However, it appears we may now know why.

My 5Ghz band is set to N-only and has been since I first got it. I've never had it set to anything else.

What happens with wireless-AC, then? Does the router use wireless-AC in N-only mode?

 

[htismaqe]

What happens with wireless-AC, then? Does the router use wireless-AC in N-only mode?

I'm assuming it doesn't do AC in N-only mode. I don't have any AC devices so I disabled it for this exact reason, I don't enable anything I don't need.

I have 2.4Ghz set to B/G-only and 5Ghz set to N-only.

UpTime:
 19:55:14 up 11 days, 23:15, load average: 1.30, 1.25, 1.27

 

[chadster766]

What happens with wireless-AC, then? Does the router use wireless-AC in N-only mode?

No AC in "N Only" but with auto channel width you can get a 300Mbit/s connection.

 

[htismaqe]

No AC in "N Only" but with auto channel width you can get a 300Mbit/s connection.

That reminds me, I have set fixed channel widths too. 20Mhz on 2.4Ghz and 40Mhz on 5Ghz.

I don't have anything set to "auto".