Sorry, another "neighbour table overflow" question

[RogerSC]

This is a question most likely for RMerlin, about the "Neighbour table overflow" issue. I see from looking around that this is most likely the result of the ARP cache overflowing. Which means that I should make the ARP cache larger. I've seen this suggested to fix the problem:

echo 512 > /proc/sys/net/ipv6/neigh/default/gc_thresh1
echo 2048 > /proc/sys/net/ipv6/neigh/default/gc_thresh2
echo 4096 > /proc/sys/net/ipv6/neigh/default/gc_thresh3

Lucky me, Comcast is my ISP, and I have IPv6 enabled on my rt-n66u. My question is how big can I make these parameters without degrading the performance of my router due to lack of RAM memory to store the table? Can I double the parameters above safely and not worry about running out of RAM?

I'm now seeing these messages at about 3:45AM and 10:45AM PDT, and while I could live with them, I'd rather my router be busy routing than dealing with these errors. I get about 500 of these messages each time I see this error. Not sure how much cache space that each ARP packet takes up, or how much extra RAM there is...one more thing, I'm just using the router for routing and wireless, no dlna, USB disks, aicloud, etc., so most likely I have minimal RAM usage at this point.

Thanks.

 

[RMerlin]

No idea, but I fear that if your ISP is allowing so many MACs to be visible from your router, I doubt there is any safe value that could be handled by this type of router. I would consider talking to Comcast about this issue, could be a misconfiguration on their end for IPv6 traffic. Good luck talking with anyone above level 1 tech support to deal with this tho

 

[RocketJSquirrel]

You can run this script to see how many neighbours you have at the moment:

#!/bin/sh
# Show ipv4 neighbour count
echo -n 'IPv4: '
ip -f inet route show cache | grep -v cache | wc -l
# Show ipv6 neighbour count
echo -n 'IPv6: '
ip -f inet6 route show cache | grep -v cache | wc -l

Maybe that will help you determine values to set. Oddly, I no longer see this problem at all, and I have Comcast also. I was under the impression they had remedied this issue somehow.

Currently, my result is:

IPv4: 46
IPv6: 9

when I used to have hundreds of IPv6 neighbours.

 

[RogerSC]

No idea, but I fear that if your ISP is allowing so many MACs to be visible from your router, I doubt there is any safe value that could be handled by this type of router. I would consider talking to Comcast about this issue, could be a misconfiguration on their end for IPv6 traffic. Good luck talking with anyone above level 1 tech support to deal with this tho

Yeah, not much luck with talking with Comcast about anything technical. That's why I was hoping to solve it on my end. Maybe I should just play around with these parameters and see if I can get anywhere. You never know. Or maybe it will just stop as suddenly as it started...

 

[RogerSC]

You can run this script to see how many neighbours you have at the moment:

#!/bin/sh
# Show ipv4 neighbour count
echo -n 'IPv4: '
ip -f inet route show cache | grep -v cache | wc -l
# Show ipv6 neighbour count
echo -n 'IPv6: '
ip -f inet6 route show cache | grep -v cache | wc -l

Maybe that will help you determine values to set. Oddly, I no longer see this problem at all, and I have Comcast also. I was under the impression they had remedied this issue somehow.

Currently, my result is:

IPv4: 46
IPv6: 9

when I used to have hundreds of IPv6 neighbours.

Well, right at the moment, only 28 for IPv6 and 44 for IPv4. Had this happen again this morning about 11:20AM...missed out on the 3AM shot today.

I'm hoping that it will just stop...I'm going to try the Comcast IPv6 forum where Chris Tuska hangs out and see what's up there, I guess.

 

[saintdev]

For me, it's actually IPv4 causing the issue (I'm also on Comcast).

admin@RT-N66U:/tmp/home/root# ip -f inet6 route show cache | grep -v cache | wc -l
22
admin@RT-N66U:/tmp/home/root# ip -f inet route show cache | grep -v cache | wc -l
2583

 

[RocketJSquirrel]

I'd be very interested to see what Ms. Tuska has to say about this.

In the meantime, to respond to the original question, I would just go ahead and raise the thresholds as long as the numbers aren't astronomical. You can keep an eye on how much memory the router is using. If you find you're starting to run out of memory, it's easy to add swap space on a USB drive.

 

[RocketJSquirrel]

There is another clue here.

Multicast snooping gets disabled when the bridge is initialized because of a bug trying to rehash the table. IGMP snooping stops the bridge from forwarding every HBH ICMPv6 multicast query reply, which results in the neighbour table filling up with ff02:: neighbours from multicast replies which it should not see (try ip -6 neigh show nud all).

The proper workaround is to attempt to re-enable snooping like:

echo 1 > /sys/class/net/eth0/bridge/multicast_snooping

The alternative is to make the neighbour table gc thresholds bigger than the number of hosts in the broadcast domain.

I don't know whether or how this translates into the version of Linux on the Asus routers.

 

[RMerlin]

Don't have any N66/AC66 running at the moment to check these, but on my RT-AC56U (running 2.6.36.4) that flag is already enabled:

admin@Stargate3:/tmp/home/root# cat /sys/class/net/br0/bridge/multicast_snooping 
1
admin@Stargate3:/tmp/home/root#

 

[RogerSC]

Yes, I ran into that as well, and it didn't map into what I see on the router. I expect that looking at the source code for the router software would tell one if this would have any effect. It would be nice if it would work with this router (rt-n66u), but I don't see that file system path or file name anywhere (using "ls" and "find").

It may be that without the file the router uses a default value, and you just need to create the file with a "1" in it in the right place, or it could be that the router software doesn't implement this functionality (which may be why we see this problem in the first place).

 

[RMerlin]

Yes, I ran into that as well, and it didn't map into what I see on the router. I expect that looking at the source code for the router software would tell one if this would have any effect. It would be nice if it would work with this router (rt-n66u), but I don't see that file system path or file name anywhere (using "ls" and "find").

It may be that without the file the router uses a default value, and you just need to create the file with a "1" in it in the right place, or it could be that the router software doesn't implement this functionality (which may be why we see this problem in the first place).

Try setting it on the bridge (br0) instead of the WAN interface (eth0), see if that works.

 

[RocketJSquirrel]

Try setting it on the bridge (br0) instead of the WAN interface (eth0), see if that works.

Doesn't work.

ASUSWRT RT-N66U_3.0.0.4 Wed Jul 24 06:22:47 UTC 2013
admin@RT-N66U:/tmp/home/root# cd /sys/class/net/br0/bridge
admin@RT-N66U:/sys/class/net/br0/bridge# ls
ageing_time               group_addr                priority                  tcn_timer
bridge_id                 gso                       root_id                   topology_change
flush                     hello_time                root_path_cost            topology_change_detected
forward_delay             hello_timer               root_port                 topology_change_timer
gc_timer                  max_age                   stp_state
admin@RT-N66U:/sys/class/net/br0/bridge# echo 1 > multicast_snooping
[B]-sh: can't create multicast_snooping: Permission denied[/B]

 

[RocketJSquirrel]

Interestingly, this is a web GUI configurable item on the wireless professional page.

 

[cmillar6]

The neighbor table overflow error actually got a little bit better for me when I upgraded to .372 They have become far less frequent but they still appear. I assumed it was ipv6 tweaks in the newer code.
Browsing these forums it does seem unique to Comcast customers, good luck dealing with any of their tech support, most don't even know what ipv6 is.

 

[saintdev]

Don't have any N66/AC66 running at the moment to check these, but on my RT-AC56U (running 2.6.36.4) that flag is already enabled:

admin@Stargate3:/tmp/home/root# cat /sys/class/net/br0/bridge/multicast_snooping 
1
admin@Stargate3:/tmp/home/root#

I thought I remembered seeing in another post on this forum the N66's kernel was too old for the multicast snooping sysfs toggle (I'm seeing 2.6.22.19).

Edit: Looks like it wasn't introduced until 2.6.34.

 

[RogerSC]

The neighbor table overflow error actually got a little bit better for me when I upgraded to .372 They have become far less frequent but they still appear. I assumed it was ipv6 tweaks in the newer code.
Browsing these forums it does seem unique to Comcast customers, good luck dealing with any of their tech support, most don't even know what ipv6 is.

I started seeing this with 372, but I just went back to 270_26b for other reasons, so I'll see if they stop...Chris Tuska from Comcast is looking into this with Asus. Hopefully they'll come up with something, although she isn't seeing the problem (of course). I keep posting the lines from my log, to show that I'm seeing it.

 

[RogerSC]

I thought I remembered seeing in another post on this forum the N66's kernel was too old for the multicast snooping sysfs toggle (I'm seeing 2.6.22.19).

Edit: Looks like it wasn't introduced until 2.6.34.

As RMerlin pointed out, Asus can include patches from later versions of the kernel if they want or need to. So if they needed this, which it is sounding like they might, they could integrate it (and test it, hopefully *smile*).