Results/Comments: Successfully replaced Google Fiber Box with RT-AC87U

[gschra11]

Hello, all,

Articles across the internet abound, some of which have broken links, some of which are specific to hardware and some of which aren't quite clear, I finally figured out how to remove my Google Fiber Network box and plug straight in to my Google Fiber Network jack... and get it to work full-speed with my ASUS RT-AC87U.

First, Google itself says that you need the following in order to make it work:

Here's the gory details if you really want to use your own router:
1. Traffic in/out of the fiberjack is vlan tagged with vlan2.
2. DHCP traffic should have 802.1p bit = 2
3. IGMP traffic should have 802.1p bit = 6
4. All other internet traffic 802.1p bit = 3

You can send data without the 802.1p bits but your performance will get throttled to something like 10Mbit.

NOTE: This data is subject to change. We are planning on changing the data in/out of the fiberjack to be untagged, which will then make it really easy for you to connect your own router.

A word of warning, most consumer routers don't have hardware forwarding (that is my feeble understanding) so you might not be happy with the performance on your network, and which will also probably affect tv service quality.
Source: http://pastebin.com/dWABB4ih

It's actually easy to apply these settings to the router, although I am simply tagging the priority bits to "3" for all protocols.

Go to the LAN button in the interface, click on the IPTV Tab, select "Manual," then put "2" for the VID and "3" for the PRIO.

That's it. Save the configuration, the router reboots and you plug WAN directly in to the Fiber Jack.

So, while it works, the technician's comment about hardware forwarding is correct. I'm actually running slower speed tests directly connected to the Fiber Jack vs. double-NAT-routing by plugging the WAN port of my router in to my Network Box and setting Network Box to DMZ the router. Surprised the hell out of me, but the speed tests don't lie. I'm running about 350 Mbps. directly connected, and was running 750 Mbps with the router connected to the Fiber Box.

From this, I have two questions:

1. Everything I have ever read (and everybody I have ever talked to) has told me that double-NAT routing is bad, bad, bad. Well, if I double-NAT route and it actually goes faster, do I really care? Am I missing something or hurting something performance-wise that doesn't show up in speed tests by putting my router behind another router?

2. Can I create a script that would tag priorities for DHCP and IGMP differently from general internet traffic? While I would be curious to see if it has any performance effects, my router processor is maxing out whenever I do a speed test. From this, I assume the "bottleneck" is the software-accelerated IP Traffic as opposed to hardware-accelerated. From what I read, tagging p-bits requires software acceleration and doesn't support hardware acceleration in the RT-AC87U. Would you agree?

Thanks for your thoughts.

 

[System Error Message]

double NAT is bad because hardware resources are wasted and its difficult if you have to do port forwarding or host servers. You may have google fibre but you arent hosting anything.

Secondly, a lot of routers (not consumer) have multi gigabit throughput. A simple example would be pfsense on an x86 box and some of mikrotik's routerboards. My CCR1036 for example would be able to have 14 google fibre accounts and get 28Gb/s (each one is 1Gb/s up + 1Gb/s down) of NAT even having to do that configuration. Not to mention since my CCR has SFP ports it doesnt need a modem to use fibre optics thus using less electricity overall.

The issue with consumer routers is that their firmware isnt as reliable or well made so they tend to lack many features. Tagging p-bits can be done in hardware but it has to be coded to do that (such as what switch chips do with vlans). You get all in ones but most do not include DSL and none include SFP. When broadcom introduced their chipsets they want with dual core ARM A9s instead of ARM A15s and there are no quad core variant. My router has 36 cores with hardware encryption acceleration and has multi gigabit VPN throughput and costs about 4 AC87Us and is many many times faster than one with the software NAT throughput at wirespeed making it so much cheaper price/performance and for features. I really dont get why people even bother to buy consumer hardware.

 

[gschra11]

I really dont get why people even bother to buy consumer hardware.

Thank you for your prompt reply. I can answer that question directly: I don't want to build the watch, I just want to know what time it is.

Allow me to offer more background. Why did I want a router beyond the Fiber Box in the first place?
1. To block porn, ad-ware, mal-ware, spy-ware, phishing sites and other illegal/grotesque sites at the network-level and not the device-level.
2. To use OpenDNS, which Google cannot support because their network box can only support IPv4 DNS servers... all IPv6 DNS traffic simply bypassed the manual DNS server configuration and thus bypassed OpenDNS.
3. The Fiber Box's WiFi was constantly crashing or, in some cases, didn't work at all because my 63-character security key was overloading its chipset. (Of course, I used that same 63-character key on my WRT54G for 8 years so I had all kinds of devices I would have had to reprogram.)
4. Support of guest networks.
5. Support of traffic logging and monitoring.

Why didn't I want to use both the Fiber Box in DMZ mode and the router behind it?
1. Double-NAT routing was akin to "crossing the streams" in all forums that I read.
2. I didn't want to burn electricity on two boxes if only one would work.
3. I wanted to free up a plug on my power strip.

Why didn't I choose a pfsense configuration?
1. I have no experience with Linux beyond what I learned in one computer science class in college that was required for my major. (I'm an engineer)
2. I have no experience with pfsense.
3. Spending the time or the money on building a Linux box, learning both sets of software minimally enough to achieve my configuration and to apply it is, frankly, more than I am willing to spend.
4. I would use more electricity and, presumably, more plugs on the power strip getting that configuration to work vs. the Fiber Box + my router.

I'm certain a pfsense configuration would provide a more-optimal experience on my home network, but I don't see it as necessary for me to achieve my goals.

At this juncture, I'm debating whether or not it would continue to make sense for me to operate without the Network Box. If I lose a substantial amount of performance from software-based p-bit tagging, but achieve better performance with hardware-based double-NAT routing, then is there any reason why I shouldn't proceed with the latter over the former?

 

[System Error Message]

pfsense and mikrotik have GUIs that if you understand networking you can configure them via GUI. What makes mikrotik better than ASUS is because you do not need to modify the linux files to configure something complicated. Everything you can do is within the GUI. They also have wiki and examples too which is worth learning and the CCR1009 is actually rated for a maximum of 25W and the SFP port means you dont need a modem assuming you can find an SFP module that fits the cable. Fibre optic SFP ports can be pricey but it mainly depends on the kind of fibre optic and are actually cheaper than buying a fibre optic modem which also means not having to put up with another 10W either.

You can find a demo of mikrotik routerOS at http://demo.mt.lv which shows how its like for one of their lower end routerboards. The CCR1009 is slightly different in some areas (switch and such) but everything else is the same. If you think you are able to learn it, give it a try and you can just use the AC87U as an AP and enjoy gigabit speeds.

The RB1100AHx2 is also fast enough to fully utilise your connection with all the configs, QoS and firewall but there is a CCR1009 that uses passive cooling. I suggest mikrotik aside from performance is because of its configurability and that it provides network services such as NTP server, RADIUS server, TFTP and can perform filtering in many ways even on L2, L4 and L7.

So while the passively cooled CCR1009 costs $500 it will fully utilise your connection and you can just get an SFP module if there is a fibre optic modem involved.

To debate if you need performance:
- electric costs
- is double NAT worth it? Do you host or use certain applications?
- is 350Mb/s of internet enough?
- do you need to use QoS and other features?

You can overclock the AC87U to 1.4Ghz. I've overclocked my asus AC68U to 1.4Ghz from 800Mhz and confirmed that it is faster. For you an additional 400Mbz will give 40% more speed. Its still not gigabit speeds but it should get you to the 500Mb/s speeds that i mention for dual core ARM A9s. Just remember that the 500Mb/s applies to total throughput so if you were uploading and downloading at the same time you would be seeing speeds of 250Mb/s per direction.

 

[gschra11]

I'm not hosting anything on my network today. That said, there may be a time when AiCloud would come in handy and/or I want to use my own cloud as opposed to Dropbox to avoid space limits, extra privacy, etc. Because I would have the Fiber Box set to DMZ if I put my router back in to a double-NAT configuration, would I still have to setup port forwarding on both? I thought a DMZ basically let everything through to a designated physical port, avoiding firewall rules, port forwarding, etc. Is my understanding incorrect?

350 Mb/s is probably enough from the standpoint that virtually every server I hit throttles me. The fastest connection I have seen has been between my home computer and my office computer when I VPN in to the office. Since that connection is Remote Desktop, however, it really doesn't improve much with greater bandwidth. Of course, latency has a large effect on it, but the latency is quite low... less than 10 ms. (Office is on Google Fiber, too, and there are all of six hops between me and work, despite being separated by 20 miles) Even Google itself throttles me. For kicks, giggles and grins I uploaded a 120 GB compressed encrypted backup file from my machine to Google drive. The upload speed was flat-lined at 200 Mbps. Downloading Windows 10 last week happened below the 100 Mbps mark.

I don't think I need QoS. With asymmetric speeds of up to 350 Mbps, is there any reason to "need" it? I very rarely use VoIP and I don't subscribe to the TV package. Netflix, YouTube, etc. have never buffered on me.

I will have to lookup overclocking, though. The marginal bump in performance might be worth it, especially on the Wi-Fi side. There is enough of a hit on the processor that Wi-Fi connections are taking a long time to authenticate. Since I have many Wi-Fi devices, I see a performance hit.

Thoughts?

 

[ASAT]

I will have to lookup overclocking, though. The marginal bump in performance might be worth it, especially on the Wi-Fi side. There is enough of a hit on the processor that Wi-Fi connections are taking a long time to authenticate. Since I have many Wi-Fi devices, I see a performance hit.

Thoughts?

I recommend overclocking the ASUS router. You'll also need a USB powered fan and some bungee cord to hold the fan against the router. Otherwise, you'll cook router.

 

[System Error Message]

I taped a laptop cooler to the back of my asus ac68U when i overclocked it to 1.4Ghz. On full load during stress test the highest temps were 15-20C lower than stock.

Make sure to check the PSU stability too and make sure the CPU temperature doesnt exceed 80C and make sure the wifi chips dont exceed 50C. Unlike the AC68U the AC87U has dedicated processors for wifi so theres nothing you can do to improve it unless you change the firmware on those tiny CPUs the router uses for wifi.

To test if google throttles you try using a torrent or try using a multi site speedtest. There are a few tests you can try to see if google bottlenecks you.
Downloading a service pack from microsoft.com that is more than 1GB big.
Downloading a huge http file from a fast server.
syncing large files with cloud
downloading ubuntu using torrent (its totally legal and they shouldnt be throttling you for this). You can download the torrent file from ubuntu website and also the image over http.

To test if you're being throttled by application or QoS, when you are experiencing slow speeds not by your router or device, ping your ISP and see how much increase there is in latency. IF when there is no traffic it takes 4ms than if your router is fully loaded it would take 100ms (pings are low priority)

 

[jevel]

Just to see if I get this right; when on Google's CPE, you get better speeds than on the RT-AC87U, but on the RT-AC87U you get the wireless stability you want?

In my book this sounds like you should be using Google's CPE for routing, disable its WiFi and then connect the RT-AC87U in AP mode to use that for WiFi.

-KJ

 

[gschra11]

Just to see if I get this right; when on Google's CPE, you get better speeds than on the RT-AC87U, but on the RT-AC87U you get the wireless stability you want?

In my book this sounds like you should be using Google's CPE for routing, disable its WiFi and then connect the RT-AC87U in AP mode to use that for WiFi.

-KJ

Sort of.

The WiFi barely works on the Network Box. Using it is not an option.

Using the Network Box as a wired router/switch works fine, but I've noticed that my router actually does better when it comes to wired traffic, at least when I have hardware mode enabled and I'm wired in a double-NAT configuration. (Using the AC87U in hardware mode directly connected to Google Fiber results in impressive download speeds, but upload speeds are throttled at 10 Mbps.)

Directly connecting to the Fiber Box and using my router as a Wi-Fi access point wouldn't do the filtering I am looking for, at least not on my wired connections.

 

[gschra11]

To test if google throttles you try using a torrent or try using a multi site speedtest. There are a few tests you can try to see if google bottlenecks you.

Google Fiber doesn't throttle me; Google Drive does. I'm able to easily break the 200 Mbps mark, but can't do it on most servers. I presume my "pipe" is just bigger than theirs.

Which laptop cooler are you using? I'm open to suggestions.

I've successfully overclocked to 1200 Mhz and am maintaining 78° processor temperatures doing nothing but stock. I won't bump higher than that until I get a fan.

Just out of curiosity, what does the 800 do in the second part of the NVRAM command? Does bumping it up higher do anything? Is it the front-side bus? RAM speed? Do I fiddle with it or leave it alone?

NVRAM set clkspd 1200,800

 

[System Error Message]

the laptop cooler i use wont work for the ac87U because of the mounting differences. I just use a cheap one thats basically a dual fan itself with a frame and just tape it to the back..

There are 2 kinds of laptop coolers you can choose from. Theres one that has fins and a fan which is placed at the side/back of the laptop if there are holes at the back of the AC87U. Theres one which you place the laptop on and it blows air in. Its better to have the fan blowing air out though.

You could use the cheap way and turn your AC87U upside down and just put a cheap dual fan on it to blow air out. dual fans are usually the right size to cover everything and you just have to tape the sidse. this can ruin the look but it is a very effective solution.

The 2nd value is the ram speed and shouldnt be changed. The highest DDR2 ram speed is 800mhz for standard ram and they didnt put performance ram in it so you cant overclock it further without causing instability.

If you're lucky you might be able to get to 1.6Ghz or more. If you overclock too far you'd have to reset so backup your settings first, overclock as far as you can till it wont boot and set the frequency a bit lower than that. just follow the overclocking way that is start with a bigger one than go with smaller changes (i.e. start with 100mhz steps, than 50mhz, than 10mhz. Each time you overclock, login via ssh and run the AES benchmark with 2 threads (there is a thread on this somewhere for RMerlin firmware where he shows the commands of verifying clock speeds).

I havent yet overclocked my AC3200 yet though but its because i havent gotten a cooler for it yet. Just remember its not the high clocks that kill a CPU, its the extra voltage, lack of power and heat. Since you cant change the voltage so you dont need to worry about it, just the PSU stability and heat.

 

[gschra11]

I've successfully, stably overclocked to 1400 Mhz on the AC87U without any issues. My processor is running at 72°C with the cooling fan I put on the router. It would seem I could bump it up a little more, but I'm not sure I want to risk it at this point. Really, the only thing I've noticed, performance-wise, is that the GUI of the router loads noticeably faster. It also seems as though my webpages are loading faster, too. My download and upload speeds remain approximately 350 Mbps, so trying to download/upload large files seems to not change in speed. My wireless performance is still sluggish and my wireless carrier's 4G is easily running better/faster than my WiFi, even if I am using mobile devices in the same room as the router. This seems to be the case on both 2.4 and 5.0 Ghz networks.

At this point, it would seem that my maximum performance is still using Google's Fiber Box, dropping my IP configuration that changes the p-bits and allowing the router to run with hardware acceleration. Even with double-NAT routing, it seems to maximize my performance to go through two routers (AC87U/Google Fiber Box) then to overclock my AC87U and directly connect to Fiber.

I'll still keep it running for a few days, though, to confirm stability. If it runs for a few more days without needing a reboot, and I'm still in the low 70s, maybe a bump it up to 1500 Mhz. Based on both 1200 and 1400 Mhz, though, I wouldn't expect any different in WiFi or throughput performance.

 

[System Error Message]

I dont think wifi performance will change however i remembered there was a design flow relating to performance in the AC87U mainly because the 2nd port (1st LAN port) is shared with wifi and has an RGMII link to the CPU/switch which means that they can interfere with each other. The hardware accelerators used for wifi could also be what slows you down too.

Try not using the 1st LAN port and see if wifi speed improves. See if you can enable/disable any hardware acceleration features including CTF.

 

[gschra11]

After trying it for about a month, I've decided to revert back to the double-NAT configuration I originally had with the Fiber Box. Simply put, the AC87U's hardware acceleration provides a noticeable boost in performance (about 33%) and stabilizes my WiFi connections. I can't provide any logical explanation for why WiFi performance is affected as much as it is, but it's fairly clear that it works with the Fiber Box and not-so-much without.

Anyways, I have my Fiber Box configured so that it doesn't use wireless, put my router in a DMZ and sets my router's WAN IP to a static number.

In my AC87U, I've configured my WAN for the same static IP I assigned at the fiber box. (Effectively disabling DHCP on the WAN-side) I disabled QoS and reverted my IPTV settings back to "disabled" and got CTF enabled. This greatly increases my performance.

 

[misterpyrrhuloxia]

........
Here's the gory details if you really want to use your own router:
1. Traffic in/out of the fiberjack is vlan tagged with vlan2.
2. DHCP traffic should have 802.1p bit = 2
3. IGMP traffic should have 802.1p bit = 6
4. All other internet traffic 802.1p bit = 3

...................................

It's actually easy to apply these settings to the router, although I am simply tagging the priority bits to "3" for all protocols.

Go to the LAN button in the interface, click on the IPTV Tab, select "Manual," then put "2" for the VID and "3" for the PRIO.

That's it. Save the configuration, the router reboots and you plug WAN directly in to the Fiber Jack.

I have the Free Google Fiber at about 5down/1up. I also have an Asus RT-AC87U but it's running DD-WRT. Do you think that I could do just these couple steps and get my Asus router running direct through the Fiber Jack without using the Network Box and not really experience any slowness or DHCP issues? If so, does anyone know how to do these steps in DD-WRT on the Asus RT-AC87U?

 

[gschra11]

I have the Free Google Fiber at about 5down/1up. I also have an Asus RT-AC87U but it's running DD-WRT. Do you think that I could do just these couple steps and get my Asus router running direct through the Fiber Jack without using the Network Box and not really experience any slowness or DHCP issues? If so, does anyone know how to do these steps in DD-WRT on the Asus RT-AC87U?

Yes, tagging p-bits is all that needs to be done for Google Fiber to bypass their box. That said, I'm not familiar with DD-WRT, so I don't know where those settings are buried.