Really Basic L2 Managed Switch Question

[randallhill]

I'm new to L2 managed switch's for home use. I recently purchased a TP-Link TL-SG3216 switch (installed latest FW). My LAN chain is ATT U-Verse RG modem (Internet only)--->Netgear R6300v2 WiFi Gigabit router--->TP-Link L2 Switch--->Windows comps Gigabit NICS/two NASs Gigabit NICs/AVR/Oppo/HDTV wired CAT 5E/6. One NAS NIC is bonded LACP (802.3ad) and LAG 1 on the switch, the switch supports 802.3ad. No Jumbo Frames enabled anywhere on the local LAN...seems to slow the throughput if added at 9000 on NAS and Windows NICS (separate issue).

My question is:
I want to use a NTP server to configure the date/time feature on the switch. I need to set up the switch for internet access to ping the NTP server. The switch GUI IP is the default 192.168.0.1 subnet 255.255.255.0, which is setup as a static IP on the switch (Static/DHCP/BOOTP). TP-Link support indicated that I need to enter an IP for the switch Gateway to access the internet, the Gateway input field is currently blank.

The local LAN gateway via the Netgear router is 10.0.0.1, same subnet. All of the devices attached to the L2 switch is in the 10.0.0.2-10.0.0.99 range. The external gateway IP (ATT modem) is 99.13.6x.xxx.

What IP address do I use for the L2 switch Gateway? I've tried a few different IPs but the switch says "invalid gateway address".

I don't want to input a bad "static" IP for the switch e.g., change from 192.168.0.1 to some other IP and then not be able to access the L2 switch via GUI IP. If I select DHCP for the switch "IP Address Mode", the Netgear router may assign a new IP to the L2 switch and then I won't know what the IP is in order to access the L2 switch GUI.

The way I access the L2 switch currently is, I go into the Windows NIC settings and then to IPV4 and change the local address to 192.168.0.2 with the same subnet 255.255.255.0 and then use my browser to access the L2 switch GUI.

Any help will be greatly appreciated!

Thanks

 

[System Error Message]

I usually configure my network devices with static IP. Sure i could use dynamic IP addresses with binding too but i havent finished setting up my network. If you have a good router like i do you can set up DHCP and bind a single IP for your switch or just look at the list to see what IP it has. automatic ip gets IP from a DHCP server. I think your router may be capable.

You can change the IP of your managed switch to any address you like. If it so happens to be used by something else unplug the switch from the network and just change the IP. Any IP address that isnt used will work fine. You could configure your DHCP server to serve IP addresses within a certain range.

For the NTP server you can try the public NTP pool or set up your own local NTP server. you can use a consumer router with 3rd party firmware as one though you will need to do a lot of configuration and installing.

You might want to read wikipedia page on bonding since it has a good explanation on different bonding modes. Use one that matches your goal whether you want increased bandwidth or failover. For best results your switch needs to support it and i have not seen tp-link managed switches before. I have seen their routers and am disappointed with their restrictions and stock firmware.


If you are playing with managed switches you should try to learn the different features your devices have. A lot of these things arent for a novice user. Try to understand more about networking. It is easy to lock yourself out if you make a mistake. Luckily L2 doesnt have much to configure. Give your switch an ip in the 10.0.0.x range. You can use 100-255 for x if your router subnet is 255.255.255.0 or /24

The problem with your IP is that it is a totally different IP class. I believe 10.xxxxx is class B while 192.168 is class C. Just change your switch IP to one in the same subnet as your router and add your router as the gateway. if you are worried about unauthorised access you can use access control list and dedicated management port instead.

192.168.0.x cannot communicate with any other IPs other than 192.168.0.1-192.168.0.254 because of the subnet nor can your router because of the same reason. You can change your subnet to handle all the IPv4 IPs but that would just ruin your NAT routes.

Edit: Im going to assume you didnt set up your router otherwise you wouldnt be asking about ip address. I would suggest inputting 10.0.0.100 as your switch IP and gateway as 10.0.0.1 (assuming its your router). It should solve your problem. You should learn more about subnets.
If for some reason it still doesnt work than try changing the subnet to 255.255.255.128

 

[randallhill]

Thanks so much for the tip!

I used 10.0.0.25 for the static IP for the switch, 10.0.0.1 for the switch gateway, using the same subnet and the NTP server pinged successfully. I can also access the switch GUI using the 10.0.0.25 IP.

RE: Jumbo Frames...it seems to make no difference in throughput when enabled. It's pretty much indifferent if it's enabled or not. My NASs, NICs and switch support it at 9000. I'll just leave it disabled.

What are the optimal settings for NICs, I have Flow Control enabled, and speed as 1000 Full Duplex as opposed to Auto Negotiate. On the switch, I have the ports configured as Flow Control enabled and 1000 Full Duplex as well.

 

[System Error Message]

flow control should be disabled because it can mess up TCP streams.


Jumbo frames allow more data to be transferred using less CPU. This helps for transferring large files for example. The server, client and switches/routers in between must all have this feature enabled. Switches that use store-and-forward switching must not use jumbo frames or you will significantly increase latency by too much.

Leave the ports to auto unless you have problems. Auto will always use the highest possible. If you get errors on the link Auto will reduce speed to reduce or eliminate errors.

 

[azazel1024]

flow control should be disabled because it can mess up TCP streams.


Jumbo frames allow more data to be transferred using less CPU. This helps for transferring large files for example. The server, client and switches/routers in between must all have this feature enabled. Switches that use store-and-forward switching must not use jumbo frames or you will significantly increase latency by too much.

Leave the ports to auto unless you have problems. Auto will always use the highest possible. If you get errors on the link Auto will reduce speed to reduce or eliminate errors.

It does significantly increase latency. From 1.5uS to 6uS. I am thinking in most applications 6 microseconds of latency isn't going to be a big deal. On a 10Mbps switch port, or even 100Mbps, sure, you can start getting in to a some serious latency, but unless you have a LOT of hops, you'll never notice the latency for jumboframes compared to standard frames. That would also only be in the case of applications that are utilizing jumbo frames.

Jumboframes reduce CPU interupts and utilization by a modest amount. The other perk is more throughput as you have less TCP overhead on your traffic. it isn't much, but IIRC it is around 3% or so gain by reduced header overhead.

I would also leave flow control disabled. I WOULD use jumboframes if you can and have no problems. If a device doesn't support jumboframes, everything to/from/passing through will drop down to the smallest supported TCP frame size. Of course this is in an IDEAL world and sometimes crap does go off standard and won't work if something is jumboframe in the network. Big networks, don't bother. The odds that something won't work properly if jumbo frames are on just gets to be too much. In a small network where troubleshooting is a lot easier, run with jumbo frames.

Only issues I've ever had are on client adapters not properly supporting jumbo frames, not that something else had jumbo frames enables and they couldn't deal with it (IE everything else having 9k jumbo on (server, desktop, switches), working great, I turn 9k jumbo on, on the adpater in another machine, it works poorly, I disable it on the adapter, it works fine)

Intel and networking gear in general I've never had a jumbo frame issue. Atheros and Realtek I've both had lots of Jumbo frame issues (leaving disabled on them seems to work the best 9 out of 10 times, one or two Realtek adapters work well with 2k and 4k jumbo, but >4k causes problems).

 

[System Error Message]

I said store and forward switching. In cut through switching jumbo frames makes no difference in latency and can improve networking performance. Some store and forward switches will not work properly and be very laggy with jumbo frames because of the entire packet having to get into the buffer, get analysed and summed before being forwarded. Refer to your switch datasheet to check which switching method it uses as it should be mentioned there. on my netgear prosafe smart switch when using jumbo frames packets time out on the network meaning that the latency from it is just too high for store and forward switching.

I've not had jumbo frame problems in anything else before and using it is actually very helpful for realtek NICs which use more CPU.

Mikrotik CCRs support jumbo frames up to 65535 bytes as they use the CPU to do the switching and the routing. Any routerboard or routerOS will be able to use 65535 bytes MTU on the CPU level but on L2 it depends on if its the CPU or the switch chip. I have made VPNs and bridges with that MTU it the performance difference is very significant. Jumbo frames are useless for FPS games and situations where more transfers are required but jumbo frames only mean the maximum packet size can be large so you can still send small packets without a problem. You can mix equipments that both support jumbo frames and dont. If only consumer routers supported those frame sizes for their VPNs they would be able to have VPN speeds at gigabit rates.

 

[azazel1024]

I've never seen an issue with jumbo frames on switches (supposing the switch supported the size). Every switch I have used is a store and forward type switch (which is most switches in general).

Again, the latency added is extremely minimal. If you were having time out errors on packets, it was because of an issue with the switch, not because of induced latency (IE the switch did not in fact support jumbo frames of the size being used).

 

[System Error Message]

yes netgear gs smart switches are actually crap. Many of them fail badly at different things, im just lucky to get one thats reliable by default but starts crapping out when using jumbo frames and such.

 

[randallhill]

Thanks again for your feedback.

My NICs are Realtek...the Flow Control option is either Tx & Rx Enabled or Disabled. I'll disable Flow Control on the switch and NICs. Seems as though most, if not all posts I've read, tend to suggest disabling Flow Control entirely in order to prevent pause packets. Not sure why switch and NIC manufacturers include it as an option. TCP has its own method for transferring packets outside of Flow Control...?

Jumbo frames are supported on all of the NICs, the maximum ranging from 7K MTU to 9K MTU. The switch supports 10,260 MTU. The NAS supports 9000. Interesting though, a lower MTU setting on a Realtek NIC may end up being optimal.

All of the NICs and the switch ports are set up for auto negotiation. I didn't notice a difference with setting the NICs and the switch ports to 1000 Full Duplex.

 

[randallhill]

Gradually Decreasing Throughput

Curious...why does my Win7 desktop slow to a snails pace when I transfer large amounts of data from / to network devices e.g., desktop to NAS, NAS to NAS, NAS to / from router USB 3.0 external HDD. I have plenty of RAM on the desktop 9GB, i7 quad core CPU, and the CPU gets maxed out with about 40% of the RAM being used. The mouse pointer barely moves. Typing this post took about 5 minutes, the cursor is extremely slow. No other programs are open.

 

[L&LD]

9GB RAM? Why?

 

[sinshiva]

Curious...why does my Win7 desktop slow to a snails pace when I transfer large amounts of data from / to network devices e.g., desktop to NAS, NAS to NAS, NAS to / from router USB 3.0 external HDD. I have plenty of RAM on the desktop 9GB, i7 quad core CPU, and the CPU gets maxed out with about 40% of the RAM being used. The mouse pointer barely moves. Typing this post took about 5 minutes, the cursor is extremely slow. No other programs are open.

couple things;

check the NIC configure options and anything that has 'offloading' in it, disable. (or enable)

when you are doing these file copies, particularly NAS to NAS or NAS to router, are you doing something like copy/paste between network shares? in this case, you're probably doing something like transferring the data from NAS > PC > NAS, etc. if you are unfamiliar with doing these file copies on the NASes/router themselves, you might want to copy them from a device to the PC before copying them to the next device, etc. when moving lots of stuff to/from the usb3 drive, you should probably connect that to the PC, too

 

[randallhill]

OK...thanks, I'll check the NICs.

Yes, I initiate the copy jobs from my desktop PC to copy or move large (GBs) files to and from NASs to NASs or NASs to USB 3.0 HDD (attached to router) or visa versa. It does seem faster when transferring directly from PC to NAS or to the UBS 3.0 HDD attached to the router. But when I'm copying or moving large files (MKV/FLAC/WAV) files between network devices via switch initiated from my PC...the throughput is much slower and the desktop is really slow...the mouse pointer barely moves, it's very laggy.

Some of the files are 5GB to 40GB each. Sometimes I'll transfer 350GB at a time.

 

[stevech]

My win7 I5 quad - doing a big transfer to/from the NAS, doesn't change in responsiveness. GigE LAN.
Performance monitor shows low CPU utilization - like 20% or less. As it should be, since that transfer is I/O bound, not CPU bound.

sounds like your IP stack in Win7 is misconfigured. Check RX window size.

 

[randallhill]

"couple things;

check the NIC configure options and anything that has 'offloading' in it, disable. (or enable)"

The NIC is enabled for:

Large Send Offload IPv4
Large Send Offload v2 IPv4
Large Send Offload v2 IPv6

UDP & TCP Checksum Offload - IPv4 & IPv6 (Tx & Rx Enabled)

 

[randallhill]

9GB RAM? Why?

I built the desktop in 2009 for HTPC/Gaming...it's a first gen X58 mobo, LGA 1366 Core i7 with an updated Core i7 960. 9GB DDR3-1333 seemed reasonable then. I have a Radeon HD 6970 for graphics...for what I do or how I use the comp, the specs seem ok for now. The desktop SATA drives are Seagate 1.5TB/1.0TB SATAII 3.0Gbps.

The main NAS (LenovoEmc px2-300d) has two WD RED 3.0TB NAS HDDs in RAID 1 w/Intellipower meaning not 7200 RPM but not less than 5400 RPM.

The desktops/notebooks and NASs are connected to the TP-Link L2 switch, I connect my AVR, Oppo BDP-103 to the switch as well as the HDTV via CAT 5E UTP/6 UTP/6A STP cables.

 

[randallhill]

Need to mention:

The px2-300d NAS is configured for NIC link aggregation bonding LACP (IEEE802.3ad) on the NAS and the TP-Link TL-SG3216 L2 switch supports LAG and LACP and it's configured for LACP on the switch.

 

[azazel1024]

A couple of things, IMHO, realtek NICs tend to be crap. Better crap than most GbE NICs, but honestly Intel is the best...by a LARGE margin when it comes to wired networking products (there is a reason why most servers use Intel NICs/chipsets).

That out of the way, if you are doing a file transfer from one NAS to another NAS, from your desktop, unless you are initiating it through something proprietary or doing it RDP/logged in to the NAS management console, then it is going from NAS, to your computer and then to the other NAS (IE if you are copying from one mapped network drive to another. Windows is smart enough that, in general, if you copy it from one mapped network drive to another on the SAME server/NAS, it'll copy it internally and not over the network, but if they are two seperate machines, it has to go through your local machine to get to the end point).

This adds significant overhead in CPU, network and RAM usage. It isn't going to hit your local disk, but it WILL hit RAM and it is going to buffer a fair amount of data, hence why you see 40% utilization. Heck, even on just a basic one machine to another machine transfer, Windows will opportunistically utilize RAM to buffer disk I/O as much as it can. My server has 8GB of RAM and my desktop 16GB. During big transfers I see around 30-60% utilization on my server and 20-35% utilization on my desktop. which then drops down to about 16-18% and 12-13% background levels of utilization after the transfer way down the line).

Windows LOVES to cache things until something else needs the space. Heck, try downloading a large file to one machine with windows and then transfer it after it is done downloading to another machine. It'll be faster/the HDD will not spin up (if you've allowed it to idle to sleep) if the file is small enough to still be cached in RAM. Oh, windows wrote it to the HDD, but it is also still cached in RAM in case you are going to open/use it.

I've played with lots of testing of that. Download 2GB file to server, then copy to desktop 220+MB/sec (I have dual NICs and SMB Multichannel running). Reboot server, copy file again to desktop, this time it has to hit the disk to get the file and the transfer is conducted at 160-180MB/sec.

As for why things slow to a crawl, my guess is copious amounts of system interupts. Try setting Jumbo frames if you can on everything and see if it'll work. Try different size jumbo frames if you can. Try disabling offload (in case it isn't working correctly). Try changing TCP autotuning to crank up the agressiveness. Turn off flow control. Increase the Tx and Rx buffers to their maximum size.

If the adapters have receive side scaling, enable it (or heck, try disabling it if it is already enabled).

In terms of overhead, during a 220+MB/sec transfer between machines I see roughly a spike of 3-7% CPU utilization across all cores on my quad core desktop (i5-3570 @4GHz). On my file server I see a spike of about 8-14% CPU utilization on the dual core processor across all cores (G1610).

 

[randallhill]

Thanks so much for your explanation. It makes a lot of sense to me now. Interestingly, I've noticed that I've had to restart Win7 routinely after very large file transfers from NAS to NAS or external USB router HDD to get things back to "normal" on the desktop. Apparently, to clear the cache. But, I'm still curious why Win7 doesn't use more RAM, only about 40%.

I use Win7 Network Monitor II and System Monitor II gadget on my desktop (assuming they are somewhat accurate) when I transfer large files from the desktop to NAS "Net Usage" can run up to 100% with more of a smooth line, but when I transfer large files from NAS to NAS it varies from 5% to 40% with a very choppy line, i.e., huge spikes and lags.

Matter of fact, there will be pauses around 1Mbps then spikes to 2.5Gbps then back and forth. Really weird behavior, maybe it's just Windows 7, the switch, the NAS, the NICs, the LAN in general or all of the above.

One thing I've learned, when it comes to Windows, lower your expectations.

 

[azazel1024]

Maybe I am extremely wrong about this, but from what I can see, a lot of stuff that Windows 7+ stores in the super cache (IE RAM) is NOT reported as utilized memory, as windows will not actually unload it to scratch/page file if the required main memory is needed for something else. It generally reports stuff that would need to be unloaded. So you MIGHT be using more than 40%, but a lot of that stuff isn't being reported as utilized, because windows has no issues bumping it from RAM without paging.

With what you are experiencing, my guess is the blame actually lies with one or both NAS. Now, granted, Windows is the one mediating the need to transfer from NAS, to Windows PC to NAS when you transfer from one network drive to another, not on the same NAS. However, my bet is you are running in to some kind of performance roadblock as a result of the NAS that is resulting in the really shirte behavior.