Port Forwarding Issues

[canadianpsyko]

Hello,
I'm having some issues that I can't seem to figure out. Port forwarding doesn't seem to be functioning correctly for me.

What I'm trying to do: Forward port 80 and 22 to my local file server.

What's happening: port 80 connects about 2/3 of the time, otherwise times out. Port 22 (ssh) connects about 1/3 of the time, and if it does, times out shortly after.

I had a working setup previously, with DD-WRT on a Buffalo router, but I wanted 802.11ac and 5GHz N, so I upgraded. Nothing has changes except the router (ac68u, currently running Merlin 3.0.0.4.374.39)

The server has a staticly assigned DHCP IP. Everything works internally, it's only when I try to access from outside there's an issue.

I've tried everything I can think of, the only thing I haven't done is take the switch between the server and the router out (too many wires running around my desk). Any ideas for troubleshooting would be great!

Things I've tried:
-Using higher number ports forwarded to SSH (2222, 522, 22222)
-Checking all SSH stuff is turned off on the router, as well as changing the port it would be set to if enabled. I see there's a section for SSH Port Forwarding, but I'm assuming that's specific to the router; a forwarded port shouldnt care what protocol is sent over it as I understand it.
-Disabling all of the options for firewalling, no difference, back to defaults


Thanks,
Cam

 

[RMerlin]

Check your list of forwards under System Log to ensure you don't have conflicting forwards. Beside that, I can't think of any technical explanation as to how a port forward could only randomly work.

 

[canadianpsyko]

Port Forward List:

Destination     Proto. Port range  Redirect to     Local port  Chain
ALL             TCP    80          192.168.11.5    80          VSERVER        
ALL             UDP    80          192.168.11.5    80          VSERVER        
ALL             TCP    22          192.168.11.5    22          VSERVER        
ALL             UDP    22          192.168.11.5    22          VSERVER        
ALL             UDP    52788       192.168.11.10   52788       VUPNP          
ALL             UDP    20530       192.168.11.10   20530       VUPNP          
ALL             TCP    20530       192.168.11.10   20530       VUPNP

Current DHCP:

Hostname                         IP Address      MAC Address       Expires  
Ipad                             192.168.11.205  74:e1:b6:cc:4c:47 21:47:16 
android-65faf4454350bc2e         192.168.11.61   bc:f5:ac:f8:4c:e1 23:29:16 
android-61c773dd8363c487         192.168.11.47   d8:50:e6:83:91:50 19:17:54 
Chromecast                       192.168.11.138  6c:ad:f8:1c:92:82 12:36:57 
CamDesk                          192.168.11.10   bc:ae:c5:74:7e:6a 14:08:58 
Cam-Serve                        192.168.11.5    00:1f:d0:ae:f9:b6 17:32:51 
HP-Print                         192.168.11.26   00:1a:4b:25:5e:39 18:22:35 
Cam-ServeVM                      192.168.11.6    00:1f:d0:ae:f9:b8 19:56:21

Manual Assignment:

BC:AE:C5:74:7E:6A	192.168.11.10	CamDesk	
00:1F:D0:AE:F9:B8	192.168.11.6	Cam-ServeVM	
00:1A:4B:25:5E:39	192.168.11.26	HP-Print	
00:1F:D0:AE:F9:B6	192.168.11.5	Cam-Serve

Nginx access logs show nothing when a time out happens. The SSH stuff I have less experience tracking down.

All of the ai-disk, etc stuff is disabled, no USB plugged into router.

 

[sinshiva]

does setting a static ip config on the server make a difference?

 

[Grump]

Check your nat table.

iptables -t nat -L -n -v

You should see something like

  910 643 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:22 to:192.168.0.2:22

The first 2 columns are the number of forwarded bytes. Try your SSH again, and if it fails, see if those numbers increment. If they do, run tcpdump on your nginx server and see if the packets are going there.

tcpdump -i any x.x.x.x

Replace x.x.x.x with the external IP. You should see the screen spit some packets if they're making it.

If they aren't hitting the nat rule, you can see if the packets are even reaching your router. You'll need entware or optware. I made a quick post here http://www.smallnetbuilder.com/forums/showthread.php?p=110094 on how to install it to /jffs if you don't have a spare usb stick.

ipkg install tcpdump
tcpdump -i eth0 host x.x.x.x

Replace x.x.x.x with the public IP of the remote host. When you try to connect, you should see it spew packets to the screen. If you don't, the problem is external to your router. (ie. your ISP or @ the remote host).

This should give you an idea where the problem is. Externally/at the router/at the nginx box.

 

[canadianpsyko]

Thanks for the ideas!
So I poked around, running tcpdump on both the router and the nginx box at the same time, the router is seeing requests, but the nginx box does not most of the time.

Example tcpdump output:

23:41:18.512894 IP <remote ip>.55093 > <local ip>.www: Flags [S], seq 1511498285, win 5840, options [mss 1460,sackOK,TS val 1718330463 ecr 0,nop,wscale 7], length 0
23:41:21.511635 IP <remote ip>.55093 > <local ip>.www: Flags [S], seq 1511498285, win 5840, options [mss 1460,sackOK,TS val 1718331213 ecr 0,nop,wscale 7], length 0
23:41:27.509977 IP <remote ip>.55093 > <local ip>.www: Flags [S], seq 1511498285, win 5840, options [mss 1460,sackOK,TS val 1718332713 ecr 0,nop,wscale 7], length 0

Or for the more verbose version:

23:48:41.831128 IP (tos 0x0, ttl 56, id 2467, offset 0, flags [DF], proto TCP (6), length 60)
    <remote ip>.55104 > <local ip>.www: Flags [S], cksum 0x9be0 (correct), seq 3728825582, win 5840, options [mss 1460,sackOK,TS val 1718441293 ecr 0,nop,wscale 7], length 0
23:48:44.836073 IP (tos 0x0, ttl 56, id 2468, offset 0, flags [DF], proto TCP (6), length 60)
    <remote ip>.55104 > <local ip>.www: Flags [S], cksum 0x98f2 (correct), seq 3728825582, win 5840, options [mss 1460,sackOK,TS val 1718442043 ecr 0,nop,wscale 7], length 0
23:48:50.825617 IP (tos 0x0, ttl 56, id 2469, offset 0, flags [DF], proto TCP (6), length 60)
    <remote ip>.55104 > <local ip>.www: Flags [S], cksum 0x9316 (correct), seq 3728825582, win 5840, options [mss 1460,sackOK,TS val 1718443543 ecr 0,nop,wscale 7], length 0

For one of the times it randomly works:

23:50:35.996853 IP (tos 0x0, ttl 56, id 64083, offset 0, flags [DF], proto TCP (6), length 60)
    <remote ip>.55114 > <local ip>.www: Flags [S], cksum 0x8290 (correct), seq 2791259800, win 5840, options [mss 1460,sackOK,TS val 1718469835 ecr 0,nop,wscale 7], length 0
23:50:36.017766 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    <local ip>.www > <remote ip>.55114: Flags [S.], cksum 0x61ba (correct), seq 3988300410, ack 2791259801, win 28960, options [mss 1460,sackOK,TS val 2341776044 ecr 1718469835,nop,wscale 7], length 0
23:50:36.056328 IP (tos 0x0, ttl 56, id 64084, offset 0, flags [DF], proto TCP (6), length 52)
    <remote ip>.55114 > <local ip>.www: Flags [.], cksum 0x016a (correct), seq 1, ack 1, win 46, options [nop,nop,TS val 1718469850 ecr 2341776044], length 0
23:50:36.057943 IP (tos 0x0, ttl 56, id 64085, offset 0, flags [DF], proto TCP (6), length 220)
    <remote ip>.55114 > <local ip>.www: Flags [P.], cksum 0x5ebd (correct), seq 1:169, ack 1, win 46, options [nop,nop,TS val 1718469850 ecr 2341776044], length 168
23:50:36.058350 IP (tos 0x0, ttl 63, id 2216, offset 0, flags [DF], proto TCP (6), length 52)
    <local ip>.www > <remote ip>.55114: Flags [.], cksum 0xffdb (correct), seq 1, ack 169, win 235, options [nop,nop,TS val 2341776085 ecr 1718469850], length 0
23:50:36.058509 IP (tos 0x0, ttl 63, id 2217, offset 0, flags [DF], proto TCP (6), length 373)
    <local ip>.www > <remote ip>.55114: Flags [P.], cksum 0xb450 (correct), seq 1:322, ack 169, win 235, options [nop,nop,TS val 2341776085 ecr 1718469850], length 321
23:50:36.095065 IP (tos 0x0, ttl 56, id 64086, offset 0, flags [DF], proto TCP (6), length 52)
    <remote ip>.55114 > <local ip>.www: Flags [.], cksum 0xff46 (correct), seq 169, ack 322, win 54, options [nop,nop,TS val 1718469859 ecr 2341776085], length 0
23:50:36.097725 IP (tos 0x0, ttl 56, id 64087, offset 0, flags [DF], proto TCP (6), length 52)
    <remote ip>.55114 > <local ip>.www: Flags [F.], cksum 0xff44 (correct), seq 169, ack 322, win 54, options [nop,nop,TS val 1718469860 ecr 2341776085], length 0
23:50:36.098112 IP (tos 0x0, ttl 63, id 2218, offset 0, flags [DF], proto TCP (6), length 52)
    <local ip>.www > <remote ip>.55114: Flags [F.], cksum 0xfe66 (correct), seq 322, ack 170, win 235, options [nop,nop,TS val 2341776125 ecr 1718469860], length 0
23:50:36.139525 IP (tos 0x0, ttl 56, id 64088, offset 0, flags [DF], proto TCP (6), length 52)
    <remote ip>.55114 > <local ip>.www: Flags [.], cksum 0xff10 (correct), seq 170, ack 323, win 54, options [nop,nop,TS val 1718469871 ecr 2341776125], length 0
23:50:36.145285 IP (tos 0x10, ttl 56, id 19724, offset 0, flags [DF], proto TCP (6), length 540)

With matching nginx box output:

00:50:36.065088 IP (tos 0x0, ttl 55, id 64083, offset 0, flags [DF], proto TCP (6), length 60)
    <remote ip>.55114 > 192.168.11.5.http: Flags [S], cksum 0xa193 (correct), seq 2791259800, win 5840, options [mss 1460,sackOK,TS val 1718469835 ecr 0,nop,wscale 7], length 0
00:50:36.065127 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    192.168.11.5.http > <remote ip>.55114: Flags [S.], cksum 0x80bd (correct), seq 3988300410, ack 2791259801, win 28960, options [mss 1460,sackOK,TS val 2341776044 ecr 1718469835,nop,wscale 7], length 0
00:50:36.104151 IP (tos 0x0, ttl 55, id 64084, offset 0, flags [DF], proto TCP (6), length 52)
    <remote ip>.55114 > 192.168.11.5.http: Flags [.], cksum 0x206d (correct), seq 1, ack 1, win 46, options [nop,nop,TS val 1718469850 ecr 2341776044], length 0
00:50:36.105957 IP (tos 0x0, ttl 55, id 64085, offset 0, flags [DF], proto TCP (6), length 220)
    <remote ip>.55114 > 192.168.11.5.http: Flags [P.], cksum 0x7dc0 (correct), seq 1:169, ack 1, win 46, options [nop,nop,TS val 1718469850 ecr 2341776044], length 168
00:50:36.105989 IP (tos 0x0, ttl 64, id 2216, offset 0, flags [DF], proto TCP (6), length 52)
    192.168.11.5.http > <remote ip>.55114: Flags [.], cksum 0x1edf (correct), seq 1, ack 169, win 235, options [nop,nop,TS val 2341776085 ecr 1718469850], length 0
00:50:36.106142 IP (tos 0x0, ttl 64, id 2217, offset 0, flags [DF], proto TCP (6), length 373)
    192.168.11.5.http > <remote ip>.55114: Flags [P.], cksum 0xd353 (correct), seq 1:322, ack 169, win 235, options [nop,nop,TS val 2341776085 ecr 1718469850], length 321
00:50:36.142915 IP (tos 0x0, ttl 55, id 64086, offset 0, flags [DF], proto TCP (6), length 52)
    <remote ip>.55114 > 192.168.11.5.http: Flags [.], cksum 0x1e4a (correct), seq 169, ack 322, win 54, options [nop,nop,TS val 1718469859 ecr 2341776085], length 0
00:50:36.145549 IP (tos 0x0, ttl 55, id 64087, offset 0, flags [DF], proto TCP (6), length 52)
    <remote ip>.55114 > 192.168.11.5.http: Flags [F.], cksum 0x1e48 (correct), seq 169, ack 322, win 54, options [nop,nop,TS val 1718469860 ecr 2341776085], length 0
00:50:36.145681 IP (tos 0x0, ttl 64, id 2218, offset 0, flags [DF], proto TCP (6), length 52)
    192.168.11.5.http > <remote ip>.55114: Flags [F.], cksum 0x1d6a (correct), seq 322, ack 170, win 235, options [nop,nop,TS val 2341776125 ecr 1718469860], length 0
00:50:36.187325 IP (tos 0x0, ttl 55, id 64088, offset 0, flags [DF], proto TCP (6), length 52)
    <remote ip>.55114 > 192.168.11.5.http: Flags [.], cksum 0x1e14 (correct), seq 170, ack 323, win 54, options [nop,nop,TS val 1718469871 ecr 2341776125], length 0

I did the test by:

watch -n 10 "curl <ip>"

It confirms that every attempt made remotely hit the router. Another thing I noticed, UPNP for torrents on my desktop work perfectly.

 

[Grump]

It sounds like an IP/arp conflict on the network. Did the iptables numbers increment?

 

[canadianpsyko]

So it seems like I fixed it, I only have 2 places I can bang on it remotely from home, but I'm getting completely reliable connection now.
It seems like some old TAP/TUN stuff I had setup for playing with VM's a couple years ago wasn't playing nice. Disabled it all, both interfaces just plain DHCP now.

I'm not sure why it only showed up after I got this router, but it's not a result of the router, or the firmware.

Thanks for all the help!