Solved Custom static secondary route on WAN interface

[ColinTaylor]

arp -an ought to show an entry for 192.168.10.1 on the appropriate interface (but I guess it doesn't for the OP).

As you say, perhaps it's a VLAN issue.

 

[HELLO_wORLD]

@ColinTaylor and @R. Gerrits

Thank you for you interest/curiosity in that.

My setup is router directly connected to internet, static IP (no modem):

ONT — (brwan) — ROUTER — (br0 ) — LAN
       Static IP          Static IP/DHCP server for LAN

The ONT is the device I try to communicate with via 192.168.10.1
It is not a router, not a modem, just a fiber <=> ethernet converter.

arping times out to.

It is getting late for me here...

 

[HELLO_wORLD]

Ok, it gets stranger...
I created iptables rules, just to be sure nothing is blocking, and to log what is going on, in filter table (INPUT, OUTPUT), nat table (PREROUTING, INPUT, OUTPUT and POSTROUTING) and mangle table (PREROUTING and POSTROUTING)

iptables logs activity (TCP and ICMP) requests cascading in the chains/tables.
tcpdump logs just this ARP activity and nothing else...
I still get no route to host, so it is like when the packet leaves iptables, it never is sent by the interface brwan!?

Here is the outputs of tcpdump and the iptables log:

1611250532.163766 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.10.1 tell 192.168.10.2, length 28
1611250533.162828 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.10.1 tell 192.168.10.2, length 28
1611250534.162828 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.10.1 tell 192.168.10.2, length 28
1611250535.225028 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.10.1 tell 192.168.10.2, length 28
1611250536.222841 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.10.1 tell 192.168.10.2, length 28
1611250537.252863 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.10.1 tell 192.168.10.2, length 28

181034:[ONT nat OUTPUT SRC] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=107 DF PROTO=TCP SPT=50988 DPT=80 WINDOW=14600 RES=0x00 SYN URGP=0 
181034:[ONT nat OUTPUT DST] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=107 DF PROTO=TCP SPT=50988 DPT=80 WINDOW=14600 RES=0x00 SYN URGP=0 
181034:[ONT filter OUTPUT SRC] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=107 DF PROTO=TCP SPT=50988 DPT=80 WINDOW=14600 RES=0x00 SYN URGP=0 
181034:[ONT filter OUTPUT DST] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=107 DF PROTO=TCP SPT=50988 DPT=80 WINDOW=14600 RES=0x00 SYN URGP=0 
181034:[ONT mangle POSTROUTING SRC] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=107 DF PROTO=TCP SPT=50988 DPT=80 WINDOW=14600 RES=0x00 SYN URGP=0 
181034:[ONT mangle POSTROUTING DST] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=107 DF PROTO=TCP SPT=50988 DPT=80 WINDOW=14600 RES=0x00 SYN URGP=0 
181034:[ONT nat POSTROUTING SRC] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=107 DF PROTO=TCP SPT=50988 DPT=80 WINDOW=14600 RES=0x00 SYN URGP=0 
181034:[ONT nat POSTROUTING DST] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=107 DF PROTO=TCP SPT=50988 DPT=80 WINDOW=14600 RES=0x00 SYN URGP=0 
181035:[ONT filter OUTPUT SRC] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=108 DF PROTO=TCP SPT=50988 DPT=80 WINDOW=14600 RES=0x00 SYN URGP=0 
181035:[ONT filter OUTPUT DST] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=108 DF PROTO=TCP SPT=50988 DPT=80 WINDOW=14600 RES=0x00 SYN URGP=0 
181035:[ONT mangle POSTROUTING SRC] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=108 DF PROTO=TCP SPT=50988 DPT=80 WINDOW=14600 RES=0x00 SYN URGP=0 
181035:[ONT mangle POSTROUTING DST] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=108 DF PROTO=TCP SPT=50988 DPT=80 WINDOW=14600 RES=0x00 SYN URGP=0 
181037:[ONT filter OUTPUT SRC] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=109 DF PROTO=TCP SPT=50988 DPT=80 WINDOW=14600 RES=0x00 SYN URGP=0 
181037:[ONT filter OUTPUT DST] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=109 DF PROTO=TCP SPT=50988 DPT=80 WINDOW=14600 RES=0x00 SYN URGP=0 
181037:[ONT mangle POSTROUTING SRC] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=109 DF PROTO=TCP SPT=50988 DPT=80 WINDOW=14600 RES=0x00 SYN URGP=0 
181037:[ONT mangle POSTROUTING DST] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=109 DF PROTO=TCP SPT=50988 DPT=80 WINDOW=14600 RES=0x00 SYN URGP=0 
181037:[ONT nat OUTPUT SRC] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=8320 DF PROTO=ICMP TYPE=8 CODE=0 ID=62530 SEQ=0 
181037:[ONT nat OUTPUT DST] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=8320 DF PROTO=ICMP TYPE=8 CODE=0 ID=62530 SEQ=0 
181037:[ONT filter OUTPUT SRC] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=8320 DF PROTO=ICMP TYPE=8 CODE=0 ID=62530 SEQ=0 
181037:[ONT filter OUTPUT DST] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=8320 DF PROTO=ICMP TYPE=8 CODE=0 ID=62530 SEQ=0 
181037:[ONT mangle POSTROUTING SRC] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=8320 DF PROTO=ICMP TYPE=8 CODE=0 ID=62530 SEQ=0 
181037:[ONT mangle POSTROUTING DST] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=8320 DF PROTO=ICMP TYPE=8 CODE=0 ID=62530 SEQ=0 
181037:[ONT nat POSTROUTING SRC] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=8320 DF PROTO=ICMP TYPE=8 CODE=0 ID=62530 SEQ=0 
181037:[ONT nat POSTROUTING DST] IN= OUT=brwan SRC=192.168.10.2 DST=192.168.10.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=8320 DF PROTO=ICMP TYPE=8 CODE=0 ID=62530 SEQ=0

And arp shows this:

IP address       HW type     Flags       HW address            Mask     Device
192.168.10.1    0x1         0x0         00:00:00:00:00:00     *        brwan

 

[ColinTaylor]

It looks like the device at 192.168.10.1 is simply not responding. Are you sure it is actually reachable? If you unplug your router from the ONT and plug in your PC instead can you get to 192.168.10.1?

 

[R. Gerrits]

If the ONT does not respond to the ARP request, then the router will not know what its MAC-address is.
And without a destination MAC-address no network traffic can be sent.

So either the IP address of the ONT is something else than 192.168.10.1, or the ONT interface that has that IP is in a different VLAN. (or something else is wrong.)

 

[HELLO_wORLD]

Thank you.

The key is probably ARP related, and that explains why tcpdump does not catch anything! Brwan does not know where to send the packets, so it does not send anything. Good catch! Thanks to both of you!

I will have to try the ONT on a PC... not simple since it cuts internet, and I have no laptop with Ethernet. I would have to unplug the ONT and bring it to my tower, or setup a Raspberry pi and bring a monitor where the ONT is...

 

[HELLO_wORLD]

Update, to close this:
I did not need to test from a PC. The ONT did not have the expected IP.
Found the right one, and it works like a charm.

Just one command ip addr add to add the subnet and route to brwan, and one iptables SNAT rule to be able to access from LAN.

 

[BritinMA]

May I interject and ask what two (?) commands you used; This seems remarkably similar to an "annoyance" I have.

MODEM (192.168.100.1) --> R9000 --> LAN (192.168.14.0)

LAN can ping 192.168.100.1 but cannot get to web interface.

Thanks & sorry if this interjection is against forum rules.

 

[HELLO_wORLD]

May I interject and ask what two (?) commands you used; This seems remarkably similar to an "annoyance" I have.

MODEM (192.168.100.1) --> R9000 --> LAN (192.168.14.0)

LAN can ping 192.168.100.1 but cannot get to web interface.

Thanks & sorry if this interjection is against forum rules.

I used ip addr add ROUTER ADDRESS/24 dev brwan
And iptables -t nat -I POSTROUTING -o brwan -d MODEM ADDRESS/24 -j SNAT --to ROUTER ADDRESS

But I don’t think it is relevant in your case.
You do have a working route since ping is working. It is more like a filtering rule...
It also depends on your setup (is your router connected as a DHCP client to your modem, or is it bridged?). What is your brwan main IP and subnet?

But you could try with 192.168.100.1 as ROUTER ADDRESS and 192.168.100.2 as MODEM ADDRESS.

 

[BritinMA]

I used ip addr add ROUTER ADDRESS/24 dev brwan
And iptables -t nat -I POSTROUTING -o brwan -d MODEM ADDRESS/24 -j SNAT --to ROUTER ADDRESS

But I don’t think it is relevant in your case.
You do have a working route since ping is working. It is more like a filtering rule...
It also depends on your setup (is your router connected as a DHCP client to your modem, or is it bridged?). What is your brwan main IP and subnet?

But you could try with 192.168.100.1 as ROUTER ADDRESS and 192.168.100.2 as MODEM ADDRESS.

I have comcast; While booting the modem gives the R9000 an IP of 192.168.100.2, and once everything is sync'd, the router gets the 'real' IP. So, as of right now, my R9000's brwan is a real IP address.

brwan     Link encap:Ethernet  HWaddr A0:04:60:**:**:** 
          inet addr:24.**.***.**  Bcast:255.255.255.255  Mask:255.255.252.0

I am happy to start a new thread, if that should be better.

 

[HELLO_wORLD]

I have comcast; While booting the modem gives the R9000 an IP of 192.168.100.2, and once everything is sync'd, the router gets the 'real' IP. So, as of right now, my R9000's brwan is a real IP address.

brwan     Link encap:Ethernet  HWaddr A0:04:60:**:**:**
          inet addr:24.**.***.**  Bcast:255.255.255.255  Mask:255.255.252.0

I am happy to start a new thread, if that should be better.

So you seem to be exactly in the same situation as @R. Gerrits
like he mentioned earlier in this thread:
http://www.snbforums.com/threads/custom-static-secondary-route-on-wan-interface.69451/post-653712

So for you, once your router has its definitive IP, that should do it:

ip addr add 192.168.100.2/24 dev brwan
iptables -t nat -I POSTROUTING -o brwan -d 192.168.100.1/24 -j SNAT --to 192.168.100.2

And if you use Aegis, make sure you whitelist 192.168.100.1

 

[BritinMA]

So you seem to be exactly in the same situation as @R. Gerrits
like he mentioned earlier in this thread:
http://www.snbforums.com/threads/custom-static-secondary-route-on-wan-interface.69451/post-653712

So for you, once your router has its definitive IP, that should do it:

ip addr add 192.168.100.2/24 dev brwan
iptables -t nat -I POSTROUTING -o brwan -d 192.168.100.1/24 -j SNAT --to 192.168.100.2

And if you use Aegis, make sure you whitelist 192.168.100.1

Worked right away - Perfect - thank you!