Solved Custom static secondary route on WAN interface

[HELLO_wORLD]

I have a technical question for experts like @R. Gerrits

Let’s say I am connected to the WAN with a static subnet 2.0.0.100/24
Let’s say I have have a device on the WAN side of the router, that has for IP 192.168.10.1 is only accessible (ping and http on port 80) from the subnet 192.168.10.0/24

I want to be able to access it from the router (and ideally from my LAN).

Here is what I do:
ip addr add 192.168.10.2/24 dev brwan
That adds the secondary subnet 192.168.10.0/24 to my brwan interface and assigns to it the IP 192.168.10.2
It also adds a route for it

## ADDR
14: brwan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
    inet 2.0.0.100/22 brd 2.0.0.255 scope global brwan
    inet 192.168.10.2/24 scope global brwan

## ROUTE
default via 2.0.0.254 dev brwan
2.0.0.0/22 dev brwan scope link  src 2.0.0.100
192.168.0.0/24 dev br0 scope link  src 192.168.0.1
192.168.10.0/24 dev brwan scope link  src 192.168.10.2

using ping or wget -o- 192.168.10.1, from the router, I get no route to host, even with --bind-address 192.168.10.2

Then, I add this iptables rule:
iptables -t nat -I POSTROUTING -o brwan -d 192.168.10.0/24 -j SNAT --to 192.168.10.3

And using ping or wget -o- 192.168.10.1 now results in a timeout (from LAN as well).
I am not sure the device is answering (timeout might be normal), so I just want to know if the rules I use should work or not or if I am forgetting a step?

 

[ColinTaylor]

Then, I add this iptables rule:
iptables -t nat -I POSTROUTING -o brwan -d 192.168.10.0/24 -j SNAT --to 192.168.10.3

Is that a typo? Presumably that ought to be:

iptables -t nat -I POSTROUTING -o brwan -d 192.168.10.0/24 -j SNAT --to 192.168.10.2

or alternatively:

iptables -t nat -I POSTROUTING -o brwan -d 192.168.10.0/24 -j MASQUERADE

Is this also a typo? Shouldn't that be 10 instead of 100?

## ROUTE
default via 2.0.0.254 dev brwan
2.0.0.0/22 dev brwan scope link src 2.0.0.100
192.168.0.0/24 dev br0 scope link src 192.168.0.1
192.168.100.0/24 dev brwan scope link src 192.168.100.2

 

[HELLO_wORLD]

Is that a typo? Presumably that ought to be:

iptables -t nat -I POSTROUTING -o brwan -d 192.168.10.0/24 -j SNAT --to 192.168.10.2

or alternatively:

iptables -t nat -I POSTROUTING -o brwan -d 192.168.10.0/24 -j MASQUERADE

Not, not a typo. With the .2 in SNAT -> no route to host
With .3 -> timeout

And I have tried masquerade as well -> no route to host

 

[ColinTaylor]

Did you see my edit above? Your route seems to be wrong:

192.168.100.0/24 dev brwan scope link src 192.168.100.2

 

[HELLO_wORLD]

Did you see my edit above? Your route seems to be wrong:

192.168.100.0/24 dev brwan scope link src 192.168.100.2

My mistake. That is a typo in my initial post. It is 192.168.10.

And results as I wrote. Strange that masquerade and SNAT are not giving the same result...

 

[HELLO_wORLD]

Also, for info,
When using
iptables -t nat -I POSTROUTING -o brwan -d 192.168.10.0/24 -j SNAT --to 192.168.10.2
Or masquerade
The no route to host is from router and LAN

With 192.168.10.3
It is timeout from router and LAN

The fact that the rule with masquerade or 192.168.10.2 says no route to host lead me to believe that I am missing a step...

 

[ColinTaylor]

The ".2" address should be correct which is why MASQUERADE produces the same error.

I suspect the issue might be with how you created the secondary interface, but I can't be sure because I don't have a Netgear router. You also appear to be editing some of the output so I don't know what you've removed.

The 192.168.10.2/24 interface doesn't appear to have a broadcast address. How did you create it?

 

[HELLO_wORLD]

The ".2" address should be correct which is why MASQUERADE produces the same error.

I suspect the issue might be with how you created the secondary interface, but I can't be sure because I don't have a Netgear router. You also appear to be editing some of the output so I don't know what you've removed.

The 192.168.10.2/24 interface doesn't appear to have a broadcast address. How did you create it?

Just: ip addr add 192.168.10.2/24 dev brwan

And yes, there is no broadcast.

This command creates the route for it automatically (the one described in first post).

 

[ColinTaylor]

OK, I'll have to leave this to other people that have experience with Netgear routers.

My final idea would be to try this method of creating the interface instead (note that the interface name is brwan:0 )

ifconfig brwan:0 192.168.10.2 netmask 255.255.255.0

EDIT: I don't know whether using this command would also require a change in the MASQUERADE rule (I'm just guessing here), i.e.

iptables -t nat -I POSTROUTING -o brwan:0 -j MASQUERADE

 

[HELLO_wORLD]

OK, I'll have to leave this to other people that have experience with Netgear routers.

My final idea would be to try this method of creating the interface instead (note that the interface name is brwan:0 )

ifconfig brwan:0 192.168.10.2 netmask 255.255.255.0

EDIT: I don't know whether using this command would also require a change in the MASQUERADE rule (I'm just guessing here), i.e.

iptables -t nat -I POSTROUTING -o brwan:0 -j MASQUERADE

Thanks for you help.

I will try the old fashion ifconfig alias.

 

[ColinTaylor]

I will try the old fashion ifconfig alias.

It occurs to me (rather belatedly ) that as you're testing this from the router itself rather than the LAN you don't need the iptables rule at all. So it might be worth removing that completely when testing. Sorry, I didn't pick up on that when you mentioned it in your first post.

 

[HELLO_wORLD]

It occurs to me (rather belatedly ) that as you're testing this from the router itself rather than the LAN you don't need the iptables rule at all. So it might be worth removing that completely when testing. Sorry, I didn't pick up on that when you mentioned it in your first post.

Well, same result:

root@HERMES:~$ ifconfig brwan:0 192.168.10.2 netmask 255.255.255.0    
root@HERMES:~$ wget -o- 192.168.10.1
--2021-01-20 13:25:33--  http://192.168.10.1/
Connecting to 192.168.10.1:80... failed: No route to host.
root@HERMES:~$ wget -o- 192.168.10.1 --bind-address 192.168.10.2
--2021-01-20 13:26:14--  http://192.168.10.1/
Connecting to 192.168.10.1:80... failed: No route to host.

I checked and the ifconfig command does create the route:
192.168.10.0/24 dev brwan scope link src 192.168.10.2

But for some reason, the route is not found...

 

[ColinTaylor]

But for some reason, the route is not found...

That is strange.

EDIT:
I think the "No route to host" from wget is misleading. I don't think it means the routing table is wrong. Try using ping 192.168.10.1 and you'll probably get "Destination Host Unreachable". There should be no routing involved because 192.168.10.1 and 192.168.10.2 are connected to the same physical network.

What do you get for ip route get 192.168.10.1

 

[HELLO_wORLD]

That is strange.

EDIT:
I think the "No route to host" from wget is misleading. I don't think it means the routing table is wrong. Try using ping 192.168.10.1 and you'll probably get "Destination Host Unreachable". There should be no routing involved because 192.168.10.1 and 192.168.10.2 are connected to the same physical network.

What do you get for ip route get 192.168.10.1

root@HERMES:~$ ip route get 192.168.10.1
192.168.10.1 dev brwan  src 192.168.10.2 
root@HERMES:~$ ping 192.168.10.1
PING 192.168.10.1 (192.168.10.1): 56 data bytes
^C
--- 192.168.10.1 ping statistics ---
27 packets transmitted, 0 packets received, 100% packet loss
root@HERMES:~$ traceroute 192.168.10.1
traceroute to 192.168.10.1 (192.168.10.1), 30 hops max, 38 byte packets
 1  192.168.10.2 (192.168.10.2)  2999.969 ms !H  3001.125 ms !H  2999.843 ms !H
root@HERMES:~$ wget -o- 192.168.10.1
--2021-01-20 15:21:35--  http://192.168.10.1/
Connecting to 192.168.10.1:80... failed: No route to host.
root@HERMES:~$ wget -o- 192.168.10.2
--2021-01-20 15:21:44--  http://192.168.10.2/
Connecting to 192.168.10.2:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2021-01-20 15:21:44 ERROR 404: Not Found.
root@HERMES:~$ wget -o- 2.0.0.2
--2021-01-20 15:22:24--  http://2.0.0.2/
Connecting to 2.0.0.2:80... ^C
root@HERMES:~$

The ping results in timeout, I had to interrupt.

When I use wget on a destination having a route (through my main subnet), but pointing to a non existing server, I get a timeout.
When I use wget on the secondary IP of my wan interface, I get 404 (router uhttpd response).
When I use wget on 192.168.10.1, I get no route to host (where I should have a timeout or an answer if the device is answering.

Traceroute first hop is 192.168.10.2, so it means hop 0 is not 192.169.10.2, and that might be the problem, because then the starting IP is either br0 or brwan primary IP, when I want it to be 192.168.10.2. So it would after all requires some masquerading or SNAT, but better if I could initiate the requests (wget, ping...) from 192.168.10.2

 

[R. Gerrits]

Are you sure that that device has a firewall which only accepts traffic from a particular range?

I have a similar situation, where my cablemodem is in bridgemode, but still has ip 192.168.100.1

If I add an additional route via ip route add 192.168.100.1 dev brwan scope link
then I can just access it, (with 192.168.100.1 being in aegis white-list).

(just to be sure, you did add the ip-addresses also to aegis whitelist ??)

With regard to all your firewall rules:
If you added an interface with IP-address 192.168.10.2 to your router AND also do your test on the router,
there is no NATting needed, as the destination IP-address is local to the route.
And also, as far as I can see, there normally should also not be any rules additional firewall rules required.
(if you also need access from LAN, then indeed you might need to do some masquarding.)

 

[HELLO_wORLD]

Are you sure that that device has a firewall which only accepts traffic from a particular range?

I have a similar situation, where my cablemodem is in bridgemode, but still has ip 192.168.100.1

If I add an additional route via ip route add 192.168.100.1 dev brwan scope link
then I can just access it, (with 192.168.100.1 being in aegis white-list).

(just to be sure, you did add the ip-addresses also to aegis whitelist ??)

With regard to all your firewall rules:
If you added an interface with IP-address 192.168.10.2 to your router AND also do your test on the router,
there is no NATting needed, as the destination IP-address is local to the route.
And also, as far as I can see, there normally should also not be any rules additional firewall rules required.
(if you also need access from LAN, then indeed you might need to do some masquarding.)

Yes I whitelisted the IP in aegis, and tried aegis totally down.

The device specifications is it has its own IP ending by .1 (here 192.168.10.1) and its management interface is accessible only from devices with their IP in the same subnet /24.
I will have no choice but to try from another device directly on Ethernet to see if it answers or not, but the ‘no route to host’ coming after one second is puzzling me because it should simply stall then timeout if there is no answer.

 

[R. Gerrits]

what about doing a tcpdump during the test? perhaps you can then at least see what is happening / which mac-addresses are responding. Perhaps that sheds some light.

 

[HELLO_wORLD]

There is definitely a problem:
secondary IP and /24 subnet added to brwan
Started tcpdump, sleep 1 then doing a wget and a ping to 192.168.10.1
All that is recorded is ARP from 192.168.10.1 to 192.168.10.2, nothing else:

tcpdump: listening on brwan, link-type EN10MB (Ethernet), capture size 262144 bytes
20:56:32.549594 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.10.1 tell 192.168.10.2, length 28
        0x0000:  0001 0800 0604 0001 bca5 1131 f7ae c0a8  ...........1....
        0x0010:  6402 0000 0000 0000 c0a8 6401            d.........d.
20:56:33.549594 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.10.1 tell 192.168.10.2, length 28
        0x0000:  0001 0800 0604 0001 bca5 1131 f7ae c0a8  ...........1....
        0x0010:  6402 0000 0000 0000 c0a8 6401            d.........d.
20:56:34.549781 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.10.1 tell 192.168.10.2, length 28
        0x0000:  0001 0800 0604 0001 bca5 1131 f7ae c0a8  ...........1....
        0x0010:  6402 0000 0000 0000 c0a8 6401            d.........d.
20:56:35.549594 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.10.1 tell 192.168.10.2, length 28
        0x0000:  0001 0800 0604 0001 bca5 1131 f7ae c0a8  ...........1....
        0x0010:  6402 0000 0000 0000 c0a8 6401            d.........d.
20:56:36.739628 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.10.1 tell 192.168.10.2, length 28
        0x0000:  0001 0800 0604 0001 bca5 1131 f7ae c0a8  ...........1....
        0x0010:  6402 0000 0000 0000 c0a8 6401            d.........d.

5 packets captured
5 packets received by filter
0 packets dropped by kernel

tcpdump command is: tcpdump -vvnnSX -i brwan net 192.168.10.0/24

No trace of the ping or the wget!!?

 

[R. Gerrits]

tried a similar thing at my end: add 192.168.100.2 as a secondary ip to brwan, started a tcpdump and did a ping from the router to 192.168.100.1.
It still uses the primary public IP instead of the secondary private IP.

But I don't see any arp request does come from the private IP (and gets a successfull response).

The strange things is that at your end, apparently even an arp request doesn't get answer.
If you have your Router, your Modem and a the device with that private IP, how did you then connect things? I'm guessing there must be a switch in between? Are you sure all the devices are then in the same vlan?

root@R7800:~$ tcpdump -vvnnSX -i brwan net 192.168.100.0/24
tcpdump: listening on brwan, link-type EN10MB (Ethernet), capture size 96 bytes
22:18:55.911791 IP (tos 0x0, ttl 64, id 7345, offset 0, flags [DF], proto ICMP (1), length 84) 94.x.x.x > 192.168.100.1: ICMP echo request, id 24656, seq 0, length 64
    0x0000:  4500 0054 1cb1 4000 4001 8312 5ed5 1767  E..T..@.@...^..g
    0x0010:  c0a8 6401 0800 d377 6050 0000 11b7 b280  ..d....w`P......
    0x0020:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0030:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0040:  0000 0000 0000 0000 0000 0000            ............
22:18:55.917102 IP (tos 0x0, ttl 64, id 1604, offset 0, flags [none], proto ICMP (1), length 84) 192.168.100.1 > 94.x.x.x: ICMP echo reply, id 24656, seq 0, length 64
    0x0000:  4500 0054 0644 0000 4001 d97f c0a8 6401  E..T.D..@.....d.
    0x0010:  5ed5 1767 0000 db77 6050 0000 11b7 b280  ^..g...w`P......
    0x0020:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0030:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0040:  0000 0000 0000 0000 0000 0000            ............
22:19:00.919757 arp who-has 192.168.100.1 tell 192.168.100.2
    0x0000:  0001 0800 0604 0001 cc40 d049 8806 c0a8  .........@.I....
    0x0010:  6402 0000 0000 0000 c0a8 6401            d.........d.
22:19:00.923756 arp reply 192.168.100.1 is-at 18:35:d1:74:82:02
    0x0000:  0001 0800 0604 0002 1835 d174 8202 c0a8  .........5.t....
    0x0010:  6401 cc40 d049 8806 c0a8 6402 0000 0000  d..@.I....d.....
    0x0020:  0000 0000 0000 0000 0000 841d c120       ..............

 

[R. Gerrits]

some other test:

root@R7800:~$ traceroute -s 192.168.100.2 192.168.100.1
traceroute to 192.168.100.1 (192.168.100.1) from 192.168.100.2, 30 hops max, 38 byte packets
1 192.168.100.1 (192.168.100.1) 12.528 ms 2.780 ms 6.405 ms
root@R7800:~$

So then it does use the correct IP and reaches the device on the first hop.

tcpdump: listening on brwan, link-type EN10MB (Ethernet), capture size 96 bytes
22:48:32.902638 arp who-has 192.168.100.1 tell 192.168.100.2
    0x0000:  0001 0800 0604 0001 cc40 d049 8806 c0a8  .........@.I....
    0x0010:  6402 0000 0000 0000 c0a8 6401            d.........d.
22:48:32.908417 arp reply 192.168.100.1 is-at 18:35:d1:74:82:02
    0x0000:  0001 0800 0604 0002 1835 d174 8202 c0a8  .........5.t....
    0x0010:  6401 cc40 d049 8806 c0a8 6402 0000 0000  d..@.I....d.....
    0x0020:  0000 0000 0000 0000 0000 841d c120       ..............
22:48:32.908448 IP (tos 0x0, ttl 1, id 39706, offset 0, flags [DF], proto UDP (17), length 38) 192.168.100.2.54752 > 192.168.100.1.33435: [bad udp cksum 8013!] UDP, length 10
    0x0000:  4500 0026 9b1a 4000 0111 9558 c0a8 6402  E..&..@....X..d.
    0x0010:  c0a8 6401 d5e0 829b 0012 4978 0101 0000  ..d.......Ix....
    0x0020:  0000 0000 0000                           ......
22:48:32.915165 IP (tos 0xc0, ttl 64, id 34587, offset 0, flags [none], proto ICMP (1), length 66) 192.168.100.1 > 192.168.100.2: ICMP 192.168.100.1 udp port 33435 unreachable, length 46
    IP (tos 0x0, ttl 1, id 39706, offset 0, flags [DF], proto UDP (17), length 38) 192.168.100.2.54752 > 192.168.100.1.33435: [udp sum ok] UDP, length 10
    0x0000:  45c0 0042 871b 0000 4001 a98b c0a8 6401  E..B....@.....d.
    0x0010:  c0a8 6402 0303 4675 0000 0000 4500 0026  ..d...Fu....E..&
    0x0020:  9b1a 4000 0111 9558 c0a8 6402 c0a8 6401  ..@....X..d...d.
    0x0030:  d5e0 829b 0012 5cf8 0101 0000 0000 0000  ......\.........
    0x0040:  0000                                     ..
22:48:32.931254 IP (tos 0x0, ttl 1, id 25918, offset 0, flags [DF], proto UDP (17), length 38) 192.168.100.2.54752 > 192.168.100.1.33436: [bad udp cksum 7f12!] UDP, length 10
    0x0000:  4500 0026 653e 4000 0111 cb34 c0a8 6402  E..&e>@....4..d.
    0x0010:  c0a8 6401 d5e0 829c 0012 4978 0201 0000  ..d.......Ix....
    0x0020:  0000 0000 0000                           ......
22:48:32.934003 IP (tos 0xc0, ttl 64, id 34588, offset 0, flags [none], proto ICMP (1), length 66) 192.168.100.1 > 192.168.100.2: ICMP 192.168.100.1 udp port 33436 unreachable, length 46
    IP (tos 0x0, ttl 1, id 25918, offset 0, flags [DF], proto UDP (17), length 38) 192.168.100.2.54752 > 192.168.100.1.33436: [udp sum ok] UDP, length 10
    0x0000:  45c0 0042 871c 0000 4001 a98a c0a8 6401  E..B....@.....d.
    0x0010:  c0a8 6402 0303 4675 0000 0000 4500 0026  ..d...Fu....E..&
    0x0020:  653e 4000 0111 cb34 c0a8 6402 c0a8 6401  e>@....4..d...d.
    0x0030:  d5e0 829c 0012 5bf7 0201 0000 0000 0000  ......[.........
    0x0040:  0000                                     ..
22:48:32.934190 IP (tos 0x0, ttl 1, id 35760, offset 0, flags [DF], proto UDP (17), length 38) 192.168.100.2.54752 > 192.168.100.1.33437: [bad udp cksum 7e11!] UDP, length 10
    0x0000:  4500 0026 8bb0 4000 0111 a4c2 c0a8 6402  E..&..@.......d.
    0x0010:  c0a8 6401 d5e0 829d 0012 4978 0301 0000  ..d.......Ix....
    0x0020:  0000 0000 0000                           ......
22:48:32.940563 IP (tos 0xc0, ttl 64, id 34589, offset 0, flags [none], proto ICMP (1), length 66) 192.168.100.1 > 192.168.100.2: ICMP 192.168.100.1 udp port 33437 unreachable, length 46
    IP (tos 0x0, ttl 1, id 35760, offset 0, flags [DF], proto UDP (17), length 38) 192.168.100.2.54752 > 192.168.100.1.33437: [udp sum ok] UDP, length 10
    0x0000:  45c0 0042 871d 0000 4001 a989 c0a8 6401  E..B....@.....d.
    0x0010:  c0a8 6402 0303 4675 0000 0000 4500 0026  ..d...Fu....E..&
    0x0020:  8bb0 4000 0111 a4c2 c0a8 6402 c0a8 6401  ..@.......d...d.
    0x0030:  d5e0 829d 0012 5af6 0301 0000 0000 0000  ......Z.........
    0x0040:  0000                                     ..