route command and non-local gateways

Hi all,

I'm struggling with a route I'm trying to install on a workstation
(Linux 2.4.22, but I think this is common to other OSes as well).

Assume that the workstation (WS1) sits on network 10.1.1.0/24 and that
there is a gateway (GW1) off this subnet at 10.1.1.1. Also directly
connected to GW1 is subnet 10.1.2.0/24, and in that subnet sits a
firewall (FW1) at 10.1.2.2, with the Internet-at-large the otherside of
it. (Sorry, I don't do ASCII Art, and if you saw my attempts at it,
you'd agree!). Internet routes are not propogated into the internal
network, nor is there a default route injected.

Now, there is an Internet connected host (H1) to which I need to make a
connection from WS1. FW1 has a rule inserted to catch the traffic from
WS1 to H1, Hide-NAT and forward the traffic. This does require that WS1
have a static route to H1 via FW1. This is where I run into
difficulties.

I've tried adding the route like:

WS1# route add -host H1 gw 10.1.2.2

but I get "SIOCADDRT: Network is unreachable" even though 10.1.2.2 is
reachable, by dint of the default route to 10.1.1.1

So, I tried:

WS1# route add -host 10.1.2.2 gw 10.1.1.1
followed by:
WS1# route add -host H1 gw 10.1.2.2
but still "SIOCADDRT: Network is unreachable"

According to the man page for route, "NOTE: The specified gateway must
be reachable first. This usually means that you have to set up a static
route to the gateway beforehand."

ISTM that I've done that, but obviously I'm missing something. Can
anybody point me at what I'm getting wrong, please?

btw, I've tried this from a box in 10.1.2.0, and all works as I expect
(ie I can connect to H1, and the log on FW1 shows the traffic being
NATed and forwarded). As a workaround, I've configured on WS1 a static
route to H1 via GW1, and on GW1 I've configured a static route to H1
via FW1, but this is not acceptable longterm.

Many thanks in anticipation of any help.

Steve

StivH wrote:
>
> Assume that the workstation (WS1) sits on network 10.1.1.0/24 and that
> there is a gateway (GW1) off this subnet at 10.1.1.1. Also directly
> connected to GW1 is subnet 10.1.2.0/24, and in that subnet sits a
> firewall (FW1) at 10.1.2.2, with the Internet-at-large the otherside of
> it. (Sorry, I don't do ASCII Art, and if you saw my attempts at it,
> you'd agree!). Internet routes are not propogated into the internal
> network, nor is there a default route injected.
>
> Now, there is an Internet connected host (H1) to which I need to make a
> connection from WS1. FW1 has a rule inserted to catch the traffic from
> WS1 to H1, Hide-NAT and forward the traffic. This does require that WS1
> have a static route to H1 via FW1. This is where I run into
> difficulties.
>
> btw, I've tried this from a box in 10.1.2.0, and all works as I expect
> (ie I can connect to H1, and the log on FW1 shows the traffic being
> NATed and forwarded). As a workaround, I've configured on WS1 a static
> route to H1 via GW1, and on GW1 I've configured a static route to H1
> via FW1, but this is not acceptable longterm.

+-----+
| H1 |
+--+--+
| y.y.y.y
| Internet
| eth0: x.x.x.x
+--+--+
| FW1 |
+--+--+
| eth1: 10.1.2.2/24
|
| eth0: 10.1.2.1/24
+--+--+
| GW1 |
+--+--+
| eth1: 10.1.1.1/24
|
| eth0: 10.1.1.x/24
+--+--+
| WS1 |
+-----+

1) WS1 should have default gw set to 10.1.1.1. GW1 should forward
traffic between eth0 and eth1 with default gw as 10.1.2.2. FW1 should
have a route for 10.1.1.0/24 pointing to 10.1.2.1 and do NAT both for
10.1.1.0/24 and 10.1.2.0/24.

2) WS1 should have default gw as 10.1.1.1. GW1 should do NAT from
10.1.1.0/24 to 10.1.2.1. FW1 should already be correct.

Personally, I don't like double NAT so I would go for no 1.

If you actually showed us your routing table it would help. We only have
your word ( the word of someone who tells us he does not know what is
happening) that it is properly set up.


"StivH" <(E-Mail Removed)> writes:

>Hi all,

>I'm struggling with a route I'm trying to install on a workstation
>(Linux 2.4.22, but I think this is common to other OSes as well).

>Assume that the workstation (WS1) sits on network 10.1.1.0/24 and that
>there is a gateway (GW1) off this subnet at 10.1.1.1. Also directly
>connected to GW1 is subnet 10.1.2.0/24, and in that subnet sits a
>firewall (FW1) at 10.1.2.2, with the Internet-at-large the otherside of
>it. (Sorry, I don't do ASCII Art, and if you saw my attempts at it,
>you'd agree!). Internet routes are not propogated into the internal
>network, nor is there a default route injected.

>Now, there is an Internet connected host (H1) to which I need to make a
>connection from WS1. FW1 has a rule inserted to catch the traffic from
>WS1 to H1, Hide-NAT and forward the traffic. This does require that WS1
>have a static route to H1 via FW1. This is where I run into
>difficulties.

>I've tried adding the route like:

>WS1# route add -host H1 gw 10.1.2.2

>but I get "SIOCADDRT: Network is unreachable" even though 10.1.2.2 is
>reachable, by dint of the default route to 10.1.1.1

>So, I tried:

>WS1# route add -host 10.1.2.2 gw 10.1.1.1
>followed by:
>WS1# route add -host H1 gw 10.1.2.2
>but still "SIOCADDRT: Network is unreachable"

>According to the man page for route, "NOTE: The specified gateway must
>be reachable first. This usually means that you have to set up a static
>route to the gateway beforehand."

>ISTM that I've done that, but obviously I'm missing something. Can
>anybody point me at what I'm getting wrong, please?

>btw, I've tried this from a box in 10.1.2.0, and all works as I expect
>(ie I can connect to H1, and the log on FW1 shows the traffic being
>NATed and forwarded). As a workaround, I've configured on WS1 a static
>route to H1 via GW1, and on GW1 I've configured a static route to H1
>via FW1, but this is not acceptable longterm.

It looks to me that this is what you were trying to do above. I have no
idea what the difference is between what you describe here in this
paragraph and what you described above.
Post your routing table(s).


>Many thanks in anticipation of any help.

>Steve

StivH wrote:
> Hi all,
>
> I'm struggling with a route I'm trying to install on a workstation
> (Linux 2.4.22, but I think this is common to other OSes as well).
>
> Assume that the workstation (WS1) sits on network 10.1.1.0/24 and that
> there is a gateway (GW1) off this subnet at 10.1.1.1. Also directly
> connected to GW1 is subnet 10.1.2.0/24, and in that subnet sits a
> firewall (FW1) at 10.1.2.2, with the Internet-at-large the otherside of
> it. (Sorry, I don't do ASCII Art, and if you saw my attempts at it,
> you'd agree!). Internet routes are not propogated into the internal
> network, nor is there a default route injected.
>
> Now, there is an Internet connected host (H1) to which I need to make a
> connection from WS1. FW1 has a rule inserted to catch the traffic from
> WS1 to H1, Hide-NAT and forward the traffic. This does require that WS1
> have a static route to H1 via FW1. This is where I run into
> difficulties.
>
> I've tried adding the route like:
>
> WS1# route add -host H1 gw 10.1.2.2
>
> but I get "SIOCADDRT: Network is unreachable" even though 10.1.2.2 is
> reachable, by dint of the default route to 10.1.1.1
>
> So, I tried:
>
> WS1# route add -host 10.1.2.2 gw 10.1.1.1
> followed by:
> WS1# route add -host H1 gw 10.1.2.2
> but still "SIOCADDRT: Network is unreachable"
>
> According to the man page for route, "NOTE: The specified gateway must
> be reachable first. This usually means that you have to set up a static
> route to the gateway beforehand."
>
> ISTM that I've done that, but obviously I'm missing something. Can
> anybody point me at what I'm getting wrong, please?
>
> btw, I've tried this from a box in 10.1.2.0, and all works as I expect
> (ie I can connect to H1, and the log on FW1 shows the traffic being
> NATed and forwarded). As a workaround, I've configured on WS1 a static
> route to H1 via GW1, and on GW1 I've configured a static route to H1
> via FW1, but this is not acceptable longterm.
>
> Many thanks in anticipation of any help.
>
> Steve
>
Hosts only need to know the next hop of a route.

WS1 (and hosts on 10.1.1.0/24) needs 10.1.1.1 as a default gateway
"route add default gw 10.1.1.1"
GW1 needs eth1 (or whatever the device attaches to 10.1.1.0/24 is
called) as the route to 10.1.1.0/24
and 10.1.2.2 as the default gateway
"route add -net 10.1.1.0/24 dev eth1"
"route add default gw 10.1.2.2"

Only the firewall needs to be set up to do NAT.

Robert

If the gateway isn't local, there's no reason the host needs to know
about it. When I send a package through the USPS, I just need to know where
the nearest mailbox is. It doesn't matter to me where the local post office
is.

Similarly, all a host does to a packet is put the destination address on
it and send it off to some local device. It doesn't care where it goes after
that, and doesn't tell that local device what to do with the packet after it
receives it.

DS

Thanks for the reply. Option 1 is broadly similar to the workaround
that I have in place. However, this is all part of a large corporate
internetwork - GW1 participates in a multi-area OSPF domain, which in
turn exchanges routes with BGP internally within the organisation. This
is why I do not want to either install on FW1 a default route via GW1,
or leave the static route on GW1 to H1 via FW1, as this is distributed
into OSPF.

I've got the option of moving WS1 into 10.1.1.x, which is looking like
my most feasible solution right now, it's just that the doc for the
route command seems (to me) to be saying that what I want to do is
feasible. Oh well.

Cheers
Steve

OK, the routing table in GW1 is > 1,000 routes, almost all of which are
completely irrelevant to this. (Btw, I made up the IP addresses for the
purpose of illustration, paranoid, really, as our real addresses are
RFC1918 compliant and are NATed anyway). I don't propose to post that.
Is there something specific that you're looking for?

WS1 routing table is like:

stivh@WS1:~#route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
10.1.1.0 * 255.255.254.0 U 0 0 0
eth0
127.0.0.0 * 255.0.0.0 U 0 0 0
lo
0.0.0.0 10.1.1.1 0.0.0.0 UG 1 0 0
eth0
stivh@WS1:~#

before I try to install any additional routes.

Cheers
Steve

StivH wrote:
> Thanks for the reply. Option 1 is broadly similar to the workaround
> that I have in place. However, this is all part of a large corporate
> internetwork - GW1 participates in a multi-area OSPF domain, which in
> turn exchanges routes with BGP internally within the organisation. This
> is why I do not want to either install on FW1 a default route via GW1,
> or leave the static route on GW1 to H1 via FW1, as this is distributed
> into OSPF.
>
> I've got the option of moving WS1 into 10.1.1.x, which is looking like
> my most feasible solution right now, it's just that the doc for the
> route command seems (to me) to be saying that what I want to do is
> feasible. Oh well.

Disregarding your BGP/OSPF setup, this is the routes you will need
(based upon option 1):

WS1: the routes in your reply to Unruh should be OK, except you
described /24 networks while your routing table is /23.

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.1.1.0 * 255.255.255.0 U 0 0 0 eth0
127.0.0.0 * 255.0.0.0 U 0 0 0 lo
0.0.0.0 10.1.1.1 0.0.0.0 UG 1 0 0 eth0

GW1: routes to 10.1.1.0/24 and 10.1.2.0/24 should be set when your
eth's come up, default gw should also be 10.1.2.2/24. If not, do

route add -net 10.1.1.0 netmask 255.255.255.0 dev eth1
route add -net 10.1.2.0 netmask 255.255.255.0 dev eth0
route add default gw 10.1.2.2 dev eth0

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.1.2.0 * 255.255.255.0 U 0 0 0 eth0
10.1.1.0 * 255.255.255.0 U 0 0 0 eth1
127.0.0.0 * 255.0.0.0 U 0 0 0 lo
0.0.0.0 10.1.2.2 0.0.0.0 UG 1 0 0 eth0

FW1: here you must add a static route to WS1 thru GW1. You also need
NAT from the 10.1.1.0/24 net (as is done with the 10.1.2.0/24 net):

route add -net 10.1.1.0/24 gw 10.1.2.1 dev eth1

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
xx.xx.xx.xx * xx.xx.xx.xx U 0 0 0 eth0
10.1.2.0 * 255.255.255.0 U 0 0 0 eth1
10.1.1.0 10.1.2.1 255.255.255.0 UG 0 0 0 eth1
127.0.0.0 * 255.0.0.0 U 0 0 0 lo
0.0.0.0 x.x.x.x 0.0.0.0 UG 1 0 0 eth0

Good luck!

Hi - thanks for the reply.
10.1.1.0/24 hosts have 10.1.1.1 as default gw.
GW1 is actually a L3 switch and has route to 10.1.1.0/24 as a directly
connected net
However, I can't have a default route for GW1 via FW1 - the closest I
can go to that is to install a route to H1 on GW1 and even that I'm not
happy about leaving on.

I'll probably have to move H1 into 10.1.1.0, but it would have been
neater if I could have just pointed it at FW1.

Cheers
Steve

Thanks.

I really cannot install a default route via FW1 into GW1, though. If I
do, that default route will be propogated through the OSPF area (in
this case, Area 0 (the backbone area)). This is where the problem lies,
and why I wanted to install the static route in WS1 via a non-local
gateway, as the doc seems to say that I can. (Btw, good spot about the
different prefix lengths - I was just testing! Actually, the host that
I took the route table off is one where I've been trying to replicate a
setup out in userland, and whilst the test net I'm using is a /23, the
user's network is actually a /22, but for the sake of keeping the
discussion as simple as I could, I used ficticious /24s).

Thanks so much for your input, though.

Steve