Three nics in Windows 2003 Std Server, two goes down 'randomly' third stays up no matter what

Hey there, be prepared for *mind boggling* strange problem -

Currently three NICs are placed in one machine, where 10.0.1.100 and
192.168.1.100 are NICs for web serving.
10.0.0.15 is connect to the internal LAN. The machine runs Windows 2003 Std.
'Gold'.

That results three different ways to access the machine -

1. From internet->firewall(212.130.x.x-/192.168.1.254)->192.168.1.100
2. From internet->firewall(212.130.x.y/10.0.1.1)->10.0.1.100
3. From LAN(10.0.0.0/24)->10.0.0.15

The problem about 192.168.1.100 is the one that confuses me the greatly, and
I think it might have something to do with the default gateway.

The situation is that sometimes, seemingly randomly, the 212.130.x.x decides
to "die", and in the process, it takes 212.130.x.z with it (or reverse, not
100% sure)!

A few times I've managed to check the LAN status (10.0.0.15), and it have
always been working while the two other interfaces have been down.
During that time I've been pinging 212.130.x.x and 212.130.x.y, 212.130.x.x
routes the pings to 192.168.1.100, and this one stops responding.
212.130.x.y always responds, but that's because I haven't figured out how to
route the pings to 10.0.1.15 yet (it's a cisco 677 if anyone got a hint on
that part).
Secondly from the machine I currently ping 192.168.1.254, and it always
responds, also when 212.130.x.x is down (stops responding on requests from
internet)! Thirdly the machine also pings 10.0.1.1 and it also always
responds back.

Suddenly (also seems randomly), 212.130.x.x decides to 'wake up' again, and
everything goes back to working state.

The curent routing table for the machine -

Network Destination Netmask Gateway Interface Metric
0.0.0.0 0.0.0.0 10.0.0.110 10.0.0.15 5
0.0.0.0 0.0.0.0 10.0.1.1 10.0.1.100 20
0.0.0.0 0.0.0.0 192.168.1.254 192.168.1.10 10
10.0.0.0 255.255.255.0 10.0.0.15 10.0.0.15 5
10.0.0.0 255.255.254.0 10.0.1.100 10.0.1.100 20
10.0.0.15 255.255.255.255 127.0.0.1 127.0.0.1 5
10.0.1.100 255.255.255.255 127.0.0.1 127.0.0.1 20
10.255.255.255 255.255.255.255 10.0.0.15 10.0.0.15 5
10.255.255.255 255.255.255.255 10.0.1.100 10.0.1.100 20
127.0.0.0 255.0.0.0 127.0.0.1 127.0.0.1 1
192.168.1.0 255.255.255.0 192.168.1.10 192.168.1.10 10
192.168.1.10 255.255.255.255 127.0.0.1 127.0.0.1 10
192.168.1.255 255.255.255.255 192.168.1.10 192.168.1.10 10
224.0.0.0 240.0.0.0 10.0.0.15 10.0.0.15 5
224.0.0.0 240.0.0.0 10.0.1.100 10.0.1.100 20
224.0.0.0 240.0.0.0 192.168.1.10 192.168.1.10 10
255.255.255.255 255.255.255.255 10.0.0.15 10.0.0.15 1
255.255.255.255 255.255.255.255 10.0.1.100 10.0.1.100 1
255.255.255.255 255.255.255.255 192.168.1.10 192.168.1.10 1
Default Gateway: 192.168.1.254

Because 212.130.x.z also goes down with 212.130.x.x it leads me to believe
that there might be a problem with the routing table, which leads to another
problem - the default gateway is wrong!

I want the machine to use 10.0.0.15 as default gateway (as the metric should
indicate), but currently Windows 2003 seems have decided to ignore that
part, but of course, only the 10.0.0.15 when "it's correct'" (as the three
networks are disjoined).

What makes this *mind boggling* strange is that I'm able to ping 10.0.1.1
and 192.168.1.254, from the machine, while the EXTERNAL access is down on
those two! The external gear is different, 3com and Cisco, and two different
ISPs!

Anybody here who can help out with this very confusing problem?

"Kim Noer" <(E-Mail Removed)> wrote in message
news:%(E-Mail Removed)...
> 1. From internet->firewall(212.130.x.x-/192.168.1.254)->192.168.1.100
> 2. From internet->firewall(212.130.x.y/10.0.1.1)->10.0.1.100
> 3. From LAN(10.0.0.0/24)->10.0.0.15

> I want the machine to use 10.0.0.15 as default gateway (as the metric
should

You can't.
You can only have *one* Default Gateway and it must be on either 192.168.1.x
Nic or the 10.0.1.x Nic,...but not both.
You can have two External (Firewall facing) Nics,..but only one will work as
an Internet connection (one with the Gateway), the other can only be used
for "Specific Destinations" via adding Static Routes in the OS's (or RRAS if
using it) Routing Table.

--

Phillip Windell [MCP, MVP, CCNA]
www.wandtv.com

"Phillip Windell" <@.> wrote in message
news:(E-Mail Removed)
> "Kim Noer" <(E-Mail Removed)> wrote in message
> news:%(E-Mail Removed)...
>> 1. From internet->firewall(212.130.x.x-/192.168.1.254)->192.168.1.100
>> 2. From internet->firewall(212.130.x.y/10.0.1.1)->10.0.1.100
>> 3. From LAN(10.0.0.0/24)->10.0.0.15
> You can't.

Figures, well, I can, but only 'from time to time' (the two external facing
cards are highly unstable).

> You can only have *one* Default Gateway and it must be on either
> 192.168.1.x Nic or the 10.0.1.x Nic,...but not both.
> You can have two External (Firewall facing) Nics,..but only one will
> work as an Internet connection (one with the Gateway), the other can
> only be used for "Specific Destinations" via adding Static Routes in
> the OS's (or RRAS if using it) Routing Table.

So what do I do, does it mean that -

1. From internet->firewall(212.130.x.x-/192.168.1.254)->192.168.1.100
2. From internet->firewall(212.130.x.y/10.0.1.1)->10.0.1.100
3. From LAN(10.0.0.0/24)->10.0.0.15

won't ever work if I want to route the packets back to where they came from?
Ie. 192.168.1.0/24 gets routed back through 192.168.1.254 and not any where
else and so on?

--
I doubt, therefore I might be.

"Kim Noer" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> >> 1. From internet->firewall(212.130.x.x-/192.168.1.254)->192.168.1.100
> >> 2. From internet->firewall(212.130.x.y/10.0.1.1)->10.0.1.100
> >> 3. From LAN(10.0.0.0/24)->10.0.0.15
> > You can't.
>
> Figures, well, I can, but only 'from time to time' (the two external
facing
> cards are highly unstable).

That's the point.

> 1. From internet->firewall(212.130.x.x-/192.168.1.254)->192.168.1.100
> 2. From internet->firewall(212.130.x.y/10.0.1.1)->10.0.1.100
> 3. From LAN(10.0.0.0/24)->10.0.0.15
>
> won't ever work if I want to route the packets back to where they came
from?
> Ie. 192.168.1.0/24 gets routed back through 192.168.1.254 and not any
where
> else and so on?

Packets are "one-way". Packets never go back where they came from. Each
direction inbound/outbound is an entirely independent communication session
as far as TCP/IP is concerned. It is in the "Application" being used that
the two are put together to create a "conversation",... which has nothing to
do with the paths they take.

The path an inbound packet takes is based on the routing scheme from the
where it originated,...the outbound packet is subject to the routing scheme
on your end,...notice there is no relationship between the two.

--

Phillip Windell [MCP, MVP, CCNA]
www.wandtv.com

"Phillip Windell" <@.> wrote in message
news:(E-Mail Removed)

> Packets are "one-way". Packets never go back where they came from.
> Each direction inbound/outbound is an entirely independent
> communication session as far as TCP/IP is concerned. It is in the
> "Application" being used that the two are put together to create a
> "conversation",... which has nothing to do with the paths they take.

Damn, for some reason I always thought the 'receiving' part knew where the
packet came from, and thus, albeit, it might need some help, could send the
packet back to the gateway it used as 'entry' point.

> The path an inbound packet takes is based on the routing scheme from
> the where it originated,...the outbound packet is subject to the
> routing scheme on your end,...notice there is no relationship between
> the two.

Hmm, then I wonder, how do one connect multiple external IPs to the internal
LAN?

If I changed the setup to the following -

1. From internet->firewall(212.130.x.x-/10.0.0.2)->10.0.0.10
1. From internet->firewall(212.130.x.y-/10.0.0.3)->10.0.0.10
3. From LAN(10.0.0.0/24)->10.0.0.10

where 10.0.0.2 & 10.0.0.3 are the two gateways, with different external IPs,
then it wouldn't work?

If so, it's impossible to have multiple gateways, and make sure the damn
packets gets kicked out again, through the 'entry' gateway?

--
I doubt, therefore I might be.

"Kim Noer" <(E-Mail Removed)> wrote in message
news:e6Cs$(E-Mail Removed)...
> "Phillip Windell" <@.> wrote in message
> news:(E-Mail Removed)
>
> > Packets are "one-way". Packets never go back where they came from.
> > Each direction inbound/outbound is an entirely independent
> > communication session as far as TCP/IP is concerned. It is in the
> > "Application" being used that the two are put together to create a
> > "conversation",... which has nothing to do with the paths they take.
>
> Damn, for some reason I always thought the 'receiving' part knew where the
> packet came from, and thus, albeit, it might need some help, could send
the
> packet back to the gateway it used as 'entry' point.

The Application is the "receiving part" and it does know where it came
from,...it just doesn't matter. The route isn't determined by "what path did
the last packet take?",...it is determined by "what does the Routing Table
say the proper path is to the place where the last packet came from?".


> Hmm, then I wonder, how do one connect multiple external IPs to the
internal > If I changed the setup to the following -
>
> 1. From internet->firewall(212.130.x.x-/10.0.0.2)->10.0.0.10
> 1. From internet->firewall(212.130.x.y-/10.0.0.3)->10.0.0.10
> 3. From LAN(10.0.0.0/24)->10.0.0.10

No. Impossible. They are all three the same subnet.

> where 10.0.0.2 & 10.0.0.3 are the two gateways, with different external
IPs,
> then it wouldn't work?

Running 1 Nic with 2 IP#s is not the same thing as two Nics with one IP#
each. If you want multiple IP#s on the external side then add them to the
same Nic and run one external Nic and one internal. But outbound packets
will still show the Primary IP# of the Nic as the IP# they came from.

> If so, it's impossible to have multiple gateways, and make sure the damn
> packets gets kicked out again, through the 'entry' gateway?

No. I think you are worrying about something that is not even a "problem" to
begin with,...it is just simply the way that TCP/IP works, ...it is the way
it was meant to work.

There can be multiple Gateways. There can only be one Default Gateway. A
Default Gateway and a Gateway are not the same thing. Gateways are for
"known" destinations, Default Gateways are the "bit bucket" for "unknown
destinations". The Internet, by nature, is one big "unknown" destination.

--

Phillip Windell [MCP, MVP, CCNA]
www.wandtv.com

Phillip Windell wrote:

> That's the point.

Just a quick followup, if that's the point, that it will be unstable, then
how come it works at all? With unstable I mean under 0.05%, last I did a
test (which is 'highly' unstable to me).

There got to be some kind of 'tricks' Windows makes use off, since I've
actually gotten this to work in the past (albeit only with two NICs).

The networks where disjoined, and each NIC had a different gateway, both
defaults...

The change is that now there are three NICs in one machine (OS changed from
Windows 2000 to Windows 2003), and three gateways, so why does it work with
two, but not three.

--
You can't trust water: Even a straight stick turns crooked in it.
(W. C. Fields)

Phillip Windell wrote:

> Running 1 Nic with 2 IP#s is not the same thing as two Nics with one
> IP# each. If you want multiple IP#s on the external side then add
> them to the same Nic and run one external Nic and one internal. But
> outbound packets will still show the Primary IP# of the Nic as the
> IP# they came from.

Isn't that really the same as two nics with two different IPs and one
default gateway, which won't work, because alas, it will use the wrong
gateway for one of the IPs? Which would result in say packets from internet
through 212.130.x.y, would be presented with 212.130.x.x as source IP when
the machine decides to answer the quest?

Like 80.70.60.50 wants to talk with 212.130.x.y (source/destination) -

80.70.60.50/212.130.x.y
NAT (10.0.0.3) mangles the packet -
80.70.60.50/10.0.0.10
Machine decides to answer
10.0.0.10/80.70.60.50
Route dictates 80.70.60.50 is unknown, thus default gateway (10.0.0.2)
applies
NAT (10.0.0.2) mangles the packet -
212.130.x.x/80.70.60.50

80.70.60.50 didn't ask 212.130.x.x for anything and refuses.

Won't the above also apply to 2 IPs on one NIC, since
- Different gear, seperate NAT tables, seperate ISPs
- Client won't accept packages from other places than what the client
originally requested

Sounds to me, that it's quite simple, that you can only have /one/ interface
facing toward internet and takes request from the internet for the entire
internal network? As in everything you got "exposed" (as in responds on TCP
on ports x,y,z etc) to the internet must use the same external facing
interface, else the routing will fail, because one default gateway rules
them all?

> No. I think you are worrying about something that is not even a
> "problem" to begin with,...it is just simply the way that TCP/IP
> works, ...it is the way it was meant to work.

Indeed, but unfortunately I have a certain habit of "breaking" the rules.

> bucket" for "unknown destinations". The Internet, by nature, is one
> big "unknown" destination.

Which I finally 'get', so thanks for the above explainations .

Btw. I appreciate the time your are taking to 'carve it out' for me!
--
You can't trust water: Even a straight stick turns crooked in it.
(W. C. Fields)

"Kim Noer" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> Phillip Windell wrote:
>
> > Running 1 Nic with 2 IP#s is not the same thing as two Nics with one
> > IP# each. If you want multiple IP#s on the external side then add
> > them to the same Nic and run one external Nic and one internal. But
> > outbound packets will still show the Primary IP# of the Nic as the
> > IP# they came from.
>
> Isn't that really the same as two nics with two different IPs and one
> default gateway,

No it is not.

One nic with 2 IP#s creates a single "set" of entries in the Routing Table
with,... I believe,... a partial entry for the additional IP#.

But 2 Nics with 1 IP# each, creates *2* full "sets" entries in the Routing
Table. Since the Routing Table is central to everything using TCP/IP
problems can be created by the additional entries.

I'm not saying that you have to use 1 Nic/2 IP#s, I'm just saying you are
better off with it then using 2 Nics/1IP#-each. But in either case, neither
one gives you what you were originally wanted.

The first IP# added to the Nic is the Primary IP#,...all addtional IP#s are
Secondary IP#s. Inbound traffic can "target" any one of the IP#s and will
work,...but outbound traffic will always show the Primary IP# as the source.
One exception might be IIS with a website where the Website can be bound to
a specific IP#,...in this case it may show the specificly bound IP# for both
directions,...but I'm not sure because I have never tested that.

--

Phillip Windell [MCP, MVP, CCNA]
www.wandtv.com

"Kim Noer" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> Phillip Windell wrote:
>
> > That's the point.
>
> Just a quick followup, if that's the point, that it will be unstable, then
> how come it works at all? With unstable I mean under 0.05%, last I did a
> test (which is 'highly' unstable to me).

I'll let MS explain it for me ;-)

175767 - Expected Behavior of Multiple Adapters on Same Network
http://support.microsoft.com/default...b;EN-US;175767

--

Phillip Windell [MCP, MVP, CCNA]
www.wandtv.com