Multi-homed MSCS servers

Hi,

I have two multi-homed servers setup in an active - passive configuration
using MSCS (on W2K3 R2). Each of these servers has 5 NIC cards (one using a
x-over cable for the heartbeat signal, the other four connected to four
different subnets servicing client requests to an Oracle database and to
shared files on the filesystem. The cluster is presently assigned an IP
adress on one of the four subnets. There is an external router that routes
between subnets.
The customer has installed a firewall between all the subnets in order to
keep certain data and information secure. This has presented a problem since
it appears that client requests coming from the subnets remote to the one
cluster IP address subnet start communication to the cluster IP and then
after several seconds the server starts responding using the NIC that is on
the clients local subnet (I beleive this is called an asymmetric routing
loop). This has effectively caused communication problems since the firewall
is performing TCP SYN and TCP SEQ checking and starts to block this
communication. We have persuaded them to disable the SYN and SEQ checking for
the time being but I am looking for other possible solutions.

Is it possible (and/or practical) to give the cluster IP addresses on the
other three subnets?
Has anyone had a similar issue?
If anyone has some advice it would be most welcomed.

If you require more information please let me know.

Thanks in Advance,

--
Henry

"Henry" <(E-Mail Removed)> wrote in message
news:6656E2CA-C583-4706-9F62-(E-Mail Removed)...
> I have two multi-homed servers setup in an active - passive configuration
> using MSCS (on W2K3 R2). Each of these servers has 5 NIC cards (one using
> a
> x-over cable for the heartbeat signal, the other four connected to four
> different subnets servicing client requests to an Oracle database and to
> shared files on the filesystem. The cluster is presently assigned an IP
> adress on one of the four subnets. There is an external router that routes
> between subnets.
> The customer has installed a firewall between all the subnets in order to
> keep certain data and information secure. This has presented a problem
> since
> it appears that client requests coming from the subnets remote to the one
> cluster IP address subnet start communication to the cluster IP and then
> after several seconds the server starts responding using the NIC that is
> on
> the clients local subnet (I beleive this is called an asymmetric routing
> loop). This has effectively caused communication problems since the
> firewall
> is performing TCP SYN and TCP SEQ checking and starts to block this
> communication. We have persuaded them to disable the SYN and SEQ checking
> for

> Is it possible (and/or practical) to give the cluster IP addresses on the
> other three subnets?

No.

> Has anyone had a similar issue?

No, but only because I would have never done it like that.

> If anyone has some advice it would be most welcomed.

I'm no expert on Clusters, but I do know how to deal with infrastructure.
Get rid of all the Nics except one. I suspect you can keep the "heardbeat
nic" since it doesn't connect anywhere else. The Cluster will be identified
by a virtual address for the cluster and it should be only one and it should
be on only one subnet.

As far as the Firewall, I really see no justification for that,...I don't
believe you should be seeking "security" at Layers 3&4 in the middle of the
LAN between the Resource and the Clients/Applications that require it. At
that point you should be looking to find your means of security in the
Database Engine Config and the Config and Design of the Application that
accesses the data. The Firewall seems to me more likely a way to just jam
up the works and screw up the functionality,...although I don't know much
about your exact situation either. But it is very common for people to
become their own worst enemy "in the name of security",...particularly when
they believe a Firewall is the universal security tool for every situation.


--
Phillip Windell
www.wandtv.com

The views expressed, are my own and not those of my employer, or Microsoft,
or anyone else associated with me, including my cats.
-----------------------------------------------------

--
Henry


"Phillip Windell" wrote:

> "Henry" <(E-Mail Removed)> wrote in message
> news:6656E2CA-C583-4706-9F62-(E-Mail Removed)...
> > I have two multi-homed servers setup in an active - passive configuration
> > using MSCS (on W2K3 R2). Each of these servers has 5 NIC cards (one using
> > a
> > x-over cable for the heartbeat signal, the other four connected to four
> > different subnets servicing client requests to an Oracle database and to
> > shared files on the filesystem. The cluster is presently assigned an IP
> > adress on one of the four subnets. There is an external router that routes
> > between subnets.
> > The customer has installed a firewall between all the subnets in order to
> > keep certain data and information secure. This has presented a problem
> > since
> > it appears that client requests coming from the subnets remote to the one
> > cluster IP address subnet start communication to the cluster IP and then
> > after several seconds the server starts responding using the NIC that is
> > on
> > the clients local subnet (I beleive this is called an asymmetric routing
> > loop). This has effectively caused communication problems since the
> > firewall
> > is performing TCP SYN and TCP SEQ checking and starts to block this
> > communication. We have persuaded them to disable the SYN and SEQ checking
> > for
>
> > Is it possible (and/or practical) to give the cluster IP addresses on the
> > other three subnets?
>
> No.
>
> > Has anyone had a similar issue?
>
> No, but only because I would have never done it like that.

I probably should have mentioned that 2 of the NIC's connect to VLAN's that
contain high speed document imaging equipment that, considering their speed
(1150 documents/minute with 6 images/document) could easily approach
saturaturation of a single NIC causing queries from other subnets to be
unacceptably slow.
>
> > If anyone has some advice it would be most welcomed.
>
> I'm no expert on Clusters, but I do know how to deal with infrastructure.
> Get rid of all the Nics except one. I suspect you can keep the "heardbeat
> nic" since it doesn't connect anywhere else. The Cluster will be identified
> by a virtual address for the cluster and it should be only one and it should
> be on only one subnet.
>
> As far as the Firewall, I really see no justification for that,...I don't
> believe you should be seeking "security" at Layers 3&4 in the middle of the
> LAN between the Resource and the Clients/Applications that require it. At
> that point you should be looking to find your means of security in the
> Database Engine Config and the Config and Design of the Application that
> accesses the data. The Firewall seems to me more likely a way to just jam
> up the works and screw up the functionality,...although I don't know much
> about your exact situation either. But it is very common for people to
> become their own worst enemy "in the name of security",...particularly when
> they believe a Firewall is the universal security tool for every situation.
>
>
> --
> Phillip Windell
> www.wandtv.com
>
> The views expressed, are my own and not those of my employer, or Microsoft,
> or anyone else associated with me, including my cats.
> -----------------------------------------------------

I agree that the firewall location is quite unncecessary but it's tough to
convince the Government of that fact. My thought is that the firewall should
not be present in the current location. I am only looking for options for the
problem at hand. My first opinion is that the firewall between these VLAN's
be replaced with a switch and the firewall moved to the perimeter of the
system. Other than that I was just looking for comments on if adding multiple
cluster IP addresses to a muti-homed server was a resonable undertaking.

Thanks for your comments.

Henry
>
>

"Henry" <(E-Mail Removed)> wrote in message
news:9535D4B5-F43A-4649-9279-(E-Mail Removed)...
>> > Has anyone had a similar issue?
>>
>> No, but only because I would have never done it like that.
>
> I probably should have mentioned that 2 of the NIC's connect to VLAN's
> that
> contain high speed document imaging equipment that, considering their
> speed
> (1150 documents/minute with 6 images/document) could easily approach
> saturaturation of a single NIC causing queries from other subnets to be
> unacceptably slow.

Actually that won't happen. On a fully switched network the Switches create
a virtual circuit between the two communicating Hosts. You can only
saturate the virtual circuit,...it is impossible to saturate the subnet
unless you yank out all the Switches and replaced them with Hubs. Only
Broadcasts can saturate a fully switched subnet and their are no big amounts
of broadcasts happening here,...the type of technology you are describing
does not create them.

Now with VLANs you can shoot yourself in the foot if you ain't careful.
VLANs do *not* conserve your bandwith and may often destroy it if they
aren't handled correctly. VLANs are usually a mix of "virtual" and
"physical",...when it is "virtual", that is two or more subnets over the
same physical cable,...they destroy your bandwidth because the traffic of
more than one segment (including the associated broadcasts) are all pounding
on the same cable. But if you keep the "virtual" part of it within the
confines of the Switch itself, and then keep things "physically" separated
where they leave the Switch (meaning a port is never part of more than one
segment) then you will gain the normal benefits of IP Segmenting.

However even IP Segmenting (subnets) only saves bandwith with respect to
Broadcasts,..nothing else. The majority of saving bandwidth comes at Layer2
by using Switches instead of Hubs.

> system. Other than that I was just looking for comments on if adding
> multiple
> cluster IP addresses to a muti-homed server was a resonable undertaking.

No. Multi-Homing does not "load balance" unless Nic Teaming is used and that
is not what this situation is. A single computer name can only be identified
by a single IP#. The same is true for the Cluster,...a Cluster "name" is
nothing more than a glorified computer name and follows the same principles.
The hosts communicating with using the cluster name do not "know" it is a
cluster name,...they "think" is is a machine name,..which is the whole point
of it.

The load balancing come from the Cluster itself, not multiple nics or
multiple IP#s.

5 machines in a Cluster = 5 nics and 5 processor sets.

The "load" is spread accross the 5,..that is where the benefit comes
from,...not from sticking extra nics in the machines. Doing so will either
do "nothing" or create a mess,...more often create a mess.

--
Phillip Windell
www.wandtv.com

The views expressed, are my own and not those of my employer, or Microsoft,
or anyone else associated with me, including my cats.
-----------------------------------------------------

I am more concerned about excess traffic on a single NIC, not a subnet.
Teaming the NIC's would be similar to using the VLAN's for performance sake.

The VLAN's are configured on the switches with each NIC connected with only
one IP to a single segment.

Thanks,
--
Henry


"Phillip Windell" wrote:

> "Henry" <(E-Mail Removed)> wrote in message
> news:9535D4B5-F43A-4649-9279-(E-Mail Removed)...
> >> > Has anyone had a similar issue?
> >>
> >> No, but only because I would have never done it like that.
> >
> > I probably should have mentioned that 2 of the NIC's connect to VLAN's
> > that
> > contain high speed document imaging equipment that, considering their
> > speed
> > (1150 documents/minute with 6 images/document) could easily approach
> > saturaturation of a single NIC causing queries from other subnets to be
> > unacceptably slow.
>
> Actually that won't happen. On a fully switched network the Switches create
> a virtual circuit between the two communicating Hosts. You can only
> saturate the virtual circuit,...it is impossible to saturate the subnet
> unless you yank out all the Switches and replaced them with Hubs. Only
> Broadcasts can saturate a fully switched subnet and their are no big amounts
> of broadcasts happening here,...the type of technology you are describing
> does not create them.
>
> Now with VLANs you can shoot yourself in the foot if you ain't careful.
> VLANs do *not* conserve your bandwith and may often destroy it if they
> aren't handled correctly. VLANs are usually a mix of "virtual" and
> "physical",...when it is "virtual", that is two or more subnets over the
> same physical cable,...they destroy your bandwidth because the traffic of
> more than one segment (including the associated broadcasts) are all pounding
> on the same cable. But if you keep the "virtual" part of it within the
> confines of the Switch itself, and then keep things "physically" separated
> where they leave the Switch (meaning a port is never part of more than one
> segment) then you will gain the normal benefits of IP Segmenting.
>
> However even IP Segmenting (subnets) only saves bandwith with respect to
> Broadcasts,..nothing else. The majority of saving bandwidth comes at Layer2
> by using Switches instead of Hubs.
>
> > system. Other than that I was just looking for comments on if adding
> > multiple
> > cluster IP addresses to a muti-homed server was a resonable undertaking.
>
> No. Multi-Homing does not "load balance" unless Nic Teaming is used and that
> is not what this situation is. A single computer name can only be identified
> by a single IP#. The same is true for the Cluster,...a Cluster "name" is
> nothing more than a glorified computer name and follows the same principles.
> The hosts communicating with using the cluster name do not "know" it is a
> cluster name,...they "think" is is a machine name,..which is the whole point
> of it.
>
> The load balancing come from the Cluster itself, not multiple nics or
> multiple IP#s.
>
> 5 machines in a Cluster = 5 nics and 5 processor sets.
>
> The "load" is spread accross the 5,..that is where the benefit comes
> from,...not from sticking extra nics in the machines. Doing so will either
> do "nothing" or create a mess,...more often create a mess.
>
> --
> Phillip Windell
> www.wandtv.com
>
> The views expressed, are my own and not those of my employer, or Microsoft,
> or anyone else associated with me, including my cats.
> -----------------------------------------------------
>
>
>

"Henry" <(E-Mail Removed)> wrote in message
news:24EB8DE6-6C2C-4206-9561-(E-Mail Removed)...
>I am more concerned about excess traffic on a single NIC, not a subnet.

Don't try to solve problems that aren't proven to be "real" in the situation
you are in. People often make real messes by trying to solve "perceived"
problems that they don't really have. Don't worry about the Nic load until
you can prove with something like Network Monitor that it is being maxed out
in a continous state (spikes are not relevant). Keep in mind that a Gigabit
Nic running full duplex actually runs at a potential 2 Gps, then you
multiply that by how many machines are in the Cluster,...that is a massive
amount of capability. It is not as easy to max that out as you might think.

In reality if there was so much action on a Nic that is staying maxed out
the CPU is probably also going to be in trouble. So the truth is that if
you are maxing out the Nic, then you should be looking at adding another
Server to the Cluster,...not another Nic to the Server.

> Teaming the NIC's would be similar to using the VLAN's for performance
> sake.

They are just the opposite.
1. VLAN's *add* to the load on a cable that is *multiplied* by the number of
VLANs on the same wire.
2. Teaming *reduces* the load on a cable that is *divided* by the number of
Nics in the "team" and the number of cables = the number of Nics.

.....exactly opposite results......

> The VLAN's are configured on the switches with each NIC connected with
> only
> one IP to a single segment.

Structurally with respect to the physical LAN's Topology that is perfectly
fine.

Performance with respect to traffic to/from the Servers,...that does
absolutely nothing.

The path taken *to* the server depends on what IP# the Host Name resolves
to, and that is always going to be the same IP#. This means that, almost no
matter what, it will take the same path over the same cable to the same
nic,...so the same path is always taken no matter how many Nics the machine
has.

The path taken *from* the server depends on the Binding Order of the Nics
and how the Destination fits into the Routing Table. Hence the traffic will
always use the first nic in the Binding Order, or the nic with the Default
Gateway. Many times both of these conditions are the same nic,...so the
same path is always taken no matter how many Nics the machine has. That can
be altered by local Static Routes, but then that will become the only path
and will not be load balanced with any other path/nic.

We are not working with water through pipes where more pipes can carry more
water and water is "self-balancing". We are working with Ethernet and
TCP/IP that follows paths based on Routing Decisions and Protocol Design.
It does not "balance out" by adding more "pipes".

It only "balances out" when you dymanicaly alter the Routing Decisions on
the fly which overrides the Protocol Design. That is what Nic Teaming does
for individual machines and what Dynamic Routing Protocols can do for pairs
of networks when there are multiple pathes available between two routers.

--
Phillip Windell
www.wandtv.com

The views expressed, are my own and not those of my employer, or Microsoft,
or anyone else associated with me, including my cats.
-----------------------------------------------------