Major problems with 2003 TCP/IP

I have a Sharepoint web farm that is NLB (only 2 nodes, Win2003
Standard SP1). One node is having some major networking problems.
Here is the rundown:

After installing the IE rollup patch, I rebooted the server and
monitored for it to come back up. When the machine never came back up,
I began to troubleshoot the problem. I have exhausted all avenues of
the problem being with our network and instead am focusing more on it
being a problem with Win2003 and possibly the TCP/IP stack.

To troubleshoot the server:

1. We broke the NLB team between the 2 servers.
2. Checked to make sure the IPs were valid and assigned to the correct
ports.
3. Removed the 2nd IP address (virtual) in the Advanced IP settings for
the NIC.
4. Re-checked all TCP/IP settings against the other front-end server
that is not experiencing problems.
5. Repaired the NIC through the OS
6. Cleared ARP cache
7. Installed updated NIC drivers.
8. Uninstalled the IE Rollup patch
9. Uninstalled SP1

To ensure that is was not the network, we hooked a client machine
(Windows 2000) up to the LAN drop, assigned it the same IP and it
connected to the network without any of the problems above.

Thinking about repairing the OS, but I would like to stick away from
completely blowing away the OS and rebuilding. Any ideas on where to
go next...

As long as you are asking for ideas....have you checked for a duplex
mis-match between the Server NICs and the Switch?
Also, you may want to post any Event Viewer error messages you may be
receiving...that may provide further clues....

<(E-Mail Removed)> wrote in message
news:(E-Mail Removed) oups.com...
>I have a Sharepoint web farm that is NLB (only 2 nodes, Win2003
> Standard SP1). One node is having some major networking problems.
> Here is the rundown:
>
> After installing the IE rollup patch, I rebooted the server and
> monitored for it to come back up. When the machine never came back up,
> I began to troubleshoot the problem. I have exhausted all avenues of
> the problem being with our network and instead am focusing more on it
> being a problem with Win2003 and possibly the TCP/IP stack.
>
> To troubleshoot the server:
>
> 1. We broke the NLB team between the 2 servers.
> 2. Checked to make sure the IPs were valid and assigned to the correct
> ports.
> 3. Removed the 2nd IP address (virtual) in the Advanced IP settings for
> the NIC.
> 4. Re-checked all TCP/IP settings against the other front-end server
> that is not experiencing problems.
> 5. Repaired the NIC through the OS
> 6. Cleared ARP cache
> 7. Installed updated NIC drivers.
> 8. Uninstalled the IE Rollup patch
> 9. Uninstalled SP1
>
> To ensure that is was not the network, we hooked a client machine
> (Windows 2000) up to the LAN drop, assigned it the same IP and it
> connected to the network without any of the problems above.
>
> Thinking about repairing the OS, but I would like to stick away from
> completely blowing away the OS and rebuilding. Any ideas on where to
> go next...
>

Barry,

Are you talking about the server NIC MAC address being mis-match at the
port? If it was incorrect, wouldn't that also prevent me from
connecting a workstation to the same ethernet drop and getting a
connection?

I have not seen any Event Error related to the problems..

The only thing I can ping is myself... I cannot ping the gateway, or
even other IP addresses on the same subnet.

In news:(E-Mail Removed) oups.com,
(E-Mail Removed) <(E-Mail Removed)> stated, which I commented
on below:
> I have a Sharepoint web farm that is NLB (only 2 nodes, Win2003
> Standard SP1). One node is having some major networking problems.
> Here is the rundown:
>
> After installing the IE rollup patch, I rebooted the server and
> monitored for it to come back up. When the machine never came back
> up, I began to troubleshoot the problem. I have exhausted all
> avenues of the problem being with our network and instead am focusing
> more on it being a problem with Win2003 and possibly the TCP/IP stack.
>
> To troubleshoot the server:
>
> 1. We broke the NLB team between the 2 servers.
> 2. Checked to make sure the IPs were valid and assigned to the correct
> ports.
> 3. Removed the 2nd IP address (virtual) in the Advanced IP settings
> for the NIC.
> 4. Re-checked all TCP/IP settings against the other front-end server
> that is not experiencing problems.
> 5. Repaired the NIC through the OS
> 6. Cleared ARP cache
> 7. Installed updated NIC drivers.
> 8. Uninstalled the IE Rollup patch
> 9. Uninstalled SP1
>
> To ensure that is was not the network, we hooked a client machine
> (Windows 2000) up to the LAN drop, assigned it the same IP and it
> connected to the network without any of the problems above.
>
> Thinking about repairing the OS, but I would like to stick away from
> completely blowing away the OS and rebuilding. Any ideas on where to
> go next...

Since you said this happening directly after installing a hotfix or update,
were the default system drive permissions ever changed?

--
Ace

This posting is provided "AS-IS" with no warranties or guarantees and
confers no rights.

If you are having difficulty in reading or finding responses to your post,
instead of the website you are using, if I may suggest to use OEx (Outlook
Express or any other newsreader of your choosing), and configure a newsgroup
account, pointing to news.microsoft.com. This is a direct link into the
Microsoft Public Newsgroups, and it is FREE and DOES NOT require a Usenet
account with your ISP. With OEx, you can easily find your post, track
threads, cross-post, and sort by date, poster's name, watched threads or
subject.

Not sure how? It's easy:
How to Configure OEx for Internet News
http://support.microsoft.com/?id=171164

Ace Fekay, MCSE 2003 & 2000, MCSA 2003 & 2000, MCSE+I, MCT, MVP
Microsoft MVP - Windows Server Directory Services
Microsoft Certified Trainer
Assimilation Imminent. Resistance is Futile.
Infinite Diversities in Infinite Combinations.
=================================

No.. The patch was applied and the system was rebooted. When the OS
came back up it could not connect to anything else on the domain. No
gateway, no DNS, no Domain Controller.

Ace Fekay [MVP] wrote:
>
> Since you said this happening directly after installing a hotfix or update,
> were the default system drive permissions ever changed?
>
> --
> Ace
>

In news:(E-Mail Removed) oups.com,
(E-Mail Removed) <(E-Mail Removed)> stated, which I commented
on below:
> No.. The patch was applied and the system was rebooted. When the OS
> came back up it could not connect to anything else on the domain. No
> gateway, no DNS, no Domain Controller.
>
> Ace Fekay [MVP] wrote:

That actually exactly mimics the main popular complaint (loss of network
connectivity which is basically what casues everything you described), that
happened after a certain patch was applied, but it is based on permissions
being altered in the past, whether on the drive and/or in the registry. The
patch protects an MSDTC issue. Here read this repost from a previous post
and see if it helps.

+++++++++++++++++++++++++++++++++++++++++
I had a client with this problem just yesterday morning. This is a
permissions problem on the %windir%\registration folder.

Systems that have changed the default Access Control List permissions on the
%windir%\registration directory may experience various problems after you
install the Microsoft Security Bulletin MS05-051 for COM+ and MS DTC
http://support.microsoft.com/kb/909444

For specific info, see:
http://blogs.technet.com/steriley/ar...08/414002.aspx
++++++++++++++++++++++++++++++++++++++++

Ace

Ace,

Thanks again for the reply... That patch was applied a while back
without causing any problems. The cumulative patch we installed after
the problems was MS05-54 (Cumulative Security Update for IE). I will
check the ACLs on that folder again to make sure it's correct. I guess
I should have checked that as well... Will keep you up to date.

Ace,

The permissions were still set up as the default windows installation.
Just to be sure, I ran the cacls command line resolution in the
article, it still didn't work.

Any other ideas?

Some more findings...

I reset the TCP/IP stack and then checked to ensure that the ICF was
off. When I clicke don settings, I received an error that said the
associated service was not started. When I went to check Services, the
IPSec Service was not started. I went to start it, but received the
following error:

"Could not start IPSec Services on local computer. Error 2: The system
cannot find the file specified".

Could it be, that the Server went to IPSec Block Mode ?

Maybe rebuilding the local policy store may help.....
Check that out: http://support.microsoft.com/kb/870910/en-us

1. Delete the local policy registry subkey. To do this, follow these
steps: a. Click Start, click Run, type regedit, and then click OK.
b. In Registry Editor, locate and then click the following
subkey:
HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Win dows\IPSec\Policy\Local
c. On the Edit menu, click Delete.
d. Click Yes to confirm that you want to delete the subkey.
e. Quit Registry Editor

2. Rebuild a new local policy store. To do this, follow this step: a.
Click Start, click Run, type regsvr32 polstore.dll, and then click OK.



MC



<(E-Mail Removed)> wrote in message
news:(E-Mail Removed) oups.com...
> Some more findings...
>
> I reset the TCP/IP stack and then checked to ensure that the ICF was
> off. When I clicke don settings, I received an error that said the
> associated service was not started. When I went to check Services, the
> IPSec Service was not started. I went to start it, but received the
> following error:
>
> "Could not start IPSec Services on local computer. Error 2: The system
> cannot find the file specified".
>