Latency and packet loss

Hi all!

I'm currently facing some issues with a Windows Server 2003 Ent. x64
part of an Oracle RAC setup. When all CPU's are under heavy load we
experience high latency and even packet loss on the network. We've
just started troubleshooting the problem and I would like to hear if
anyone had comments as to whether this is normal behaviour or not.

When pinging the loopback address we experienced the same issues so I
guess that rules out the physical network.

Memory and disk bottlenecks does not seem to be an issue.
I setup some performance monitoring and saw that the processors often
are busy processing deferred procedure calls (DPCs). - Any ideas to
this ?

When handling for instance ICMP packets some switches will throw these
away when busy serving other types of data. Will the same happen in a
Windows server 2003 environment or is it not that intelligent ?


Best regards. Jacob Christensen

Hello,

do you have installed the Service pack 2 for Windows 2003 ?

As it is an entreprise version, do you bind oracle to a defined set of CPU ?

What about the network traffic ?
Do you have the latest network's driver ?


--
Cordialement,
Mathieu CHATEAU
English blog: http://lordoftheping.blogspot.com
French blog: http://www.lotp.fr


"Jacob Christensen" wrote in message
news:(E-Mail Removed)...
> Hi all!
>
> I'm currently facing some issues with a Windows Server 2003 Ent. x64
> part of an Oracle RAC setup. When all CPU's are under heavy load we
> experience high latency and even packet loss on the network. We've
> just started troubleshooting the problem and I would like to hear if
> anyone had comments as to whether this is normal behaviour or not.
>
> When pinging the loopback address we experienced the same issues so I
> guess that rules out the physical network.
>
> Memory and disk bottlenecks does not seem to be an issue.
> I setup some performance monitoring and saw that the processors often
> are busy processing deferred procedure calls (DPCs). - Any ideas to
> this ?
>
> When handling for instance ICMP packets some switches will throw these
> away when busy serving other types of data. Will the same happen in a
> Windows server 2003 environment or is it not that intelligent ?
>
>
> Best regards. Jacob Christensen
>

Hi Mathieu.

Thanks for the reply! :-)

I should have mentioned that of course. On my list for tomorrow is
upgrading the NIC drivers and applying SP2. ( on the same model blade
servers in a test environment) The drivers for the NIC ( Broadcom
nc7/ something i forgot) are version 8.48.0.0 and the newest are
10.42.0.0 . However the release notes for the drivers didn't mention
anything the made any bells ring.

SP2 however does contain a lot of kernels patches resolving issues
with high cpu load and eventual bluescreens. However the system
doesn't crash, it's just heavy loaded with the result mentioned
before.

Furthermore it contains the Scalable Networking Pack, but I'm unsure
if I will benefit from this. Does it require newer NIC's are could i
expect a performance boost from my existing hardware?

Best regards, Jacob


On Thu, 11 Oct 2007 20:30:08 +0200, "Mathieu CHATEAU"
<(E-Mail Removed)> wrote:

>Hello,
>
>do you have installed the Service pack 2 for Windows 2003 ?
>
>As it is an entreprise version, do you bind oracle to a defined set of CPU ?
>
>What about the network traffic ?
>Do you have the latest network's driver ?

From SP2, you would expect better network performance as all CPU can
participate in network tasks.
But you may also have issues:

You may experience network-related problems after you install Windows Server
2003 SP2 or the Scalable Networking Pack
http://support.microsoft.com/kb/936594/en-us

test, test and test again

--
Cordialement,
Mathieu CHATEAU
English blog: http://lordoftheping.blogspot.com
French blog: http://www.lotp.fr


"Jacob Christensen" wrote in message
news:(E-Mail Removed)...
> Hi Mathieu.
>
> Thanks for the reply! :-)
>
> I should have mentioned that of course. On my list for tomorrow is
> upgrading the NIC drivers and applying SP2. ( on the same model blade
> servers in a test environment) The drivers for the NIC ( Broadcom
> nc7/ something i forgot) are version 8.48.0.0 and the newest are
> 10.42.0.0 . However the release notes for the drivers didn't mention
> anything the made any bells ring.
>
> SP2 however does contain a lot of kernels patches resolving issues
> with high cpu load and eventual bluescreens. However the system
> doesn't crash, it's just heavy loaded with the result mentioned
> before.
>
> Furthermore it contains the Scalable Networking Pack, but I'm unsure
> if I will benefit from this. Does it require newer NIC's are could i
> expect a performance boost from my existing hardware?
>
> Best regards, Jacob
>
>
> On Thu, 11 Oct 2007 20:30:08 +0200, "Mathieu CHATEAU"
> <(E-Mail Removed)> wrote:
>
>>Hello,
>>
>>do you have installed the Service pack 2 for Windows 2003 ?
>>
>>As it is an entreprise version, do you bind oracle to a defined set of CPU
>>?
>>
>>What about the network traffic ?
>>Do you have the latest network's driver ?

On Thu, 11 Oct 2007 20:30:08 +0200, "Mathieu CHATEAU"
<(E-Mail Removed)> wrote:


>As it is an entreprise version, do you bind oracle to a defined set of CPU ?

Forgot to answer this one - I actually don't know, I'll check it up
tomorrow. Are you thinking of any particular issue when assigning
hard affinity ?

Best regards, Jacob

By default, process can go on all CPU.

With entreprise edition, you can WSRM to restrict binding to some CPU. So
you may keep a CPU out of Oracle for the system (network, backup ...). That
would be a poor workaround for your issue anyway.



--
Cordialement,
Mathieu CHATEAU
English blog: http://lordoftheping.blogspot.com
French blog: http://www.lotp.fr


"Jacob Christensen" wrote in message
news:(E-Mail Removed)...
> On Thu, 11 Oct 2007 20:30:08 +0200, "Mathieu CHATEAU"
> <(E-Mail Removed)> wrote:
>
>
>>As it is an entreprise version, do you bind oracle to a defined set of CPU
>>?
>
> Forgot to answer this one - I actually don't know, I'll check it up
> tomorrow. Are you thinking of any particular issue when assigning
> hard affinity ?
>
> Best regards, Jacob