NFS high load when writing

In our setup, we have a fileserver and a login server. The file server
houses all /home directories on a raid 5 array, the login server maps
this using nfs. When locally on the file server, pushing a 15mb file
from /tmp to /home (the raid 5 array) it goes extremely fast and is done
almost instantly, however when on the logon server pushing the 15mb file
from /tmp to /home (which is mounted to the file server via nfs) it
takes a good 45 seconds to a minute. however, reads from /home are fine,
its only writing. But the raid array speed is fine locally so I dont
believe it is that.

Would appreciate all help

Thanks

On Tue, 21 Sep 2004 15:10:38 -0400, Justin wrote:

> In our setup, we have a fileserver and a login server. The file server
> houses all /home directories on a raid 5 array, the login server maps this
> using nfs. When locally on the file server, pushing a 15mb file from /tmp
> to /home (the raid 5 array) it goes extremely fast and is done almost
> instantly, however when on the logon server pushing the 15mb file from
> /tmp to /home (which is mounted to the file server via nfs) it takes a
> good 45 seconds to a minute. however, reads from /home are fine, its only
> writing. But the raid array speed is fine locally so I dont believe it is
> that.

Have you got "sync" in the mount options on the client? If so try removing
it.

Regards, Ian

Ian Northeast wrote:
> On Tue, 21 Sep 2004 15:10:38 -0400, Justin wrote:
>
>
>>In our setup, we have a fileserver and a login server. The file server
>>houses all /home directories on a raid 5 array, the login server maps this
>>using nfs. When locally on the file server, pushing a 15mb file from /tmp
>>to /home (the raid 5 array) it goes extremely fast and is done almost
>>instantly, however when on the logon server pushing the 15mb file from
>>/tmp to /home (which is mounted to the file server via nfs) it takes a
>>good 45 seconds to a minute. however, reads from /home are fine, its only
>>writing. But the raid array speed is fine locally so I dont believe it is
>>that.
>
>
> Have you got "sync" in the mount options on the client? If so try removing
> it.
>
> Regards, Ian
>
nope, sync is not in there.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
NotDashEscaped: You need GnuPG to verify this message

In comp.os.linux.networking Justin <(E-Mail Removed)> suggested:
> In our setup, we have a fileserver and a login server. The file server
> houses all /home directories on a raid 5 array, the login server maps
> this using nfs. When locally on the file server, pushing a 15mb file
> from /tmp to /home (the raid 5 array) it goes extremely fast and is done
> almost instantly, however when on the logon server pushing the 15mb file
> from /tmp to /home (which is mounted to the file server via nfs) it
> takes a good 45 seconds to a minute. however, reads from /home are fine,
{..]

Might be just a network problem, I'd check 'ifconfig <device>'
for growing numbers in these lines during file transfer for
what's going on.

RX .. errors:4 dropped:0 overruns:0 frame:0
TX .. errors:0 dropped:0 overruns:8 carrier:0

Please don't remove TX RX numbers like I did, so we can have a
look at them.

--
Michael Heiming (GPG-Key ID: 0xEDD27B94)
mail: echo (E-Mail Removed) | perl -pe 'y/a-z/n-za-m/'
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFBUI+6AkPEju3Se5QRAiYUAKCDHkKCYFyidc5OWRTx18 X6XINMHQCgqaJ2
2D0q5J2qpulbNn+gKUGDVXk=
=J7So
-----END PGP SIGNATURE-----

Michael Heiming wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> NotDashEscaped: You need GnuPG to verify this message
>
> In comp.os.linux.networking Justin <(E-Mail Removed)> suggested:
>
>>In our setup, we have a fileserver and a login server. The file server
>>houses all /home directories on a raid 5 array, the login server maps
>>this using nfs. When locally on the file server, pushing a 15mb file
>>from /tmp to /home (the raid 5 array) it goes extremely fast and is done
>>almost instantly, however when on the logon server pushing the 15mb file
>>from /tmp to /home (which is mounted to the file server via nfs) it
>>takes a good 45 seconds to a minute. however, reads from /home are fine,
>
> {..]
>
> Might be just a network problem, I'd check 'ifconfig <device>'
> for growing numbers in these lines during file transfer for
> what's going on.
>
> RX .. errors:4 dropped:0 overruns:0 frame:0
> TX .. errors:0 dropped:0 overruns:8 carrier:0
>
> Please don't remove TX RX numbers like I did, so we can have a
> look at them.
>
0 errors in ifconfig.

the network link to the server remains up fine. its only the load that
skyrockets.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
NotDashEscaped: You need GnuPG to verify this message

In comp.os.linux.networking Justin <(E-Mail Removed)> suggested:
> Michael Heiming wrote:
>> In comp.os.linux.networking Justin <(E-Mail Removed)> suggested:
[..]
>> Might be just a network problem, I'd check 'ifconfig <device>'
>> for growing numbers in these lines during file transfer for
>> what's going on.
>>
>> RX .. errors:4 dropped:0 overruns:0 frame:0
>> TX .. errors:0 dropped:0 overruns:8 carrier:0
>>
>> Please don't remove TX RX numbers like I did, so we can have a
>> look at them.
>>
> 0 errors in ifconfig.

> the network link to the server remains up fine. its only the load that
> skyrockets.

Sorry, but why don't you simply post the two complete lines?
People want to see data not your impressions.

--
Michael Heiming (GPG-Key ID: 0xEDD27B94)
mail: echo (E-Mail Removed) | perl -pe 'y/a-z/n-za-m/'
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFBUKS6AkPEju3Se5QRAgleAKC9J/96ouvX5DLLx0SNk167vbhCoACfRlks
NN1nN8BpAblpj9MXt6jkqrk=
=hL/Z
-----END PGP SIGNATURE-----

On 2004-09-21, Justin wrote:
> Ian Northeast wrote:
>> On Tue, 21 Sep 2004 15:10:38 -0400, Justin wrote:
>>
>>
>>>In our setup, we have a fileserver and a login server. The file server
>>>houses all /home directories on a raid 5 array, the login server maps this
>>>using nfs. When locally on the file server, pushing a 15mb file from /tmp
>>>to /home (the raid 5 array) it goes extremely fast and is done almost
>>>instantly, however when on the logon server pushing the 15mb file from
>>>/tmp to /home (which is mounted to the file server via nfs) it takes a
>>>good 45 seconds to a minute. however, reads from /home are fine, its only
>>>writing. But the raid array speed is fine locally so I dont believe it is
>>>that.
>>
>>
>> Have you got "sync" in the mount options on the client? If so try removing
>> it.
>>
>> Regards, Ian
>>
> nope, sync is not in there.

Of course there is explicit async?

--
Stanislaw Klekot

>
> nope, sync is not in there.

The default is sync.

--
Angel R. Rivera aka Wolf
----------------------------------------------------------------
Please post all reponses to UseNet. All email cheerfully and automagically
routed to Dave Null

On Wed, 22 Sep 2004 04:28:23 +0000, Wolf wrote:

>
>
>> nope, sync is not in there.
>
> The default is sync.

Not on any system I have seen it isn't. But the man page doesn't say and
nor does "mount" - this is just from experience. My systems are Debian
Woody and SuSE Enterprise 7 and 8. So there is no harm in the OP's trying
an explicit async to see if it has any effect. The default for an export
from an NFS server is usually sync nowadays, but this is completely
different. That means that the server will wait until the I/O is complete
before signalling so to the client. It doesn't make the client wait for
it. This is a good thing.

On the subject of defaults, this reminds me, there is one which could be
relevant here. The default number of bytes read and written at a time is
1024. It is recommended to increase this by specifying
"rsize=8192,wsize=8192" and this does indeed speed things a lot IME. Some
people recommend even larger sizes but I've not seen that make a lot of
difference.

Regards, Ian

Ian Northeast wrote:

> On Wed, 22 Sep 2004 04:28:23 +0000, Wolf wrote:
>
>>
>>
>>> nope, sync is not in there.
>>
>> The default is sync.
>
> Not on any system I have seen it isn't.

For newer versions of nfs-utils, yes, it's true:

See http://nfs.sourceforge.net/ in "Quick Overview":

* NFS Version 2 and 3 servers now default to synchronous IO (the "sync"
export option), if nfs-utils-1.0.1 (or later) is installed. (see questions
B1, B4, B5, and B6).

-- Rex