reads slow, writes fast, on server.

since a couple of months i'm using a new fileserver but i have very
slow reads from the server while i can write to it in normal speeds.
the reads are only 1/20 of the write speed.
at first i thought i was using a bad network card (onboard Yukon
Gigabit Ethernet 10/100/1000Base-T Adapter) so i bought a seperate one
(3com 3c2000-T) but the problem was the same.
my next guess was that the software raid had troubles so i mounted a
ramdisk to read from but that didn't help either.
i ruled out bad infrastructure by using a twisted cable to directly
connect to a client. but still..

any suggestions on what to do next?

janm
--
I am the Lizard King
I can do anything

Le Tue, 27 Sep 2005 15:47:55 +0200, Jan Mannoury a écrit*:

> since a couple of months i'm using a new fileserver but i have very
> slow reads from the server while i can write to it in normal speeds.
> the reads are only 1/20 of the write speed.
> at first i thought i was using a bad network card (onboard Yukon
> Gigabit Ethernet 10/100/1000Base-T Adapter) so i bought a seperate one
> (3com 3c2000-T) but the problem was the same.
> my next guess was that the software raid had troubles so i mounted a
> ramdisk to read from but that didn't help either.
> i ruled out bad infrastructure by using a twisted cable to directly
> connect to a client. but still..
>
> any suggestions on what to do next?
>
> janm

Hi!

You can check the operating mode of the network interfaces
(auto-negotation issue ?), also check if you don't have iptables
loaded .

what kind of hardware/distro do you use ?

--
=Gilles=

On Tue, 27 Sep 2005 17:28:40 +0200, gilles <(E-Mail Removed)>
wrote:

>Le Tue, 27 Sep 2005 15:47:55 +0200, Jan Mannoury a écrit*:
>> any suggestions on what to do next?
>Hi!
>
>You can check the operating mode of the network interfaces
>(auto-negotation issue ?), also check if you don't have iptables
>loaded .

i have this now (dmesg):
eth0: network connection up using port A
speed: 100
autonegotiation: yes
duplex mode: half
flowctrl: none
irq moderation: disabled
scatter-gather: enabled
NET: Registered protocol family 17
NET: Registered protocol family 10
Disabled Privacy Extensions on device c02e60a0(lo)
IPv6 over IPv4 tunneling driver
eth0: no IPv6 routers present

no iptables on this machine.

>what kind of hardware/distro do you use ?

its an athlon XP on an asus mainboard. 1G ram (dmesg: "Warning only
896MB will be used. Use a HIGHMEM enabled kernel." <-- still have to
fix that) it runs debian unstable with stockkernel 2.6.11-1-386

janm
--
I am the Lizard King
I can do anything

On Tue, 27 Sep 2005 20:31:06 +0200, Jan Mannoury wrote:

> eth0: network connection up using port A
> speed: 100
> autonegotiation: yes
> duplex mode: half

You should investigate why eth0 is not running full duplex. See if you
can force both sides to full duplex using mii-tool.

Check your transfer speeds in both directions using ttcp.

"Jan Mannoury" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)

> since a couple of months i'm using a new fileserver but i have very
> slow reads from the server while i can write to it in normal speeds.
> the reads are only 1/20 of the write speed.
....
> any suggestions on what to do next?

Perhaps to illustrate the problem you can provide some simple metrics for
people to consider, e.g. the output of:

yes | time dd of=junk count=8192
time if=junk of=/dev/null

On Tue, 27 Sep 2005 12:11:16 -0700, "ynotssor" <(E-Mail Removed)>
wrote:

>"Jan Mannoury" <(E-Mail Removed)> wrote in message
>news:(E-Mail Removed)
>
>> since a couple of months i'm using a new fileserver but i have very
>> slow reads from the server while i can write to it in normal speeds.
>> the reads are only 1/20 of the write speed.
>...
>> any suggestions on what to do next?
>
>Perhaps to illustrate the problem you can provide some simple metrics for
>people to consider, e.g. the output of:
>
>yes | time dd of=junk count=8192
>time if=junk of=/dev/null

i guess the second command is dd too? shouldn't i be doing this over
the network? i think this is what you asked:
===
pengi:~# yes | time dd of=junk count=8192
8192+0 records in
8192+0 records out
4194304 bytes transferred in 0.122484 seconds (34243719 bytes/sec)
0.00user 0.03system 0:00.12elapsed 27%CPU (0avgtext+0avgdata
0maxresident)k
0inputs+0outputs (0major+165minor)pagefaults 0swaps
pengi:~# time dd if=junk of=/dev/null
8192+0 records in
8192+0 records out
4194304 bytes transferred in 0.011347 seconds (369638103 bytes/sec)

real 0m0.012s
user 0m0.005s
sys 0m0.008s
===
i see a significant difference but i don't really understand what it
means.

janm
--
I am the Lizard King
I can do anything

On Tue, 27 Sep 2005 16:04:27 -0300, Snowbat <(E-Mail Removed)>
wrote:

>On Tue, 27 Sep 2005 20:31:06 +0200, Jan Mannoury wrote:
>
>> eth0: network connection up using port A
>> speed: 100
>> autonegotiation: yes
>> duplex mode: half
>
>You should investigate why eth0 is not running full duplex. See if you
>can force both sides to full duplex using mii-tool.

i get to read a lot of manpages tonight. :-)
i don't know if i can force both sides cause the other side is a
switch or a lot of windows-clients. i noticed there was full duplex
mode with the 3-com i tried today with a crossed cable to my laptop.
they both were in gigabit mode then (i had never done that before) and
it was a bit strange to see an 800mb file sent in under a minute and
get it back in an hour or so.

>Check your transfer speeds in both directions using ttcp.

another tool i don't know yet. i'll dive into that tomorrow.

janm
--
I am the Lizard King
I can do anything

"Jan Mannoury" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)

>> Perhaps to illustrate the problem you can provide some simple
>> metrics for people to consider, e.g. the output of:
>>
>> yes | time dd of=junk count=8192
>> time if=junk of=/dev/null
>
> i guess the second command is dd too? shouldn't i be doing this over
> the network?

Yes, my apologies for the command omission which you clearly perceived, and
of course you should be doing this in whatever context the problem exists,
as the purpose is to measure the problem; alter the of= and if= accordingly.

> i think this is what you asked:
> ===
> pengi:~# yes | time dd of=junk count=8192
> 8192+0 records in
> 8192+0 records out
> 4194304 bytes transferred in 0.122484 seconds (34243719 bytes/sec)
> 0.00user 0.03system 0:00.12elapsed 27%CPU (0avgtext+0avgdata
> 0maxresident)k
> 0inputs+0outputs (0major+165minor)pagefaults 0swaps
> pengi:~# time dd if=junk of=/dev/null
> 8192+0 records in
> 8192+0 records out
> 4194304 bytes transferred in 0.011347 seconds (369638103 bytes/sec)
>
> real 0m0.012s
> user 0m0.005s
> sys 0m0.008s
> ===
> i see a significant difference but i don't really understand what it
> means.

It shows that reads are faster than writes when performed locally on the
drive. You would want to see the difference "over the network" as you say.
It's just a simple diagnostic tool to help identify the nature and magnitude
of the problem.

Of course the duplex mode of the network interface is of interest, as others
have mentioned.

On Tue, 27 Sep 2005 13:51:05 -0700, "ynotssor" <(E-Mail Removed)>
wrote:

>"Jan Mannoury" <(E-Mail Removed)> wrote in message
>news:(E-Mail Removed)

>> i see a significant difference but i don't really understand what it
>> means.
>
>It shows that reads are faster than writes when performed locally on the
>drive. You would want to see the difference "over the network" as you say.
>It's just a simple diagnostic tool to help identify the nature and magnitude
>of the problem.

i'll set up some nfs shares tomorrow and try it.

>Of course the duplex mode of the network interface is of interest, as others
>have mentioned.

i'll look into that too.

thanks,
janm
--
I am the Lizard King
I can do anything

"Jan Mannoury" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...

> since a couple of months i'm using a new fileserver but i have very
> slow reads from the server while i can write to it in normal speeds.
> the reads are only 1/20 of the write speed.
> at first i thought i was using a bad network card (onboard Yukon
> Gigabit Ethernet 10/100/1000Base-T Adapter) so i bought a seperate one
> (3com 3c2000-T) but the problem was the same.
> my next guess was that the software raid had troubles so i mounted a
> ramdisk to read from but that didn't help either.
> i ruled out bad infrastructure by using a twisted cable to directly
> connect to a client. but still..
>
> any suggestions on what to do next?

What are the read and write speeds you observed? And is this gigabit on
both ends?

Write speeds can be deceptively high because you may be measuring how
long it takes to *start* the write. Read speeds generally measure how long
it takes to complete the read because the data isn't available until the
read completes.

Most likely the problem is in your expectations or how you are
measuring. (Assuming you don't have a duplex mismatch or something like
that.)

DS