Slowness of Apache

Hi,

In the past couple of days our webserver has been quite slow loading
up websites. The weird thing is that nothing has been done to the
apache (v1.3.12 running on linux redhat 6.2) configuration and it
usually happens in the highest traffic time of day being early to late
morning. I figured it would be apache running out of client processes
but the httpd error logs reports no errors. What I have been doing is
restart apache almost every 10 minutes. It's not a bandwidth
situation as I've tested speed using ftp transferring files to and
from the web server. Also figured it could be a switch problem as we
are on 10Mb full duplex to the co-location provider's router. Last
time we had this problem was because our switch would not negotiate to
the router's 10Mb (It would only go 100Mb) so had to force it to 10Mb.
If the problem was the switch then it should affect FTP as well, or
any other protocol.
I've played around with apache modifying maxservers and startservers,
but to no avail!! Any help greatly appreciated.

- Trevor

In article <(E-Mail Removed) >,
(E-Mail Removed) (Trevor) wrote:

> Hi,
>
> In the past couple of days our webserver has been quite slow loading
> up websites. The weird thing is that nothing has been done to the
> apache (v1.3.12 running on linux redhat 6.2) configuration and it
> usually happens in the highest traffic time of day being early to late
> morning. I figured it would be apache running out of client processes
> but the httpd error logs reports no errors. What I have been doing is
> restart apache almost every 10 minutes. It's not a bandwidth
> situation as I've tested speed using ftp transferring files to and
> from the web server. Also figured it could be a switch problem as we
> are on 10Mb full duplex to the co-location provider's router. Last
> time we had this problem was because our switch would not negotiate to
> the router's 10Mb (It would only go 100Mb) so had to force it to 10Mb.
> If the problem was the switch then it should affect FTP as well, or
> any other protocol.
> I've played around with apache modifying maxservers and startservers,
> but to no avail!! Any help greatly appreciated.
>
> - Trevor

First, I'd advise updating to the latest on both ends, system and Apache-
but that's what most would say. You also might look into faster, more macho
hardware. As to the particulars- some very general things to look into.

High traffic sluggishness is usually due to the system hitting its limits as
it parcels out cpu to the processes and the files start flying. Running more
of the things is just going to add to the overhead. Allowing Apache to start
more servers isn't always going to fix that. There's a fine tradeoff in
responsiveness to the client's requests and keeping the machine chugging along
at top speed when things start getting busy.

Running something like 'top' when things go all slowed up will let you see
what's eating resources, and might give you a clue as to what's necessary to
give the server breathing space. Also, if your RAM is limited, swapping will
eat a good deal of time when things get busy. Having a huge number of files
being channeled through the machine simultaneously does tend to make things
hectic.

Bandwidth is not usually the problem here (well, except in extreme cases
where you've got the entirety of North America supping at your server). It's
usually the server going overloaded as it tries to deal with everything going
on at the same time. It'll handle things under those conditions, but at its
own pace. As for measuring bandwidth; keep in mind that a ftp transfer is
(essentially) a single process and a single port being used. However, in your
case, cutting the pipe back to 10M from 100M might have had an impact.

As to specifics: visit the Apache site. I recall seeing a number of articles
there, with links to others, dealing with similar cases of high-load
sluggishness and advice upon how to handle it. Oh yes, just in case: I could
be entirely wrong about this.


- Doug

On Tue, 16 Sep 2003 08:32:30 -0700, Trevor wrote:

> Hi,
>
> In the past couple of days our webserver has been quite slow loading
> up websites. The weird thing is that nothing has been done to the
> apache (v1.3.12 running on linux redhat 6.2) configuration and it
> usually happens in the highest traffic time of day being early to late
> morning. I figured it would be apache running out of client processes
> but the httpd error logs reports no errors. What I have been doing is
> restart apache almost every 10 minutes. It's not a bandwidth
> situation as I've tested speed using ftp transferring files to and
> from the web server. Also figured it could be a switch problem as we
> are on 10Mb full duplex to the co-location provider's router. Last
> time we had this problem was because our switch would not negotiate to
> the router's 10Mb (It would only go 100Mb) so had to force it to 10Mb.
> If the problem was the switch then it should affect FTP as well, or
> any other protocol.
> I've played around with apache modifying maxservers and startservers,
> but to no avail!! Any help greatly appreciated.
>
> - Trevor

Possibilities:

1) It could be doing a rebuild of it's locate database. Is the clock spot on?

2) Check the system log for hardware problems ( like disk full! ) .

3) Check if it's doing any dns lookups before writing it's logs. Your dns
server may be under heavy load as well and keeping Apache waiting on the
log writing section.

hth

joseph philip wrote:
> On Tue, 16 Sep 2003 08:32:30 -0700, Trevor wrote:
>>I've played around with apache modifying maxservers and startservers,
>>but to no avail!! Any help greatly appreciated.
>>
>> - Trevor
>
>
> Possibilities:
>
> 1) It could be doing a rebuild of it's locate database. Is the clock spot on?
>
> 2) Check the system log for hardware problems ( like disk full! ) .
>
> 3) Check if it's doing any dns lookups before writing it's logs. Your dns
> server may be under heavy load as well and keeping Apache waiting on the
> log writing section.


How many simultaneous connections are being made? Do you have any idea?
http://www.acme.com/software/thttpd/notes.html#listen
Will explain another bottleneck.

Hi,

Thanks all for the replies. To answer some questions you've asked,
our server is a dual pentium3 800mhz with 1GbRAM so it should handles
requests no problems. We get about 5million pageviews a month if
that'll help. The problem has just arisen late last week just all of a
sudden and the traffic doesnt appear to be any different than usual
judging from the website logs analysis.
I did however have apache set to 'keepalive' requests enabled which
was like that for 5 years and then we started having kernel panic
issues where apache was still running fine serving pages, but you
couldnt do anything else: ftp, top etc would come up as seg faults.
Someone on a newsgroup suggested that keepalive was not a good idea
for heavy load sites so I disabled that which immediately fixed the
problem of seg faults for quite a few months, until now!!! I ended up
re-enabling keepalives but drop the timeouts down to 10 seconds and
dropped the number of keepalive requests. At this stage it has seemed
to fix the problem as far as page loading speeds, but to the sacrifice
of high CPU usage and number of processes running.
The locate cron task wasnt running at the times when the page loading
was slow. Also I disabled hostnames lookup to see if it was the DNS
server causing the delays. This did not have any effect on the page
loading speed. I did however run tcpdump and grepp'ed just port 80 get
requests and when the pages were not loading or timed out, I noticed
tcpdump displayed nothing on the console.
The HDs definitely are not full.

This is a paste from the server-status page:-
Current Time: Thursday, 18-Sep-2003 11:02:12 EDT
Restart Time: Thursday, 18-Sep-2003 10:35:35 EDT
Parent Server Generation: 0
Server uptime: 26 minutes 37 seconds
Total accesses: 226599 - Total Traffic: 789.6 MB
CPU Usage: u534.33 s463.15 cu49.76 cs8.77 - 66.1% CPU load
142 requests/sec - 506.3 kB/second - 3653 B/request
240 requests currently being processed, 192 idle servers

Note though this is with keepalive turn on and max clients set to 512
(I recompiled apache with higher hard limit than 256).

This is what I have set for apache server limits:-
MinSpareServers 5
MaxSpareServers 15
StartServers 10
MaxClients 512
MaxRequestsPerChild 20000

This is quite a mystery as I've never had this problem before, plus
our traffic is fairly consistent so its not a sudden surge in traffic.

In article <(E-Mail Removed) >,
(E-Mail Removed) (Trevor) wrote:

> Hi,
>
> Thanks all for the replies. To answer some questions you've asked,
> our server is a dual pentium3 800mhz with 1GbRAM so it should handles
> requests no problems. We get about 5million pageviews a month if
> that'll help. The problem has just arisen late last week just all of a
> sudden and the traffic doesnt appear to be any different than usual
> judging from the website logs analysis.

[Snippage}

>
> This is quite a mystery as I've never had this problem before, plus
> our traffic is fairly consistent so its not a sudden surge in traffic.

A long shot: was any of the content that made use of one of the Apache
support modules changed recently? Perl, cgi, php, et al?


- Doug

Im a new user .. So which one do u suggest to start ..

somebody says that mandrake was good for new personel user

thans

Doug Winger <(E-Mail Removed)> wrote in message news:<justdoug-(E-Mail Removed)>...
> A long shot: was any of the content that made use of one of the Apache
> support modules changed recently? Perl, cgi, php, et al?
>

Nope..nothing had been changed recently.

- Trevor