Proftpd - High Load averages and strange performance problems.

Hello,

We recently upgraded a server with a few hundred ftp users from a P4
1.7Ghz to P4 2.4Ghz with HyperThreading, and since that time there
seems to be something wrong with ProFTPD. The server is RedHat 9
(latest glibc, smp kernel, sendmail etc.etc.etc...) We compiled
proftpd-1.2.9rc2 to start with, but when it wasn't allowing users to
overwrite their own files (for no apparent reason), we then replaced
this with proftpd-1.2.8, which seemed to fix that problem.

Whenever I try to log into a users account, if they have more than 72
files and directories, the ls only lists 72 then hangs. When I watch
TOP on the server, the loads skyrocket as high as 66.xx at which time
my pkill -9 proftpd finally kicked in and shut the whole thing down.
Normally this server sites at 0.05 to 0.50 during peak hours.

Standalone configs:
################### MAIN ######################
ServerName xxxxxx
ServerType standalone
ServerAdmin xxxxx
TimesGMT off
DefaultServer on
Port 21
Umask 022
MaxInstances 25
TimeoutLogin 60
TimeoutIdle 600
TimeoutNoTransfer 600
UseReverseDNS off
IdentLookups off
User root
Group root
<Directory /*>
AllowOverwrite on
<Limit ALL>
IgnoreHidden off
AllowUser root
</Limit ALL>
</Directory>

<Global>
HiddenStor off
ListOptions "-a"
MaxClients 25
RLimitCPU 15 30 #Just added to try and keep loads down...
ServerIdent off
AllowOverwrite on
ShowSymlinks off
ExtendedLog /var/log/ftp.log
</Global>

Include /etc/proftpd.conf

################## INCLUDE FILE ################

<VirtualHost 192.168.0.10>
ServerName domain.com

<Limit LOGIN>
DenyUser !mruser1
</Limit LOGIN>

DefaultRoot /directory/mruser1
DefaultChdir /directory/mruser1/www/
ExtendedLog /directory/mruser1/httpd/logs/ext-ftp.log ALL
TransferLog /directory/mruser1/httpd/logs/ftp.log
MaxClients 10
</VirtualHost 192.168.0.10>

<VirtualHost 192.168.1.20>
# seconduser
ServerName anotherdomain.com
DefaultRoot /directory/seconduser
DefaultChdir /directory/seconduser/www
ExtendedLog /directory/seconduser/httpd/logs/ext-ftp.log ALL
TransferLog /directory/seconduser/httpd/logs/ftp.log

#<Limit LOGIN>
# DenyUser !seconduser !anonymous !ftp
#</Limit LOGIN>

<Anonymous /directory/seconduser/anonftp>
User ftp
Group siteuser
UserAlias anonymous ftp
MaxClients 10
# TransferRate 51200

TransferLog /directory/seconduser/httpd/logs/anon-ftp.log

<Directory /directory/seconduser/anonftp>
<Limit DIRS READ CWD>
AllowAll
</Limit DIRS READ CWD>
</Directory>

<Directory /directory/seconduser/anonftp/outgoing/*>
<Limit WRITE DELE RMD STOR>
DenyUser !seconduser
</Limit WRITE DELE RMD STOR>
</Directory>

<Directory /directory/seconduser/anonftp/incoming/>
AllowOverwrite on
<Limit WRITE STOR DIRS>
AllowAll
</Limit WRITE STOR DIRS>
<Limit RETR READ DELE MKD>
DenyUser !seconduser
</Limit RETR READ DELE MKD>
</Directory>
</Anonymous>
# seconduser
</VirtualHost 192.168.1.20>

########################

There are an assortment of a few hundred users setup with one or the
other of the above user configs...

LOG ENTRIES: Seem normal but one thing that I don't understand is this
UNKNOWN portion:
192.168.10.10 UNKNOWN mruser1 [11/Oct/2003:08:57:19 -0700] "LIST" 226
1687
^^^^^^^ Seems strange, not sure what that's about.

FTP Client: I have tried command-line FTP from a machine in the same
network, and as well from remote locations etc... all get the same
results. There are generally 2-3 users logged in at one time, so the
ftp related loads are pretty nominal.

Thanks in advance for any help... I've searched many groups but
haven't found any answers (at least in English anyway)...

Mike

Well... after downgrading from 1.2.9rc2 --> 1.2.8p --> 1.2.7p the
problem is now resolved. So it seems there is something significantly
wrong with the latest distributions of Proftpd.

Good Luck...

> (latest glibc, smp kernel, sendmail etc.etc.etc...) We compiled
> proftpd-1.2.9rc2 to start with, but when it wasn't allowing users to
> overwrite their own files (for no apparent reason), we then replaced
> this with proftpd-1.2.8, which seemed to fix that problem.

> Whenever I try to log into a users account, if they have more than 72
> files and directories, the ls only lists 72 then hangs.

I posted the following on bugzilla when I noticed a similar problem on my
own server. It sounds like it may be the same bug:

"Hanging during LIST command, high CPU usage"
http://bugs.proftpd.org/show_bug.cgi?id=2226

Can you please append your information to that bug report, so the
developers can realize the significance of the bug and collect more data on
how to reproduce it?

--
Jem Berkes
http://www.sysdesign.ca/