Hacked, now trying to disinfect

Yeah, i know, it can't happen in Linux. But it has been happening to
our work servers for several months.

Due to poor security practices of the past catching up to us, three of
our servers (www,mail, and internal business software) got hacked
into, and now there's some bug installed that monitors and logs ssh
logins. It writes login information to /etc/host.

The www and mail servers are running FC6 and Cent4.4, respectively,
and the other is running Slack 10. Suggesting a different distro for
the Slackware box isn't an option at this point. The software that
runs on it is 20+ years old and barely runs even on that OS. The
others seem to work fine aside from the password logger and any other,
more subtle infections present.

i've been looking around, and can't find any references to "/etc/
host" (most links refer to the valid "/etc/hosts" or "host.conf" or
"host.allow/deny" ... Does anyone have any info on this type of
logger? It's clear enough that whoever is doing this is managing to
catch other credentials beyond just the ssh sessions, and the worst
actual damage we've seen has been creating phishing pages. We're
trying to beef up security, but now it's an even steeper uphill battle
with the enemy already inside.

Any help or info on this type of attack would be appreciated.
-joe t.

On Wed, 01 Aug 2007 21:07:45 +0000, joe t. wrote:

> Yeah, i know, it can't happen in Linux. But it has been happening to our
> work servers for several months.

Sure it can. Most, probably all, Linux distros are shipped with *root*
login enabled in sshd. If you expose such a system to the Internet you
are almost certain to get successfully attacked.

> i've been looking around, and can't find any references to "/etc/ host"
> (most links refer to the valid "/etc/hosts" or "host.conf" or
> "host.allow/deny" ... Does anyone have any info on this type of logger?
> It's clear enough that whoever is doing this is managing to catch other
> credentials beyond just the ssh sessions, and the worst actual damage
> we've seen has been creating phishing pages. We're trying to beef up
> security, but now it's an even steeper uphill battle with the enemy
> already inside.

Any of that effort is futile. Backup what good data you have and
reinstall, this time blocking root ssh login.

"joe t." <(E-Mail Removed)> writes:

>Yeah, i know, it can't happen in Linux. But it has been happening to
>our work servers for several months.

Of course it can happen. The usual way is for your password to get hacked
from one of your users. There are password bots out there whcih try to
attack ssh with a guessing attack.

>Due to poor security practices of the past catching up to us, three of
>our servers (www,mail, and internal business software) got hacked
>into, and now there's some bug installed that monitors and logs ssh
>logins. It writes login information to /etc/host.

Yup. The best thing to do now is to a) backup your data, b) do a complete
reformat and reinstall, and c) Do a scan of all of the backups looking for
suid programs.d( change all passwords.ALL.
The hacker knows them all . And once you have done that only then let the
machines back on the net.



>The www and mail servers are running FC6 and Cent4.4, respectively,
>and the other is running Slack 10. Suggesting a different distro for
>the Slackware box isn't an option at this point. The software that
>runs on it is 20+ years old and barely runs even on that OS. The
>others seem to work fine aside from the password logger and any other,
>more subtle infections present.

>i've been looking around, and can't find any references to "/etc/
>host" (most links refer to the valid "/etc/hosts" or "host.conf" or
>"host.allow/deny" ... Does anyone have any info on this type of
>logger? It's clear enough that whoever is doing this is managing to

No they will just grab filenames that look innocuous. Mine had a
/tmp/banana, /dev/cron, and various other files as suid root shells.
(I got broken into because I used telnet and some of my users were in Korea
and got sniffed)
>catch other credentials beyond just the ssh sessions, and the worst
>actual damage we've seen has been creating phishing pages. We're
>trying to beef up security, but now it's an even steeper uphill battle
>with the enemy already inside.

>Any help or info on this type of attack would be appreciated.
>-joe t.

If you want make a system backup that you can study, but first get things
back on track.

Dave Uhring <(E-Mail Removed)> writes:

>On Wed, 01 Aug 2007 21:07:45 +0000, joe t. wrote:

>> Yeah, i know, it can't happen in Linux. But it has been happening to our
>> work servers for several months.

>Sure it can. Most, probably all, Linux distros are shipped with *root*
>login enabled in sshd. If you expose such a system to the Internet you
>are almost certain to get successfully attacked.

Now that is nonesense. You will get attacked, but with a proper password,
the guessing can go on forever.



>> i've been looking around, and can't find any references to "/etc/ host"
>> (most links refer to the valid "/etc/hosts" or "host.conf" or
>> "host.allow/deny" ... Does anyone have any info on this type of logger?
>> It's clear enough that whoever is doing this is managing to catch other
>> credentials beyond just the ssh sessions, and the worst actual damage
>> we've seen has been creating phishing pages. We're trying to beef up
>> security, but now it's an even steeper uphill battle with the enemy
>> already inside.

>Any of that effort is futile. Backup what good data you have and
>reinstall, this time blocking root ssh login.

Unruh wrote:
> Dave Uhring <(E-Mail Removed)> writes:
>
>> On Wed, 01 Aug 2007 21:07:45 +0000, joe t. wrote:
>
>>> Yeah, i know, it can't happen in Linux. But it has been happening to our
>>> work servers for several months.
>
>> Sure it can. Most, probably all, Linux distros are shipped with *root*
>> login enabled in sshd. If you expose such a system to the Internet you
>> are almost certain to get successfully attacked.
>
> Now that is nonesense. You will get attacked, but with a proper password,
> the guessing can go on forever.

Not forever. I had a "strong password" on a system I installed. The
sysadmin failed to notice an attack that started on a Friday afternoon;
by Sunday the system had been compromised. The attack used a
coordinated approach from compromised machines in Romania and Korea, mostly.

Unfortunately the sysadmin also removed the local firewall on that
machine as they had just installed a new hardware firewall, which did
not include a rate-limiter for ssh connections.

*Any* machine can be compromised, given slack enough security in other
areas, even with a strong password, if your pipe is big enough, your CPU
fast enough, and you don't rate-limit new connections.

CptDondo <(E-Mail Removed)> writes:

> Unruh wrote:
>> Dave Uhring <(E-Mail Removed)> writes:
>>
>>> On Wed, 01 Aug 2007 21:07:45 +0000, joe t. wrote:
>>
>>>> Yeah, i know, it can't happen in Linux. But it has been happening to our
>>>> work servers for several months.
>>
>>> Sure it can. Most, probably all, Linux distros are shipped with
>>> *root* login enabled in sshd. If you expose such a system to the
>>> Internet you are almost certain to get successfully attacked.
>> Now that is nonesense. You will get attacked, but with a proper
>> password,
>> the guessing can go on forever.
>
> Not forever. I had a "strong password" on a system I installed. The
> sysadmin failed to notice an attack that started on a Friday
> afternoon; by Sunday the system had been compromised. The attack used
> a coordinated approach from compromised machines in Romania and Korea,
> mostly.

Either they got lucky or your password wasn't that strong.
Here's how I calculated it.

A strong password should be immunune to dictionary attacks. In such a
case, the number of possibilities in an exhaustive search assuming an
8-character password is (52+10+10)^8 = 7.2.E14 password guesses,
assuming 10 symbols are available in addition to 52 letters and 10 numbers.

Now let's assume the machine had a 100 Mbit/sec connection to the internet,
and let's assume that it takes 10 bytes to query and 10 bytes to respond
to the sshd server with a username/password. That means you can make
100E6 / (20*8) = 625000 username/password attempts per second.

Assume the password is guessed in 1/100 of the total possible
attempts. Then it would take

(7.22E14 / 100) [password guesses] * 1 sec / (625000 [password guesses])
= 133 days

to guess.

Have I reasoned something incorrectly? If anything, I think I erred
on the side of the hacker.
--
% Randy Yates % "Midnight, on the water...
%% Fuquay-Varina, NC % I saw... the ocean's daughter."
%%% 919-577-9882 % 'Can't Get It Out Of My Head'
%%%% <(E-Mail Removed)> % *El Dorado*, Electric Light Orchestra
http://home.earthlink.net/~yatescr

CptDondo <(E-Mail Removed)> writes:

>Unruh wrote:
>> Dave Uhring <(E-Mail Removed)> writes:
>>
>>> On Wed, 01 Aug 2007 21:07:45 +0000, joe t. wrote:
>>
>>>> Yeah, i know, it can't happen in Linux. But it has been happening to our
>>>> work servers for several months.
>>
>>> Sure it can. Most, probably all, Linux distros are shipped with *root*
>>> login enabled in sshd. If you expose such a system to the Internet you
>>> are almost certain to get successfully attacked.
>>
>> Now that is nonesense. You will get attacked, but with a proper password,
>> the guessing can go on forever.

>Not forever. I had a "strong password" on a system I installed. The
>sysadmin failed to notice an attack that started on a Friday afternoon;
>by Sunday the system had been compromised. The attack used a
>coordinated approach from compromised machines in Romania and Korea, mostly.

I am sorry, but you can only try about 2 passwords per second. Two days is
4x10^5 trials. That is very small even of a 8 character password ( andall
current systems allow an arbitrary length). even at only 40 character, that
is about 10^13 passwords. A strong one would be a random selection so in 2
days the chances of breaking it is 10^-8. Ie, you should consider entering
the lottery.


>Unfortunately the sysadmin also removed the local firewall on that
>machine as they had just installed a new hardware firewall, which did
>not include a rate-limiter for ssh connections.

>*Any* machine can be compromised, given slack enough security in other
>areas, even with a strong password, if your pipe is big enough, your CPU
>fast enough, and you don't rate-limit new connections.

The ssh daemon/pam daemon is not that fast.
10^8 trials per second means you have a terabit network connection to
Romania and Korea. Pretty good.

joe t. schrieb:
> Yeah, i know, it can't happen in Linux. But it has been happening to
> our work servers for several months.

>
> Any help or info on this type of attack would be appreciated.
> -joe t.
>

do not disinfect. save/rescue any important data and configurations (no
binaries!!!!! ASCII configs only, no scripts neither!) and better
reinstall the whole system. the chance to get back a clean system from a
hacked one is small and sometimes nearly impossible.

it usualy is easier , safer and faster to reinstall the whole system
with TRUSTED Installation-Media, Sources and with higher security policies.

then also install and maintenance a host based IDS or some programm that
tracks changes to important system areas and files in there.. like
tripwire and similar.

thats just my recommentations.

On Aug 2, 6:36 am, Axel Werner <axel.wer...@akadpol.bwl.de> wrote:
> joe t. schrieb:
>
> > Yeah, i know, it can't happen in Linux. But it has been happening to
> > our work servers for several months.
>
> > Any help or info on this type of attack would be appreciated.
> > -joe t.
>
> do not disinfect. save/rescue any important data and configurations (no
> binaries!!!!! ASCII configs only, no scripts neither!) and better
> reinstall the whole system. the chance to get back a clean system from a
> hacked one is small and sometimes nearly impossible.
>
> it usualy is easier , safer and faster to reinstall the whole system
> with TRUSTED Installation-Media, Sources and with higher security policies.
>
> then also install and maintenance a host based IDS or some programm that
> tracks changes to important system areas and files in there.. like
> tripwire and similar.
>
> thats just my recommentations.

That's what i thought would end up being the case. i appreciate
everyone's responses. Looks like a long weekend ahead.
-joe t.

V Wed, 01 Aug 2007 23:39:58 -0400, Randy Yates napsal(a):

> CptDondo <(E-Mail Removed)> writes:
>
>> Unruh wrote:
>>> Dave Uhring <(E-Mail Removed)> writes:
>>>
>>>> On Wed, 01 Aug 2007 21:07:45 +0000, joe t. wrote:
>>>
>>>>> Yeah, i know, it can't happen in Linux. But it has been happening to our
>>>>> work servers for several months.
>>>
>>>> Sure it can. Most, probably all, Linux distros are shipped with
>>>> *root* login enabled in sshd. If you expose such a system to the
>>>> Internet you are almost certain to get successfully attacked.
>>> Now that is nonesense. You will get attacked, but with a proper
>>> password,
>>> the guessing can go on forever.
>>
>> Not forever. I had a "strong password" on a system I installed. The
>> sysadmin failed to notice an attack that started on a Friday
>> afternoon; by Sunday the system had been compromised. The attack used
>> a coordinated approach from compromised machines in Romania and Korea,
>> mostly.
>
> Either they got lucky or your password wasn't that strong.
> Here's how I calculated it.
>
> A strong password should be immunune to dictionary attacks. In such a
> case, the number of possibilities in an exhaustive search assuming an
> 8-character password is (52+10+10)^8 = 7.2.E14 password guesses,
> assuming 10 symbols are available in addition to 52 letters and 10 numbers.
>
> Now let's assume the machine had a 100 Mbit/sec connection to the internet,
> and let's assume that it takes 10 bytes to query and 10 bytes to respond
> to the sshd server with a username/password. That means you can make
> 100E6 / (20*8) = 625000 username/password attempts per second.
>
> Assume the password is guessed in 1/100 of the total possible
> attempts. Then it would take
>
> (7.22E14 / 100) [password guesses] * 1 sec / (625000 [password guesses])
> = 133 days
>
> to guess.
>
> Have I reasoned something incorrectly? If anything, I think I erred
> on the side of the hacker.

Well,they got lucky. The password was *not* a dictionary password, and
was composed of upper and lower case letters. Not entirely random, but
still pretty strong.

My point is, don't bet security on luck.....

That's the only time a system I've worked on got hacked. Multiple layers,
multiple defenses - but I learned and now disable root logins by default
on any exposed system. What I really would like to see is a two-password
option for root, with a timeout for entering the second password and a
timed lockout if multiple attemps fail.

I actually tested that password with JtR and it came up as pretty good....

--Yan