Linux Console Hangs

Hi,

I have a problem on my RHEL 3 Linux. It starts well but the console
hangs as soon as I type the password even as root. I have Samba and
Webmin there and they both work very well but sshd and sendmail don't
- every login hangs the session the same way as on the console. I can
login on Webmin as root without problems.

/etc/passwd and /etc/shadow files look good and I can create or delete
user through Webmin command shell. I can also execute some scripts but
not all.

crond doesn't work neither.

The first time when the problem appeared I was able to "solve" it
with simple reboot but since the third or forth reboot it doesn't
help anymore.

Does anybody have an idea what could be the reason for that? Thank you.

Svet

On Tue, 10 Jan 2006 10:18:01 -0800, Svet wrote:

> Hi,
>
> I have a problem on my RHEL 3 Linux. It starts well but the console
> hangs as soon as I type the password even as root.

Could it be hardware related? One of my boxes kept hanging randomly in
parts so I ran memtest on it and found one of the memory sticks had gone
bad. Or perhaps you have a dodgy HDD?

--
What is everyone raving about?
https://profits4investingtoo.com/?ref=jafar00

In comp.os.linux.networking Svet <(E-Mail Removed)>:
> Hi,

> I have a problem on my RHEL 3 Linux. It starts well but the console
> hangs as soon as I type the password even as root. I have Samba and
> Webmin there and they both work very well but sshd and sendmail don't
> - every login hangs the session the same way as on the console. I can
> login on Webmin as root without problems.

What do you mean with console? Directly from a keyboard connected
to the box?

> /etc/passwd and /etc/shadow files look good and I can create or delete
> user through Webmin command shell. I can also execute some scripts but
> not all.

What do you mean with look good? Unless you ran md5sum over them
and compare with a trusted backup none can say.

Can you write a script, put it on the box and run it as root?

> crond doesn't work neither.

> The first time when the problem appeared I was able to "solve" it
> with simple reboot but since the third or forth reboot it doesn't
> help anymore.

This isn't doze, a reboot is pretty unlikely to fix real problems
on *nix.

> Does anybody have an idea what could be the reason for that? Thank you.

There are many, provide more info.

It doesn't really look like a network problem, since smb is
working fine we can presume ping goes through.

--
Michael Heiming (X-PGP-Sig > GPG-Key ID: EDD27B94)
mail: echo (E-Mail Removed) | perl -pe 'y/a-z/n-za-m/'
#bofh excuse 168: le0: no carrier: transceiver cable problem?

"Svet" <(E-Mail Removed)> writes:

> I have a problem on my RHEL 3 Linux. It starts well but the console
> hangs as soon as I type the password even as root. I have Samba and
> Webmin there and they both work very well but sshd and sendmail don't
> - every login hangs the session the same way as on the console. I can
> login on Webmin as root without problems.

Try to get a rootshell and look into the logs.

When the machine boots, modify the boot parameters and add "init=/bin/sh".

Then you should get a root shell immediately.

Once there you can also run the gettys by hand, you may run them
through strace, etc.

Vilmos

When I say console I mean from the keyboard connected directly to the
box.

/etc/passwd and /etc/shadow look good for me because I can create or
delete a user with useradd or userdel commands and the files are
changed every time.

I can write a simple script and run it as root. By simple I mean a bash
script without function definitions inside - I have one; it doesn't
work.

Ping works well. As I said the only way that I can see or run something
on the box is through a Webmin interface from the network.

I didn't mention that I have 2 similar servers: I mean same Linux
version, same function in the network (Samba server). The problem
appeared on both servers almost in the same time (within a day or too
difference) just before Christmas.

I checked one of them with chkrootkit but it didn't find anything
suspicious.

What do you need as more info? Ask me a question, please.

Thanks

What is gettys? How I can use it and what I can get from it? I can try
it tomorrow. Thanks.

Svet

"Svet" <(E-Mail Removed)> writes:

> What is gettys?

getty is the program which gives you the login prompt on the console.
Then it calls the login program.

Look at /etc/inittab to see which getty is run and how. It looks
something like this (multiple lines, possibly not exactly like this):

2:23:respawn:/sbin/getty 38400 tty2

So if you get a rootshell (by adding "init=/bin/sh" to the end of
the kernel boot line), execute "/sbin/getty 38400 tty2".

Then look into your logs and see what happens.

Actually get the line from *YOUR* inittab (I run Debian now),
and execute that command. You are not interested in the 2:23:respawn:
stuff now.

Vilmos

Hi,

I tried the rootshell by adding "init=/bin/sh" and I got
init-2.05b# prompt.
Then I tried /sbin/mingetty tty3 (as it was in my /etc/inittab) but
nothing happens. 2 seconds latter I had the same prompt.
I have nothing in my log files.
I noticed than I cannot modify a file in /etc - it was read-only even
when I modified ro to rw in the kernel boot line.

Something new: even if I try a login with a foo password, the console
hangs.

I still have access trough Webmin and it uses /etc/shadow for
authentication, according to /etc/webmin/miniserv.conf file (I suppose
this is the Webmin config file)

Any other ideas? Can be a corrupted version of /bin/bash or /bin/sh? Or
some kind of hack? The server is not exposed to Internet and the last
thing is unlikely. Unfortunately, I cannot reboot the servers more than
2-3 times a day during lunch time, so I need more options to try next
time. Thanks

Svet

"Svet" <(E-Mail Removed)> writes:

> I tried the rootshell by adding "init=/bin/sh" and I got
> init-2.05b# prompt.

OK.

> Then I tried /sbin/mingetty tty3 (as it was in my /etc/inittab) but
> nothing happens. 2 seconds latter I had the same prompt.

Yes. The "tty3" means that it was supposed to open tty3. Hit Alt-Ctrl-F3
to get the login prompt. Then hit Alt-Ctrl-F1 to get back. (The Ctrl
is not necessary, an Alt-Fx is enough unless switching away from X).

> I noticed than I cannot modify a file in /etc - it was read-only even
> when I modified ro to rw in the kernel boot line.

You might to remount rw.

figure out what is your root partition. Say, /dev/hda1

# mount -o remount,rw /dev/hda1

When finished, don't forget to remount "ro", and then run "sync".

> Something new: even if I try a login with a foo password, the console
> hangs.

I start to have the feeling that it will be something really small thing.

> Any other ideas? Can be a corrupted version of /bin/bash or /bin/sh?

Then you wouldn't have got the init shell. Lemme ask you. Is it possible
that you have something like /etc/nologin present? That would explain,
but not as root.

> Or some kind of hack? The server is not exposed to Internet and the
> last thing is unlikely.

If somebody rooted your box, then the last thing they would do is to
make it obvious. Or if they *DO* want it to be obvious, then there
are far better ways...

Vilmos

Hi,

I tried the rootshell again, as you suggested.
>From my init-2.05b# I did:
1) /sbin/mingetty tty3
2) I switched with Alt-F3 but there was nothing there, except a
blinking cursor
3) There was nothing in the log files.
4) I also tried the same thing with a copy of mingetty from another
linux without success.

I've tried something else: I logged as single user and I was able to
restore /etc/passwd, shadow, group and gshadow from my original backup.
It doesn't help either.

Anything else?

As I sad, I have some non-ascii things in /var/log/boot.log file every
time when I reboot. Does it mean something?

Also, as I sad I have access to the server through Webmin and from its
Command Shell module I can execute some commands. I tried adduser, but
it did work. I had all 4 passwd files locked.

There isn't /etc/nologin or something else like it, there.

Do you have any idea?

Thanks.

Svet