Windows 2003 Server RDP: Black Screen

I have this problem. To add to it, the following happens:
1. If I place a monitor of the server, I can actually see the mouse moving
on the monitor through the remote control session, but it ignores any input
(mouse clicks etc).
2. I have tried closing the session down and accessing it again, but to no
avail.
3. The firewall is turned off now (after I tried it with the firewall on and
using the 2701 and 2702 exceptions), but no further advance is made.
4. RDP is off, but if I have the remote desktop connected, then connect to
it using RDP, the SMS remote control window shows the screen, but I still
cannot see or click on anything there.

Note: The same is ONLY happenning on all the Windows 2003 Server boxes.
Windows Server 2000 access is fine.

My configuration is as follows:
Connecting to:
Windows Server 2003 SP1 (with all updates current)
2GB RAM and 240GB HD free
CPU is pretty much idle.

Connecting From:
Windows XP SP2 Workstation with all updates current.
1GB RAM and 120GB free

cheers,
Ronnie