VPN Problem - Error 678 the remote computer did not respond

I shall try and be as detailed as possible.

Windows Server 2003 SBS R2 Standard (non-isa)
- Inside a Netgear DG834 Wired router on ADSL.
- 1701, 50,51, 500, 4500 ports forwarded too the server.
Windows XP SP2 Laptop client on a remote network
- Inside a Linksys Router.

2003 server setup for L2TP/IPSEC VPN with computer/user certificate
authentication.

Tested VPN inside the SBS network - works fine.....

Outside the network you get "Error 678 the remote computer did not respond"

Microsoft Network Monitor 3.1 is installed on the server and on the client
to troubleshoot this.

You can see the client sending data on UDP 500 and UDP 4500 towards the
server.
You can see the server receive data on UDP 500 and UDP 4500.
You can see the server send data on UDP 500 and UDP 4500
You can then see the client recieve data on UDP 500 and UDP 4500

No other ports show up, whist some traffic is clearly flowing I still get:
Error 678 the remote computer did not respond.

Any ideas?