Setting up a home network for learning Linux

I am working on learning Linux, and somebody suggested setting up Linux on a separate computer, and searching for answers to whatever may be needed, on a different computer plugged in to the Internet.

I have a Windows 7 PC, plugged in to a cable modem, and an old notebook, Compaq Presario R3000 (it was good, but it is already 10 years old, or so). I was thinking about installing the newest CentOS on the notebook (from a CD version), and connecting both machines to the Internet, through an xDSL router (this is what is needed, according to my ISP), which I would get for around $15-$20 (this is what it costs).

My overall goal is to learn server administration. I dont necessarily need to learn Linux as a whole (although I may go in this direction), and it is only the server administration part, which is important. I want to be able to get any UNMANAGED hosting plan (something like dedicated, which may cost $100-$300+ per month) and do everything by myself, on a professional level (administration, maintenence, updates, security, backups), similar to something that I would get with a regular hosting plan (so I want to learn everything, very good).

With this I would have a Linux and a Windows computer, both connected to the Internet, so I could work on the Linux one, and also check whatever would be needed, through the Windows one.

With this, I was wondering, if I could set up actual server on the Linux computer (LAMP, lets say), and start experimenting with hosting some websites there. I am not sure if the router would be what is needed for this, and how would this work. Router will computer both computers to the Internet, but will it connect them to each other, probably not...

I would like to be able to have both of the machines turn on, have websites on the Linux machine, and be able to access them from the Windows PC, lets say. This way I could experiment with it, just like I could with a regular hosting plan on the Internet.

So, the question is, is it possible not only to have two separate machines connected to the Internet, but also have these machines connected to each other, and work like I would work with a regular Linux server, on the Internet.

Thanks.