Linksys BEFSR81, multiple routers/subnet, single Internet, Port Forwarding

The setup:

Roommate situation, he's got his toys, I've got mine. Cablemodem. 2
befsr81s, etc. We are sharing the cable connection, and are set up in
the following way (mostly so his gaming traffic stays off _my_
network:

RouterA
WAN->Cablemodem->Public IP
LAN->192.168.10.0/24
His toys->192.168.10.1,2,3

RouterB
WAN->RouterA->192.168.10.254 (cabled to RouterA's LAN side)
LAN->192.168.20.0/24
My toys->192.168.20.1..etc

Routing protocol is RIP2. RouterA is set as Gateway, RouterB as
Router.

Both networks can see each other fine, both can get to the Internet.
The problem arises in that I want certain connections to certain ports
coming from the public side of RouterA to hit any of 2 machines on my
subnet, either 192.168.20.3 or .4. As an example, I set up a forward
for auth port 113 on RouterA to forward to 192.168.10.254 (connected
to the WAN port on RouterB). In turn, configured forwarding on
RouterB to forward traffic inbound on that port to the intended
destination: 192.168.20.3. No luck. I have verified that RouterA is
actually doing port forwarding by forwarding 113 traffic to a host
local to its own LAN subnet, and it works just dandy.

Am I expecting too much out of these routers? Is it impossible to
maintain couple routed subnets in the house and be able to route
traffic from the Internet to a machine on another subnet? Does the
packet stop on the LAN side of the Internet router? WTF can't they
open those destination addresses up to any subnet present in the
routing tables, rather than just the local subnet?

Am I out of luck here? Any help appreciated.

Tim