Changing source address of packets

Greetings,

I am trying to set up a server to change the source address of
outgoing packets for a particular user to be a secondary address. I
am limited to eth0, but so far as I can tell this should not be a huge
problem.

I am using Red Hat Enterprise Linux 3.0, it's fully updated etc. RHEL
is unfortunately limited to iptables v1.2.8, which doesn't have the
ROUTE module that could override the core routing decision with a
different source address. I'm therefore trying to use iproute2 to do
this for me.

Background: I want to keep services separate from servers, because in
our organization services tend to move servers from time to time.
Generally, this is easy: a service (say finance.example.com) is
deployed on a server (machine1.example.com). People accessing the
service go to finance.example.com, which resolves to an IP address
provided by an IP alias on machine1.example.com. If we move finance
to machine2.example.com, the IP alias moves to machine2 as well,
without affecting the primary IP addresses on machine1 or machine2.

The problem here is that in this case the service goes out and talks
to other services. So, when the service connects to machine3, I want
it to come from finance.example.com, and not machine1.example.com.

Here's the basic procedure I have used to get this to work:

1. Set up the alias on the device:

# ip address add dev $DEV $ADDR/24 broadcast $BCAST label $DEV

2. The service is tied to a particular user, so I can use iptables to
mark any packets coming from that user:

# iptables -I OUTPUT 1 -t mangle -m owner --uid-owner $USER -j MARK
--set-mark 0xf1

3. I set up a special routing table in which any packets routed
through it have their source address changed:

# ip route add to default table $TABLE src $ADDR dev $DEV

4. And finally I set up a rule to route marked packets to this table:

# ip rule add fwmark 0xf1 table $TABLE

This setup is the closest I have gotten. The packets get recognized
and marked, but unfortunately always seem to wind up coming from the
primary address.

Any thoughts?

Thanks,
Drew.