iptable rule for bypassing netfilter queue for a matching address.

I have two iptable rules for userspace modification :

iptable -t mangle -A PREROUTING -p udp --dport 9090 -j NFQUEUE
iptable -t mangle -A OUTPUT -p udp --sport 9090 -j NFQUEUE

I have the following network setup:

client ---------------->Linux Box or router--------------------->server. What i'm trying to achieve is modifying all packets which comes from client to 9090 port of the Linux Box. Also which packets will go from Linux Box to client. I do not want to manipulate both from or to the server.

So the rules i need will work like this:

If packet comes from client to port 9090 i will modify it. Now the router opens symmetric port when it communicates with the server. That means it opens 9090 port when it communicates with the server and hence get modified bythe second rule(--sport 9090). I want to avoid this.

The rules will be like this:

1.If packet comes from a specific ip block i will accept it. then i will queue the remaining packets based on the destination port it comes to.
2.If packet goes to a specific ip block i will forward it. I will queue theremaining packets based on the source port information.

Thanks in advance.