iptables port forwarding with SNAT and DNAT

Assume we have a web server having node.js (or any other client-server software) installed behind a load balancer and teh web server has only internal IP address. We need to allow clients to connect to node.js. So we need to do port forwarding with iptables from load balancer’s 8000 to web server’s 8000.

So the ideal model will look like on the figure below.

Doing a basic google search you’ll find handful DNAT rule for this:

iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 8000 -j DNAT --to 192.168.1.1:8000
iptables -A FORWARD -p tcp -d 192.168.1.1 --dport 8000 -j ACCEPT

The second rule is needed only if you have default FORWARD policy = DROP or REJECT.

But having done that you may figure out it doesn’t work. If you try to telnet 1:1:1:1 8000 (load balancer 8000 port) you’ll get your connection dropped. Why? Because it might be so that the web server to where your packets are forwarded has default router set to another one, not your load balancer. And it is very likely. See the figure below illustrating it:

So the forwarded packets are gone from the web server to it’s default router and not to load balancer – that’s the most important step here in the model. And you’re receiving them from router’s ip address.Obviously your client does not expect this but rather expects them to return from load balancer.

In order to solve this, first of all make sure that for local IPs your web server does not route to its default gateway. You can do that by looking at route -n output. Once you do that, you need to add SNAT rule to load balancer’s iptables. I.e. you should replace the source IP of all packets going to 8000 port with load balancer’s internal IP address. So that response packets are gone to load balancer. This can be done by adding the following iptables rules (to the mentioned ones above):

#marking all packets going to  8000 port with label = 1 
iptables -A PREROUTING -t mangle -i eth0 -p tcp --dport 8000 -j MARK --set-mark 1
# mark those having label = 1 with source IP = internal load balancer IP
iptables -A POSTROUTING -t nat --match mark --mark 1 -j SNAT --to-source 192.168.1.2
#allow forwarding back returning packets from web server's 8000 port
iptables -A FORWARD -p tcp -s 192.168.1.1 --sport 8000 -j ACCEPT

Note few things:
[list style=”upper-alpha tick”]
[list_item]Your web server won’t know external client’s IP address in this scheme, it will see 192.168.1.2 as source[/list_item]
[list_item]You need to make sure that packet forwarding is allowed on your load balancer by doing cat /proc/sys/net/ipv4/ip_forward. It should show 1, otherwise, do echo 1 > /proc/sys/net/ipv4/ip_forward
[/list_item]
[list_item]You need to allow input connections at your web server to port 8000 in case of DROP/REJECT policy of iptables there. This can be done by:

iptables -A OUTPUT -p tcp --sport 8000 -m state --state ESTABLISHED -j ACCEPT
iptables -A INPUT -p tcp --dport 8000 -m state --state NEW,ESTABLISHED -j ACCEPT

[/list_item]
[/list]

Remember you can forward any other application port to web server, such as https 443 port.