Redundant routers, same subnets

Question

I have inherited a network with a bunch of subnets. I have an Ubuntu box ("router1") that has the external connection and functions as a router eth1 is the external connection eth2 and eth3 connect to internal subnets (net2 and net3). Routing works: machines on an internal subnet can see machines on the other subnet when required, and external connections work as they should.

I have another Ubuntu box ("router2") with another external connection and external IP address. It's basically identical to the first one. I'd like to be able to use it as a redundant router, working the same way as the first one. It has the same eth1, eth2 and eth3 setups as the first router (with different IP addresses - router1 is n.n.1.1, x.y.2.1, x.y.3.1; router2 has n.n.1.3, x.y.2.3, x.y.3.3). Router2 works, kind of.

Machines on the net2 subnet that use router2 as gateway are unable to see (ping, ssh, http etc) machines on net3 subnet that use router1 as their gateway, and vice versa. A machine on net2 can see machine on net3 if they both have the same gateway, either router1 or router2. All machines can access the Internet, whether they have router1 or router2 as their gateway.

I suppose my first question is: is it at all possible to have two redundant routers serving the same subnet the way this is described? If this should be possible, any idea where I should look to fix this?

Answer 1

A common issue is that stateful (conntrack-based) firewalls are unable to deal with asymmetric routing without some way to synchronize their knowledge.

For example, your router's "forward" chain might have --state ESTABLISHED -j ACCEPT (or equivalent) so that it allows communications in one direction but also allows replies to come back. This however relies on replies coming back through the same router as the original request.

In your case, the original request (ping or TCP SYN) between subnets goes through router1, but the response goes through router2. When router2 handles the response it can't find a matching state entry (only router1 knows about it) and rejects the packet.

To avoid this, use a program that synchronizes connection tracking state between both firewalls. For Linux you need conntrackd, for OpenBSD or FreeBSD you would use pfsync.

(I am not sure if such a system is 100% reliable, as the reply packet can arrive faster than the states can be transferred... Unless conntrackd actually makes the firewall queue the original packet until it becomes "safe" to release it?)

Alternatively, for TCP only, you can allow packets with the --tcp-flags ACK ACK flag to go in both directions as these will always be replies. (This also applies to SCTP, but obviously won't work for UDP.) For "ping", I would just allow ICMP in both directions unconditionally.