Linux IP Policy Routing: Route by Source, Interface or Mark with ip rule

Useful answer

Linux can forward packets based on criteria other than the destination address by using multiple routing tables and the ip rule command. A rule matches source address, incoming interface, or firewall mark and selects an alternate table where the normal destination lookup occurs.

How policy routing works

When a packet arrives, the kernel walks the list of rules in priority order (lower numbers first). The first rule whose selector matches the packet determines which routing table to consult for the route lookup. If no rule matches, the kernel falls back to the default tables: local (255), main (254) and unspec (253). Custom tables use IDs 1‑252 or names defined in /etc/iproute2/rt_tables.

Example: route a subnet via a second ISP

1. Log in as root or a user with CAP_NET_ADMIN.
2. Add a name for the new table, e.g. isp2.
3. Add a rule that selects this table for packets from a given source subnet.
4. Populate the table with a default route pointing to the second ISP’s gateway.
5. Flush the route cache so the new rule takes effect immediately.

# 1. Define table name (requires root)
echo "200 isp2" >> /etc/iproute2/rt_tables

# 2. Rule: source 198.51.100.0/24 uses table 200, priority 100
ip rule add from 198.51.100.0/24 table 200 priority 100

# 3. Route in table 200: default via ISP2 gateway on eth1
ip route add default via 203.0.113.2 dev eth1 table 200

# 4. Make the kernel re‑evaluate routes
ip route flush cache

After these steps, any packet whose source address falls in 198.51.100.0/24 will be looked up in table 200 and sent out eth1 toward 203.0.113.2, regardless of the destination address.
Verification

List rules: ip rule show – you should see the new rule with priority 100.
Test lookup: ip route get 8.8.8.8 iif eth0 src 198.51.100.10 (replace interfaces and addresses as needed). The output shows the table used, gateway and device.
Observe egress: run tcpdump -i eth1 -n icmp while pinging from a host in the source subnet; packets should leave on eth1.

Limits and common mistakes

Asymmetric return paths. Policy routing only influences the outbound decision; reply packets may follow the normal routing table and arrive on a different interface. If rp_filter is set to 1 (strict) on the incoming interface, those replies are dropped. Use rp_filter=2 (loose) on multi-homed hosts only after assessing spoofing risk, or pair the rule with SNAT/DNAT or connection-tracking marks to enforce symmetry.
No built-in load balancing. Adding multiple rules does not spread traffic across links; you need separate tools (e.g., ECMP, conntrack‑based load balancers) for that.
Rule persistence. Commands above affect the running kernel only. To survive a reboot, place the echo line in /etc/iproute2/rt_tables and the ip rule and ip route commands in a network‑up script (e.g., /etc/network/if-up.d/ on Debian‑based systems or a systemd networkd dispatcher).
Flushing the cache. On modern kernels the route cache is largely obsolete, but ip route flush cache still forces a re‑evaluation of the rule list and is harmless. Forgetting it may leave old lookups in use for a short time.
Reserved table IDs. IDs 253‑255 are reserved for unspec, main and local. Using them for custom tables will cause undefined behavior.

Rollback
To return to the default behavior, remove the rule and route, delete the table name if desired, and flush the cache:
ip rule del priority 100
ip route del default table 200
ip route flush cache
# Optional: remove the name from rt_tables if no longer needed
sed -i '/^200 isp2$/d' /etc/iproute2/rt_tables