Backup internet connection for specific process

I ended up achieving this by configuring an alternate route table and a routing policy rule for the source address of the backup interface.

The USB LTE modem I have presents as a NDIS device, so it simply shows up as eth1 with an IP of 192.168.0.190 and it does NAT routing internally. I've configured eth1 with a static IP and manually configured routes.

1. The default configuration uses DHCP, so bring down the interface and ensure that any automatically added routes are deleted.

2. Add a static IP configuration for the interface and bring it up.

3. Add entries to an alternate routing table (I've chosen 1) for the subnet and default gateway.

   # ip route add 192.168.0.0/24 dev eth1 src 192.168.0.190 table 1
   # ip route add default via 192.168.0.1 table 1
   

4. Set routing policy rules so that apps which explicitly use 192.168.0.190 as their source address will use routing table 1 instead of the default.

   # ip rule add from 192.168.0.190/32 table 1
   # ip rule add to 192.168.0.190/32 table 1
   

At this point, you should be able to test your connectivity.

$ curl https://wtfismyip.com/text
1.2.3.4  # primary ISP external IP
$ curl --interface 192.168.0.190 https://wtfismyip.com/text
5.6.7.8  # backup LTE external IP

If all looks good, make the configuration permanent. I added to /etc/network/interfaces:

iface eth1 inet static
        address 192.168.0.190
        netmask 255.255.255.0
        post-up ip route add 192.168.0.0/24 dev eth1 src 192.168.0.190 table 1
        post-up ip route add default via 192.168.0.1 table 1
        post-up ip rule add from 192.168.0.190/32 table 1
        post-up ip rule add to 192.168.0.190/32 table 1

Now only apps that explicitly bind to 192.168.0.190 when making outgoing connections will be routed over the backup connection. All other traffic gets routed over eth0 (or whatever is configured in the main [default] routing table).

It is possible that you have something that enumerates all available IPs and attempts to send traffic from them, which could result in unexpected traffic over the backup connection, but that's unlikely. I've not observed any such traffic.

Note that this does not address DNS resolution. In a situation where the primary connection is offline, you may get lucky and get a lookup result from cache, but that's not good to rely on. I wouldn't configure the systemwide resolver to send requests over the LTE interface either. Instead, your app could manually handle DNS resolution when making backup requests.

---

With node, making HTTP requests (or any TCP connection) from a specific source address is easy. Simply specify the localAddress option, eg:

https.get('https://wtfismyip.com/text', { localAddress: '192.168.0.190' }, …);

Solving the DNS lookup is slightly trickier. A lookup option is also available, which allows you to override the default DNS resolution process. You can use a custom dns.Resolver to make lookups. Unfortunately, node did not have a way to specify the source address for DNS lookups, so I added it. With that in place, you can put the pieces together:

const resolver = new dns.Resolver();
resolver.setServers(['8.8.8.8']);
resolver.setLocalAddress('192.168.0.190'); // requires node > v15.0.0

https.get('https://wtfismyip.com/text', {
  localAddress: '192.168.0.190',
  lookup: function(hostname, opts, cb) {
    resolver.resolve(hostname, function(err, records) {
      if (err) cb(err);
      else if (!records[0]) cb(new Error('Missing DNS record'));
      else cb(null, records[0], 4);
    });
  }
}, function(res) { … });