All traffic in our network has to pass through our core-routers, additionally switch ports are configured in such a way that they only accept ethernet frames with mac addresses originating from these routers. Consequently, no layer-2 traffic is forwarded between switch ports and direct communication between servers, even when on the same subnet, is not possible. If you have multiple servers with us which use IPv4 addresses from the same subnet (usually /24), and you want these servers to exchange traffic between each other, you will have to create static routes on these servers to each other.
The following examples assume that the server with the IPv4 address 192.51.100.10 and the server with the IPv4 address 192.51.100.42 want to communicate with each other.
CentOS
If you’re not sure about your admin rights in Linux, always enter the command sudo -i at the beginning of every session:
sudo -i
This command will grant you the rights of a power user, so you don’t have to write the command sudo at the beginning of every command line.
In CentOS, you can add static routes to the route-file of the respective interface. In case of eth0, the file is called route-eth0 and located under /etc/sysconfig/network-scripts/. The below entry on server 192.51.100.10 adds a static to server 192.51.100.42:
#/etc/sysconfig/network-scripts/route-eth0
...
192.51.100.42/32 via 192.51.100.1 dev eth0
Vice versa, the server 192.51.100.42 needs a corresponding route to server 192.51.100.10:
#/etc/sysconfig/network-scripts/route-eth0
...
192.51.100.10/32 via 192.51.100.1 dev eth0
Alternatively, both servers can have a single route added to the whole /24 subnet:
#/etc/sysconfig/network-scripts/route-eth0
...
192.51.100.0/24 via 192.51.100.1 dev eth0
Debian / Ubuntu (until 17.04)
Static routes in Debian and Ubuntu are saved to /etc/network/interfaces. Entries are added to the end of the file or under the iface sections of the respective interface. Assuming the interface to be eth0, the entry on server 192.51.100.10 would look like this:
#/etc/network/interfaces
...
up ip route add 192.51.100.42/32 via 192.51.100.1 dev eth0
down ip route del 192.51.100.42/32 via 192.51.100.1 dev eth0
On server 192.51.100.42, this would be the static route to 192.51.100.10:
#/etc/network/interfaces
...
up ip route add 192.51.100.10/32 via 192.51.100.1 dev eth0
down ip route del 192.51.100.10/32 via 192.51.100.1 dev eth0
It is also possible to add a static route to the whole /24 on both servers instead:
#/etc/network/interfaces
...
up ip route add 192.51.100.0/24 via 192.51.100.1 dev eth0
down ip route del 192.51.100.0/24 via 192.51.100.1 dev eth0
Ubuntu (17.10 and above)
In Ubuntu beginning with version 17.10, static routes are added to the file /etc/netplan/01-netcfg.yaml . On the server with the IP 192.51.100.10,the entry has to be added in the correct indentation to the network interface, in this case eth0 . Because of some particularities in Netplan, there are two routes that have to be added for each server ip. Please note the prefixes that that got changed to /25:
#/etc/netplan/01-netcfg.yaml
...
eth0:
routes:
- to: 192.51.100.0/25
via: 192.51.100.1
- to: 192.51.100.128/25
via: 192.51.100.1
Finally it is required to reset the routing table and to apply the changes. Please use the whole command at once to prevent a network connection loss:
ip route flush table main; ip route flush cache; netplan apply
openSUSE
Static routes in openSUSE can be set in /etc/sysconfig/network/routes. On server 192.51.100.10, the static route to 192.51.100.42 is as follows:
#/etc/sysconfig/network/routes
...
192.51.100.42/32 192.51.100.1 - eth0
Conversely, the static route on 192.51.100.42 to 192.51.100.10:
#/etc/sysconfig/network/routes
...
192.51.100.10/32 192.51.100.1 - eth0
As an alternative, both servers can have a static route to the whole /24 subnet:
#/etc/sysconfig/network/routes
...
192.51.100.0/24 192.51.100.1 - eth0
Windows Server
To create a static route under Windows Server 2012 or above, please open the “Command Prompt” with administrative privileges. Simply right-click on the Command Prompt icon in your Start panel and then click on “Run as administrator”:
On server 192.51.100.10 enter the following:
route -p add 192.51.100.42 mask 255.255.255.255 192.51.100.1
The corresponding entry on server 192.51.100.42 is:
route -p add 192.51.100.10 mask 255.255.255.255 192.51.100.1
Setting the -p option makes the route persistent across reboots. If the route is meant to be temporary, you may omit -p.
Since Windows automatically adds a default route to the whole subnet with a higher metric, it is necessary to also deactive this default route. Otherwise, the new static route will not have any effect.
Use the following command to delete the default route:
route delete 192.51.100.0
If you often reboot your server, it’s advisable to create a little .bat script to perform this step automatically.