Routing
From SolusVM
Contents |
Introducation
It is common for some service providers to not allow bridged interface on their servers. In these cases you need to use NAT and route the domU traffic over the dom0's if. Enabling this support in XEN is very easy
What will this do ?
It will masquerade the VM MACs, this way only 1 MAC will be visible on the switch for all of the VM's. still allowing for them to use public and individual IP's.
Immediately after finishing the installation of solusvm do the following steps:
Step 1: Change from a bridged to a routed configuration
vi /etc/xen/xend-config.sxp
Comment out all other vif and network scripts except for:
(network-script network-route)
(vif-script vif-route)
Step 2: Setup the kernel for forwarding
Setup forwarding and arp proxy in kernel
Edit the file /etc/sysctl.conf
And change the ip_forward to = 1 and add the proxy_arp line
# Controls IP packet forwarding
net.ipv4.ip_forward = 1
net.ipv4.conf.all.proxy_arp = 1
Step 3: Setup NAT & Firewall
Setup Masquerading nat using iptables
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
Setup IN/OUT & Forwarding policies. Assuming eth0 is the if to be used for NAT
iptables -A OUTPUT -t filter -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -t filter -i eth0 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -t filter -i eth0 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
After this is done you just need to set the GW on solusvm for this node to be the nodes IP.
Notes
Use the appropriate eth# above. In many cases eth0 is added in a vlan with private non routable IP's to be used for node comms only.
Troubleshooting
- In some cases the providers switch will have the last MAC cached in its arp(mostly due to running a XEN VM with its own MAC), ask for a flap of the port to clear it.
- In any other case where your MAC will change you will need to ask for a flush. (Motherboard or NIC changes due to malfunction etc)
- Most providers although will state that they allow 1 MAC per phy dev, will not have a problem extending this to 2 (on CISCO and Juniper).
- if cat /proc/cpuinfo | grep vmx doesnt give you a result than either your kernel or your CPU doesnt support it.
- Kernel 2.6.18-194.8.1.el5 doesnt enable the vmx extension required for XEN. Instead use 2.6.18-164.el5 or build your own kernel.
- XEN requires a CPU with the AMD-V or Intel VT extensions.
Warnings
The above policies allow all trafic, in a normal situation you should add to the chain the filters you need to protect both the dom0 and the domU.
