From the last week I am working on Katello (systems management software for subscriptions and content) and Foreman (system management software for provisioning and configuration) integration. I need to get everything running on one physical box using libvirt. I had to do three rounds until I finally configured everything correctly (actually doing my third round now). Anyway, let me share one libvirt-related idea that came to my mind today.
I need to have both Katello and Foreman and my own DHCP, DNS and TFTP services on the same network, but my hypervisor is on a corporate network. I use my libvirt mainly for development purposes and I have a bridged interface to get my development instances on the local LAN, so I can access it externally. This is widely used libvirt configuration I guess. I use my snap-guest script to bring my instances quickly to life.
But for Katello-Foreman integration I need my own (virtual) network, therefore I have created new one using virt-manager: 192.168.100.0/24 with IPs from 192.168.100.10 to 192.168.100.200. One could reuse the default network called "network", but I have created new one since I will be disabling dnsmasq later on to let my own DHCP daemon into the play. And here comes the big idea.
This virtual network is behind NAT, so I cannot access guests externally. I was trying to realize how to solve this situation for few minutes, until I've heard the click in my head! All I need to do is to add another NIC, but bridged. So I did this using virt-manager, two mouse-clicks. And I realized I don't even need to restart the system, the new interface is immediately available from the bridged network. Now, I can access my development instances from the external network while having my own one with all the DHCP/DNS/TFTP and other provisioning stuff I am working on.
But if I'd need to access any port behind NAT (because of testing purposes or whatever), I created the following bash bit which creates NAT rules for all possible IP addresses for ssh (port 22), so I can access guests also via NAT. Feel free to replace port 22 with any other service you need to test against.
#!/bin/bash
# Create ssh forwards for libvirt virtual network 192.168.100.0 (10..200).
# Make sure to run this after libvirtd is started. This is not permanent,
# after firewall restart those rules are dropped.
#
# Example: port 2242 -> 192.168.100.42
# port 2299 -> 192.168.100.99
# Port (if >63 use different scheme because of TCP port limitations)
PORT=22
# Network part (only class C otherwise you need to change lines bellow)
NET=192.168.100
for i in {10..200}; do
iptables -t nat -I PREROUTING -p tcp --dport $PORT$i -j DNAT --to-destination $NET.$i:$PORT
done
iptables -I FORWARD -m state -d $NET.0/24 --state NEW,RELATED,ESTABLISHED -j ACCEPT
For the SSH example the following command lets you in:
ssh -p2299 your.bridged.hypervisor.hostname.com
I use NAT forwarding to get access to my testing guests which are created with foreman.
Update: I have added new option to snap-guest -k for second NIC, so you can start them with two. Also if you follow my two-NICs approach, make sure the bridged NIC has the default route! You want to go everything via bridged interface, while your virtual LAN is still accessible (route entry will be there). With red hat systems, you need to set DEFROUTE, PEERDNS and PEERROUTES settings in the ifcfg-eth* configurations properly (no for NAT interface, yes for bridged interface).
If you must have both NAT and bridged interfaces accessible from outside (using the above trick), you will probably need two default routes.