This post turned out to be much longer than I planned. So I decided to split it into two posts.
This part covers:
- Simple scenario walkthrough (connecting a virtual machine through ovs bridge to the internet)
- Basic commands ( add/delete bridge, add/delete ports, show configuration, etc)
The second part covers:
- Components (ovsdb-server, ovs-vswitch, ovs kernel module)
- Utilities (ovs-vsctl, ovs-ofctl, ovs-appctl, etc)
- Modes (normal & flow)
What is Open vSwitch?
Open vSwitch is a multilayer software/virtual switch used to interconnect virtual machines in the same host and between different hosts.
The goal of the project (as specified in the official documentation): “Implement a production quality switch platform that supports standard management interfaces and opens the forwarding functions to programmatic extension and control.”
It is a very popular project, used in a variety of SDN solutions. You’ll probably bump into it soon or later when deep diving into projects such as OpenStack and OpenDaylight.
OpenvSwitch supports many of the features you already familiar with, assuming you worked with switches before:
- VLAN tagging
- Tunneling protocols (GRE, VXLAN)
- SPAN, RSPAN
Virtual Switch Overview
Before we deep dive into OpenvSwitch (I’ll refer to it as ovs from now on) and learn how to use ovs commands, let’s start with a basic overview of how it looks when you configured one bridge with one interface and multiple ports on your system
As you can see in the drawing, there is one bridge named ‘my_bridge’ which was created using openvswitch. You can have more than one bridge on your system, using ‘ovs-vsctl add-br’ command, which we will cover later.
Each bridge can have multiple ports and each port consists of one or more interfaces. In our example, there is one port named ‘Bond’, which is an actual bond of two physical interfaces (eth0 and eth1).
First ovs command that we’ll use is ‘ovs-vsctl show’. This command will print the contents of the ovs database (in other words, your switches configuration).
Basically everything that you can see in the drawing (note that in a fresh new environment, you would not see anything except for ovs version and id.).
> ovs-vsctl show 79ec4909-0d98-489b-a81b-a667cf26a25a Bridge my_bridge Port "Bond" Interface "eth0" Interface "eth1" ovs_version: "2.5.0"
Before you can start using ovs, you need to install it on your system. The following commands should work on Fedora/CentOS/RHEL.
> yum install -y openvswitch > systemctl start openvswitch
If openvswitch isn’t provided by one of your repositories, try to download it directly from here.
You should now be able to use the ‘ovs-vsctl show’ command which we already used in the overview section.
> sudo ovs-vsctl show 9e72385f-ed0a-40fd-97f3-21d49cbf60f3 ovs_version: "2.5.0"
Connect a virtual machine through Open vSwitch bridge
I can show you several random ovs commands, but I believe in learning through specific exercises. So let’s say our goal is to connect a newly created VM to the internet, but we want to connect it through an ovs bridge. Our goal is to achieve something similar to the following diagram
I’m going to assume you already have a VM with eth0, so we will not cover the creation of it, in this post.
Add a new bridge
The first thing we should do, is to create an ovs bridge. The way to create and configure bridges is by using the ‘ovs-vsctl add-br’ command.
> sudo ovs-vsctl add-br my_bridge
By using the above command, we added a new bridge, named ‘my_bridge’.
We can then use then ‘ovs-vsctl show’ command to verify our bridge was created.
> ovs-vsctl show 9e72385f-ed0a-40fd-97f3-21d49cbf60f3 Bridge my_bridge Port my_bridge Interface my_bridge type: internal ovs_version: "2.5.0"
As you can see in the above output, there is now a new bridge named ‘my_bridge’. It has one port, which is an internal (according to the type) and it mapped to an interface called ‘my_bridge’.
Now let’s bring the ‘my_bridge’ interface up.
> ip link set my_bridge up > ip a 3: my_bridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1 link/ether 9a:15:b7:cc:29:4e brd ff:ff:ff:ff:ff:ff inet6 fe80::9815:b7ff:fecc:294e/64 scope link valid_lft forever preferred_lft forever
Note that our newly created bridge is not directly connected to our physical interface (eth0). Let’s change it.
Add existing interface to the bridge
Let’s connect eth0 interface to ‘my_bridge’ (Warning: you’ll lose any connectivity to the internet after executing the following command)
> sudo ovs-vsctl add-port my_bridge eth0
We just lost connectivity to the external world. Check by yourself (the most popular check today is probably pinging 18.104.22.168).
This is because eth0 is now connected to our bridge and not to the default IP stack of the system. Our system still trying to reach the external network directly through eth0. In order to recover our connectivity to the external network, we need to do two things:
- Remove eth0 address, since we no longer reach the internet directly through eth0
- Assign my_bridge with address so we can reach the internet through it ( the flow would be: IP stack -> my_bridge -> eth0).
Let’s start by removing eth0 current address
> ip addr del 192.168.121.52/24 dev eth0
Verify with ‘ip a’ that eth0 indeed has no IP address.
Next, we will run dhclient to configure ‘my_bridge’, so it can be allocated with an IP address
> dhclient my_bridge
Now that ‘my_bridge’ has an IP address, we should be able to reach the internet once again.
> ip a 4: my_bridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN link/ether ce:8b:5b:a7:a8:4f brd ff:ff:ff:ff:ff:ff inet 192.168.121.195/24 brd 192.168.121.255 scope global dynamic my_bridge valid_lft 3478sec preferred_lft 3478sec inet6 fe80::cc8b:5bff:fea7:a84f/64 scope link valid_lft forever preferred_lft forever > ping 22.214.171.124 PING 126.96.36.199 (188.8.131.52) 56(84) bytes of data. 64 bytes from 184.108.40.206: icmp_seq=1 ttl=56 time=105 ms 64 bytes from 220.127.116.11: icmp_seq=2 ttl=56 time=105 ms
Add TAP device
Let’s add a new interface which we’ll later connect to our virtual machine.
> ip tuntap add mode tap virt_port # Bring the interface up > ip link set virt_port up
You might not be familiar with TUN, TAP interfaces, so let’s take a second to explain what they are exactly.
TUN, TAP devices are entirely virtual in contrast to other devices on your system (e.g eth0) which associated with a physical address.
A TUN device operates in the third OSI layer (network) and used mostly for routing traffic, while a TAP device operates in the second OSI layer (data link) and used to process Ethernet frames.
Now let’s add our newly created device to our ovs bridge
> sudo ovs-vsctl add-port my_bridge virt_port
We can verify now with ‘ovs-vsctl show’ that our ports are connected to ‘my_bridge
> sudo ovs-vsctl show 9e72385f-ed0a-40fd-97f3-21d49cbf60f3 Bridge my_bridge Port my_bridge Interface my_bridge type: internal Port "virt_port" Interface "virt_port" ovs_version: "2.5.0"
This is the state we reached
Connect a bridge port to the VM
The last step is to connect our newly created device ‘virt_port’ to our virtual machine.
This step implementation depends on the virtualization solution you are using. Personally, I’m using libvirt.
With libvirt, you go to the virtual machine properties (the light bulb icon) and click at the bottom on ‘add hardware’ button.
Next, you choose ‘Network’ and in ‘Network source’ you choose the ‘virt _port’ device and ‘Bridge’ for ‘Source mode’ as can seen in the following image
Congrats, we reached our goal. The virtual machine is now able to reach the internet, through our ovs bridge ‘my_bridge’.
I highly recommend to test the connectivity with a simple check of ‘ping 18.104.22.168’. In case it doesn’t work, try to repeat the steps above or post a comment on this post and I’ll try to assist.
Delete a bridge
Now that we finished with our small exercise, we can cleanup the environment.
To delete an existing bridge, use the ‘ovs-vsctl del-br’ command.
> ovs-vsctl del-br <bridge_name>
Note that it will also remove any related ports and interfaces.
To remove only a specific port, you can use the following command
> ovs-vsctl del-port <port_name>
I gathered all the commands we used in this post, for an easier reference
ovs-vsctl show # Print summary of the ovs database content (bridges, interfaces, ports, etc) ovs-vsctl add-br <bridge_name> # Add a new bridge ovs-vsctl del-br <bridge_name> # Delete existing bridge ovs-vsctl add-port <bridge_name> <port_name> # Add a new port in the specified bridge ip a (= 'ip addr' = 'ip address') # Displays addresses and their properties ip addr del <IP address/CIDR> dev <device> # Remove the specified address from the specified device ip set link <interface_name> up # Bring an interface up ip tuntap add mode tap <device_name> # Add TAP device
Q: You described a virtual switch and the technology is called ‘openvswitch’, but the actual commands use the word ‘bridge’, why?
A: Switch is a bridge with multiple ports, don’t let it confuse you, ‘switch’ and ‘bridge’ are usually used interchangeably.
Q: Is it possible to add two ports to a bridge with one command?
A: Yes. It can be done with ‘ovs-vsctl add-port <bridge_name> <port1> — add-port <bridge> <port2>