您的位置：首页 > 运维架构

Configuring VXLAN and GRE Tunnels on OpenvSwitch

2013-05-05 08:42 393 查看

CONFIGURING VXLAN AND GRE TUNNELS ON OPENVSWITCH

UPDATE: VXLAN ENCAPSULATION

VXLan is now upstreamed into the Master build. It is worth mentioning that is the encapsulation only, not the Multicast control plane functionality. That is being worked on by Mestery currently. So basically no need to pull from the fork but pull directly from
the Open vSwitch Master. ‘git clone git://openvswitch.org/openvswitch’ have the VXLan encap included.

I have done a couple of GRE tunnel how-tos using OpenvSwitch (OVS). I ran across the git tree of the VXlan implementation which is still in development I am sure. I don’t want anyone to think this is anything other than me seeing the tree on GitHub. I had been
itching to give VXLan a spin in OVS so why not ferret out someones tree on GitHub. I believe VXLan is still scheduled to officially release soon in OpenvSwitch. So here are the steps for installing, configuring tunnels on OpenvSwitch with VXLan and GRE encapsulations.
At the end we will compare some of the protocols with difference MTU sizes. The results were interesting I think (for a nerd). We will be installing and then configure both GRE and VXLan encapsulated tunnels using Open vSwitch.

I like seeing some collaboration of really smart people from different companies as displayed in the GPL below.

/* * Copyright (c) 2011 Nicira Networks.

* Copyright (c) 2012 Cisco Systems Inc.

* Distributed under the terms of the GNU GPL version 2.

* * Significant portions of this file may be copied from parts of the Linux

* kernel, by Linus Torvalds and others. */

Figure 1. Example of how tunnels can be leveraged as overlays.

By the way I should probably disclaim now that huge Layer2 networks do not scale and huge Layer3 networks do. Host count in a broadcast domain/Vlan/network should be kept to a reasonable 3 digit number. Cisco is quick to point out that OTV is the solution over
WAN’s for extending Layer 2 networks with OTV as the solution to extend Layer 2 Vlans. That said Overlays
are flat out required to overcome Vlan number limitations and have lots of potential with programmatic orchestration.

Figure 2. OVS punts the first packet to user land for the forwarding decision and passes the data path back to the data path in the kernel for subsequent packets in the flow. slow path for the first packet then fast path for the rest.

I do some simple Iperf tests at the end of the post with 1500 and 9000 byte MTUs. The numbers are kind of fun to look at from the result. For a really nice analysis take a look at Martin’s post at
Network Heresy on comparing STT, Linux Bridging and GRE. I am going to spin up some VMs on the VXLan tunnel later this week and measure the speeds a little closer and see how GRE and VXL
4000
an stack up to one another from hosts using overlays. I just ran some
Iperfs from the hypervisor itself rather than VMs. OVS also supports Capwap encapsulation that performs mac in GRE which is rather slick. Wonder why we do not hear much about that. I am going to dig in when I get back to home from the road later next week.

Figure 3. Lab Setup basically has a fake interface up with Br1. Real world Br1 would have VMs tapped on it. The video uses br1 and br2 but it got to be confusing for people so I changed it to br0 and br1 to match most peoples eth0
= br0 NIC naming in ifconfig.

Quick Video of the fast install below.

For those familiar with the build you can just paste the following in your bash shell as root. To walk through the install skip
the following snippet.

1234567891011121314151617181920

apt-get updateapt-get install -y git python-simplejson python-qt4 python-twisted-conch automake autoconf gcc uml-utilities libtool build-essential git pkg-config linux-headers-`uname -r`git clone git://openvswitch.org/openvswitchcd openvswitch./boot.sh./configure --with-linux=/lib/modules/`uname -r`/buildmake && make installinsmod datapath/linux/openvswitch.kotouch /usr/local/etc/ovs-vswitchd.confmkdir -p /usr/local/etc/openvswitchovsdb-tool create /usr/local/etc/openvswitch/conf.db vswitchd/vswitch.ovsschemaovsdb-server /usr/local/etc/openvswitch/conf.db \--remote=punix:/usr/local/var/run/openvswitch/db.sock \--remote=db:Open_vSwitch,manager_options \--private-key=db:SSL,private_key \--certificate=db:SSL,certificate \--bootstrap-ca-cert=db:SSL,ca_cert --pidfile --detach --log-fileovs-vsctl --no-wait initovs-vswitchd --pidfile --detachovs-vsctl show

OPEN VSWITCH SYSTEM PREPERATIONThis is on two boxes with 64 cores and Broadcom Corporation NetXtreme II BCM57810 10G Nics to a random 10G TOR switch.
Install dependencies

1 2	; html-script: false ]apt-get update apt-get install python-simplejson python-qt4 python-twisted-conch automake autoconf gcc uml-utilities libtool build-essential git pkg-config

Download the Open vSwitch latest build

12	git clone git://openvswitch.org/openvswitchcd openvswitch

COMPILE OPENVSWITCH FROM SOURCE

1

2

3

4

5

6

;
html-script:
false
]./boot.sh

./configure
--with-linux=/lib/modules/`uname
-r`/build

make

make
install

#Load the OVS Kernel Module

insmod
datapath/linux/openvswitch.ko

1234	; html-script: false ]#If you are running a stripped version of Nix like an EC2 cloud image and get an error along the lines of this:configure: error: source dir /lib/modules/3.2.0-23-virtual/build doesn't exist#Pull down the headers for your kernel. sudo apt-get install linux-headers-`uname -r`

INITIAL OPEN VSWITCH CONFIGURATION

1

2

3

;
html-script:
false
]touch
/usr/local/etc/ovs-vswitchd.conf

mkdir
-p
/usr/local/etc/openvswitch

ovsdb-tool
create
/usr/local/etc/openvswitch/conf.db
vswitchd/vswitch.ovsschema

Run the following commands, note some dashes are two ‘- -’ e.g. – -remote=db & – -private-key

Start ovsdb-server, this stores the config into a file that is persistent even after restarts.

123456

; html-script: false ]ovsdb-server /usr/local/etc/openvswitch/conf.db \--remote=punix:/usr/local/var/run/openvswitch/db.sock \--remote=db:Open_vSwitch,manager_options \--private-key=db:SSL,private_key \--certificate=db:SSL,certificate \--bootstrap-ca-cert=db:SSL,ca_cert --pidfile --detach --log-file

1

2

3

4

5

6

7

8

;
html-script:
false
]#Only
need to run this the first time.

ovs-vsctl
--no-wait
init

#Start vswitch

ovs-vswitchd
--pidfile
--detach

#Verify the kernel module(s) in case you didn't earlier and get
errors.

#lsmod | grep br

#brcompat 13512 0

#openvswitch 98196 1 brcompat

*note* “brcompat” is depreciated since the OVS upstream. Output should just be “openvswitch” as a loaded kernel module. If they are not there try loading again and check your path to the kernel module.

1	insmod datapath/linux/openvswitch.ko

At this point you have a fucntioning vanilla OVS install. Output should look something like this.

1

2

3

4

5

;
html-script:
false
]ovs-vsctl
show

b6d574d7-5582-4cc0-93e5-a90cf0eb0a38

root@demo-139:/home/ubuntu/ovs-vxlan#
ovs-vsctl --version

ovs-vsctl
(Open
vSwitch)
1.8.90

Compiled
Aug
19
2012
06:23:36

CONFIGURE LINUX NETWORKING

I have one NIC (eth0) on the same LAN segment/network/vlan.

We are attaching eth0 to br1 and applying an IP to the bridge interface.

We are attaching an IP to br1. br1 is the island that we are building a tunnel for hosts to connect on. Without the VXLAN tunnel, the two br1 interfaces should not be able to ping one another. Note: This is being setup on the same subnet, it is important to
keep in mind that the VXLAN framing will allow for the tunnel to be established over disparate networks. E.g. can be done over the Internet etc.

Configuration for Host 1

12345678

; html-script: false ]#Host 1 Configurationovs-vsctl add-br br0ovs-vsctl add-br br1ovs-vsctl add-port br0 eth0ifconfig eth0 0 && ifconfig br0 192.168.1.10 netmask 255.255.255.0route add default gw 192.168.1.1 br0ifconfig br1 10.1.2.10 netmask 255.255.255.0ovs-vsctl add-port br1 gre1 -- set interface gre1 type=gre options:remote_ip=192.168.1.11

Configuration for Host 2

1

2

3

4

5

6

7

8

;
html-script:
false
]#Host
1 Configuration

ovs-vsctl
add-br
br0

ovs-vsctl
add-br
br1

ovs-vsctl
add-port
br0
eth0

ifconfig
eth0
0
&&
ifconfig
br0
192.168.1.11
netmask
255.255.255.0

route
add
default
gw
192.168.1.1
br0

ifconfig
br1
10.1.2.11
netmask
255.255.255.0

ovs-vsctl
add-port
br1
gre1
--
set
interface
gre1
type=gre
options:remote_ip=192.168.1.10

If you have issues getting the bridge built you may need to kill the OVS processes and restart them depending on your step order.

Your Linux routing table and ifconfig should now look something like this:

1234567891011121314151617181920212223242526272829303132333435363738394041

root@ub64:/home/brent/openvswitch# route -nKernel IP routing tableDestination Gateway Genmask Flags Metric Ref Use Iface0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 br010.1.2.0 0.0.0.0 255.255.255.0 U 0 0 0 br1192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 br0root@ub64:/home/brent/openvswitch# ifconfigbr0 Link encap:Ethernet HWaddr 00:0c:29:65:fd:82 inet addr:192.168.1.10 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::285f:3ff:fea2:dafc/64 Scope:Link UP BROADCAST RUNNING MTU:1500 Metric:1 RX packets:622 errors:0 dropped:0 overruns:0 frame:0 TX packets:54 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:171756 (171.7 KB) TX bytes:3832 (3.8 KB) br1 Link encap:Ethernet HWaddr ea:34:7d:49:e7:49 inet addr:10.1.2.10 Bcast:10.1.2.255 Mask:255.255.255.0 inet6 addr: fe80::c0a7:aaff:fe90:29dc/64 Scope:Link UP BROADCAST RUNNING MTU:1500 Metric:1 RX packets:12 errors:0 dropped:0 overruns:0 frame:0 TX packets:13 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:896 (896.0 B) TX bytes:986 (986.0 B) eth0 Link encap:Ethernet HWaddr 00:0c:29:65:fd:82 inet6 addr: fe80::20c:29ff:fe65:fd82/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1222493 errors:0 dropped:0 overruns:0 frame:0 TX packets:53824 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:653399936 (653.3 MB) TX bytes:4089412 (4.0 MB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:7088 errors:0 dropped:0 overruns:0 frame:0 TX packets:7088 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:537609 (537.6 KB) TX bytes:537609 (537.6 KB)

TROUBLESHOOTING OPEN VSWITCH INSTALLATIONAn error was sent to me by someone that looked like this below, restarting the OVS procs will clear it up:

1

2

3

4

5

6

7

8

9

10

12345678910111213141516171819

; html-script: false ]#Find the OVS PIDs from PSps -ea | grep ovs16369 ? 00:00:00 ovsdb-server16372 ? 00:00:00 ovs-vswitchd16373 ? 00:00:00 ovs-vswitchd16375 ? 00:00:00 ovs_workq#Terminate themkill 16369kill 16372kill 16375#Then restart the OVS serverovsdb-server /usr/local/etc/openvswitch/conf.db \--remote=punix:/usr/local/var/run/openvswitch/db.sock \--remote=db:Open_vSwitch,manager_options \--private-key=db:SSL,private_key \--certificate=db:SSL,certificate \--bootstrap-ca-cert=db:SSL,ca_cert --pidfile --detach --log-file#Start vswitchd (JSON DB)ovs-vswitchd --pidfile --detach

BUILD A GRE TUNNEL IN OPEN VSWITCH

1 2	; html-script: false ]ovs-vsctl add-port br1 gre1 -- set interface gre1 type=gre options:remote_ip=192.168.1.10 ovs-vsctl add-port br1 gre1 -- set interface gre1 type=gre options:remote_ip=192.168.1.11

Adjust MTUs with % ifconfig <interface e.g. br0 eth0 etc> mtu 9000

#(Remember your switch needs to have >= your MTU.

Figure 4. Output of GRE Tunnels running with 1500 byte MTUs.

Figure 5. Output of GRE Tunnels running with 9000 byte MTUs.

Now delete the GRE ports

1	ovs-vsctl del-port gre1

CONFIGURE THE VXLAN TUNNELThe difference here is the “type” specifying the tunnel encapsulation.

1 2	ovs-vsctl add-port br1 vx1 -- set interface vx1 type=vxlan options:remote_ip=192.168.1.10 ovs-vsctl add-port br1 vx1 -- set interface vx1 type=vxlan options:remote_ip=192.168.1.11

Figure 6 Output of VXLan Tunnels running with 1500 byte MTUs.

内容来自用户分享和网络整理，不保证内容的准确性，如有侵权内容，可联系管理员处理

标签：

相关文章推荐

新的分享

章节导航