B Cisco n3k Interfaces Configuration Guide 602 U11
B Cisco n3k Interfaces Configuration Guide 602 U11
B Cisco n3k Interfaces Configuration Guide 602 U11
Release 6.x
First Published: April 09, 2013
Last Modified: June 16, 2014
Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 527-0883
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH
THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY,
CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain version
of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS" WITH ALL FAULTS.
CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT
LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS
HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network
topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional
and coincidental.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: http://
www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership
relationship between Cisco and any other company. (1110R)
Preface Preface xi
Audience xi
Document Conventions xi
Related Documentation for Cisco Nexus 3000 Series NX-OS Software xii
Documentation Feedback xiii
Obtaining Documentation and Submitting a Service Request xiv
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 iii
Contents
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
iv OL-29480-06
Contents
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 v
Contents
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
vi OL-29480-06
Contents
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 vii
Contents
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
viii OL-29480-06
Contents
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 ix
Contents
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
x OL-29480-06
Preface
This preface contains the following sections:
• Audience, page xi
• Document Conventions, page xi
• Related Documentation for Cisco Nexus 3000 Series NX-OS Software, page xii
• Documentation Feedback, page xiii
• Obtaining Documentation and Submitting a Service Request, page xiv
Audience
This publication is for network administrators who configure and maintain Cisco Nexus devices.
Document Conventions
Command descriptions use the following conventions:
Convention Description
bold Bold text indicates the commands and keywords that you enter literally
as shown.
Italic Italic text indicates arguments for which the user supplies the values.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 xi
Preface
Related Documentation for Cisco Nexus 3000 Series NX-OS Software
Convention Description
[x {y | z}] Nested set of square brackets or braces indicate optional or required
choices within optional or required elements. Braces and a vertical bar
within square brackets indicate a required choice within an optional
element.
variable Indicates a variable for which you supply values, in context where italics
cannot be used.
string A nonquoted set of characters. Do not use quotation marks around the
string or the string will include the quotation marks.
Convention Description
screen font Terminal sessions and information the switch displays are in screen font.
boldface screen font Information you must enter is in boldface screen font.
italic screen font Arguments for which you supply values are in italic screen font.
Note Means reader take note. Notes contain helpful suggestions or references to material not covered in the
manual.
Caution Means reader be careful. In this situation, you might do something that could result in equipment damage
or loss of data.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
xii OL-29480-06
Preface
Documentation Feedback
http://www.cisco.com/en/US/products/ps11541/tsd_products_support_series_home.html
Release Notes
The release notes are available at the following URL:
http://www.cisco.com/en/US/products/ps11541/prod_release_notes_list.html
License Information
For information about feature licenses in NX-OS, see the Cisco NX-OS Licensing Guide, available at the
following URL: http://www.cisco.com/en/US/docs/switches/datacenter/sw/nx-os/licensing/guide/b_Cisco_
NX-OS_Licensing_Guide.html.
For the NX-OS end user agreement and copyright information, see License and Copyright Information for
Cisco NX-OS Software, available at the following URL: http://www.cisco.com/en/US/docs/switches/datacenter/
sw/4_0/nx-os/license_agreement/nx-ossw_lisns.html.
Configuration Guides
The configuration guides are available at the following URL:
http://www.cisco.com/en/US/products/ps11541/products_installation_and_configuration_guides_list.html
Programming Guides
The XML Interface User Guide and other programming guides are available at the following URL:
http://www.cisco.com/en/US/products/ps11541/products_programming_reference_guides_list.html
Technical References
The technical references are available at the following URL:
http://www.cisco.com/en/US/products/ps11541/prod_technical_reference_list.html
Documentation Feedback
To provide technical feedback on this document, or to report an error or omission, please send your comments
to:
• [email protected]
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 xiii
Preface
Obtaining Documentation and Submitting a Service Request
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
xiv OL-29480-06
CHAPTER 1
New and Changed Information
This chapter contains the following sections:
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 1
New and Changed Information
New and Changed Information in this Release
DHCP Client You can now configure 6.0(2)U3(1) DHCP Client Discovery
configuration on SVIs the IP address of a DHCP on SVIs, on page 37
client by using the ip
address dhcp command.
Dynamic Port Breakout The dynamic breakout 6.0(2)U2(3) Port Modes, on page 8
for Cisco Nexus 3172 feature is now supported
by Cisco Nexus 3172.
Hashing for NVGRE Hashing for NVGRE 6.0(2)U2(1) Hashing for NVGRE
Traffic traffic allows the switch Traffic, on page 56
to include the GRE Key
field present in the GRE
header in hash
computations when
NVGRE traffic is
forwarded over a port
channel or an Equal Cost
Multipath (ECMP).
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
2 OL-29480-06
New and Changed Information
New and Changed Information in this Release
SVI Autostate Disable The SVI Autostate 6.0(2)U2(1) SVI Autostate Disable,
Disable feature enables on page 37
the Switch Virtual
Interface (SVI) to be in
the “up” state even if no
interface is in the “up”
state in the corresponding
VLAN.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 3
New and Changed Information
New and Changed Information in this Release
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
4 OL-29480-06
CHAPTER 2
Configuring Layer 2 Interfaces
This chapter contains the following sections:
Interface Command
You can enable the various capabilities of the Ethernet interfaces on a per-interface basis using the interface
command. When you enter the interface command, you specify the following information:
• Interface type—All physical Ethernet interfaces use the ethernet keyword.
• Slot number:
◦Slot 1 includes all the fixed ports.
◦Slot 2 includes the ports on the upper expansion module (if populated).
◦Slot 3 includes the ports on the lower expansion module (if populated).
◦Slot 4 includes the ports on the lower expansion module (if populated).
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 5
Configuring Layer 2 Interfaces
Unidirectional Link Detection Parameter
The interface numbering convention is extended to support use with a Cisco Nexus Fabric Extender as follows:
switch(config)# interface ethernet [chassis/]slot/port
• The chassis ID is an optional entry that you can use to address the ports of a connected Fabric Extender.
The chassis ID is configured on a physical Ethernet or EtherChannel interface on the switch to identify
the Fabric Extender discovered through the interface. The chassis ID ranges from 100 to 199.
Note By default, UDLD is locally disabled on copper LAN ports to avoid sending unnecessary control traffic
on this type of media.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
6 OL-29480-06
Configuring Layer 2 Interfaces
Unidirectional Link Detection Parameter
The following figure shows an example of a unidirectional link condition. Device B successfully receives
traffic from Device A on the port. However, Device A does not receive traffic from Device B on the same
port. UDLD detects the problem and disables the port.
UDLD per-port enable state for fiber-optic media Enabled on all Ethernet fiber-optic LAN ports
UDLD per-port enable state for twisted-pair (copper) Disabled on all Ethernet 10/100 and 1000BASE-TX
media LAN ports
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 7
Configuring Layer 2 Interfaces
Interface Speed
In these cases, the UDLD aggressive mode disables one of the ports on the link, which prevents traffic from
being discarded.
Interface Speed
Cisco Nexus 3000 Series switches have a number of fixed 10-Gigabit ports; each is equipped with SFP+
interface adapters. Cisco Nexus 3100 Series switches have 32 Quad Same Factor Pluggable (QSFP) ports and
4 SFP+ interface adapters. The default speed for these 32 ports is 40 Gbps.
Note When you break out from 40-Gigabit Ethernet to 10-Gigabit Ethernet, or break in from 10-Gigabit Ethernet
to 40-Gigabit Ethernet, all interface configurations are reset, and the affected ports are administratively
unavailable. To make these ports available, use the no shut command.
The ability to break out a 40-Gigabit Ethernet port into four 10-Gigabit Ethernet ports and break in four
10-Gigabit Ethernet ports into a 40-Gigabit Ethernet port dynamically allows you to use any of the
breakout-capable ports to work in the 40-Gigabit Ethernet or 10-Gigabit Ethernet modes without permanently
defining them.
For Cisco Nexus 3132Q switches, when the Ethernet interface 1/1 is in the 40-Gigabit Ethernet mode, the
first QSFP port is active. After breakout, when the Ethernet interface 1/1/1-4 is in the 10-Gigabit Ethernet
mode, you can choose to use either QSFP ports or SFP+ ports. However, both the first QSFP port and the
four SFP+ ports cannot be active at the same time.
Port Modes
Cisco Nexus 3100 Series switches have various port modes. In Cisco NX-OS Release 6.0(2)U(2)1, only the
Cisco Nexus 3132Q switch has port modes that support breakout. Cisco NX-OS Release 6.0(2)U(3)1 introduces
breakout port modes for the Cisco Nexus 3172PQ and 3172CR switches.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
8 OL-29480-06
Configuring Layer 2 Interfaces
SVI Autostate
Cisco Nexus 6 x QSFP ports and 48 SFP+ The following is the default port mode and supports
3172PQ and ports breakout:
Cisco Nexus
3172CR • 48x10G+breakout6x40G
SVI Autostate
The Switch Virtual Interface (SVI) represents a logical interface between the bridging function and the routing
function of a VLAN in the device. By default, when a VLAN interface has multiple ports in the VLAN, the
SVI goes to the down state when all the ports in the VLAN go down.
Autostate behavior is the operational state of an interface that is governed by the state of the various ports in
its corresponding VLAN. An SVI interface on a VLAN comes up when there is at least one port in that vlan
that is in STP forwarding state. Similarly, this interface goes down when the last STP forwarding port goes
down or goes to another STP state.
By default, Autostate calculation is enabled. You can disable Autostate calculation for an SVI interface and
change the default value.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 9
Configuring Layer 2 Interfaces
Cisco Discovery Protocol
Note Nexus 3000 Series switches do not support bridging between two VLANs when an SVI for one VLAN
exists on the same device as the bridging link. Traffic coming into the device and bound for the SVI is
dropped as a IPv4 discard. This is because the BIA MAC address is shared across VLANs/SVIs with no
option to modify the MAC of the SVI.
Error-Disabled State
An interface is in the error-disabled (err-disabled) state when the inteface is enabled administratively (using
the no shutdown command) but disabled at runtime by any process. For example, if UDLD detects a
unidirectional link, the interface is shut down at runtime. However, because the interface is administratively
enabled, the interface status displays as err-disabled. Once an interface goes into the err-disabled state, you
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
10 OL-29480-06
Configuring Layer 2 Interfaces
Default Interfaces
must manually reenable it or you can configure an automatic timeout recovery value. The err-disabled detection
is enabled by default for all causes. The automatic recovery is not configured by default.
When an interface is in the err-disabled state, use the errdisable detect cause command to find information
about the error.
You can configure the automatic err-disabled recovery timeout for a particular err-disabled cause by changing
the time variable.
The errdisable recovery cause command provides automatic recovery after 300 seconds. To change the
recovery period, use the errdisable recovery interval command to specify the timeout period. You can specify
30 to 65535 seconds.
To disable recovery of an interface from the err-disabled state, use the no errdisable recovery cause command.
The various options for the errdisable recover cause command are as follows:
• all—Enables a timer to recover from all causes.
• bpduguard—Enables a timer to recover from the bridge protocol data unit (BPDU) Guard error-disabled
state.
• failed-port-state—Enables a timer to recover from a Spanning Tree Protocol (STP) set port state failure.
• link-flap—Enables a timer to recover from linkstate flapping.
• pause-rate-limit—Enables a timer to recover from the pause rate limit error-disabled state.
• udld—Enables a timer to recover from the Unidirectional Link Detection (UDLD) error-disabled state.
• loopback—Enables a timer to recover from the loopback error-disabled state.
If you do not enable the err-disabled recovery for the cause, the interface stays in the err-disabled state until
you enter the shutdown and no shutdown commands. If the recovery is enabled for a cause, the interface is
brought out of the err-disabled state and allowed to retry operation once all the causes have timed out. Use
the show interface status err-disabled command to display the reason behind the error.
Default Interfaces
You can use the default interface feature to clear the configured parameters for both physical and logical
interfaces such as the Ethernet, loopback, management, VLAN, and the port-channel interface.
Caution Enabling the debounce timer causes the link-down detections to be delayed, which results in a loss of
traffic during the debounce period. This situation might affect the convergence and reconvergence of some
Layer 2 and Layer 3 protocols.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 11
Configuring Layer 2 Interfaces
MTU Configuration
MTU Configuration
The Cisco Nexus device switch does not fragment frames. As a result, the switch cannot have two ports in
the same Layer 2 domain with different maximum transmission units (MTUs). A per-physical Ethernet interface
MTU is not supported. Instead, the MTU is set according to the QoS classes. You modify the MTU by setting
class and policy maps.
Note When you show the interface settings, a default MTU of 1500 is displayed for physical Ethernet interfaces.
Downlink Delay
You can operationally enable uplink SFP+ ports before downlink RJ-45 ports after a reload on a Cisco Nexus
3048 switch. You must delay enabling the RJ-45 ports in the hardware until the SFP+ ports are enabled.
You can configure a timer that during reload enables the downlink RJ-45 ports in hardware only after the
specified timeout. This process allows the uplink SFP+ ports to be operational first. The timer is enabled in
the hardware for only those ports that are admin-enable.
Downlink delay is disabled by default and must be explicitly enabled. When enabled, if the delay timer is not
specified, it is set for a default delay of 20 seconds.
Encapsulation ARPA
1 MTU cannot be changed per-physical Ethernet interface. You modify MTU by selecting maps of QoS classes.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
12 OL-29480-06
Configuring Layer 2 Interfaces
Configuring Ethernet Interfaces
Note Before you begin, UDLD must be enabled for the other linked port and its device.
Procedure
Step 4 switch(config)# show udld global Displays the UDLD status for the device.
Step 5 switch(config)# interface type slot/port Specifies an interface to configure, and enters
interface configuration mode.
Step 6 switch(config-if)# udld {enable | disable Enables the normal UDLD mode, disables
| aggressive} UDLD, or enables the aggressive UDLD mode.
Step 7 switch(config-if)# show udld interface Displays the UDLD status for the interface.
This example shows how to enable the normal UDLD mode for an Ethernet port:
This example shows how to enable the aggressive UDLD mode for an Ethernet port:
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 13
Configuring Layer 2 Interfaces
Triggering the Link State Consistency Checker
Procedure
This example shows how to trigger a Link State consistency check and display its results:
switch# show consistency-checker link-state module 1
Link State Checks: Link state only
Consistency Check: FAILED
No inconsistencies found for:
Ethernet1/1
Ethernet1/2
Ethernet1/3
Ethernet1/4
Ethernet1/5
Ethernet1/6
Ethernet1/7
Ethernet1/8
Ethernet1/9
Ethernet1/10
Ethernet1/12
Ethernet1/13
Ethernet1/14
Ethernet1/15
Inconsistencies found for following interfaces:
Ethernet1/11
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
14 OL-29480-06
Configuring Layer 2 Interfaces
Changing an Interface Port Mode
Procedure
Step 2 switch(config)# copy running-config Copies the running configuration to the bootflash. You
bootflash: my-config.cfg can use this file to configure your device later.
This example shows how to change the port mode to 48x10g+breakout6x40g for QSFP+ ports:
switch# configure terminal
switch(config)# copy running-config bootflash:my-config.cfg
switch(config)# write erase
switch(config)# reload
WARNING: This command will reboot the system
Do you want to continue? (y/n) [n] y
switch(config)# hardware profile portmode 48x10g+breakout6x40g
Warning: This command will take effect only after saving the configuration and reload!
Port configurations could get lost when port mode is changed!
switch(config)# copy running-config startup-config
switch(config)# reload
WARNING: This command will reboot the system
Do you want to continue? (y/n) [n] y
This example shows how to change the port mode to 48x10g+4x40g for QSFP+ ports:
switch# configure terminal
switch(config)# copy running-config bootflash:my-config.cfg
switch(config)# write erase
switch(config)# reload
WARNING: This command will reboot the system
Do you want to continue? (y/n) [n] y
switch(config)# hardware profile portmode 48x10g+4x40g
Warning: This command will take effect only after saving the configuration and reload!
Port configurations could get lost when port mode is changed!
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 15
Configuring Layer 2 Interfaces
Configuring the Interface Speed
This example shows how to change the port mode to 48x10g+4x40g for QSFP+ ports and verify the changes:
switch# configure terminal
switch(config)# hardware profile portmode 48x10g+4x40g
Warning: This command will take effect only after saving the configuration and r
eload! Port configurations could get lost when port mode is changed!
switch(config)# show running-config
!Command: show running-config
!Time: Thu Aug 25 07:39:37 2011
version 5.0(3)U2(1)
feature telnet
no feature ssh
feature lldp
username admin password 5 $1$OOV4MdOM$BAB5RkD22YanT4empqqSM0 role network-admin
ip domain-lookup
switchname BLR-QG-5
ip access-list my-acl
10 deny ip any 10.0.0.1/32
20 deny ip 10.1.1.1/32 any
class-map type control-plane match-any copp-arp
class-map type control-plane match-any copp-bpdu
:
:
control-plane
service-policy input copp-system-policy
hardware profile tcam region arpacl 128
hardware profile tcam region ifacl 256
hardware profile tcam region racl 256
hardware profile tcam region vacl 512
hardware profile portmode 48x10G+4x40G
snmp-server user admin network-admin auth md5 0xdd1d21ee42e93106836cdefd1a60e062
<--Output truncated-->
switch#
This example shows how to restore the default port mode for QSFP+ ports:
switch# configure terminal
switch(config)# no hardware profile portmode
Warning: This command will take effect only after saving the configuration and r
eload! Port configurations could get lost when port mode is changed!
switch(config)#
Note If the interface and transceiver speed is mismatched, the SFP validation failed message is displayed when
you enter the show interface ethernet slot/port command. For example, if you insert a 1-Gigabit SFP
transceiver into a port without configuring the speed 1000 command, you will get this error. By default,
all ports are 10 Gbps.
Procedure
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
16 OL-29480-06
Configuring Layer 2 Interfaces
Configuring Break-Out 10-Gigabit Interface Speed Ports
This example shows how to set the speed for a 1-Gigabit Ethernet port:
switch# configure terminal
switch(config)# interface ethernet 1/4
switch(config-if)# speed 1000
Procedure
Step 2 switch(config)# interface type Enters interface configuration mode for the specified
slot/port-range interface.
Note Interface range is not supported for 40-Gigabit
Ethernet interfaces. For example, Eth 1/2-5 is
not supported.
Step 3 switch(config-if)# speed 10000 Sets the speed on the interface to 10-Gigabit per second.
This example shows how to set the speed to 10-Gigabit per second on Ethernet interface 1/2:
switch# configure terminal
switch(config)# interface ethernet 1/2
switch(config-if)# speed 10000
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 17
Configuring Layer 2 Interfaces
Configuring Break-In 40-Gigabit Ethernet Interface Speed Ports
Procedure
Step 2 switch(config)# interface type Enters interface configuration mode for the specified
slot/port interface.
Note The Interface range is supported for 10-Gigabit
Ethernet interfaces. For example, Eth 1/2/1-4
is supported.
Step 3 switch(config-if)# speed 40000 Sets the speed on the interface to 40 Gbps.
This example shows how to set the speed to 40 Gbps on Ethernet interface 1/2/1:
switch# configure terminal
switch(config)# interface ethernet 1/2/1
switch(config-if)# speed 40000
Procedure
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
18 OL-29480-06
Configuring Layer 2 Interfaces
Disabling Link Negotiation
Note Autonegotiation configuration is not applicable on 10-Gigabit Ethernet ports. When autonegotiation is
configured on a 10-Gigabit port the following error message is displayed:
ERROR: Ethernet1/40: Configuration does not match the port capability
Procedure
Step 2 switch(config)# interface ethernet Selects the interface and enters interface mode.
slot/port
Step 3 switch(config-if)# no negotiate Disables link negotiation on the selected Ethernet interface
auto (1-Gigabit port).
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 19
Configuring Layer 2 Interfaces
Disabling SVI Autostate
This example shows how to disable autonegotiation on a specified Ethernet interface (1-Gigabit port):
switch# configure terminal
switch(config)# interface ethernet 1/1
switch(config-if)# no negotiate auto
switch(config-if)#
This example shows how to enable autonegotiation on a specified Ethernet interface (1-Gigabit port):
switch# configure terminal
switch(config)# interface ethernet 1/5
switch(config-if)# negotiate auto
switch(config-if)#
Procedure
Step 3 switch(config)# system default Configures the system to enable or disable the
interface-vlan [no] autostate Autostate default behavior.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
20 OL-29480-06
Configuring Layer 2 Interfaces
Configuring a Default Interface
This example shows how to disable the systems Autostate default for all the SVIs on the switch:
switch# configure terminal
switch(config)# feature interface-vlan
switch(config)# system default interface-vlan no autostate
switch(config)# interface vlan 50
switch(config-if)# no autostate
switch(config)# copy running-config startup-config
Procedure
Step 2 switch(config)# default interface type Deletes the configuration of the interface and restores
interface number the default configuration. The following are the
supported interfaces:
• ethernet
• loopback
• mgmt
• port-channel
• vlan
This example shows how to delete the configuration of an Ethernet interface and revert it to its default
configuration:
switch# configure terminal
switch(config)# default interface ethernet 1/3
.......Done
switch(config)# exit
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 21
Configuring Layer 2 Interfaces
Enabling or Disabling CDP
Procedure
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
22 OL-29480-06
Configuring Layer 2 Interfaces
Enabling the Error-Disabled Detection
Procedure
Step 2 switch(config)# interface type slot/port Enters interface configuration mode for the specified
interface.
Procedure
Step 2 switch(config)# errdisable detect Specifies a condition under which to place the interface
cause {all | link-flap | loopback} in an err-disabled state. The default is enabled.
Step 4 switch(config)# no shutdown Brings the interface up administratively and enables the
interface to recover manually from the err-disabled state.
Step 5 switch(config)# show interface status Displays information about err-disabled interfaces.
err-disabled
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 23
Configuring Layer 2 Interfaces
Enabling the Error-Disabled Recovery
This example shows how to enable the err-disabled detection in all cases:
switch# configure terminal
switch(config)# errdisable detect cause all
switch(config)# shutdown
switch(config)# no shutdown
switch(config)# show interface status err-disabled
switch(config)# copy running-config startup-config
Procedure
Step 2 switch(config)# errdisable recovery Specifies a condition under which the interface
cause {all | udld | bpduguard | link-flap automatically recovers from the err-disabled state,
| failed-port-state | pause-rate-limit | and the device retries bringing the interface up. The
loopback} device waits 300 seconds to retry. The default is
disabled.
Step 3 switch(config)# show interface status Displays information about err-disabled interfaces.
err-disabled
Step 4 switch(config)# copy running-config (Optional)
startup-config Saves the change persistently through reboots and
restarts by copying the running configuration to the
startup configuration.
This example shows how to enable err-disabled recovery under all conditions:
switch# configure terminal
switch(config)# errdisable recovery cause loopback
switch(config)# show interface status err-disabled
switch(config)# copy running-config startup-config
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
24 OL-29480-06
Configuring Layer 2 Interfaces
Configuring the Error-Disabled Recovery Interval
Procedure
Step 2 switch(config)# errdisable recovery Specifies the interval for the interface to recover from
interval interval the err-disabled state. The range is from 30 to 65535
seconds. The default is 300 seconds.
Step 3 switch(config)# show interface status Displays information about err-disabled interfaces.
err-disabled
Step 4 switch(config)# copy running-config (Optional)
startup-config Saves the change persistently through reboots and
restarts by copying the running configuration to the
startup configuration.
This example shows how to enable err-disabled recovery under all conditions:
switch# configure terminal
switch(config)# errdisable recovery interval 32
switch(config)# show interface status err-disabled
switch(config)# copy running-config startup-config
Procedure
Step 2 switch(config)# no errdisable recovery Specifies a condition under which the interface
cause {all | udld | bpduguard | link-flap | reverts back to the default err-disabled state.
failed-port-state | pause-rate-limit |
loopback}
Step 3 switch(config)# show interface status (Optional)
err-disabled Displays information about err-disabled
interfaces.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 25
Configuring Layer 2 Interfaces
Configuring the Debounce Timer
Procedure
Step 2 switch(config)# interface type slot/port Enters interface configuration mode for the specified
interface.
Step 3 switch(config-if)# link debounce time Enables the debounce timer for the amount of time
milliseconds (1 to 5000 ms) specified.
Disables the debounce timer if you specify 0
milliseconds.
This example shows how to enable the debounce timer and set the debounce time to 1000 ms for an Ethernet
interface:
switch# configure terminal
switch(config)# interface ethernet 3/1
switch(config-if)# link debounce time 1000
This example shows how to disable the debounce timer for an Ethernet interface:
switch# configure terminal
switch(config)# interface ethernet 3/1
switch(config-if)# link debounce time 0
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
26 OL-29480-06
Configuring Layer 2 Interfaces
Configuring the Description Parameter
Procedure
Step 2 switch(config)# interface type slot/port Enters interface configuration mode for the
specified interface.
Step 3 switch(config-if)# description test Specifies the description for the interface.
This example shows how to set the interface description to Server 3 interface:
switch# configure terminal
switch(config)# interface ethernet 1/3
switch(config-if)# description Server 3 Interface
Procedure
Step 2 switch(config)# interface type slot/port Enters interface configuration mode for the
specified interface.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 27
Configuring Layer 2 Interfaces
Configuring Downlink Delay
switch(config-if)# no shutdown
Procedure
Step 2 switch(config)# downlink delay enable | Enables or disables downlink delay and
disable [timeout time-out] configures the timeout.
This example shows how to enable downlink delay and configure the delay timeout on the switch:
switch# configure terminal
switch(config)# downlink delay enable timeout 45
Command Purpose
switch# show interface type slot/port Displays the detailed configuration of the specified
interface.
switch# show interface type slot/port capabilities Displays detailed information about the capabilities
of the specified interface. This option is available
only for physical interfaces.
switch# show interface type slot/port transceiver Displays detailed information about the transceiver
connected to the specified interface. This option is
available only for physical interfaces.
switch# show interface flowcontrol Displays the detailed listing of the flow control
settings on all interfaces.
The show interface command is invoked from EXEC mode and displays the interface configurations. Without
any arguments, this command displays the information for all the configured interfaces in the switch.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
28 OL-29480-06
Configuring Layer 2 Interfaces
Displaying Interface Information
MDIX: no
FEX Fabric: yes
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 29
Configuring Layer 2 Interfaces
Displaying Input Packet Discard Information
This example shows how to display a brief interface status (some of the output has been removed for brevity):
switch# show interface brief
--------------------------------------------------------------------------------
Ethernet VLAN Type Mode Status Reason Speed Port
Interface Ch #
--------------------------------------------------------------------------------
Eth1/1 200 eth trunk up none 10G(D) --
Eth1/2 1 eth trunk up none 10G(D) --
Eth1/3 300 eth access down SFP not inserted 10G(D) --
Eth1/4 300 eth access down SFP not inserted 10G(D) --
Eth1/5 300 eth access down Link not connected 1000(D) --
Eth1/6 20 eth access down Link not connected 10G(D) --
Eth1/7 300 eth access down SFP not inserted 10G(D) --
...
+-----------------------------------------+-----------------+----------------+
| Counter Description | Count | |
+-----------------------------------------+-----------------+----------------+
IPv4 Discards 0
STP Discards 0
Policy Discards 100
ACL Drops 0
Receive Drops 0
Vlan Discards 33
+-----------------------------------------+-----------------+----------------+
Counter Information:
• IPv4 Discards—IPv4 Discards represent errors at the IP layer, for example the IP checksum error.
• STP Discards—STP Discards are incremented when the receive interface STP state is not forwarding
the packets received.
• Policy Discards—Policy Discards are incremented when there are discards because of the input policy
on the interface.
• ACL Drops—ACL drops indicate that incoming packets match an ACL entry with a drop action.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
30 OL-29480-06
Configuring Layer 2 Interfaces
MIBs for Layer 2 Interfaces
• Receive Drops—This drop increment represents a condition when no output port is determined for an
ingress packet. Receive drops happen because of variety of reasons including IPv4, STP, and policy
discards. The drop counter increments with one of the above counters or separately.
• VLAN Discard—VLAN Discard indicates VLAN-based discards. For example, a VLAN-tagged packet
that is ingressing on a port that is not a member of the VLAN.
This example shows how to clear all the input discard counters:
switch# show hardware internal interface indiscard-stats front-port 1 clear
+-----------------------------------------+-----------------+----------------+-------------------------------------+
| Counter Description | Count | Last Increment | Last
Increment Time |
+-----------------------------------------+-----------------+----------------+-------------------------------------+
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 31
Configuring Layer 2 Interfaces
MIBs for Layer 2 Interfaces
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
32 OL-29480-06
CHAPTER 3
Configuring Layer 3 Interfaces
This chapter contains the following sections:
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 33
Configuring Layer 3 Interfaces
Routed Interfaces
Routed Interfaces
You can configure a port as a Layer 2 interface or a Layer 3 interface. A routed interface is a physical port
that can route IP traffic to another device. A routed interface is a Layer 3 interface only and does not support
Layer 2 protocols, such as the Spanning Tree Protocol (STP).
All Ethernet ports are Layer 2 (switchports) by default. You can change this default behavior using the no
switchport command from interface configuration mode. To change multiple ports at one time, you can
specify a range of interfaces and then apply the no switchport command.
You can assign an IP address to the port, enable routing, and assign routing protocol characteristics to this
routed interface.
You can assign a static MAC address to a Layer 3 interface. The default MAC address for a Layer 3 interface
is the MAC address of the virtual device context (VDC) that is associated with it. You can change the default
MAC address of the Layer 3 interface by using the mac-address command from the interface configuration
mode. A static MAC address can be configured on SVI, Layer 3 interfaces, port channels, Layer 3 subinterfaces,
and tunnel interfaces. You can also configure static MAC addresses on a range of ports and port channels.
However, all ports must be in Layer 3. Even if one port in the range of ports is in Layer 2, the command is
rejected and an error message appears. For information on configuring MAC addresses, see the Layer 2
Switching Configuration Guide for your device.
You can also create a Layer 3 port channel from routed interfaces.
Routed interfaces and subinterfaces support exponentially decayed rate counters. Cisco NX-OS tracks the
following statistics with these averaging counters:
• Input packets/sec
• Output packets/sec
• Input bytes/sec
• Output bytes/sec
Subinterfaces
You can create virtual subinterfaces on a parent interface configured as a Layer 3 interface. A parent interface
can be a physical port or a port channel.
Subinterfaces divide the parent interface into two or more virtual interfaces on which you can assign unique
Layer 3 parameters such as IP addresses and dynamic routing protocols. The IP address for each subinterface
should be in a different subnet from any other subinterface on the parent interface.
You create a subinterface with a name that consists of the parent interface name (for example, Ethernet 2/1)
followed by a period and then by a number that is unique for that subinterface. For example, you could create
a subinterface for Ethernet interface 2/1 named Ethernet 2/1.1 where .1 indicates the subinterface.
Cisco NX-OS enables subinterfaces when the parent interface is enabled. You can shut down a subinterface
independent of shutting down the parent interface. If you shut down the parent interface, Cisco NX-OS shuts
down all associated subinterfaces as well.
One use of subinterfaces is to provide unique Layer 3 interfaces to each VLAN that is supported by the parent
interface. In this scenario, the parent interface connects to a Layer 2 trunking port on another device. You
configure a subinterface and associate the subinterface to a VLAN ID using 802.1Q trunking.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
34 OL-29480-06
Configuring Layer 3 Interfaces
VLAN Interfaces
The following figure shows a trunking port from a switch that connects to router B on interface E 2/1. This
interface contains three subinterfaces that are associated with each of the three VLANs that are carried by the
trunking port.
VLAN Interfaces
A VLAN interface or a switch virtual interface (SVI) is a virtual routed interface that connects a VLAN on
the device to the Layer 3 router engine on the same device. Only one VLAN interface can be associated with
a VLAN, but you need to configure a VLAN interface for a VLAN only when you want to route between
VLANs or to provide IP host connectivity to the device through a virtual routing and forwarding (VRF)
instance that is not the management VRF. When you enable VLAN interface creation, Cisco NX-OS creates
a VLAN interface for the default VLAN (VLAN 1) to permit remote switch administration.
You must enable the VLAN network interface feature before you can configure it. The system automatically
takes a checkpoint prior to disabling the feature, and you can roll back to this checkpoint. For information
about rollbacks and checkpoints, see the System Management Configuration Guide for your device.
You can route across VLAN interfaces to provide Layer 3 inter-VLAN routing by configuring a VLAN
interface for each VLAN that you want to route traffic to and assigning an IP address on the VLAN interface.
For more information on IP addresses and IP routing, see the Unicast Routing Configuration Guide for your
device.
The following figure shows two hosts connected to two VLANs on a device. You can configure VLAN
interfaces for each VLAN that allows Host 1 to communicate with Host 2 using IP routing between the VLANs.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 35
Configuring Layer 3 Interfaces
Loopback Interfaces
VLAN 1 communicates at Layer 3 over VLAN interface 1and VLAN 10 communicates at Layer 3 over VLAN
interface 10.
Loopback Interfaces
A loopback interface is a virtual interface with a single endpoint that is always up. Any packet that is transmitted
over a loopback interface is immediately received by this interface. Loopback interfaces emulate a physical
interface.
You can use loopback interfaces for performance analysis, testing, and local communications. Loopback
interfaces can act as a termination address for routing protocol sessions. This loopback configuration allows
routing protocol sessions to stay up even if some of the outbound interfaces are down.
Tunnel Interfaces
Cisco NX-OS supports tunnel interfaces as IP tunnels. IP tunnels can encapsulate a same- layer or higher
layer protocol and transport the result over IP through a tunnel that is created between two routers.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
36 OL-29480-06
Configuring Layer 3 Interfaces
Default Settings for Layer 3 Interfaces
• If you change a Layer 2 interface to a Layer 3 interface, Cisco NX-OS shuts down the interface, reenables
the interface, and deletes all configuration specific to Layer 2.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 37
Configuring Layer 3 Interfaces
Configuring Layer 3 Interfaces
• The DNS server and default router option-related configurations are saved in the startup configuration
when you enter the copy running-config startup-config command. When you reload the switch, if this
configuration is not applicable, you might have to remove it.
• You can configure a maximum of six DNS servers on the switch, which is a switch limitation. This
maximum number includes the DNS servers configured by the DHCP client and the DNS servers
configured manually.
• If the number of DNS servers configured on the switch is more than six, and if you get a DHCP offer
for an SVI with DNS option set, the IP address is not assigned to the SVI.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
38 OL-29480-06
Configuring Layer 3 Interfaces
Configuring a Subinterface
Configuring a Subinterface
Before You Begin
• Configure the parent interface as a routed interface.
• Create the port-channel interface if you want to create a subinterface on that port channel.
Procedure
Step 2 switch(config)# interface ethernet Enters interface configuration mode. The range for the
slot/port.number slot is from 1 to 255. The range for the port is from 1
to 128.
Step 3 switch(config-if)# [ip | ipv6] address Configures an IP address for this interface.
ip-address/length
Step 4 switch(config-if)# encapsulation Configures IEEE 802.1Q VLAN encapsulation on the
dot1Q vlan-id subinterface. The range for the vlan-id is from 2 to
4093.
Step 5 switch(config-if)# show interfaces (Optional)
Displays the Layer 3 interface statistics.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 39
Configuring Layer 3 Interfaces
Configuring the Bandwidth on an Interface
Procedure
Step 2 switch(config)# interface Enters interface configuration mode. The range for the slot is
ethernet slot/port from 1 to 255. The range for the port is from 1 to 128.
Step 3 switch(conifg-if)# bandwidth Configures the bandwidth parameter for a routed interface,
[value | inherit [value]] port channel, or subinterface, as follows:
• value—Size of the bandwidth in kilobytes. The range is
from 1 to 10000000.
• inherit—Indicates that all subinterfaces of this interface
inherit either the bandwidth value (if a value is specified)
or the bandwidth of the parent interface (if a value is not
specified).
This example shows how to configure Ethernet interface 2/1 with a bandwidth value of 80000:
Step 3 switch(config)# interface vlan number Creates a VLAN interface. The number range is
from 1 to 4094.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
40 OL-29480-06
Configuring Layer 3 Interfaces
Configuring a Loopback Interface
Procedure
Step 2 switch(config)# interface loopback Creates a loopback interface. The instance range is
instance from 0 to 1023.
Step 3 switch(config-if)# [ip | ipv6 ] address Configures an IP address for this interface.
ip-address/length
Step 4 switch(config-if)# show interface (Optional)
loopback instance Displays the loopback interface statistics. The
instance range is from 0 to 1023.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 41
Configuring Layer 3 Interfaces
Assigning an Interface to a VRF
Procedure
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
42 OL-29480-06
Configuring Layer 3 Interfaces
Configuring an Interface MAC Address
Procedure
Step 4 switch(config-if)# show interface (Optional) Displays all information for the interface.
ethernet slot/port
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 43
Configuring Layer 3 Interfaces
Configuring a DHCP client on an SVI
Procedure
Step 2 switch(config)# system default Reenables the system default autostate behavior on
interface-vlan autostate Switching Virtual Interface (SVI) in a VLAN. Use
the no form of the command to disable the autostate
behavior on SVI.
Step 3 switch(config)# feature Enables the creation of VLAN interfaces SVI.
interface-vlan
Step 4 switch(config)# interface vlan vlan Disables the VLAN interface and enters interface
id configuration mode.
Step 5 (config-if)# [no] autostate Disables the default autostate behavior of SVIs on the
VLAN interface.
Step 6 (config-if)# end Returns to privileged EXEC mode.
Step 7 show running-config interface vlan (Optional) Displays the running configuration for a
vlan id specific port channel.
This example shows how to configure the SVI Autostate Disable feature:
switch# configure terminal
switch(config)# system default interface-vlan autostate
switch(config)# feature interface-vlan
switch(config)# interface vlan 2
switch(config-if)# no autostate
switch(config-if)# end
Procedure
Step 2 switch(config)# interface vlan vlan id Creates a VLAN interface. The range of vlan id is
from 1 to 4094
Step 3 switch(config-if)# ip address dhcp Requests the DHCP server for an IP address.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
44 OL-29480-06
Configuring Layer 3 Interfaces
Verifying the Layer 3 Interfaces Configuration
This example shows how to configure the IP address of a DHCP client on an SVI:
switch# configure terminal
switch(config)# interface vlan 15
switch(config-if)# ip address dhcp
Command Purpose
show interface ethernet slot/port Displays the Layer 3 interface configuration, status,
and counters (including the 5-minute exponentially
decayed moving average of inbound and outbound
packet and byte rates).
show interface ethernet slot/port brief Displays the Layer 3 interface operational status.
show interface ethernet slot/port capabilities Displays the Layer 3 interface capabilities, including
port type, speed, and duplex.
show interface ethernet slot/port description Displays the Layer 3 interface description.
show interface ethernet slot/port status Displays the Layer 3 interface administrative status,
port mode, speed, and duplex.
show interface ethernet slot/port.number Displays the subinterface configuration, status, and
counters (including the f-minute exponentially
decayed moving average of inbound and outbound
packet and byte rates).
show interface loopback number Displays the loopback interface configuration, status,
and counters.
show interface loopback number brief Displays the loopback interface operational status.
show interface loopback number description Displays the loopback interface description.
show interface loopback number status Displays the loopback interface administrative status
and protocol status.
show interface vlan number Displays the VLAN interface configuration, status,
and counters.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 45
Configuring Layer 3 Interfaces
Triggering the Layer 3 Interface Consistency Checker
Command Purpose
show interface vlan number brief Displays the VLAN interface operational status.
show interface vlan number description Displays the VLAN interface description.
show interface vlan number private-vlan mapping Displays the VLAN interface private VLAN
information.
show interface vlan number status Displays the VLAN interface administrative status
and protocol status.
Procedure
This example shows how to trigger the Layer 3 interface consistency check and display its results:
switch# show consistency-checker l3-interface module 1
L3 LIF Checks: L3 Vlan, CML Flags, IPv4 Enable
Consistency Check: PASSED
No inconsistencies found for:
Ethernet1/17
Ethernet1/49
Ethernet1/50
Command Purpose
show interface ethernet slot/port counters Displays the Layer 3 interface statistics (unicast,
multicast, and broadcast).
show interface ethernet slot/port counters brief Displays the Layer 3 interface input and output
counters.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
46 OL-29480-06
Configuring Layer 3 Interfaces
Configuration Examples for Layer 3 Interfaces
Command Purpose
show interface ethernet slot/port counters detailed Displays the Layer 3 interface statistics. You can
[all] optionally include all 32-bit and 64-bit packet and
byte counters (including errors).
show interface ethernet slot/port counters error Displays the Layer 3 interface input and output
errors.
show interface ethernet slot/port counters snmp Displays the Layer 3 interface counters reported
by SNMP MIBs. You cannot clear these counters.
show interface ethernet slot/port.number counters Displays the subinterface statistics (unicast,
multicast, and broadcast).
show interface loopback number counters Displays the loopback interface input and output
counters (unicast, multicast, and broadcast).
show interface loopback number counters detailed Displays the loopback interface statistics. You can
[all] optionally include all 32-bit and 64-bit packet and
byte counters (including errors).
show interface loopback number counters errors Displays the loopback interface input and output
errors.
show interface vlan number counters Displays the VLAN interface input and output
counters (unicast, multicast, and broadcast).
show interface vlan number counters detailed [all] Displays the VLAN interface statistics. You can
optionally include all Layer 3 packet and byte
counters (unicast and multicast).
show interface vlan counters snmp Displays the VLAN interface counters reported
by SNMP MIBs. You cannot clear these counters.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 47
Configuring Layer 3 Interfaces
Related Documents for Layer 3 Interfaces
This example shows how to configure Switching Virtual Interface (SVI) Autostate Disable:
switch# configure terminal
switch(config)# system default interface-vlan autostate
switch(config)# feature interface-vlan
switch(config)# interface vlan 2
switch(config-if)# no autostate
switch(config-if)# end
switch# show running-config interface vlan 2
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
48 OL-29480-06
Configuring Layer 3 Interfaces
Standards for Layer 3 Interfaces
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 49
Configuring Layer 3 Interfaces
Feature History for Layer 3 Interfaces
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
50 OL-29480-06
CHAPTER 4
Configuring Port Channels
This chapter contains the following sections:
Related Topics
LACP Overview, on page 57
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 51
Configuring Port Channels
Understanding Port Channels
Note Cisco NX-OS does not support Port Aggregation Protocol (PAgP) for port channels.
A port channel bundles individual links into a channel group to create a single logical link that provides the
aggregate bandwidth of several physical links. If a member port within a port channel fails, traffic previously
carried over the failed link switches to the remaining member ports within the port channel.
Each port can be in only one port channel. All the ports in a port channel must be compatible; they must use
the same speed and operate in full-duplex mode. When you are running static port channels without LACP,
the individual links are all in the on channel mode; you cannot change this mode without enabling LACP.
Note You cannot change the mode from ON to Active or from ON to Passive.
You can create a port channel directly by creating the port-channel interface, or you can create a channel
group that acts to aggregate individual ports into a bundle. When you associate an interface with a channel
group, Cisco NX-OS creates a matching port channel automatically if the port channel does not already exist.
You can also create the port channel first. In this instance, Cisco NX-OS creates an empty channel group with
the same channel number as the port channel and takes the default configuration.
Note A port channel is operationally up when at least one of the member ports is up and that port’s status is
channeling. The port channel is operationally down when all member ports are operationally down.
Compatibility Requirements
When you add an interface to a port channel group, Cisco NX-OS checks certain interface attributes to ensure
that the interface is compatible with the channel group. Cisco NX-OS also checks a number of operational
attributes for an interface before allowing that interface to participate in the port-channel aggregation.
The compatibility check includes the following operational attributes:
• Port mode
• Access VLAN
• Trunk native VLAN
• Allowed VLAN list
• Speed
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
52 OL-29480-06
Configuring Port Channels
Compatibility Requirements
Use the show port-channel compatibility-parameters command to see the full list of compatibility checks
that Cisco NX-OS uses.
You can only add interfaces configured with the channel mode set to on to static port channels. You can also
only add interfaces configured with the channel mode as active or passive to port channels that are running
LACP. You can configure these attributes on an individual member port.
When the interface joins a port channel, the following individual parameters are replaced with the values on
the port channel:
• Bandwidth
• MAC address
• Spanning Tree Protocol
The following interface parameters remain unaffected when the interface joins a port channel:
• Description
• CDP
• LACP port priority
• Debounce
After you enable forcing a port to be added to a channel group by entering the channel-group force command,
the following two conditions occur:
• When an interface joins a port channel, the following parameters are removed and they are operationally
replaced with the values on the port channel; however, this change will not be reflected in the running
configuration for the interface:
• QoS
• Bandwidth
• Delay
• STP
• Service policy
• ACLs
• When an interface joins or leaves a port channel, the following parameters remain unaffected:
• Beacon
• Description
• CDP
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 53
Configuring Port Channels
Load Balancing Using Port Channels
Note You have the option to include the source and destination port number for the Layer 4
frame.
You can configure the switch to use one of the following methods (see the following table for more details)
to load balance across the port channel:
• Destination MAC address
• Source MAC address
• Source and destination MAC address
• Destination IP address
• Source IP address
• Source and destination IP address
• Destination TCP/UDP port number
• Source TCP/UDP port number
• Source and destination TCP/UDP port number
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
54 OL-29480-06
Configuring Port Channels
Resilient Hashing
Source and destination Source and destination Source and destination Source and destination
MAC MAC MAC MAC
Source and destination IP Source and destination Source and destination Source and destination
MAC MAC, source and MAC, source and
destination IP destination IP
Source TCP/UDP port Source MAC Source MAC, source IP Source MAC, source IP,
source port
Source and destination Source and destination Source and destination Source and destination
TCP/UDP port MAC MAC, source and MAC, source and
destination IP destination IP, source and
destination port
Use the option that provides the balance criteria with the greatest variety in your configuration. For example,
if the traffic on a port channel is going only to a single MAC address and you use the destination MAC address
as the basis of port-channel load balancing, the port channel always chooses the same link in that port channel;
using source addresses or IP addresses might result in better load balancing.
Resilient Hashing
With the exponential increase in the number of physical links used in data centers, there is also the potential
for an increase in the number of failed physical links. In static hashing systems that are used for load balancing
flows across members of port channels or Equal Cost Multipath (ECMP) groups, each flow is hashed to a
link. If a link fails, all flows are rehashed across the remaining working links. This rehashing of flows to links
results in some packets being delivered out of order even for those flows that were not hashed to the failed
link.
This rehashing also occurs when a link is added to the port channel or Equal Cost Multipath (ECMP) group.
All flows are rehashed across the new number of links, which results in some packets being delivered out of
order. Resilient hashing supports only unicast traffic.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 55
Configuring Port Channels
Hashing for NVGRE Traffic
The resilient hashing system in Cisco Nexus 3100 Series switches maps flows to physical ports. In case a link
fails, the flows assigned to the failed link are redistributed uniformly among the working links. The existing
flows through the working links are not rehashed and their packets are not delivered out of order.
Resilient hashing is supported only by ECMP groups and on port channel interfaces. When a link is added to
the port channel or ECMP group, some of the flows hashed to the existing links are rehashed to the new link,
but not across all existing links.
Resilient hashing does not support IPv6 traffic and IPv4 multicast traffic.
Symmetric Hashing
To be able to effectively monitor traffic on a port channel, it is essential that each interface connected to a
port channel receives both forward and reverse traffic flows. Normally, there is no guarantee that the forward
and reverse traffic flows will use the same physical interface. However, when you enable symmetric hashing
on the port channel, bidirectional traffic is forced to use the same physical interface and each physical interface
in the port channel is effectively mapped to a set of flows.
Cisco NX-OS Release 6.0(2)U2(3) introduces symmetric hashing. When symmetric hashing is enabled, the
parameters used for hashing, such as the source and destination IP address, are normalized before they are
entered into the hashing algorithm. This process ensures that when the parameters are reversed (the source
on the forward traffic becomes the destination on the reverse traffic), the hash output is the same. Therefore,
the same interface is chosen.
Symmetric hashing is supported only on Cisco Nexus 3100 Series switches.
Only the following load-balancing algorithms support symmetric hashing:
• source-dest-ip-only
• source-dest-port-only
• source-dest-ip
• source-dest-port
• source-dest-ip-gre
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
56 OL-29480-06
Configuring Port Channels
Understanding LACP
Understanding LACP
LACP Overview
Note You must enable the LACP feature before you can configure and use LACP functions.
The following figure shows how individual links can be combined into LACP port channels and channel
groups as well as function as individual links.
With LACP, just like with static port channels, you can bundle up to 16 interfaces in a channel group.
Note When you delete the port channel, Cisco NX-OS automatically deletes the associated channel group. All
member interfaces revert to their previous configuration.
You cannot disable LACP while any LACP configurations are present.
LACP ID Parameters
LACP uses the following parameters:
• LACP system priority—Each system that runs LACP has an LACP system priority value. You can
accept the default value of 32768 for this parameter, or you can configure a value between 1 and 65535.
LACP uses the system priority with the MAC address to form the system ID and also uses the system
priority during negotiation with other devices. A higher system priority value means a lower priority.
Note The LACP system ID is the combination of the LACP system priority value and the MAC address.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 57
Configuring Port Channels
Understanding LACP
• LACP port priority—Each port configured to use LACP has an LACP port priority. You can accept the
default value of 32768 for the LACP port priority, or you can configure a value between 1 and 65535.
LACP uses the port priority with the port number to form the port identifier. LACP uses the port priority
to decide which ports should be put in standby mode when there is a limitation that prevents all compatible
ports from aggregating and which ports should be put into active mode. A higher port priority value
means a lower priority for LACP. You can configure the port priority so that specified ports have a lower
priority for LACP and are most likely to be chosen as active links, rather than hot-standby links.
• LACP administrative key—LACP automatically configures an administrative key value equal to the
channel-group number on each port configured to use LACP. The administrative key defines the ability
of a port to aggregate with other ports. A port’s ability to aggregate with other ports is determined by
these factors:
◦Port physical characteristics, such as the data rate, the duplex capability, and the point-to-point or
shared medium state
◦Configuration restrictions that you establish
Channel Modes
Individual interfaces in port channels are configured with channel modes. When you run static port channels,
with no protocol, the channel mode is always set to on. After you enable LACP globally on the device, you
enable LACP for each channel by setting the channel mode for each interface to active or passive. You can
configure either channel mode for individual links in the LACP channel group.
Note You must enable LACP globally before you can configure an interface in either the active or passive
channel mode.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
58 OL-29480-06
Configuring Port Channels
Understanding LACP
Both the passive and active modes allow LACP to negotiate between ports to determine if they can form a
port channel, based on criteria such as the port speed and the trunking state. The passive mode is useful when
you do not know whether the remote system, or partner, supports LACP.
Ports can form an LACP port channel when they are in different LACP modes as long as the modes are
compatible as in the following examples:
• A port in active mode can form a port channel successfully with another port that is in active mode.
• A port in active mode can form a port channel with another port in passive mode.
• A port in passive mode cannot form a port channel with another port that is also in passive mode because
neither port will initiate negotiation.
• A port in on mode is not running LACP.
Table 5: Port Channels with LACP Enabled and Static Port Channels
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 59
Configuring Port Channels
Configuring Port Channels
Note The MinLinks feature works only with LACP port channels. The device allows you to configure this
feature in non-LACP port channels, but the feature is not operational.
Note If you want LACP-based port channels, you need to enable LACP.
Procedure
Step 2 switch(config)# interface port-channel Specifies the port-channel interface to configure, and
channel-number enters the interface configuration mode. The range is
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
60 OL-29480-06
Configuring Port Channels
Adding a Port to a Port Channel
Step 3 switch(config)# no interface Removes the port channel and deletes the associated
port-channel channel-number channel group.
Note If you want LACP-based port channels, you need to enable LACP.
Procedure
Step 2 switch(config)# interface type Specifies the interface that you want to add to a channel
slot/port group and enters the interface configuration mode.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 61
Configuring Port Channels
Configuring Load Balancing Using Port Channels
This example shows how to add an Ethernet interface 1/4 to channel group 1:
switch# configure terminal
switch (config)# interface ethernet 1/4
switch(config-if)# switchport mode trunk
switch(config-if)# channel-group 1
Note If you want LACP-based port channels, you need to enable LACP.
Procedure
Step 2 switch(config)# port-channel Specifies the load-balancing algorithm and hash for the
load-balance ethernet device. The range depends on the device. The default is
{[destination-ip | destination-ip-gre source-dest-mac.
| destination-mac | destination-port | Note The optional destination-ip-gre,
source-dest-ip | source-dest-ip-gre | source-dest-ip-gre and source-ip-gre keywords
source-dest-mac | source-dest-port | are used to include the NVGRE key in the hash
source-ip | source-ip-gre | source-mac computation. Inclusion of the NVGRE key is not
| source-port] symmetric | crc-poly} enabled by default in the case of port channels. You
must configure it explicitly by using these optional
keywords.
The optional symmetric keyword is used to enable
or disable symmetric hashing. Symmetric hashing
forces bi-directional traffic to use the same physical
interface. Only the following load-balancing
algorithms support symmetric hashing:
• source-dest-ip-only
• source-dest-port-only
• source-dest-ip
• source-dest-port
• source-dest-ip-gre
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
62 OL-29480-06
Configuring Port Channels
Configuring Hardware Hashing for Multicast Traffic
This example shows how to configure source IP load balancing for port channels:
switch# configure terminal
switch (config)# port-channel load-balance ethernet source-ip
This example shows how to configure symmetric hashing for port channels:
switch# configure terminal
switch (config)# port-channel load-balance ethernet source-dest-ip-only symmetric
Procedure
Step 2 switch(config)# interface port-channel Selects the port channel and enters the interface
channel-number configuration mode.
Step 3 switch(config-if)# [no] hardware multicast Configures hardware hashing for the specified
hw-hash port channel.
This example shows how to remove hardware hashing from a port channel:
switch# configure terminal
switch (config)# interface port-channel 21
switch(config-if)# no hardware multicast hw-hash
Enabling LACP
LACP is disabled by default; you must enable LACP before you begin LACP configuration. You cannot
disable LACP while any LACP configuration is present.
LACP learns the capabilities of LAN port groups dynamically and informs the other LAN ports. Once LACP
identifies correctly matched Ethernet links, it facilitates grouping the links into an port channel. The port
channel is then added to the spanning tree as a single bridge port.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 63
Configuring Port Channels
Configuring the Channel Mode for a Port
Procedure
Procedure
Step 2 switch(config)# interface type Specifies the interface to configure, and enters the interface
slot/port configuration mode.
Step 3 switch(config-if)# Specifies the port mode for the link in a port channel. After LACP
channel-group is enabled, you configure each link or the entire channel as active
channel-number [force] or passive.
[mode {on | active | passive}]
force—Specifies that the LAN port be forcefully added to the
channel group.
mode—Specifies the port channel mode of the interface.
active—Specifies that when you enable LACP, this command
enables LACP on the specified interface. The interface is in an
active negotiating state in which the port initiates negotiations with
other ports by sending LACP packets.
on—(Default mode) Specifies that all port channels that are not
running LACP remain in this mode.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
64 OL-29480-06
Configuring Port Channels
Configuring LACP Port Channel MinLinks
Step 4 switch(config-if)# no Returns the port mode to on for the specified interface.
channel-group number mode
This example shows how to set the LACP-enabled interface to active port-channel mode for Ethernet interface
1/4 in channel group 5:
switch# configure terminal
switch (config)# interface ethernet 1/4
switch(config-if)# channel-group 5 mode active
This example shows how to forcefully add an interface to the channel group 5:
switch(config)# interface ethernet 1/1
switch(config-if)# channel-group 5 force
switch(config-if)#
Important We recommend that you configure the LACP MinLink feature on both ends of your LACP port channel,
that is, on both the switches. Configuring the lacp min-links command on only one end of the port channel
might result in link flapping.
Procedure
Step 2 switch(config)# interface Specifies the interface to configure and enters interface
port-channel number configuration mode.
Step 3 switch(config-if)# [no] lacp min-links Specifies the port channel interface to configure the
number number of minimum links and enters the interface
configuration mode.
The default value for number is 1. The range is from 1
to 16.
Use the no form of this command to disable this feature.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 65
Configuring Port Channels
Configuring the LACP Fast Timer Rate
This example shows how to configure the minimum number of port channel interfaces on module 3:
switch# configure terminal
switch(config) # interface port-channel 3
switch(config-if) # lacp min-links 3
switch(config-if) #
Procedure
Step 2 switch(config)# interface type Specifies the interface to configure and enters the
slot/port interface configuration mode.
Step 3 switch(config-if)# lacp rate fast Configures the fast rate (one second) at which LACP
control packets are sent to an LACP-supported
interface.
This example shows how to configure the LACP fast rate on Ethernet interface 1/4:
This example shows how to restore the LACP default rate (30 seconds) on Ethernet interface 1/4.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
66 OL-29480-06
Configuring Port Channels
Configuring the LACP System Priority and System ID
Procedure
Step 2 switch(config)# lacp system-priority Configures the system priority for use with LACP.
priority Valid values are 1 through 65535, and higher numbers
have lower priority. The default value is 32768.
This example shows how to set the LACP system priority to 2500:
switch# configure terminal
switch(config)# lacp system-priority 2500
Procedure
Step 2 switch(config)# interface type Specifies the interface to configure, and enters the
slot/port interface configuration mode.
Step 3 switch(config-if)# lacp port-priority Configures the port priority for use with LACP. Valid
priority values are 1 through 65535, and higher numbers have
lower priority. The default value is 32768.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 67
Configuring Port Channels
Verifying Port Channel Configuration
This example shows how to set the LACP port priority for Ethernet interface 1/4 to 40000:
switch# configure terminal
switch (config)# interface ethernet 1/4
switch(config-if)# lacp port priority 40000
Command Purpose
show interface port channelchennal-number Displays the status of a port channel interface.
show port-channel database [interface Displays the aggregation state for one or more
port-channel channel-number] port-channel interfaces.
show port-channel summary Displays a summary for the port channel interfaces.
show port-channel traffic Displays the traffic statistics for port channels.
show port-channel usage Displays the range of used and unused channel
numbers.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
68 OL-29480-06
Configuring Port Channels
Verifying the Load-Balancing Outgoing Port ID
Procedure
This example shows how to trigger a port channel membership consistency check and display its results:
switch# show consistency-checker membership port-channels
Checks: Trunk group and trunk membership table.
Consistency Check: PASSED
No Inconsistencies found for port-channel1111:
Module:1, Unit:0
['Ethernet1/4', 'Ethernet1/5', 'Ethernet1/6']
No Inconsistencies found for port-channel2211:
Module:1, Unit:0
['Ethernet1/7', 'Ethernet1/8', 'Ethernet1/9', 'Ethernet1/10']
No Inconsistencies found for port-channel3311:
Module:1, Unit:0
['Ethernet1/11', 'Ethernet1/12', 'Ethernet1/13', 'Ethernet1/14']
No Inconsistencies found for port-channel4095:
Module:1, Unit:0
['Ethernet1/33', 'Ethernet1/34', 'Ethernet1/35', 'Ethernet1/36', 'Ethernet1
/37', 'Ethernet1/38', 'Ethernet1/39', 'Ethernet1/40', 'Ethernet1/41', 'Ethernet1
/42', 'Ethernet1/43', 'Ethernet1/44', 'Ethernet1/45', 'Ethernet1/46', 'Ethernet1
/47', 'Ethernet1/48', 'Ethernet1/29', 'Ethernet1/30', 'Ethernet1/31', 'Ethernet1
/32']
Note Certain traffic flows are not subject to hashing such as when there is a single port in a port-channel.
The show port-channel load-balance command supports only unicast traffic hashing. Multicast traffic hashing
is not supported.
To display the load-balancing outgoing port ID, perform one of the tasks:
Command Purpose
switch# show port-channel load-balance Displays the outgoing port ID.
forwarding-path interface port-channel
port-channel-id vlan vlan-id dst-ip src-ip dst-mac
src-mac l4-src-port port-id l4-dst-port port-id
ether-type ether-type ip-proto ip-proto
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 69
Configuring Port Channels
Feature History for Port Channels
Example
This example shows how to display the load balancing outgoing port ID:
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
70 OL-29480-06
CHAPTER 5
Configuring IP Tunnels
This chapter contains the following sections:
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 71
Configuring IP Tunnels
GRE Tunnels
An IP tunnel takes a passenger protocol, such as IPv4, and encapsulates that protocol within a carrier protocol,
such as GRE. The device then transmits this carrier protocol over a transport protocol, such as IPv4.
You configure a tunnel interface with matching characteristics on each end of the tunnel.
You must enable the tunnel feature before you can configure it.
GRE Tunnels
You can use GRE as the carrier protocol for a variety of passenger protocols. The selection of tunnel interfaces
can also be based on the PBR policy.
The figure shows the IP tunnel components for a GRE tunnel. The original passenger protocol packet becomes
the GRE payload and the device adds a GRE header to the packet. The device then adds the transport protocol
header to the packet and transmits it.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
72 OL-29480-06
Configuring IP Tunnels
Licensing Requirements for IP Tunnels
• Each tunnel will consume one Equal Cost Multipath (ECMP) adjacency.
• The Cisco Nexus device does not support the following features:
• Path maximum transmission unit (MTU) discovery
• Tunnel interface statistics
• Access control lists (ACLs)
• Unicast reverse path forwarding (URPF)
• Multicast traffic and associated multicast protocols such as Internet Group Management Protocol
(IGMP) and Protocol Independent Multicast (PIM)
• Cisco NX-OS software does not support the Web Cache Control Protocol (WCCP) on tunnel interfaces.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 73
Configuring IP Tunnels
Default Settings for IP Tunneling
Parameters Default
Tunnel feature Disabled
Configuring IP Tunnels
Enabling Tunneling
Before You Begin
You must enable the tunneling feature before you can configure any IP tunnels.
Procedure
Step 2 switch(config)# feature tunnel Enables the tunnel feature on the switch.
Step 4 switch(config)# show feature Displays the tunnel feature on the switch.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
74 OL-29480-06
Configuring IP Tunnels
Creating a Tunnel Interface
switch(config)# exit
switch(config)# copy running-config startup-config
Procedure
Step 4 switch(config)# tunnel source {ip Configures the source address for this IP tunnel.
address | interface-name}
Step 5 switch(config)# tunnel destination Configures the destination address for this IP tunnel.
{ip address | host-name}
Step 6 switch(config)# tunnel use-vrf (Optional)
vrf-name Uses the configured VRF instance to look up the tunnel
IP destination address.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 75
Configuring IP Tunnels
Configuring a Tunnel Interface Based on Policy Based Routing
Procedure
Step 5 switch(config-route-map)# match ip Matches an IPv4 address against one or more IP access
address access-list-name name control lists (ACLs). This command is used for
policy-based routing and is ignored by route filtering or
redistribution.
Step 6 switch(config-route-map)# set ip Sets the IPv4 next-hop address for policy-based routing.
next-hop address Tunnel IP addresses can be specified as next-hop
addresses to select tunnel interfaces. This command uses
the first valid next-hop address if multiple addresses are
configured. Use the load-share option to select ECMP
across next-hop entries.
This example shows how to configure a tunnel interface that is based on PBR:
switch# configure terminal
switch(config)# interface tunnel 1
switch(config)# ip address 1.1.1.1/24
switch(config)# route-map pbr1
switch(config-route-map)# match ip address access-list-name pbr1
switch(config-route-map)# set ip next-hop 1.1.1.1
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
76 OL-29480-06
Configuring IP Tunnels
Configuring a Tunnel Interface
Procedure
Step 5 switch(config-if)# mtu value Sets the maximum transmission unit (MTU) of IP packets
sent on an interface.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 77
Configuring IP Tunnels
Assigning VRF Membership to a Tunnel Interface
Procedure
Step 4 switch(config)# ip address ip-prefix/length Configures an IP address for this interface. You
must do this step after you assign this interface to
a VRF.
Command Purpose
show interface tunnel number Displays the configuration for the tunnel interface
(MTU, protocol, transport, and VRF). Displays input
and output packets, bytes, and packet rates.
show interface tunnel number brief Displays the operational status, IP address,
encapsulation type, and MTU of the tunnel interface.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
78 OL-29480-06
Configuring IP Tunnels
Configuration Examples for IP Tunneling
Command Purpose
show interface tunnel number description Displays the configured description of the tunnel
interface.
show interface tunnel number status Displays the operational status of the tunnel interface.
show interface tunnel number status err-disabled Displays the error disabled status of the tunnel
interface.
router B:
feature tunnel
interface tunnel 0
ip address 209.165.20.1/8
tunnel source ethernet 1/3
tunnel destination 192.0.2.55
tunnel mode gre ip
interface ethernet 1/3
ip address 192.0.2.2/8
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 79
Configuring IP Tunnels
Feature History for Configuring IP Tunnels
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
80 OL-29480-06
CHAPTER 6
Configuring VXLANs
This chapter contains the following sections:
• Overview, page 81
• Configuring VXLAN Traffic Forwarding, page 87
• Verifying the VXLAN Configuration, page 93
• Displaying MAC Addresses, page 95
• Clearing MAC Addresses, page 99
Overview
VXLAN Overview
The Cisco Nexus 3100 Series switches are designed for a hardware-based Virtual Extensible LAN (VXLAN)
function. These switches can extend Layer 2 connectivity across the Layer 3 boundary and integrate between
VXLAN and non-VXLAN infrastructures. Virtualized and multitenant data center designs can be shared over
a common physical infrastructure.
VXLANs enable you to extend Layer 2 networks across the Layer 3 infrastructure by using MAC-in-UDP
encapsulation and tunneling. In addition, you can use a VXLAN to build a multitenant data center by decoupling
tenant Layer 2 segments from the shared transport network.
When deployed as a VXLAN gateway, the Cisco Nexus 3100 Series switches can connect VXLAN and classic
VLAN segments to create a common forwarding domain so that tenant devices can reside in both environments.
A VXLAN has the following benefits:
• Flexible placement of multitenant segments throughout the data center.
It extends Layer 2 segments over the underlying shared network infrastructure so that tenant workloads
can be placed across physical pods in the data center.
• Higher scalability to address more Layer 2 segments.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 81
Configuring VXLANs
VXLAN Encapsulation and Packet Format
A VXLAN uses a 24-bit segment ID called the VXLAN network identifier (VNID). The VNID allows
a maximum of 16 million VXLAN segments to coexist in the same administrative domain. (In comparison,
traditional VLANs use a 12-bit segment ID that can support a maximum of 4096 VLANs.)
• Utilization of available network paths in the underlying infrastructure.
VXLAN packets are transferred through the underlying network based on its Layer 3 header. It uses
equal-cost multipath (ECMP) routing and link aggregation protocols to use all available paths.
A VXLAN uses an 8-byte VXLAN header that consists of a 24-bit VNID and a few reserved bits. The VXLAN
header and the original Ethernet frame are in the UDP payload. The 24-bit VNID identifies the Layer 2
segments and maintains Layer 2 isolation between the segments. A VXLAN can support 16 million LAN
segments.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
82 OL-29480-06
Configuring VXLANs
VXLAN Packet Forwarding Flow
A VTEP device is identified in the IP transport network by using a unique IP address, which is a loopback
interface IP address. The VTEP device uses this IP address to encapsulate Ethernet frames and transmits the
encapsulated packets to the transport network through the IP interface. A VTEP device learns the remote
VTEP IP addresses and the remote MAC address-to-VTEP IP mapping for the VXLAN traffic that it receives.
The VXLAN segments are independent of the underlying network topology; conversely, the underlying IP
network between VTEPs is independent of the VXLAN overlay. The IP network routes the encapsulated
packets based on the outer IP address header, which has the initiating VTEP as the source IP address and the
terminating VTEP or multicast group IP address as the destination IP address.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 83
Configuring VXLANs
VXLAN Layer 2 Gateway as a Transit Multicast Router
When Layer 2 traffic is received on the access side, a MAC address lookup is performed for the destination
MAC address in the frame. If the lookup is successful, VXLAN forwarding is done based on the information
retrieved as a result of the lookup. The lookup result provides the IP address of the remote VTEP from which
this MAC address is learned. This Layer 2 frame is then UDP/IP encapsulated with the destination IP address
as the remote VTEP IP address and is forwarded out of the appropriate network interface. In the Layer 3 cloud,
this IP packet is forwarded to the remote VTEP through the route to that IP address in the network.
For unicast-learned traffic, you must ensure the following:
• The route to the remote peer is known through a routing protocol or through static routes in the network.
• Adjacency is resolved.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
84 OL-29480-06
Configuring VXLANs
Guidelines and Limitations for VXLANs
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 85
Configuring VXLANs
vPC Guidelines and Limitations for VXLAN Deployment
• VXLAN multicast traffic should always use the RPT shared tree.
• An RP for the multicast group on the VTEP is a supported configuration. However, you must configure
the RP for the multicast group at the spine layer/upstream device. Because all multicast traffic traverses
the RP, it is more efficient to have this traffic directed to a spine layer/upstream device.
• For multicast, the vPC node that receives the (S,G) join from the RP becomes the designated forwarder
(DF). On the DF node, both encapsulation and decapsulation routes are installed for multicast. The other
vPC node does not initiate or terminate multicast traffic.
• Multicast traffic on a vPC that is hashed toward the non-DF switch traverses the multichassis EtherChannel
trunk (MCT) and is encapsulated on the DF node.
• When MCT is shut, the loopback interface on the secondary vPC is brought down and the status is
Admin Shut. The route to the loopback is withdrawn on the upstream and the upstream can divert all
traffic to the primary vPC.
Note Orphans that are connected to the secondary vPC experience a loss of traffic when the
MCT is shut down. This situation is similar to Layer 2 orphans in a secondary vPC of
a traditional vPC setup.
• In a VXLAN vPC, consistency checks are performed to ensure that NVE configurations and VN-Segment
configurations are identical across vPC peers.
• The router ID for unicast routing protocols must be different from the loopback IP address used for
VTEP.
• When MCT is no-shut, the NVE loopback interface is brought up again and the route is advertised
upstream to attract traffic.
• Configure an SVI between vPC peers and advertise routes between the vPC peers by using a routing
protocol with higher routing metric. This action ensures that the IP connectivity of the vPC node does
not go down if one vPC node fails.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
86 OL-29480-06
Configuring VXLANs
Configuring VXLAN Traffic Forwarding
Note On a Cisco Nexus 3100 Series switch that functions as a VXLAN Layer 2 gateway,
note that traffic that is received on the access side cannot trigger an ARP on the network
side. ARP for network side interfaces should be resolved either by using a routing
protocol such as BGP, or by using static ARP. This requirement is applicable for ingress
replication cases alone, not for multicast replication cases.
Procedure
Step 3 switch(config)# ip pim spt-threshold Creates the IPv4 Protocol Independent Multicast (PIM)
infinity group-list route-map-name (*, G) state only. Allows selection of the RPT only
and not the SPT.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 87
Configuring VXLANs
Configuring a Rendezvous Point
version 6.0(2)U3(1)
feature pim
Procedure
Step 2 switch(config)# ip pim rp-address Configures a PIM RP address for a multicast group
rp-address [group-list ip-prefix | range. You can specify a route-map policy name that
route-map policy-name] lists the group prefixes to use with the match ip
multicast command. The default mode is ASM. The
default group range is 224.0.0.0 through
239.255.255.255.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
88 OL-29480-06
Configuring VXLANs
Enabling a VXLAN
Enabling a VXLAN
Enabling VXLANs involves the following:
• Enabling the VXLAN feature
• Enabling VLAN to VN-Segment mapping
Procedure
This example shows how to enable a VXLAN and configure VLAN to VN-Segment mapping:
switch# configure terminal
switch(config)# feature nv overlay
switch(config)# feature vn-segment-vlan-based
switch(config)# copy running-config startup-config
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 89
Configuring VXLANs
Configuring a Routing Protocol for NVE Unicast Addresses
Note Open shortest path first (OSPF) is used as the routing protocol in the examples.
Procedure
Step 2 switch(config)# interface loopback Creates a dedicated loopback interface for the NVE
instance interface. The instance range is from 0 to 1023.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
90 OL-29480-06
Configuring VXLANs
Creating a VXLAN Destination UDP Port
Step 6 switch(config-if)# ip pim sparse-mode Enables PIM sparse mode on this interface. The
default is disabled.
Enable the PIM sparse mode in case of multicast
replication.
This example shows how to configure a routing protocol for NVE unicast addresses:
switch# configure terminal
switch(config)# interface loopback 10
switch(config-if)# ip address 222.2.2.1/32
switch(config-if)# ip ospf network point-to-point
switch(config-if)# ip router ospf 1 area 0.0.0.0
switch(config-if)# ip pim sparse-mode
Note If the configuration must be changed while the NVE interface is enabled, ensure that you shut down the
NVE interface, make the UDP configuration change, and then reenable the NVE interface.
Ensure that the UDP port configuration is done network-wide before the NVE interface is enabled on the
network.
The VXLAN UDP source port is determined based on the VNID and source and destination IP addresses.
Procedure
Step 2 switch(config)# vxlan udp port Specifies the destination UDP port number for VXLAN
number encapsulated packets. The default destination UDP
port number is 4789.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 91
Configuring VXLANs
Creating and Configuring an NVE Interface
Procedure
Step 2 switch(config)# interface nve Creates a VXLAN overlay interface that initiates and
instance terminates VXLAN tunnels.
Note Only one NVE interface is allowed on the
switch.
Step 3 switch(config-if-nve)# Specifies a source interface.
source-interface loopback The source interface must be a loopback interface that is
instance configured on the switch with a valid /32 IP address. This
/32 IP address must be known by the transit routers in the
transport network and the remote VTEPs.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
92 OL-29480-06
Configuring VXLANs
Verifying the VXLAN Configuration
Procedure
This example shows how to map a VNI to an NVE interface and assign it to a multicast group:
switch(config-if-nve)# member vni 5000 mcast-group 225.1.1.1
Procedure
This example shows how to map a VNI to an NVE interface and create a unicast tunnel:
switch(config-if-nve)# member vni 5001
switch(config-if-nve-vni)# ingress-replication 111.1.1.1
Command Purpose
show nve interface nve id Displays the configuration of an NVE interface.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 93
Configuring VXLANs
Verifying the VXLAN Configuration
Command Purpose
show nve vni Displays the VNI that is mapped to an NVE interface.
show interface nve id counters Displays all the counters for an NVE interface.
This example shows how to display the VNI that is mapped to an NVE interface for multicast replication:
switch# show nve vni
Interface VNI Multicast-group VNI State
---------------- -------- --------------- ---------
nve1 5000 225.1.1.1 Up
This example shows how to display the VNI that is mapped to an NVE interface for ingress replication:
switch# show nve vni
Interface VNI Multicast-group VNI State
---------------- -------- --------------- ---------
nve1 5000 0.0.0.0 Up
--------------------------------------------------------------------------------
Port InOctets InUcastPkts
--------------------------------------------------------------------------------
nve1 0 0
--------------------------------------------------------------------------------
Port InMcastPkts InBcastPkts
--------------------------------------------------------------------------------
nve1 0 0
--------------------------------------------------------------------------------
Port OutOctets OutUcastPkts
--------------------------------------------------------------------------------
nve1 0 0
--------------------------------------------------------------------------------
Port OutMcastPkts OutBcastPkts
--------------------------------------------------------------------------------
nve1 0 0
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
94 OL-29480-06
Configuring VXLANs
Displaying MAC Addresses
This example shows how to display the VXLAN UDP port configured:
switch# show nve vxlan-params
VxLAN Dest. UDP Port: 4789
Command Purpose
show mac address-table Displays both VLAN and VXLAN MAC addresses.
show mac address-table vlan vlan-id Displays all the VxLAN MAC addresses that are
learned on the specified VLAN. For VN-Segment
mapped VLANs, it displays both local and remote
MAC addresses.
show mac address-table local Displays only locally learned MAC addresses on all
VLANs that are mapped to VN-Segments.
show mac address-table local vlan vlan-id Displays only locally learned MAC addresses on the
specified VLAN, which is mapped to a VN-Segment.
show mac address-table interface nve nve-id Displays all remote MAC addresses learned on NVE.
show mac address-table interface nve nve-id vni Displays all remote MAC addresses learned on the
vni-id VNI.
show mac address-table interface ethernet slot/port Displays all MAC addresses learned on the VLAN
vlan vlan-id on this interface.
show mac address-table interface nve nve-id peer Displays all MAC addresses learned on NVE from
ip-address the specified peer.
show mac address-table interface nve nve-id peer
vrf vrf-name ip-address
show mac address-table interface nve nve-id peer Displays all MAC addresses learned on NVE from
ip-address vni vni-id the specified peer on the specified VNI.
show mac address-table interface nve nve-id peer
vrf vrf-name ip-address vni vni-id
show mac address-table count local Displays the number of locally learned MAC address
table entries.
show mac address-table count local vlan vlan-id Displays the number of locally learned MAC address
table entries on the specified VLAN, which is mapped
to a VN-segment.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 95
Configuring VXLANs
Displaying MAC Addresses
Command Purpose
show mac address-table count interface nve nve-id Displays the number of remote MAC address table
entries learned on NVE.
show mac address-table count interface nve nve-id Displays the number of remote MAC address table
vni vni-id entries learned on the VNI.
show mac address-table count interface nve nve-id Displays the number of MAC address table entries
peer ip-address learned on NVE from the specified peer.
show mac address-table count interface nve nve-id
peer ip-address vrf vrf-name
show mac address-table count interface nve nve-id Displays the number of MAC address table entries
peer ip-address vni vni-id learned on NVE from the specified peer on the
specified VNI.
show mac address-table count interface nve nve-id
peer ip-address vrf vrf-name vni vni-id
This example shows how to display both VLAN and VXLAN MAC addresses:
switch# show mac address-table
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since first seen,+ - primary entry using vPC Peer-Link
VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
* 109 0000.0410.0902 dynamic 470 F F Po2233
* 109 0000.0410.0912 dynamic 470 F F Po2233
* 109 0000.0410.0912 dynamic 470 F F nve1(1.1.1.200)
* 108 0000.0410.0802 dynamic 470 F F Po2233
* 108 0000.0410.0812 dynamic 470 F F Po2233
* 107 0000.0410.0702 dynamic 470 F F Po2233
* 107 0000.0410.0712 dynamic 470 F F Po2233
* 107 0000.0410.0712 dynamic 470 F F nve1(1.1.1.200)
* 106 0000.0410.0602 dynamic 470 F F Po2233
* 106 0000.0410.0612 dynamic 470 F F Po2233
* 105 0000.0410.0502 dynamic 470 F F Po2233
* 105 0000.0410.0512 dynamic 470 F F Po2233
* 105 0000.0410.0512 dynamic 470 F F nve1(1.1.1.200)
* 104 0000.0410.0402 dynamic 470 F F Po2233
* 104 0000.0410.0412 dynamic 470 F F Po2233
This example shows how to display all the VXLAN MAC addresses learned on the specified VLAN:
switch# show mac address-table vlan 107
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since first seen,+ - primary entry using vPC Peer-Link
VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
* 107 0000.0410.0702 dynamic 470 F F Po2233
* 107 0000.0410.0712 dynamic 470 F F Po2233
* 107 0000.0410.0712 dynamic 470 F F nve1(1.1.1.200)
This example shows how to display only locally learned MAC addresses on all VLANs that are mapped to
VN-Segments:
switch# show mac address-table local
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since first seen,+ - primary entry using vPC Peer-Link
VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
96 OL-29480-06
Configuring VXLANs
Displaying MAC Addresses
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 97
Configuring VXLANs
Displaying MAC Addresses
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
98 OL-29480-06
Configuring VXLANs
Clearing MAC Addresses
Command Purpose
clear mac address-table dynamic Clears all MAC address entries in the MAC address
table.
clear mac address-table dynamic vlan vlan-id Clears all VLAN and VXLAN MAC address entries
from the MAC address table.
clear mac address-table dynamic local Clears all locally learned MAC address entries on all
VLANs mapped to VN -Segments.
clear mac address-table dynamic local vlan vlan-id Clears all locally learned MAC address entries on the
specified VLAN.
clear mac address-table dynamic interface nve Clears all overlay learned MAC addresses.
nve-id
clear mac address-table dynamic interface nve Clears all network-learnt MAC addresses on the
nve-id vni vni-id specified VNI.
clear mac address-table dynamic interface Clears all MAC addresses on the specified interface
Ethernet slot/port vlan vlan-id and VLAN.
clear mac address-table dynamic interface nve Clears all MAC addresses on the NVE interface for
nve-id peer ip-address the specified peer.
clear mac address-table dynamic interface nve
nve-id peer ip-address vrf vrf-name
clear mac address-table dynamic interface nve Clears all MAC addresses on the NVE interface from
nve-id peer ip-address vni vni-id the specified peer on the specified VNI.
clear mac address-table dynamic interface nve
nve-id peer ip-address vrf vrf-name vni vni-id
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 99
Configuring VXLANs
Clearing MAC Addresses
This example shows how to clear all MAC address entries in the MAC address table:
switch# clear mac address-table dynamic
switch#
This example shows how to clear all VLAN and VXLAN MAC address entries from the MAC address table:
switch# clear mac address-table dynamic vlan 3100
switch#
This example shows how to clear all locally learned MAC address entries on all VLANs mapped to VN
-Segments:
switch# clear mac address-table dynamic local
switch#
This example shows how to clear all locally learned MAC address entries on the specified VLAN:
switch# clear mac address-table dynamic local vlan 3100
switch#
This example shows how to clear all overlay learned MAC addresses:
switch# clear mac address-table dynamic interface nve 1
switch#
This example shows how to clear all network-learnt MAC addresses on the specified VNI:
switch# clear mac address-table dynamic interface nve 1 vni 5000
switch#
This example shows how to clear all MAC addresses on the specified interface and VLAN:
switch# clear mac address-table dynamic interface Ethernet 1/1 vlan 3100
switch#
This example shows how to clear all MAC addresses on the NVE interface for the specified peer:
switch# clear mac address-table dynamic interface nve 1 peer 222.1.1.1 vrf default
switch#
This example shows how to clear all MAC addresses on the NVE interface from the specified peer on the
specified VNI:
switch# clear mac address-table dynamic interface nve 1 peer 222.1.1.1 vrf default vni 5000
switch#
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
100 OL-29480-06
CHAPTER 7
Configuring Virtual Port Channels
This chapter contains the following sections:
When you configure the EtherChannels in a vPC—including the vPC peer link channel—each switch can
have up to 16 active links in a single EtherChannel.
Note You must enable the vPC feature before you can configure or run the vPC functionality.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 101
Configuring Virtual Port Channels
Terminology
To enable the vPC functionality, you must create a peer-keepalive link and a peer-link under the vPC domain
for the two vPC peer switches to provide the vPC functionality.
To create a vPC peer link you configure an EtherChannel on one Cisco Nexus device by using two or more
Ethernet ports. On the other switch, you configure another EtherChannel again using two or more Ethernet
ports. Connecting these two EtherChannels together creates a vPC peer link.
Note We recommend that you configure the vPC peer-link EtherChannels as trunks.
The vPC domain includes both vPC peer devices, the vPC peer-keepalive link, the vPC peer link, and all of
the EtherChannels in the vPC domain connected to the downstream device. You can have only one vPC
domain ID on each vPC peer device.
Note Always attach all vPC devices using EtherChannels to both vPC peer devices.
Terminology
vPC Terminology
The terminology used in vPCs is as follows:
• vPC—combined EtherChannel between the vPC peer devices and the downstream device.
• vPC peer device—One of a pair of devices that are connected with the special EtherChannel known as
the vPC peer link.
• vPC peer link—link used to synchronize states between the vPC peer devices.
• vPC member port—Interfaces that belong to the vPCs.
• vPC domain—domain that includes both vPC peer devices, the vPC peer-keepalive link, and all of the
port channels in the vPC connected to the downstream devices. It is also associated to the configuration
mode that you must use to assign vPC global parameters. The vPC domain ID must be the same on both
switches.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
102 OL-29480-06
Configuring Virtual Port Channels
vPC Domain
• vPC peer-keepalive link—The peer-keepalive link monitors the vitality of a vPC peer Cisco Nexus
device. The peer-keepalive link sends configurable, periodic keepalive messages between vPC peer
devices.
No data or synchronization traffic moves over the vPC peer-keepalive link; the only traffic on this link
is a message that indicates that the originating switch is operating and running vPCs.
vPC Domain
To create a vPC domain, you must first create a vPC domain ID on each vPC peer switch using a number
from 1 to 1000. This ID must be the same on a set of vPC peer devices.
You can configure the EtherChannels and vPC peer links by using LACP or no protocol. When possible, we
recommend that you use LACP on the peer-link, because LACP provides configuration checks against a
configuration mismatch on the EtherChannel.
The vPC peer switches use the vPC domain ID that you configure to automatically assign a unique vPC system
MAC address. Each vPC domain has a unique MAC address that is used as a unique identifier for the specific
vPC-related operations, although the switches use the vPC system MAC addresses only for link-scope
operations, such as LACP. We recommend that you create each vPC domain within the contiguous network
with a unique domain ID. You can also configure a specific MAC address for the vPC domain, rather than
having the Cisco NX-OS software assign the address.
The vPC peer switches use the vPC domain ID that you configure to automatically assign a unique vPC system
MAC address. The switches use the vPC system MAC addresses only for link-scope operations, such as LACP
or BPDUs. You can also configure a specific MAC address for the vPC domain.
We recommend that you configure the same VPC domain ID on both peers and, the domain ID should be
unique in the network. For example, if there are two different VPCs (one in access and one in aggregation)
then each vPC should have a unique domain ID.
After you create a vPC domain, the Cisco NX-OS software automatically creates a system priority for the
vPC domain. You can also manually configure a specific system priority for the vPC domain.
Note If you manually configure the system priority, you must ensure that you assign the same priority value on
both vPC peer switches. If the vPC peer switches have different system priority values, the vPC will not
come up.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 103
Configuring Virtual Port Channels
Compatibility Parameters for vPC Peer Links
The vPC peer-keepalive can be carried either in the management or default VRF on the Cisco Nexus device.
When you configure the switches to use the management VRF, the source and destination for the keepalive
messages are the mgmt 0 interface IP addresses. When you configure the switches to use the default VRF, an
SVI must be created to act as the source and destination addresses for the vPC peer-keepalive messages.
Ensure that both the source and destination IP addresses used for the peer-keepalive messages are unique in
your network and these IP addresses are reachable from the VRF associated with the vPC peer-keepalive link.
Note We recommend that you configure the vPC peer-keepalive link on the Cisco Nexus device to run in the
management VRF using the mgmt 0 interfaces. If you configure the default VRF, ensure that the vPC
peer link is not used to carry the vPC peer-keepalive messages.
Note You must ensure that all interfaces in the vPC have the identical operational and configuration parameters
listed in this section.
Enter the show vpc consistency-parameters command to display the configured values on all interfaces
in the vPC. The displayed configurations are only those configurations that would limit the vPC peer link
and vPC from coming up.
The switch automatically checks for compatibility of these parameters on the vPC interfaces. The per-interface
parameters must be consistent per interface, and the global parameters must be consistent globally.
• Port-channel mode: on, off, or active
• Link speed per channel
• Duplex mode per channel
• Trunk mode per channel:
◦Native VLAN
◦VLANs allowed on trunk
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
104 OL-29480-06
Configuring Virtual Port Channels
Compatibility Parameters for vPC Peer Links
If any of these parameters are not enabled or defined on either switch, the vPC consistency check ignores
those parameters.
Note To ensure that none of the vPC interfaces are in the suspend mode, enter the show vpc brief and show
vpc consistency-parameters commands and check the syslog messages.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 105
Configuring Virtual Port Channels
Per-VLAN Consistency Check
◦BPDU Guard
◦Cost
◦Link type
◦Priority
◦VLANs (Rapid PVST+)
To ensure that all the configuration parameters are compatible, we recommend that you display the
configurations for each vPC peer switch once you configure the vPC.
vPC Auto-Recovery
When both vPC peer switches reload and only one switch reboots, auto-recovery allows that switch to assume
the role of the primary switch and the vPC links will be allowed to come up after a predetermined period of
time. The reload delay period in this scenario can range from 240 to 3600 seconds.
When vPCs are disabled on a secondary vPC switch due to a peer-link failure and then the primary vPC switch
fails or is unable to forward traffic, the secondary switch reenables the vPCs. In this scenario, the vPC waits
for three consecutive keepalive failures to recover the vPC links.
The vPC auto-recovery feature is disabled by default.
Note You must configure the peer-keepalive link before you configure the vPC peer link or the peer link will
not come up.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
106 OL-29480-06
Configuring Virtual Port Channels
vPC Number
Many operational parameters and configuration parameters must be the same in each switch connected by a
vPC peer link. Because each switch is completely independent on the management plane, you must ensure
that the switches are compatible on the critical parameters. vPC peer switches have separate control planes.
After configuring the vPC peer link, you should display the configuration on each vPC peer switch to ensure
that the configurations are compatible.
Note You must ensure that the two switches connected by the vPC peer link have certain identical operational
and configuration parameters.
When you configure the vPC peer link, the vPC peer switches negotiate that one of the connected switches
is the primary switch and the other connected switch is the secondary switch. By default, the Cisco NX-OS
software uses the lowest MAC address to elect the primary switch. The software takes different actions on
each switch—that is, the primary and secondary—only in certain failover conditions. If the primary switch
fails, the secondary switch becomes the operational primary switch when the system recovers, and the previously
primary switch is now the secondary switch.
You can also configure which of the vPC switches is the primary switch. If you want to configure the role
priority again to make one vPC switch the primary switch, configure the role priority on both the primary and
secondary vPC switches with the appropriate values, shut down the EtherChannel that is the vPC peer link
on both switches by entering the shutdown command, and reenable the EtherChannel on both switches by
entering the no shutdown command.
MAC addresses that are learned over vPC links are also synchronized between the peers.
Configuration information flows across the vPC peer links using the Cisco Fabric Services over Ethernet
(CFSoE) protocol. All MAC addresses for those VLANs configured on both switches are synchronized
between vPC peer switches. The software uses CFSoE for this synchronization.
If the vPC peer link fails, the software checks the status of the remote vPC peer switch using the peer-keepalive
link, which is a link between vPC peer switches, to ensure that both switches are up. If the vPC peer switch
is up, the secondary vPC switch disables all vPC ports on its switch. The data then forwards down the remaining
active links of the EtherChannel.
The software learns of a vPC peer switch failure when the keepalive messages are not returned over the
peer-keepalive link.
Use a separate link (vPC peer-keepalive link) to send configurable keepalive messages between the vPC peer
switches. The keepalive messages on the vPC peer-keepalive link determines whether a failure is on the vPC
peer link only or on the vPC peer switch. The keepalive messages are used only when all the links in the peer
link fail.
vPC Number
Once you have created the vPC domain ID and the vPC peer link, you can create EtherChannels to attach the
downstream switch to each vPC peer switch. That is, you create one single EtherChannel on the downstream
switch with half of the ports to the primary vPC peer switch and the other half of the ports to the secondary
peer switch.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 107
Configuring Virtual Port Channels
vPC Interactions with Other Features
On each vPC peer switch, you assign the same vPC number to the EtherChannel that connects to the downstream
switch. You will experience minimal traffic disruption when you are creating vPCs. To simplify the
configuration, you can assign the vPC ID number for each EtherChannel to be the same as the EtherChannel
itself (that is, vPC ID 10 for EtherChannel 10).
Note The vPC number that you assign to the EtherChannel that connects to the downstream switch from the
vPC peer switch must be identical on both vPC peer switches.
Note When you manually configure the system priority, you must ensure that you assign the same priority value
on both vPC peer switches. If the vPC peer switches have different system priority values, vPC does not
come up.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
108 OL-29480-06
Configuring Virtual Port Channels
Guidelines and Limitations for vPCs
The Bridge Protocol Data Units (BPDUs) use the MAC address set for the vPC for the STP bridge ID in the
designated bridge ID field. The vPC primary switch sends these BPDUs on the vPC interfaces.
Note Display the configuration on both sides of the vPC peer link to ensure that the settings are identical. Use
the show spanning-tree command to display information about the vPC.
CFSoE
The Cisco Fabric Services over Ethernet (CFSoE) is a reliable state transport mechanism that you can use to
synchronize the actions of the vPC peer devices. CFSoE carries messages and packets for many features linked
with vPC, such as STP and IGMP. Information is carried in CFS/CFSoE protocol data units (PDUs).
When you enable the vPC feature, the device automatically enables CFSoE, and you do not have to configure
anything. CFSoE distributions for vPCs do not need the capabilities to distribute over IP or the CFS regions.
You do not need to configure anything for the CFSoE feature to work correctly on vPCs.
You can use the show mac address-table command to display the MAC addresses that CFSoE synchronizes
for the vPC peer link.
Note Do not enter the no cfs eth distribute or the no cfs distribute command. CFSoE must be enabled for
vPC functionality. If you do enter either of these commands when vPC is enabled, the system displays an
error message.
When you enter the show cfs application command, the output displays "Physical-eth," which shows the
applications that are using CFSoE.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 109
Configuring Virtual Port Channels
Verifying the vPC Configuration
• You should configure all port channels in the vPC using LACP with the interfaces in active mode.
• You might experience traffic disruption when the first member of a vPC is brought up.
• OSPF over vPC and BFD with OSPF are supported on Cisco Nexus 3000 and 3100 Series switches.
However, BFD with OSPF over vPC peer links is not currently supported.
Command Purpose
switch# show feature Displays whether vPC is enabled or not.
switch# show port-channel capacity Displays how many EtherChannels are configured
and how many are still available on the switch.
switch# show running-config vpc Displays running configuration information for vPCs.
switch# show vpc consistency-parameters Displays the status of those parameters that must be
consistent across all vPC interfaces.
switch# show vpc role Displays the peer status, the role of the local switch,
the vPC system MAC address and system priority,
and the MAC address and priority for the local vPC
switch.
For information about the switch output, see the Command Reference for your Cisco Nexus Series switch.
vPC domain id : 10
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status: success
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
110 OL-29480-06
Configuring Virtual Port Channels
Viewing a Global Type-1 Inconsistency
vPC domain id : 10
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status: failed
Per-vlan consistency status : success
Configuration consistency reason: vPC type-1 configuration incompatible - STP
Mode inconsistent
Type-2 consistency status : success
vPC role : secondary
Number of vPCs configured : 2
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
20 Po20 down* failed Global compat check failed -
30 Po30 down* failed Global compat check failed -
The example shows how to display the inconsistent status ( the VLANs on the primary vPC are not suspended)
on the primary switch:
switch(config)# show vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 10
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status: failed
Per-vlan consistency status : success
Configuration consistency reason: vPC type-1 configuration incompatible - STP Mo
de inconsistent
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 2
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 111
Configuring Virtual Port Channels
Viewing an Interface-Specific Type-1 Inconsistency
vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
20 Po20 up failed Global compat check failed 1-10
30 Po30 up failed Global compat check failed 1-10
vPC domain id : 10
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status: success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : secondary
Number of vPCs configured : 2
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
20 Po20 up success success 1
30 Po30 down* failed Compatibility check failed -
for port mode
This example shows how to display the inconsistent status (the VLANs on the primary vPC are not suspended)
on the primary switch:
switch(config-if)# show vpc brief
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 10
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status: success
Per-vlan consistency status : success
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
112 OL-29480-06
Configuring Virtual Port Channels
Viewing a Per-VLAN Consistency Status
vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
20 Po20 up success success 1
30 Po30 up failed Compatibility check failed 1
for port mode
vPC domain id : 10
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status: success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : secondary
Number of vPCs configured : 2
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
20 Po20 up success success 1-10
30 Po30 up success success 1-10
Entering no spanning-tree vlan 5 command triggers the inconsistency on the primary and secondary VLANs:
switch(config)# no spanning-tree vlan 5
This example shows how to display the per-VLAN consistency status as Failed on the secondary switch:
switch(config)# show vpc brief
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 113
Configuring Virtual Port Channels
Viewing a Per-VLAN Consistency Status
vPC domain id : 10
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status: success
Per-vlan consistency status : failed
Type-2 consistency status : success
vPC role : secondary
Number of vPCs configured : 2
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
20 Po20 up success success 1-4,6-10
30 Po30 up success success 1-4,6-10
This example shows how to display the per-VLAN consistency status as Failed on the primary switch:
switch(config)# show vpc brief
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 10
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status: success
Per-vlan consistency status : failed
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 2
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
20 Po20 up success success 1-4,6-10
30 Po30 up success success 1-4,6-10
This example shows the inconsistency as STP Disabled:
switch(config)# show vpc consistency-parameters vlans
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
114 OL-29480-06
Configuring Virtual Port Channels
vPC Default Settings
VLAN Mapping
STP Loopguard 1 success 0-4095
STP Bridge Assurance 1 success 0-4095
STP Port Type, Edge 1 success 0-4095
BPDUFilter, Edge BPDUGuard
STP MST Simulate PVST 1 success 0-4095
Pass Vlans - 0-4,6-4095
Parameters Default
vPC system priority 32667
Configuring vPCs
Enabling vPCs
You must enable the vPC feature before you can configure and use vPCs.
Procedure
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 115
Configuring Virtual Port Channels
Disabling vPCs
Disabling vPCs
You can disable the vPC feature.
Note When you disable the vPC feature, the Cisco Nexus device clears all the vPC configurations.
Procedure
Procedure
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
116 OL-29480-06
Configuring Virtual Port Channels
Configuring a vPC Keepalive Link and Messages
Note We recommend that you configure a separate VRF instance and put a Layer 3 port from each vPC peer
switch into that VRF instance for the vPC peer-keepalive link. Do not use the peer link itself to send vPC
peer-keepalive messages.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 117
Configuring Virtual Port Channels
Configuring a vPC Keepalive Link and Messages
Procedure
Step 2 switch(config)# vpc domain domain-id Creates a vPC domain on the switch if it does
not already exist, and enters the vpc-domain
configuration mode.
Step 3 switch(config-vpc-domain)# peer-keepalive Configures the IPv4 address for the remote end
destination ipaddress [hold-timeout secs | of the vPC peer-keepalive link.
interval msecs {timeout secs} | precedence Note The system does not form the vPC peer
{prec-value | network | internet | critical | link until you configure a vPC
flash-override | flash | immediate priority | peer-keepalive link.
routine} | tos {tos-value | max-reliability | The management ports and VRF are the defaults.
max-throughput | min-delay |
min-monetary-cost | normal} | tos-byte
tos-byte-value} | source ipaddress | vrf {name
| management vpc-keepalive}]
Step 4 switch(config-vpc-domain)# vpc (Optional)
peer-keepalive destination ipaddress source Configures a separate VRF instance and puts a
ipaddress Layer 3 port from each vPC peer device into that
VRF for the vPC peer-keepalive link.
This example shows how to configure the destination IP address for the vPC-peer-keepalive link:
This example shows how to set up the peer keepalive link connection between the primary and secondary
vPC device:
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
118 OL-29480-06
Configuring Virtual Port Channels
Creating a vPC Peer Link
interface Vlan123
vrf member vpc_keepalive
ip address 123.1.1.2/30
no shutdown
vpc domain 1
peer-keepalive destination 123.1.1.1 source 123.1.1.2 vrf
vpc_keepalive
Procedure
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 119
Configuring Virtual Port Channels
Checking the Configuration Compatibility
Step 3 switch(config-if)# vpc peer-link Configures the selected EtherChannel as the vPC peer
link, and enters the vpc-domain configuration mode.
In the case of a Type 2 mismatch, the vPC is not suspended. Type 1 mismatches suspend the vPC.
Procedure
This example shows how to check that the required configurations are compatible across all the vPC interfaces:
switch# show vpc consistency-parameters global
Legend:
Type 1 : vPC will be suspended in case of mismatch
Name Type Local Value Peer Value
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
120 OL-29480-06
Configuring Virtual Port Channels
Enabling vPC Auto-Recovery
Step 2 switch(config)# vpc domain domain-id Enters vpc-domain configuration mode for an
existing vPC domain.
This example shows how to enable the auto-recovery feature in vPC domain 10 and set the delay period for
240 seconds:
switch(config)# vpc domain 10
switch(config-vpc-domain)# auto-recovery reload-delay 240
Warning:
Enables restoring of vPCs in a peer-detached state after reload, will wait for 240 seconds
(by default) to determine if peer is un-reachable
This example shows how to view the status of the auto-recovery feature in vPC domain 10:
switch(config-vpc-domain)# show running-config vpc
!Command: show running-config vpc
!Time: Tue Dec 7 02:38:44 2010
version 5.0(3)U2(1)
feature vpc
vpc domain 10
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 121
Configuring Virtual Port Channels
Configuring the Restore Time Delay
Procedure
Step 2 switch(config)# vpc domain Creates a vPC domain on the switch if it does not already
domain-id exist, and enters vpc-domain configuration mode.
Step 3 switch(config-vpc-domain)# delay Configures the time delay before the vPC is restored.
restore time The restore time is the number of seconds to delay
bringing up the restored vPC peer device. The range is
from 1 to 3600. The default is 30 seconds.
This example shows how to configure the delay reload time for a vPC link:
switch(config)# vpc domain 1
switch(config-vpc-domain)# delay restore 10
switch(config-vpc-domain)#
Excluding VLAN Interfaces from Shutting Down a vPC Peer Link Fails
When a vPC peer-link is lost, the vPC secondary switch suspends its vPC member ports and its switch virtual
interface (SVI) interfaces. All Layer 3 forwarding is disabled for all VLANs on the vPC secondary switch.
You can exclude specific SVI interfaces so that they are not suspended.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
122 OL-29480-06
Configuring Virtual Port Channels
Configuring the VRF Name
Procedure
Step 2 switch(config)# vpc domain Creates a vPC domain on the switch if it does not already
domain-id exist, and enters vpc-domain configuration mode.
This example shows how to keep the interfaces on VLAN 10 up on the vPC peer switch if a peer link fails:
Procedure
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 123
Configuring Virtual Port Channels
Moving Other Port Channels into a vPC
Procedure
Step 2 switch(config)# interface Selects the port channel that you want to put into the vPC to
port-channel channel-number connect to the downstream switch, and enters interface
configuration mode.
Note A vPC can be configured on a normal port channel
(physical vPC topology) and on a port channel host
interface (host interface vPC topology)
Step 3 switch(config-if)# vpc number Configures the selected port channel into the vPC to connect
to the downstream switch. The range is from 1 to 4096.
The vPC number that you assign to the port channel that
connects to the downstream switch from the vPC peer switch
must be identical on both vPC peer switches.
This example shows how to configure a port channel that will connect to the downstream device:
switch# configure terminal
switch(config)# interface port-channel 20
switch(config-if)# vpc 5
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
124 OL-29480-06
Configuring Virtual Port Channels
Manually Configuring the System Priority
Procedure
Step 2 switch(config)# vpc domain Selects an existing vPC domain on the switch, or creates
domain-id a new vPC domain, and enters the vpc-domain
configuration mode. There is no default domain-id; the
range is from 1 to 1000.
Step 3 switch(config-vpc-domain)# Enters the MAC address that you want for the specified
system-mac mac-address vPC domain in the following format: aaaa.bbbb.cccc.
Procedure
Step 2 switch(config)# vpc domain Selects an existing vPC domain on the switch, or creates
domain-id a new vPC domain, and enters the vpc-domain
configuration mode. There is no default domain-id; the
range is from 1 to 1000.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 125
Configuring Virtual Port Channels
Manually Configuring a vPC Peer Switch Role
Procedure
Step 2 switch(config)# vpc domain Selects an existing vPC domain on the switch, or creates
domain-id a new vPC domain, and enters the vpc-domain
configuration mode. There is no default domain-id; the
range is from 1 to 1000.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
126 OL-29480-06
Configuring Virtual Port Channels
Manually Configuring a vPC Peer Switch Role
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 127
Configuring Virtual Port Channels
Manually Configuring a vPC Peer Switch Role
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
128 OL-29480-06
CHAPTER 8
Configuring Q-in-Q VLAN Tunnels
This chapter contains the following sections:
Note Q-in-Q is supported on port channels. To configure a port channel as an asymmetrical link, all ports in
the port channel must have the same tunneling configuration.
Using the 802.1Q tunneling feature, service providers can use a single VLAN to support customers who have
multiple VLANs. Customer VLAN IDs are preserved and traffic from different customers is segregated within
the service-provider infrastructure even when they appear to be on the same VLAN. The 802.1Q tunneling
expands VLAN space by using a VLAN-in-VLAN hierarchy and tagging the tagged packets. A port configured
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 129
Configuring Q-in-Q VLAN Tunnels
Information About Q-in-Q Tunnels
to support 802.1Q tunneling is called a tunnel port. When you configure tunneling, you assign a tunnel port
to a VLAN that is dedicated to tunneling. Each customer requires a separate VLAN, but that VLAN supports
all of the customer’s VLANs.
Customer traffic tagged in the normal way with appropriate VLAN IDs come from an 802.1Q trunk port on
the customer device and into a tunnel port on the service-provider edge switch. The link between the customer
device and the edge switch is an asymmetric link because one end is configured as an 802.1Q trunk port and
the other end is configured as a tunnel port. You assign the tunnel port interface to an access VLAN ID that
is unique to each customer.
Note Selective Q-in-Q tunneling is not supported. All frames entering the tunnel port are subjected to Q-in-Q
tagging.
Packets that enter the tunnel port on the service-provider edge switch, which are already 802.1Q-tagged with
the appropriate VLAN IDs, are encapsulated with another layer of an 802.1Q tag that contains a VLAN ID
that is unique to the customer. The original 802.1Q tag from the customer is preserved in the encapsulated
packet. Therefore, packets that enter the service-provider infrastructure are double-tagged.
The outer tag contains the customer’s access VLAN ID (as assigned by the service provider), and the inner
VLAN ID is the VLAN of the incoming traffic (as assigned by the customer). This double tagging is called
tag stacking, Double-Q, or Q-in-Q.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
130 OL-29480-06
Configuring Q-in-Q VLAN Tunnels
Native VLAN Hazard
The following figure shows the differences between the untagged, tagged and double-tagged ethernet frames.
By using this method, the VLAN ID space of the outer tag is independent of the VLAN ID space of the inner
tag. A single outer VLAN ID can represent the entire VLAN ID space for an individual customer. This
technique allows the customer’s Layer 2 network to extend across the service provider network, potentially
creating a virtual LAN infrastructure over multiple sites.
Note Hierarchical tagging, that is multi-level dot1q tagging Q-in-Q, is not supported.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 131
Configuring Q-in-Q VLAN Tunnels
Information About Layer 2 Protocol Tunneling
Note The vlan dot1q tag native command is a global command that affects the tagging
behavior on all trunk ports.
• Ensure that the native VLAN ID on the edge switch trunk port is not within the customer VLAN range.
For example, if the trunk port carries traffic of VLANs 100 to 200, assign the native VLAN a number
outside that range.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
132 OL-29480-06
Configuring Q-in-Q VLAN Tunnels
Information About Layer 2 Protocol Tunneling
discover neighboring Cisco devices from local and remote sites, and the VLAN Trunking Protocol (VTP)
must provide consistent VLAN configuration throughout all sites in the customer network.
When protocol tunneling is enabled, edge switches on the inbound side of the service-provider infrastructure
encapsulate Layer 2 protocol packets with a special MAC address and send them across the service-provider
network. Core switches in the network do not process these packets, but forward them as normal packets.
Bridge protocol data units (BPDUs) for CDP, STP, or VTP cross the service-provider infrastructure and are
delivered to customer switches on the outbound side of the service-provider network. Identical packets are
received by all customer ports on the same VLANs.
If protocol tunneling is not enabled on 802.1Q tunneling ports, remote switches at the receiving end of the
service-provider network do not receive the BPDUs and cannot properly run STP, CDP, 802.1X, and VTP.
When protocol tunneling is enabled, Layer 2 protocols within each customer’s network are totally separate
from those running within the service-provider network. Customer switches on different sites that send traffic
through the service-provider network with 802.1Q tunneling achieve complete knowledge of the customer’s
VLAN.
Note Layer 2 protocol tunneling works by tunneling BPDUs in the software. A large number of BPDUs that
comes into the supervisor module cause the CPU load to go up. The load It is controlled by Control Plane
Policing CoPP configured for packets marked as BPDU.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 133
Configuring Q-in-Q VLAN Tunnels
Information About Layer 2 Protocol Tunneling
For example, the following figure shows Customer X has four switches in the same VLAN that are connected
through the service-provider network. If the network does not tunnel BPDUs, the switches on the far ends of
the network cannot properly run the STP, CDP, 802.1X, and VTP protocols.
In the preceding example, STP for a VLAN on a switch in Customer X, Site 1 will build a spanning tree on
the switches at that site without considering convergence parameters based on Customer X’s switch in Site
2.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
134 OL-29480-06
Configuring Q-in-Q VLAN Tunnels
Licensing Requirements for Q-in-Q Tunnels
The following figure shows the resulting topology on the customer’s network when BPDU tunneling is not
enabled.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 135
Configuring Q-in-Q VLAN Tunnels
Configuring Q-in-Q Tunnels and Layer 2 Protocol Tunneling
Note You must set the 802.1Q tunnel port to an edge port with the spanning-tree port type edge command.
The VLAN membership of the port is changed when you enter the switchport access vlan vlan-id
command.
You should disable IGMP snooping on the access VLAN allocated for the dot1q-tunnel port to allow
multicast packets to traverse the Q-in-Q tunnel.
Procedure
Step 2 switch(config)# interface ethernet Specifies an interface to configure, and enters interface
slot/port configuration mode.
Step 4 switch(config-if)# [no] switchport Creates an 802.1Q tunnel on the port. The port will go
mode dot1q-tunnel down and reinitialize (port flap) when the interface mode
is changed. BPDU filtering is enabled and CDP is
disabled on tunnel interfaces.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
136 OL-29480-06
Configuring Q-in-Q VLAN Tunnels
Enabling the Layer 2 Protocol Tunnel
Procedure
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 137
Configuring Q-in-Q VLAN Tunnels
Configuring Thresholds for Layer 2 Protocol Tunnel Ports
This example shows how to enable protocol tunneling on an 802.1Q tunnel port:
switch# configure terminal
switch(config)# interface ethernet 7/1
switch(config-if)# switchport
switch(config-if)# switchport mode dot1q-tunnel
switch(config-if)# l2protocol tunnel stp
switch(config-if)# exit
switch(config)# exit
Procedure
Step 2 switch(config)# interface ethernet Specifies an interface to configure, and enters interface
slot/port configuration mode.
Step 6 switch(config-if)# [no] l2protocol Specifies the maximum number of packets that can be
tunnel shutdown-threshold [cdp | processed on an interface. When the number of packets is
stp | vtp] exceeded, the port is put in error-disabled state. Optionally,
you can specify the Cisco Discovery Protocol (CDP),
Spanning Tree Protocol (STP), or VLAN Trunking
Protocol (VTP). Valid values for the packets is from 1 to
4096.
This example shows how to configure a threshold for a Layer 2 protocol tunnel port:
switch# configure terminal
switch(config)# interface ethernet 7/1
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
138 OL-29480-06
Configuring Q-in-Q VLAN Tunnels
Verifying the Q-in-Q Configuration
switch(config-if)# switchport
switch(config-if)# switchport mode dot1q-tunnel
switch(config)# l2protocol tunnel drop-threshold 3000
switch(config)# l2protocol tunnel shutdown-threshold 3000
switch(config)# exit
switch# copy running-config startup-config
Command Purpose
clear l2protocol tunnel counters [interface if-range] Clears all the statistics counters. If no interfaces are
specified, the Layer 2 protocol tunnel statistics are
cleared for all interfaces.
show dot1q-tunnel [interface if-range] Displays a range of interfaces or all interfaces that
are in dot1q-tunnel mode.
show l2protocol tunnel [interface if-range | vlan Displays Layer 2 protocol tunnel information for a
vlan-id] range of interfaces or all dot1q-tunnel interfaces that
are part of a specified VLAN or all interfaces.
show l2protocol tunnel summary Displays a summary of all ports that have Layer 2
protocol tunnel configurations.
show running-config l2pt Displays the current Layer 2 protocol tunnel running
configuration.
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 139
Configuring Q-in-Q VLAN Tunnels
Feature History for Q-in-Q Tunnels and Layer 2 Protocol Tunneling
switch(config)# exit
switch#
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
140 OL-29480-06
INDEX
D
C debounce timer 11
changed information 1 parameters 11
description 1 debounce timer, configuring 26
channel mode 64 Ethernet interfaces 26
port channels 64 default interface 11
channel modes 58 default settings 37, 74
port channels 58 ip tunnels 74
clearing MAC addresses 99 Layer 3 interfaces 37
configuration 45 DHCP client configuration 37
Layer 3 interfaces 45 DHCP client configuration limitations 37
verifying 45 DHCP client discovery 37
configuration examples 47, 79 disabling 19, 22, 25, 27, 116
ip tunneling 79 CDP 22
Layer 3 interfaces 47 error-disabled recovery 25
configuring 25, 27, 38, 39, 40, 41, 66, 67 ethernet interfaces 27
description parameter 27 link negotiation 19
error-disabled recovery interval 25 vPCs 116
interface bandwidth 40 displaying MAC addresses 95
LACP fast timer rate 66 downlink delay 12
LACP port priority 67
loopback interfaces 41
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 IN-1
Index
E interfaces 5, 6, 33, 34, 35, 36, 40, 41, 42, 46, 47, 71, 72, 75, 77, 78, 129, 132,
135, 136, 137, 138, 139
enabling 22, 23, 24 802.1q tunnel port, creating 136
CDP 22 assigning to a VRF 42
error-disabled detection 23 chassis ID 5
error-disabled recovery 24 configuring bandwidth 40
enabling feature nv overlay 89 gre tunnel, configuring 77
enabling PIM 87 gre tunnels 72
enabling VLAN to vn-segment mapping 89 ip tunnel configuration, verifying 78
Ethernet interfaces 8, 26 ip tunnels 71
debounce timer, configuring 26 ipip tunnel decapsulation-only, configuring 77
interface speed 8 ipip tunnel, configuring 77
layer 2 protocol tunnel 137
layer 2 protocol tunnel ports, thresholds configuring 138
F layer 2 protocol tunneling 132
Layer 3 33, 46, 47
feature history 49, 70, 80, 140 configuration examples 47
ip tunnels 80 monitoring 46
Layer 3 interfaces 49 loopback 36, 41
port channels 70 options 5
q-in-q tunnels, layer 2 protocol tunneling 140 q-in-q configuration, verifying 139
q-in-q tunneling, guidelines 135
q-in-q tunnels 129
q-in-q tunnels, licensing 135
G routed 34
gre tunnel, configuring 77 tunnel 36
interfaces 77 tunnel interface, creating 75
gre tunnels 72 UDLD 6
interfaces 72 VLAN 35, 40
guidelines 73 configuring 40
ip tunnels 73 ip tunnel configuration, verifying 78
guidelines and limitations 36, 109 interfaces 78
Layer 3 interfaces 36 ip tunneling 79
vPCs 109 configuration examples 79
guidelines and limitations for VXLAN 85 ip tunnels 71, 73, 74, 79, 80
default settings 74
feature history 80
guidelines 73
H interfaces 71
hardware hashing 63 licensing requirements 73
multicast traffic 63 prerequisites 73
standards 79
ipip decapsulate-only 72
I
ingress replication 93 L
interface information, displaying 28
layer 2 28 LACP 52, 57, 59, 60, 63, 65
interface MAC address, configuring 43 configuring 63
interface speed 8, 16 marker responders 59
configuring 16 port channel, minlinks 60, 65
Ethernet interfaces 8 port channels 57
system ID 57
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
IN-2 OL-29480-06
Index
M Q
MIBs 31, 48 q-in-q configuration, verifying 139
Layer 2 interfaces 31 interfaces 139
Layer 3 interfaces 48 q-in-q tunneling, guidelines 135
interfaces 135
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
OL-29480-06 IN-3
Index
R
related documents 48 U
Layer 3 interfaces 48
resilient hashing 55 UDLD 6, 7
restarting 27 aggressive mode 7
ethernet interfaces 27 defined 6
routed interfaces 34, 38, 40 nonaggressive mode 7
configuring 38 UDLD modeA 13
configuring bandwidth 40 configuring 13
Unidirectional Link Detection 6
S
V
SFP+ transceiver 8
Small form-factor pluggable (plus) transceiver 8 verifying 45
standards 49, 79 Layer 3 interface configuration 45
ip tunnels 79 VLAN 35
Layer 3 interfaces 49 interfaces 35
STP 51 VLAN interfaces 40
port channel 51 configuring 40
subinterfaces 34, 39, 40 VLAN to VXLAN VNI mapping 89
configuring 39 VNI to multicast group mapping 92
configuring bandwidth 40 vPC terminology 102
svi autostate 9 vPCs 109, 124
layer 2 9 guidelines and limitations 109
SVI autostate disable 37 moving port channels into 124
SVI autostate disable, configuring 43 VRF 42
svi autostate, disabling 20 assigning an interface to 42
layer 2 20 vrf membership, assigning 78
symmetric hashing 56 tunnel interface 78
Cisco Nexus 3000 Series NX-OS Interfaces Configuration Guide, Release 6.x
IN-4 OL-29480-06