Originally published at: https://indeni.com/deployment-of-the-cisco-nexus-vpc-technology-and-analysis-by-indeni-part-1/
The vPC enables the deployment of a link aggregation from a downstream network device to two individual and independent Cisco NX-OS switches (vPC peers). The downstream devices can be another blade switch, server, or any other device such as a firewall which supports a link aggregation technology protocol such as LACP or PagP. It should be noticed that the vPC is supported since NX-OS version 4.1(4) is included in the base NX-OS software license.
This article is the first of a series of vPC Nexus-Indeni articles highlighting how Indeni can analyze vPC configuration parameters upon the discovery of a Nexus switch. Besides, vPC alerts are triggered by Indeni providing remediation steps so that the Network Administrator can easily identify and resolve several vPC issues proactively or to implement the best practices configuration recommended by Cisco.
Are you a Network Administrator, System Engineer, Software Engineer, Indeni Knowledge Expert (IKEs), Indeni Software Developers or Tech Geek? If yes, then read on! This article is published for you!
Cisco Nexus vPC Main Features Overview
Let’s have a quick overview to the NX-OS vPC which is one of the most widely deployed Cisco DC technologies. The vPC belongs to the Multichassis EtherChannel (MEC) family of technology and has the following features:- Allows dual-homed servers (dual uplinks) to operate in active-active mode
- Provides fast convergence in case of link or a device failure
- Improves performance by offering dual active/active default gateways for servers
- Maintains independent control planes in contrast to the Cisco Catalyst similar technology named VSS which has a single Control Plane
- Simplifies Network Design
- Offers a loop free topology by eliminating Spanning Tree Protocol (STP) blocked ports. The STP is used only as a fallback mechanism for deployment with only vPC peers.
Cisco Nexus vPC Main Components
As is illustrated to the figure the vPC architecture consists of the following main components:
vPC Peer: This is the adjacent (peer) device which is connected via the vPC Peer-link. A vPC setup consists of two Nexus devices in a pair highlighted as S1 and S2 to the figure.
vPC Peer-link: The vPC peer-link is the most critical connectivity element in the vPC architecture. This link is used to synchronize the state between vPC peer devices via vPC control packets. The vPC peer-link should not be used for production traffic but limited to only multicast, broadcast, unknown unicast traffic and for the traffic of orphaned ports.
vPC Peer Keepalive Link: The peer keepalive link provides a Layer 3 connectivity path which is used as a secondary test in order to determine whether the remote peer is operational or down. In particular it is used as an extra protection mechanism to determine whether the peer link itself has failed or whether the vPC peer is down. It should be clear that neither data nor synchronization traffic is sent over the vPC peer keepalive link but just keepalive packets. The frequency of these packets can be set with relevant configuration change.
vPC Domain: This is the common domain configured across two vPC peer devices and this value identifies the vPC. It should be noticed that this is only allowed on one vPC domain id per device
vPC Member Port: This is the interface that is a member of one of the vPCs configured on the vPC peers.
Cisco Fabric Services (CFS): This protocol is important for the vPC operation and is used for the state full synchronization and configuration. It utilizes the peer link and is enabled by default.
Orphan Device: It is a device such as a Server which is on a vPC VLAN but only connected to one VPC peer and not to both.
Orphan Port: An orphan port is an interface which connects to an orphan device e.g. Server vPC VLAN.
non-vPC VLAN : Any of the VLANs not carried over the peer-link. In this case the STP is utilized to offer link redundancy and convergence in case of link or node failure.
Cisco Nexus & Indeni Lab Environment
This section describes that the Cisco Nexus switches, Indeni platform and Software versions are deployed at the lab environment in order to setup the vPC technology. Then, the Nexus switches are discovered and analyzed by Indeni. The deployed CISCO Nexus lab physical topology is illustrated in the next diagram. Cisco Nexus - Indeni Lab Physical Topology
The software versions information of the Network devices and the Indeni equipped to the lab can be found in the following table.| Cisco Nexus Switch | Cisco Bug ID | Fixed Release Availability |
|---|---|---|
| N5k-UP | N5K-C5548UP-SUP | 7.0(2)N1(1) |
| N5k-down | N5K-C5548UP-SUP | 7.0(7)N1(1) |
| FEX-100 | N5K-C2248TP-1GE | 7.0(7)N1(1) |
| FEX-101 | N5K-C2248TP-1GE | 7.0(7)N1(1) |
| ISR-DC-Lab | cisco ISR4331/K9 | Cisco IOS XE Software Version 03.16.02.S |
| Indeni | Indeni | Installed Indeni packages: indeni-collector 6.1.4.65 indeni-ds 6.1.4.65 indeni-server 6.1.4.65 indeni-triton 6.1.4.65 indeni-vigile 6.1.4.65 indeni-walt 6.1.4.65 |
Nexus Hardware and Software revisions
The following figure provides an easy mnemonic in order to elaborate the numbering convention of the deployed NX-OS versions. Figure: 2: NX-OS Release explanation
Finally, a brief description for the used Cisco Nexus switches for deploying the vPC technology and testing the Indeni vPC analysis capabilities is provided below.Nexus 5548UP: A powerful and widely deployed Nexus switch commonly used as “End of Row” at a Data Center with thirty-two fixed Unified ports 1/10 Gbps fixed SFP+ on the base chassis along with one expansion slot. In addition to the fixed interfaces, the Nexus 5548UP has one expansion module. The expansion module supports native Fibre Channel, Ethernet, and FCoE interface for a total of 48 interfaces. Unified ports on the Nexus 5500 platforms enable an interface to have one of the following characteristics depending on the licensing and pluggable transceiver installed: traditional Ethernet, Fibre Channel (FC), or FCoE. Depending on the configuration, the interface can have the following physical characteristics: 1-Gigabit Ethernet, 10-Gigabit Ethernet, 10-Gigabit Ethernet with FCoE, and 1/2/4/8-G native Fibre Channel. In addition to these expansion modules, the 5548UP supports a Layer 3 daughter card that can be ordered with the system or as a spare.
Nexus 2248TP-E: A popular Nexus FEX with forty-eight ports 100M/1000BaseT Enhanced Host interfaces (server interfaces) and four 10-Gigabit Ethernet Fabric uplinks. NOTE: The FEX do not have any actual role to this configuration/test scenario and the FEX technology will be discussed in an upcoming article.
Nexus vPC Configuration
The NX-OS vPC technology is configured to the pair of the N5k switches deployed at the lab. A basic configuration of the vPC technology as well as the normal operation can be verified by following the next 6 steps mentioned below. It should be noted that the order of the vPC configuration is important and that a basic vPC setup is established by using just the first 4 steps.Step 1: Enable the vPC feature and configure the vPC domain ID on both Nexus switches.
Step 2: Select a Peer Keepalive deployment option (this connectivity is provided via the management interface at the lab).
Step 3: Establish the vPC peer keepalive link.
Step 4: Configure the vPC peer link.
Note: This step completes the global vPC configuration on both vPC peer switches.
Step 5: Configure individual vPCs to downstream switches or devices (a CISCO ISR router is used as a downstream device at the lab)
Step 6: Verify operation of the vPC and vPC consistency parameters.
Upon completion of the aforementioned steps it can be verified that the vPC technology has been successfully deployed to the Nexus switches installed at the Lab as is described in the next section.
Cisco Nexus vPC & Indeni Analysis
Indeni currently monitors several important metrics related to the NX-OS vPC technology as is described below:✓ HSRP & VRRP Active/Standby state change of the vPC cluster members ✓ vPC role configuration/operation status ✓ Connected networks do not match across vPC cluster members ✓ vPC type 2 consistency status ✓ vPC advanced features status such as peer-gateway, vPC graceful consistency check, auto-recovery ✓ vPC peer link status or any of the members link vPC status ✓ vPC peer keepalive status ✓ Checks for Domain name of the vPC peers mismatch ✓ Checks for Login banner of the vPC peers mismatch ✓ Checks for NX-OS version of the vPC peers mismatch ✓ Checks for enabled features of the vPC peers mismatch ✓ Checks for Timezone of the vPC peers mismatch
The show vpc brief command is executed at the Nexus 5k switch which are deployed at the lab to check the vPC domain ID, the peer-link status, the keepalive message status, whether the configuration consistency is successful, and whether a peer link has formed. It also states the status of the vPC Port Channels. The output of this NX-OS command proves that the vPC between the pair of the N5k switches is operational without any issue!N5k-UP# sh vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 1
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : secondary, operational primary
Number of vPCs configured : 2
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled (timeout = 240 seconds)
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po23 up 10,100
vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
10 Po10 down* success success -
11 Po11 down* Not Consistency Check Not -
Applicable Performed
The “show vpc consistency-parameters” NX-OS command is executed to identify specific parameters that might have caused the consistency check to fail either on the vPC peer link or to the vPC enabled Portchannels. The output is provided to the next table and it seems that there isn’t any inconsistency.
N5k-UP# show vpc consistency-parameters global Legend: Type 1 : vPC will be suspended in case of mismatch Name Type Local Value Peer Value ------------- ---- ---------------------- ----------------------- QoS 2 ([], [], [], [], [], ([], [], [], [], [], []) []) Network QoS (MTU) 2 (1538, 0, 0, 0, 0, 0) (1538, 0, 0, 0, 0, 0) Network Qos (Pause) 2 (F, F, F, F, F, F) (F, F, F, F, F, F) Input Queuing (Bandwidth) 2 (100, 0, 0, 0, 0, 0) (100, 0, 0, 0, 0, 0) Input Queuing (Absolute 2 (F, F, F, F, F, F) (F, F, F, F, F, F) Priority) Output Queuing (Bandwidth) 2 (100, 0, 0, 0, 0, 0) (100, 0, 0, 0, 0, 0) Output Queuing (Absolute 2 (F, F, F, F, F, F) (F, F, F, F, F, F) Priority) STP Mode 1 Rapid-PVST Rapid-PVST STP Disabled 1 None None STP MST Region Name 1 "" "" STP MST Region Revision 1 0 0 STP MST Region Instance to 1 VLAN Mapping STP Loopguard 1 Disabled Disabled STP Bridge Assurance 1 Enabled Enabled STP Port Type, Edge 1 Normal, Disabled, Normal, Disabled, BPDUFilter, Edge BPDUGuard Disabled Disabled STP MST Simulate PVST 1 Enabled Enabled IGMP Snooping Group-Limit 2 4000 4000 Interface-vlan admin up 2 10 - Interface-vlan routing 2 10 - capability Allowed VLANs - 10,100-101 10,100 Local suspended VLANs - 101 -
Finally, it should be noticed that it is feasible to set the role priority under vpc domain configuration. The “show vpc role” NX-OS command is executed and it is proved that the N5k-UP switch has been assigned as operational primary.
N5k-UP# sh vpc role
vPC Role status
vPC Role status
vPC role : secondary, operational primary
Dual Active Detection Status : 0
vPC system-mac : 00:23:04:ee:be:01
vPC system-priority : 32667
vPC local system-mac : 8c:60:4f:aa:c2:3c
vPC local role-priority : 32667
N5k-DOWN# show vpc role
vPC Role status
vPC Role status
vPC role : primary, operational secondary
Dual Active Detection Status : 0
vPC system-mac : 00:23:04:ee:be:01
vPC system-priority : 32667
vPC local system-mac : 8c:60:4f:2c:b3:01
vPC local role-priority : 32667
N5k-DOWN#
Let’s move into action!
We are connecting to the Indeni server and reviewing the discovered Nexus switches deployed at the lab. The dual Nexus 5k switches previously configured with the vPC technology have been discovered successfully and several alerts have already been triggered. Then, we review the vPC Indeni analysis of one of the aforementioned Nexus 5k series switches. As is illustrated in the next screen capture, you must choose the discovered Nexus switch e.g. N5k-UP and then press the “More Device Info” button on the right side. Indeni has completed the vPC analysis which does not take more than two minutes in most cases. We review the indeni analysis by selecting the first tab of the vPC named “vPC Consistency Descriptions”. It is verified that there isn’t any vPC inconsistency between the vPC peers as was displayed also by running the relevant NX-OS command. We are moving to the next tab named “vPC information” which provides an in depth analysis of the Nexus switch by Indeni. Indeni provides critical information about the stability of the vPC and checks if recommended by Cisco vPC features/technologies have been deployed to the Nexus switch. Below is a summary of the most critical vPC features which are monitored by Indeni.
✓ Peer-gateway: This capability allows for a vPC switch the forwarding of traffic local to the vPC peer device and avoids the use of the peer-link (by not bridging the traffic to the other vPC peer device). ✓ Graceful consistency check: It has been introduced since NX-OS version 5.2 to soften vPC system reaction in occurrence to type 1 consistency check. With vPC graceful type-1 check capability, only member ports on secondary vPC peer device are brought down. vPC member ports on primary vPC peer device remain up and process all traffic coming from (or going out to) the access device. ✓ Auto-recovery: It was included with NX-OS version 5.2(1) and is used mainly for two reasons. First, it is used to provide a backup mechanism in case of vPC peer-link failure followed by vPC primary peer device unavailability. Finally, it is important to handle a specific case where both vPC peer devices reload but only one comes back to life. ✓ Peer Keepalive Link status: The keepalive link is used as a secondary test mechanism to confirm the vPC peer is live in case the Peer-Link goes down. It should be mentioned that in the event the Peer Keepalive Link fails it will not affect the operation of the vPC.
The list of all the NX-OS features and critical metrics analyzed by Indeni can be found in this screen capture. We move forward by selecting the next tab named “vPC Role Status” which provides an analysis of the Nexus with the configured and actual operational role of each Nexus switch. Let’s see how this feature works. Between the pair of vPC peer switches, an election is held to determine a primary and secondary vPC device. This election is non-preemptive. The vPC primary or secondary role is primarily a control plane role that determines which of the two switches will primarily be by default responsible for the generation and processing of spanning-tree BPDUs for the vPCs.The primary and secondary roles are also important in certain failure scenarios, most notably in a peer link failure. It is recommended for this reason that the orphan ports (ports connecting to only one switch) are connected to the vPC primary switch. It is possible that the configured vPC primary role does not match with the assigned operation role as is also illustrated in the following screen capture where the nexus switch has not been configured with the primary role but has been assigned as primary. This is also shown in the relevant NX-OS show commands outputs.
NOTE: It is feasible to set the role priority under vpc domain configuration with the command role priority to affect the election of the primary vPC switch
Summary
This article describes the main components, features and several best practices of a Nexus vPC Data Center architecture. The steps to configure vPC have been described before this configuration has been applied to the Lab network equipment. The Nexus 5k switches are used for analysis by Indeni to receive a dynamic report for the vPC technology and the status of the vPC features. Finally, instructions on how to get access to the vPC Indeni Analysis tabs and captures are provided to describe the NX-OS vPC capabilities offered by Indeni via the Live Config tab.This article is the first part in discussing the cool features and powerful capabilities that Indeni offers for the NX-OS vPC technology. The following articles are going to describe how we can get live alerts and notifications from Indeni by performing vPC configuration changes to the Nexus lab environment. Finally, instructions are given on the best practices and the important vPC features which are highly recommended by Cisco to be deployed to a Nexus Data Center environment. Stay tuned 
Learn more by joining the Indeni Community. If you liked this article please share it by clicking on one of the social media sharing icons at the top of this page. Thank you to Vasileios Bouloukos for his contributions to this article.