Guide IWAN Design

2014 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Page 1 of 66
Cisco Intelligent WAN (IWAN)
Deployment Guide
Deployment Guide

2014 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 2 of 66
Contents
Introduction .............................................................................................................................................................. 3
IWAN Solution Overview ....................................................................................................................................... 3
Design Overview ................................................................................................................................................... 5
Transport-Independent Design ......................................................................................................................... 5
Internet as WAN Transport .......................................................................................................................... 6
WAN-Aggregation Designs .......................................................................................................................... 6
IWAN Hybrid Design Model ......................................................................................................................... 7
WAN Remote-Site Designs ......................................................................................................................... 8
IP Multicast ....................................................................................................................................................... 9
Deploying Transport-Independent Topology ........................................................................................................ 9
Design Overview ................................................................................................................................................... 9
DMVPN Hub Routers ....................................................................................................................................... 9
Remote Sites - DMVPN Spoke Router Selection ........................................................................................... 10
VRFs and Front Door VRF ............................................................................................................................. 11
Design Details ..................................................................................................................................................... 12
EIGRP ............................................................................................................................................................ 13
Encryption and MTU ....................................................................................................................................... 14
DMVPN .......................................................................................................................................................... 15
Deployment Details ............................................................................................................................................. 17
DMVPN Hub Router Configuration ................................................................................................................. 18
Example .................................................................................................................................................... 33
Example .................................................................................................................................................... 44
Deploying Intelligent Path Control ....................................................................................................................... 51
Design Overview ................................................................................................................................................. 51
PfR Components ............................................................................................................................................ 51
PfR Classes of Service ................................................................................................................................... 52
Design Details ..................................................................................................................................................... 53
Deployment Details ............................................................................................................................................. 53

Introduction
The Cisco
Intelligent WAN (IWAN) solution provides design and implementation guidance for organizations
looking to deploy WAN transport with intelligent path control, application optimization, and secure encrypted
communications between branch locations while reducing the operating cost of the WAN. IWAN takes full
advantage of cost-effective Internet services to increase bandwidth capacity without compromising performance,
reliability, or security of collaboration or cloud-based applications.
IWAN Solution Overview
With the advent of globalization, wide area networks (WAN) have become a major artery for communication
between remote offices and customers in any corner of the world. Additionally, with data center consolidation,
applications are moving to centralized data centers and clouds. WAN now plays an even more critical role as
business survival is dependent on the availability and performance of the network.
Until now, the only way to get reliable connectivity with predictable performance was to take advantage of a private
WAN using MPLS or leased line service. However, carrier-based MPLS and leased line service can be expensive
and are not always cost-effective for an organization to use for WAN transport to support growing bandwidth
requirements for remote-site connectivity. Organizations are looking for ways to lower operating budget while
adequately providing the network transport for a remote site.
Nearly half of businesses (46 percent) are migrating, or are planning to migrate their WAN to the Internet
(Nemertes Research, Benchmark 2012-13 Emerging WAN Trends). Why? As bandwidth demands have increased,
the Internet has become a much more stable platform, and the price-to-performance gains are very attractive.
However, businesses are primarily deploying Internet as WAN in their smaller sites or as a backup path because
of the risks. Now this cost-effective, performance-enhancing opportunity can be realized at all your branch offices
with Cisco IWAN.
Cisco IWAN can enable organizations to deliver an uncompromised experience over any connection. With Cisco
IWAN IT organizations can provide more bandwidth to their branch office connections using less expensive WAN
transport options without affecting performance, security, or reliability. With the IWAN solution, traffic is dynamically
routed based on application service-level agreement (SLA), endpoint type, and network conditions to deliver the
best quality experience. The realized savings from IWAN not only pays for the infrastructure upgrades, but also
frees resources for business innovation.

Figure 1 outlines the components of the IWAN solution. Each component is explained in more detail below.
Figure 1. Cisco IWAN Components

Secure and flexible transport-independent design: Using Dynamic Multipoint VPN (DMVPN) IWAN
provides capabilities for easy multi-homing over any carrier service offering, including Multiprotocol Label
Switching (MPLS), broadband, and cellular 3G/4G/LTE. More importantly, the design simplifies the routing
design with a single routing control plane and minimal peering to providers, making it easy for organizations
to mix and match and change providers and transport options. Two or more WAN transport providers are
recommended to increase network availability up to 99.999%. Additionally, the Cisco DMVPN solution
provides an industry-proven and U.S. government FIPS 140-2 certified IPsec solution for data privacy and
integrity protection, and automatic site-to-site IP Security (IPsec) tunnels.
Intelligent path control: By using Cisco Performance Routing (PfR), this component improves application
delivery and WAN efficiency. PfR dynamically controls data packet forwarding decisions by looking at
application type, performance, policies, and path status. PfR protects business applications from fluctuating
WAN performance while intelligently load-balancing traffic over the best performing path based on the
application policy. PfR monitors the network performance - jitter, packet loss, delay - and makes decisions
to forward critical applications over the best performing path based on the application policy. Cisco PfR
consists of border routers that connect to the broadband service, and a master controller application
supported by Cisco IOS
Software on a router. The border routers collect traffic and path information and
send it to the master controller, which detects and enforces the service policies to match the application
requirement. Cisco PfR can select an egress WAN path to intelligently load-balance traffic based on circuit
costs, to reduce a company's overall communications expenses. IWAN intelligent path control is the key to
providing a business-class WAN over Internet transport.

Application optimization: Cisco Application Visibility and Control (AVC) and Cisco Wide Area Application
Services (WAAS) provide application performance visibility and optimization over the WAN. With
applications becoming increasingly opaque due to increase reuse of well-known ports such as HTTP (port
80), static port classification of application is no longer sufficient. Cisco AVC provides application awareness
with deep packet inspection of traffic to identify and monitor applications' performance. Visibility and control
at the application level (layer 7) is provided through AVC technologies such as Network-Based Application
Recognition 2 (NBAR2), NetFlow, quality of service (QoS), Performance Monitoring, Medianet, and more.
Cisco AVC allows IT to determine what traffic is running across the network, tune the network for business-
critical services, and resolve network problems. With increased visibility into the applications on the
network, better QoS and PfR policies can be enabled to help ensure that critical applications are properly
prioritized across the network. Cisco WAAS provides application-specific acceleration capabilities that
improve response times while reducing WAN bandwidth requirements.
Secure connectivity protects the WAN and offloads user traffic directly to the Internet. Strong IPsec
encryption, zone-based firewalls, and strict access lists are used to protect the WAN over the public
Internet. Routing branch users directly to the Internet improves public cloud application performance while
reducing traffic over the WAN. Cisco Cloud Web Security (CWS) service provides a cloud-based web proxy
to centrally manage and secure user traffic accessing the Internet.
Design Overview
The Intelligent WAN Design Guide provides a design that supports highly available, secure, and optimized
connectivity for multiple remote branches.
The primary focus of the design is to allow active/active usage of WAN transports in the following deployment
scenarios:
Hybrid WAN design
Dual Internet WAN design
Transport-Independent Design
Dynamic Multipoint VPN (DMVPN) is the building block for the transport-independent design. It provides easy
multi-homing over any carrier service offering while providing a single routing control plane with minimal peering to
providers. It is a solution for building scalable, automatic, site-to-site IPsec VPNs that support a variety of
applications. DMVPN is widely used for encrypted site-to-site connectivity over public or private IP networks and
can be implemented on all WAN routers used in this design guide.
DMVPN is used as the solution for data privacy and integrity protection over Internet transport because it supports
dynamic, on-demand, full meshed connectivity with a simple hub-and-spoke configuration and a zero-touch hub
deployment model for adding remote sites. Additionally, DMVPN supports spoke routers that have dynamically
assigned Internet IP addresses.
DMVPN makes use of multipoint generic routing encapsulation (mGRE) tunnels to interconnect the hub to all of the
spoke routers. These mGRE tunnels are also sometimes referred to as DMVPN clouds in this context. This
technology combination supports unicast, multicast, and broadcast IP, including the ability to run routing protocols
within the tunnels.

Internet as WAN Transport
The Internet is essentially a large-scale public WAN composed of multiple interconnected service providers. The
Internet can provide reliable, high-performance connectivity between various locations. However, it lacks any
explicit guarantees for these connections. Despite its best effort nature, the Internet is a sensible choice for a
primary transport when it is not feasible to connect with another transport option. Additional resiliency is provided
by using the Internet as an alternate transport option.
Internet connections are typically included in discussions relevant to the Internet edge, specifically for the primary
site. Remote-site routers also commonly have Internet connections, but do not provide the same breadth of
services using the Internet.
The WAN uses the Internet for VPN site-to-site connections and MPLS transport, both as primary WAN transport.
For security and other reasons, Internet access at remote sites is traditionally routed through the primary site for
centralized security policy control.
WAN-Aggregation Designs
There are two WAN-aggregation design models that are documented in this design guide. The first one is the
IWAN Hybrid design model, which uses Internet VPN as transport in conjunction with traditional WAN to provide
more bandwidth for key applications while balancing SLA guarantees for the applications. The second is the IWAN
Dual Internet design model, which uses dual Internet service providers to further reduce cost while maintaining five
nines of reliability for the network transport.
This guide documents the Hybrid deployment module as organizations adopt the IWAN solution to complement
their existing WAN connection. From the WAN-aggregation perspective, there are no functional differences
between these two methods.
Following figure shows the deployment models for IWAN today.
Figure 2. IWAN Deployment Model

In all of the WAN-aggregation designs, tasks such as IP route summarization are performed at the distribution
layer. There are other various devices supporting WAN edge services, and these devices should also connect into
the distribution layer.
IWAN Hybrid Design Model
The IWAN Hybrid design model:
Has a single MPLS VPN carrier or a single layer 2 WAN
Uses a single Internet link
Uses Front Door VRF (FVRF) with static routes into MPLS VPN carrier or Internet carrier for IPSec VPN
setup
Figure 3 shows the Hybrid design.
Figure 3. IWAN Hybrid Design Model for MPLS WAN

In the IWAN Hybrid design model, the DMVPN hub router connects to the Internet indirectly through a firewall
demilitarized zone (DMZ) interface contained within the Internet edge. For details about the connection to the
Internet, see the Firewall and IPS Design Guide. The VPN hub routers are connected into the firewall DMZ
interface, rather than connected directly with Internet service provider routers.

WAN Remote-Site Designs
This guide documents multiple WAN remote-site designs (Figure 4). They are based on various combinations of
WAN transports mapped to the site specific requirements for service levels and redundancy.
Figure 4. WAN Remote-Site Designs

The remote-site designs include single or dual WAN edge routers. These can be a customer edge router (for MPLS
or Layer 2 WAN) and a VPN spoke router. In some cases, a single WAN edge router can perform the role of both a
customer edge router and VPN spoke router.
Most remote sites are designed with a single router WAN edge; however, certain remote-site types require a dual
router WAN edge. Dual router candidate sites include regional office or remote campus locations with large user
populations, or sites with business-critical needs that justify additional redundancy to remove single points of
failure.
The overall WAN design methodology is based on a primary WAN-aggregation site design that can accommodate
all of the remote-site types that map to the various link combinations.
The modular nature of the IWAN network design helps you to create design elements that can be replicated
throughout the network.
The WAN-aggregation designs and all of the WAN remote-site designs are standard building blocks in the overall
design. Replication of the individual building blocks provides an easy way to scale the network and allows for a
consistent deployment method.

IP Multicast
IP Multicast allows a single IP data stream to be replicated by the infrastructure (routers and switches) and sent
from a single source to multiple receivers. IP Multicast is much more efficient than multiple individual unicast
streams or a broadcast stream that would propagate everywhere. IP telephony music on hold (MOH) and IP video
broadcast streaming are two examples of IP Multicast applications.
To receive a particular IP Multicast data stream, end hosts must join a multicast group by sending an Internet
Group Management Protocol (IGMP) message to their local multicast router. In a traditional IP Multicast design, the
local router consults another router in the network that is acting as a rendezvous point to map the receivers to
active sources so that they can join their streams.
The rendezvous point is a control-plane operation that should be placed in the core of the network or close to the
IP Multicast sources on a pair of layer 3 switches or routers. IP Multicast routing begins at the distribution layer if
the access layer is layer 2 and provides connectivity to the IP Multicast rendezvous point. In designs without a core
layer, the distribution layer performs the rendezvous point function.
This design is fully enabled for a single global scope deployment of IP Multicast. The design uses an Anycast
rendezvous point implementation strategy. This strategy provides load sharing and redundancy in Protocol
Independent Multicast sparse mode (PIM SM) networks. Two rendezvous points share the load for source
registration and the ability to act as hot backup routers for each other.
The benefit of this strategy from the WAN perspective is that all IP routing devices within the WAN use an identical
configuration referencing the Anycast rendezvous points. IP PIM SM is enabled on all interfaces, including
loopbacks, VLANs and subinterfaces.
Deploying Transport-Independent Topology
Design Overview
DMVPN Hub Routers
The most critical devices are the WAN routers that are responsible for reliable IP forwarding and QoS. The amount
of bandwidth required at the WAN-aggregation site determines which model of router to use. The choice of whether
to implement a single router or dual router is determined by the number of DMVPN clouds that are required in
order to provide connections to all of the remote sites.
Cisco ASR 1000 Series Aggregation Services Routers represent the next-generation, modular, services-integrated
Cisco routing platform. They are specifically designed for WAN aggregation, with the flexibility to support a wide
range of 3- to 130-Mpps (millions of packets per second) packet-forwarding capabilities, 2.5- to 200-Gbps system
bandwidth performance, and scaling.
The Cisco ASR 1000 Series is fully modular from both hardware and software perspectives, and the routers have
all the elements of a true carrier-class routing product that serves both enterprise and service provider networks.
This design uses the following routers as DMVPN hub routers:
Cisco ASR 1002-X Router configured with an Embedded Services Processor (ESP) default bandwidth of
5 Gbps upgradable with software licensing options to 10 Gbps, 20 Gbps, and 36 Gbps
Cisco ASR 1002 Router configured with an Embedded Services Processor 5 (ESP5)
Cisco ASR 1001 Router fixed configuration with a 2.5 Gbps ESP

All of the design models can be constructed using any of the DMVPN hub routers listed in Table 1. You should
consider the following: the forwarding performance of the router using an Ethernet WAN deployment with broad
services enabled, the router's alignment with the suggested design model, and the number of remote sites.
Table 1. DMVPN Hub Router Options
Option ISR 4451-X ASR 1001 ASR 1002 ASR 1002-X
Ethernet WAN with IWAN services 430 Mbps 250 Mbps 500 Mbps 500 Mbps - 1.5 Gbps
Software redundancy option No Yes Yes Yes
Redundant power supply Optional Default Default Default
Supported design models All All All All
Suggested number of remote sites 100 100 250 250+
Remote Sites - DMVPN Spoke Router Selection
There are many factors to consider in the selection of the WAN remote-site routers. Among those, and key to the
initial deployment, is the ability to process the expected amount and type of traffic. You also need to make sure that
you have enough interfaces, enough module slots, and a properly licensed Cisco IOS Software image that
supports the set of features that is required by the topology. Cisco tested multiple integrated service router models
as DMVPN spoke routers. Expected performance is shown in Table 2
Table 2. WAN Remote-Site Cisco Integrated Service Router Options
Option 892fsp 2911+ISM 2951+ISM 3945+ISM 4451-X
Ethernet WAN with IWAN services
1
18 Mbps 36 Mbps 85 Mbps 136 Mbps 430Mbps
On-board GE ports
2
10 3 3 3 4
Service module slots 0 1 2 4 2
Redundant power supply option No No No Yes Yes
Notes:
1. The performance numbers are conservative numbers obtained when the router is passing IMIX traffic with
IWAN services (DMVPN, EIGRP, NBAR2/DSCP Marking, PfRv2, H-QOS) enabled.
2. A single-router, dual-link remote site requires four router interfaces when using a port channel to connect to an
access or distribution layer. Add the EHWIC-1GE-SFP-CU to the Cisco 2900 and 3900 Series Integrated
Services Routers (ISRs) in order to provide the additional WAN-facing interface.
The DMVPN spoke routers at the WAN remote sites connect to the Internet directly through a router interface.
More details about the security configuration of the remote-site routers connected to the Internet are discussed
later in this guide. The single link DMVPN remote site is the basic of building blocks for any remote location.
The IP routing is straightforward and can be handled entirely by static routes at the WAN-aggregation site and
static default routes at the remote site in conjunction with FVRF configuration on the WAN interface. This ensures
separation of the routing plane used for IPSec tunnel setup in the FVRF and the data plane for carrying user traffic
inside the DMVPN tunnel. The dual-router, dual-link design continues to improve upon the level of high availability
for the site. This design can tolerate the loss of the primary router and traffic can be rerouted using the secondary
router (through the alternate path). Figure 5 outlines the IWAN Hybrid spoke design.

Figure 5. IWAN Hybrid Spoke Design

For IWAN Hybrid Transport Independent design, DMVPN is deployed across MPLS and Internet Transport. This
greatly simplifies the routing by using a single routing protocol, creating a succine routing domain. The design
provides active-active WAN paths that take full advantage of DMVPN for consistent IPsec overlay. The MPLS and
Internet connections can be terminated on a single router, or terminated on two separate routers for additional
resiliency. The same design can be used over MPLS, Internet, or 3G/4G transports, making the design tranport-
independent. The single-router and dual-router options are shown respectively in Figure 5.
VRFs and Front Door VRF
Virtual Routing and Forwarding (VRF) is a technology used in computer networks that allows multiple instances of
a routing table to co-exist within the same router at the same time. Because the routing instances are independent,
you can use the same or overlapping IP addresses without conflicting with each other.
You can implement VRF in a network device by having distinct routing tables, also known as Forwarding
Information Bases (FIBs), one per VRF. Alternatively, a network device may have the ability to configure different
virtual routers, where each one has its own FIB that is not accessible to any other virtual router instance on the
same device.
The simplest form of VRF implementation is VRF Lite. In this implementation, each router within the network
participates in the virtual routing environment on a peer-by-peer basis. VRF Lite configurations are only locally
significant.
The IP routing policy used in this design for the WAN remote sites does not allow direct Internet access for web
browsing or other uses; any remote-site hosts that access the Internet must do so through the Internet edge at the
primary site. The end hosts require a default route for all Internet destinations; however, this route must force traffic
across the primary or secondary WAN transport DMVPN tunnels. This requirement conflicts with the more general
VPN spoke router requirement for an Internet-facing default route to bring up the VPN tunnel.

The multiple default routes conundrum is solved through the use of VRFs on the router. A router can have multiple
routing tables that are kept logically separate on the device. This separation is similar to a virtual router from the
forwarding plane perspective. The global VRF corresponds to the traditional routing table, and additional VRFs are
given names and route descriptors. Certain features on the router are VRF-aware, including static routing and
routing protocols, interface forwarding, and IPsec tunneling. This set of features is used in conjunction with DMVPN
to permit the use of multiple default routes for both the DMVPN hub routers and DMVPN spoke routers. This
combination of features is referred to as front-door VRF (FVRF); because the VRF faces the Internet and the router
internal interfaces and the mGRE tunnel all remain in the global VRF (see Figure 6). More technical details
regarding FVRF can be found in the Technical Feature Supplement appendix.
Figure 6. Front-Door VRF (FVRF)

Design Details
The DMVPN hub routers connect to a resilient switching device in the distribution layer and in the DMZ. The
DMVPN routers use EtherChannel connections consisting of two port bundles. This design provides both resiliency
and additional forwarding performance. Additional forwarding performance can be accomplished by increasing the
number of physical links within an EtherChannel.

The DMVPN hub routers must have sufficient IP-routing information to provide end-to-end reachability. Maintaining
this routing information typically requires a routing protocol. Cisco Enhanced Interior Gateway Routing Protocol
(EIGRP) is used for this purpose. Multiple EIGRP processes are used - one for internal routing on the LAN
(EIGRP-100) and a separate one for the DMVPN networks (EIGRP-IWAN). The primary reason for the separate
EIGRP processes is to ensure compatibility with the route selection process at the WAN-aggregation site when
deploying other Cisco Validated Deign WAN designs. This method ensures DMVPN learned routes appear as
EIGRP external routes after they are redistributed into the EIGRP-100 process used on the campus LAN.
An example of the routing details for the IWAN Hybrid design model is shown in figure 7.
Figure 7. IWAN Hybrid Design - Routing Details

At the WAN-aggregation site, the Internet DMVPN router is connected to the DMZ-VPN that provides Internet
connectivity. The Internet DMVPN hub router uses FVRF and has a static default route with the INET-PUBLIC VRF
pointing to the firewall DMZ interface. On the MPLS DMVPN router, it also uses FVRF and has a static default
route with the INET-MPLS pointing to the provider edge router interface.
EIGRP
IWAN solution uses EIGRP as the primary routing protocol because it is easy to configure, and does not require a
large amount of planning. EIGRP has flexible summarization and filtering capability, and can scale to large
networks. As networks grow, the number of IP prefixes or routes in the routing tables grows as well. By performing
IP summarization, you can reduce the amount of bandwidth, processor, and memory necessary to carry large route
tables, and reduce convergence time associated with a link failure.

In this design, EIGRP process IWAN is the primary EIGRP process used for the DMVPN tunnels and is referred to
as EIGRP-IWAN. Unlike the classic configuration, where an autonomous system number is used for EIGRP, in
IWAN design the EIGRP name mode is used for creating an EIGRP routing instance.
Router eigrp <virtual-instance-name>
There are several benefits for running EIGRP name mode:
Name mode allows you to create a single instance of EIGRP that can be used for all address family types,
such as IPv4, IPv6, VRFs, SAF, etc.
It supports multiple VRFs and is limited only by available system resources.
It is a single place for all commands needed to completely define an instance. You no longer have to split
configuration between the router EIGRP and the interface configuration in the classic mode.
Tech Tip
You can use the eigrp upgrade-cli command to convert an existing EIGRP configuration from classic mode to
named mode. If multiple classic mode configurations exist, you must use this command per EIGRP autonomous
system number in classic mode.
Encryption and MTU
The encryption process creates an additional header and trailer on to the original IP packet. The original packet,
payload, and header are encrypted and then encapsulated with a new header (or multiple headers) and transmitted
across the network. The additional headers introduce a certain amount of overhead to the overall packet length.
Table 3 highlights the packet overhead associated with encryption based on the additional headers required for
various combinations of IPsec and GRE.
Table 3. Overhead Associated with IPsec and GRE
Encapsulation Overhead
GRE only 24 bytes
IPsec (transport mode) 36 bytes
IPsec (tunnel mode) 52 bytes
IPsec (transport mode) plus GRE 60 bytes
IPsec (tunnel mode) plus GRE 76 bytes
There is a maximum transmission unit (MTU) parameter for every link in an IP network and typically the MTU is
1500 bytes. IP packets larger than 1500 bytes must be fragmented when transmitted across these links.
Fragmentation is not desirable and can impact network performance. To avoid fragmentation, the original packet
size plus overhead must be 1500 bytes or less, which means that the sender must reduce the original packet size.
To account for other potential overhead, Cisco recommends that you configure tunnel interfaces with a 1400 byte
MTU.
There are dynamic methods for network clients to discover the path MTU, which allow the clients to reduce the size
of packets they transmit. However, in many cases, these dynamic methods are unsuccessful, typically because
security devices filter the necessary discovery traffic. This failure to discover the path MTU creates the need for a
method that can reliably inform network clients of the appropriate packet size. The solution is to implement the
ip tcp adjust-mss [size] command on the WAN routers, which influences the TCP maximum segment size (MSS)
value reported by end hosts.

The MSS defines the maximum amount of data that a host is willing to accept in a single TCP/IP datagram. The
MSS value is sent as a TCP header option only in TCP SYN segments. Each side of a TCP connection reports its
MSS value to the other side. The sending host is required to limit the size of data in a single TCP segment to a
value less than or equal to the MSS reported by the receiving host.
The IP and TCP headers combine for 40 bytes of overhead, so the typical MSS value reported by network clients
will be 1460. This design includes encrypted tunnels with a 1400-byte MTU, so the MSS used by endpoints should
be configured to be 1360 to minimize any impact of fragmentation. In this solution, you implement the ip tcp
adjust-mss 1360 command on all tunnel interfaces.
DMVPN
This solution uses the Internet for WAN transport. For data security and privacy concerns any site-to-site traffic that
traverses the Internet must be encrypted. Multiple technologies can provide encryption, but the method that
provides the best combination of performance, scale, application support, and ease of deployment is DMVPN.
The IWAN independent transport solution requires a DMVPN dual-cloud design, each with a single hub router. The
DMVPN routers use tunnel interfaces that support IP unicast as well as IP multicast and broadcast traffic, including
the use of dynamic routing protocols. After the initial spoke-to-hub tunnel is active, it is possible to create dynamic
spoke-to-spoke tunnels when site-to-site IP traffic flows require it.
The information required by a spoke to set up dynamic spoke-to-spoke tunnel (Figure 8) is provided by the Next
Hop Resolution Protocol (NHRP). Spoke-to-spoke tunnels allow for the optimal routing of traffic between locations
without traffic hair-pinning at the hub. Idle spoke-to-spoke tunnels gracefully time out after a period of inactivity.
It is common for a firewall to be placed between the DMVPN hub routers and the Internet. In many cases, the
firewall may provide Network Address Translation (NAT) from an internal RFC-1918 IP address (such as
10.4.128.33) to an Internet-routable IP address. The DMVPN solution works well with NAT but requires the use of
IPsec transport mode to support a DMVPN hub behind static NAT.
DMVPN requires the use of Internet Security Association and Key Management Protocol (ISAKMP) keep alive for
Dead Peer Detection (DPD), which is essential to facilitate fast reconvergence and for spoke registration to
function properly in case a DMVPN hub is reloaded. This design enables a spoke to detect that an encryption peer
has failed and that the ISAKMP session with that peer is stale, which then allows a new one to be created. Without
DPD, the IPsec security association must time out (the default is 60 minutes) and when the router cannot
renegotiate a new security association, a new ISAKMP session is initiated. The maximum wait time is
approximately 60 minutes.

Figure 8. DMVPN Dynamic Spoke-Spoke Tunnel

One of the key benefits of the DMVPN solution is that the spoke routers can use dynamically assigned addresses,
often using DHCP from an Internet provider. The spoke routers can take advantage of an Internet default route for
reachability to the hub routers and also other spoke addresses.
The DMVPN hub routers have static IP addresses assigned to their public-facing interfaces. This configuration is
essential for proper operation as each of the spoke routers have these IP addresses embedded in their
configurations. Figure 9 displays a hub technology and associated addresses.

Deployment Details
Figure 9. Hub Topology and Addresses

The procedures in this section provide examples for some settings (Table 4). The actual settings and values that
you use are determined by your current network configuration. This process is used for the Dual DMVPN design
(repeat for each DMVPN hub router).
Table 4. Examples of Parameters Used in the Deployment
Host Name Loopback IP Address Port Channel IP Address
VPN-ASR1002-1 10.4.32.241/32 10.4.32.2/28
VPN-ASR1002-2 10.4.32.243/32 10.4.32.3/28

DMVPN Hub Router Configuration
Procedure 1: Configure the WAN Aggregation Routers
This process applies to both routers connecting to the MPLS network customer edge router and the router
connecting to the Internet. The example shown includes the configuration example for Internet DMVPN router, but
the same procedure is reused for MPLS DMVPN router.
Reader Tip
This process assumes that the distribution switch has already been configured following the guidance in the
Campus Wired LAN Design Guide. Only the procedures required to support the integration of the WAN
aggregation router into the deployment are included.
The LAN distribution switch is the path to the organizations main campus and data center. A layer 3 port-channel
interface connects to the distribution switch to the WAN aggregation router and the internal routing protocol peers
across this interface.
Tech Tip
As a best practice, use the same channel numbering on both sides of the link where possible.
Within this design, there are features and services that are common across all WAN aggregation routers. These
are system settings that simplify and secure the management of the solution.
Step 1. Configure the common features and services for the routers, including:
Device host name for ease of identifying the device
The local login account and password, which provides basic access
Network Time Protocol (NTP) to synchronize system clock network devices
service timestamps debug datetime msec localtime
service timestamps log datetime msec localtime
service password-encryption
!
hostname VPN-ASR1002-1
!
username admin password c1sco123
enable secret c1sco123
aaa new-model
!
clock timezone PST -8
clock summer-time PDT recurring
ntp server 10.4.48.17
Step 2. Configure device management protocols
Use Secure Shell (SSH) for secure management of the network device
Specify the transport preferred none on vty lines to prevent errant connection attempts from the mistyped
command-line interface (CLI) commands
Enable Simple Network Management Protocol version 2c (SNMPv2c) for the read-only and read-write
community string

ip domain-name cisco.local
!
ip ssh version 2
no ip http server
ip http secure-server
snmp-server community cisco RO
snmp-server community cisco123 RW
!
line vty 0 15
transport input ssh
transport preferred none
line con 0
logging synchronous
(Optional) In networks where network operational support is centralized you can increase network security by using
an access list to limit the networks that can access your device. In this example, only devices on the 10.4.48.0/24
network will be able to access the device through SSH or SNMP.
access-list 55 permit 10.4.48.0 0.0.0.255
line vty 0 15
access-class 55 in
!
snmp-server community cisco RO 55
snmp-server community cisco123 RW 55
Tech Tip
If you configure an access list on the vty interface you may lose the ability to use SSH to log in from one router to the next for hop-by-hop
troubleshooting.
Step 3. (Optional) Configure centralized user authentication.
As networks scale in the number of devices, Cisco recommends the best practice to use a centralized
Authentication, Authorization, and Accounting (AAA) service to reduce operational tasks per device and provides
an audit log of user access for security compliance and root-cause analysis. When AAA is enabled for access
control, all management access to the network infrastructure devices (SSH and HTTPS) is controlled by AAA.
TACACS+ is the primary protocol used to authenticate management logins on the infrastructure devices to the AAA
server. A local AAA user database is also defined on each network infrastructure device to provide a fallback
authentication source in case the centralized TACACS+ server is unavailable.
tacacs server TACACS-SERVER-1
address ipv4 10.4.48.15
key SecretKey
!
aaa group server tacacs+ TACACS-SERVERS
server name TACACS-SERVER-1
!
aaa authentication login default group TACACS-SERVERS local
aaa authorization exec default group TACACS-SERVERS local

aaa authorization console
ip http authentication aaa
Step 4. Configure an in-band management interface.
The loopback interface is a logical interface that is always reachable as long as the device is powered on and any
IP interface is reachable to the network. Because of this capability, the loopback address is the best way to
manage the switch in band. Layer 3 processes and features are also bound to the loopback interface to ensure
process resiliency.
The loopback address is commonly a host address with a 32-bit address mask. Allocate the loopback address from
the IP address block that the distribution switch summarizes to the rest of the network.
interface Loopback 0
ip address 10.4.32.243 255.255.255.255
ip pim sparse-mode
The ip pim sparse-mode command will be explained further in the process.
Bind the device processes for SNMP, SSH, Protocol Independent Multicast (PIM), TACACS+ and Network Time
Protocol (NTP) to the loopback interface address for optimal resiliency:
snmp-server trap-source Loopback0
ip ssh source-interface Loopback0
ip pim register-source Loopback0
ip tacacs source-interface Loopback0
ntp source Loopback0
Step 5. Enable IP Multicast routing.
from a single source to multiple receivers. Using IP Multicast is much more efficient than using multiple individual
unicast streams or a Broadcast stream that would propagate everywhere. IP telephony MOH and IP video
To receive a particular IP Multicast data stream, end hosts must join a multicast group by sending an Internet
Group Management Protocol (IGMP) message to their local multicast router. In a traditional IP Multicast design, the
local router consults another router in the network that is acting as a rendevous point to map the receivers to active
sources so they can join their streams.
In this design, which is based on sparse mode multicast operation, auto-rendevous point is used to provide a
simple, yet scalable way to provide a highly resilient rendevous point environment.
Enable IP Multicast routing on the platforms in the global configuration mode.
ip multicast-routing
The Cisco ASR 1000 Series router requires the distributed keyword.
ip multicast-routing distributed
Every layer 3 switch and router must be configured to discover the IP Multicast rendevous point with autorp. Use
the ip pim autorp listener command to allow for discovery across sparse mode links. This configuration provides
for future scaling and control of the IP Multicast environment and can change based on network needs and design.
ip pim autorp listener

All layer 3 interfaces in the network must be enabled for sparse mode multicast operation.
ip pim sparse-mode
Procedure 2: Configure VRF Lite
Table 5. VRF and Crypto Parameters Example for DMVPN Hub
Parameter DMVPN-INTERNET Cloud DMVPN-MPLS Cloud
vrf INET-PUBLIC INET-MPLS
rd 65512:1 65512:2
crypto keyring DMVPN-KEYRING1 DMVPN-KEYRING2
crypto isakmp profile FVRF-ISAKMP-INET-PUBLIC FVRF-ISAKMP-INET-MPLS
crypto ipsec profile DMVPN-INTERNET DMVPN-MPLS
Tunnel interface Tunnel 10 Tunnel 20
A dedicated Internet-facing and MPLS-facing VRF is created to support FVRF for DMVPN-Internet cloud and
DMVPN-MPLS cloud respectively. The VRF name is arbitrary, but it is useful to select a name that describes the
VRF. This design uses VRF Lite so that the route distinguisher value can be chosen arbitrarily. It is a best practice
to use the same VRF and route distinguisher combination across multiple devices when using VRFs in a similar
manner. However, this convention is not strictly required.
ip vrf INET-PUBLIC
rd 65512:1
Tech Tip
Command Reference:
A route distinguisher is either access service network (ASN)-related (composed of an ASN and an arbitrary
number) or IP-address-related (composed of an IP address and an arbitrary number)
You can enter a route distinguisher in either of these formats:
16-bit autonomous-system-number: your 32-bit number
For example, 65512:1
32-bit IP address: your 16-bit number
For example, 192.168.122.15:1
Procedure 3: Connect WAN Aggregation Router to the Service Provider
Option 1: Connect to Internet DMZ
The DMVPN hub requires a connection to the Internet, and in this design the DMVPN hub is connected through a
Cisco AS A5500 Adaptive Security Appliance using a DMZ interface specifically created and configured for a VPN
termination router.
Step 1. Enable the interface, select the VRF, and assign the IP address.
The IP address that you use for the Internet-facing interface of the DMVPN hub router must be an Internet routable
address. There are two possible methods to accomplish this task:
Assign a routable IP address directly to the router
Assign a non-routable RFC-1918 address directly to the router and use a static NAT on the Cisco ASA 5500
to translate the router IP address to a routable IP address.
This design assumes that the Cisco ASA 5500 is configured for static NAT for the DMVPN hub router.

The DMVPN design is using FVRF so this interface must be placed into the VRF configured in the previous
procedure.
interface GigabitEthernet0/0/3
ip vrf forwarding INET-PUBLIC
ip address 192.168.18.10 255.255.255.0
no shutdown
Step 2. Configure the VRF specific default routing
The VRF created for FVRF must have its own default route to the Internet. This default route points to the ASA
5500 DMZ interface IP address. Figure 10 offers different views for DMZ connection.
ip route vrf INET-PUBLIC 0.0.0.0 0.0.0.0 192.168.18.1
Figure 10. Physical and Logical Views for DMZ Connection

Option 2: Connect to MPLS Provider Edge Router
Step 1. Assign the interface bandwidth.
The bandwidth value should correspond to the actual interface speed. Or, if you are using a subrate service, use
the policed rate from the carrier.
The example shows a Gigabit interface (1000 Mbps) with a subrate of 300 Mbps.
bandwidth 300000

Tech Tip
Command reference:
bandwidth kbps
(300 Mbps = 300,000 kbps)
Step 2. Assign the IP address and netmask of the WAN interface.
The IP addressing used between customer edge and provider edge routers must be negotiated with your MPLS
carrier. Typically, a point-to-point netmask of 255.255.255.252 is used.
ip vrf forwarding INET-MPLS
ip address 192.168.3.1 255.255.255.252
no shutdown
Step 3. Administratively enable the interface and disable Cisco Discovery Protocol (CDP).
We do not recommend the use of CDP on external interfaces.
no cdp enable
Step 4. Configure the VRF-Specific Default Routing
The VRF created for FVRF must have its own default route to the MPLS network. This default route points to the
MPLS PE interface IP address.
ip route vrf INET-MPLS 0.0.0.0 0.0.0.0 192.168.3.2
Procedure 4: Configure ISAKMP and IPsec
Step 1. Configure the crypto keyring.
The crypto keyring defines a pre-shared key (or password) valid for IP sources that are reachable within a
particular VRF. This key is a wildcard pre-shared key if it applies to any IP source. A wildcard key is configured
using the 0.0.0.0 0.0.0.0 network/mask combination.
crypto keyring DMVPN-KEYRING vrf INET-PUBLIC
pre-shared-key address 0.0.0.0 0.0.0.0 key cisco123
Step 2. Configure the ISAKMP policy.
The ISAKMP policy for DMVPN uses:
Advanced Encryption Standard (AES) with a 256-bit key
Secure Hash Standard (SHA)
Authentication by Pre-Shared Key (PSK)
Diffie-Hellman group: 2
crypto isakmp policy 10
encr aes 256
hash sha
authentication pre-share
group 2

Step 3. Create the ISAKMP profile.
The ISAKMP profile creates an association between an identity address, a VRF, and a crypto keyring. A wildcard
address within a VRF is referenced with 0.0.0.0.
crypto isakmp profile FVRF-ISAKMP-INET-PUBLIC
keyring DMVPN-KEYRING
match identity address 0.0.0.0 INET-PUBLIC
Step 4. Define the IPsec transform set.
A transform set is an acceptable combination of security protocols, algorithms, and other settings to apply to IPsec-
protected traffic. Peers agree to use a particular transform set when protecting a particular data flow.
The IPsec transform set for DMVPN uses the following:
Encapsulating Security Payload (ESP) with the 256-bit AES encryption algorithm
ESP with the SHA (Hashed Message Authentication Code [HMAC] variant) authentication algorithm
Because the DMVPN hub router is behind a NAT device, the IPsec transform must be configured for transport
mode.
crypto ipsec transform-set AES256/SHA/TRANSPORT esp-aes 256 esp-sha-hmac
mode transport
Step 5. Create the IPSec profile.
The IPsec profile creates an association between an ISAKMP profile and an IPsec transform-set.
crypto ipsec profile DMVPN-INTERNET
set transform-set AES256/SHA/TRANSPORT
set isakmp-profile FVRF-ISAKMP-INET-PUBLIC
Procedure 5: Configure the mGRE Tunnel
Table 6. DMVPN Tunnel Parameters
DMVPN Cloud Tunnel Interface Tunnel IP Address EIGRP AS NHRP Network ID
Internet DMVPN Tunnel 10 10.4.34.1/23 IWAN 101
MPLS DMVPN Tunnel 20 10.4.36.1/23 IWAN 102
Step 1. Configure basic DMVPN tunnel interface settings.
Tunnel interfaces are created as they are configured. The tunnel number is arbitrary, but it is best to begin tunnel
numbering at 10 or above, because other features deployed in this design may also require tunnels and they may
select lower numbers by default.
The bandwidth setting should be set to match the Internet bandwidth of the respective primary or secondary
carrier.
Configure the IP MTU to 1400 and the ip tcp adjust-mss to 1360. There is a 40-byte difference, which corresponds
to the combined IP and TCP header length.
interface Tunnel10
bandwidth 10000
ip address 10.4.34.1 255.255.254.0
no ip redirects
ip mtu 1400

ip tcp adjust-mss 1360
DMVPN uses multipoint GRE (mGRE) tunnels. This type of tunnel requires a source interface only. Use the same
source interface that you use to connect to the Internet. Set the tunnel vrf command to the VRF defined previously
for FVRF.
Enabling encryption on this interface requires the application of the IPsec profile configured in the previous
procedure.
interface Tunnel10
tunnel source GigabitEthernet0/0/3
tunnel mode gre multipoint
tunnel vrf INET-PUBLIC
tunnel protection ipsec profile DMVPN-INTERNET
Step 2. Configure NHRP.
NHRP is used by remote routers for setting up dynamic spoke-to-spoke tunnels.
NHRP requires all devices within a DMVPN cloud to use the same network ID and authentication key. The NHRP
cache holdtime should be configured to 600 seconds.
EIGRP (configured in the following procedure) relies on a multicast transport and requires NHRP to automatically
add routers to the multicast NHRP mappings.
interface Tunnel10
ip nhrp authentication cisco123
ip nhrp map multicast dynamic
ip nhrp network-id 101
ip nhrp holdtime 600
Step 3. Enable PIM non-broadcast multiple access (NBMA) mode for the DMVPN tunnel.
Spoke-to-spoke DMVPN networks present a unique challenge because the spokes cannot directly exchange
information with one another, even though they are on the same logical network. This inability to directly exchange
information can also cause problems when running IP Multicast.
To resolve this issue requires a method where each remote PIM neighbor has its join messages tracked
separately. A router in PIM nonbroadcast multaccess (NBMA) mode treats each remote PIM neighbor as if it were
connected to the router through a point-to-point link.
Tech Tip
Do not enable PIM on the Internet DMZ interface, as no multicast traffic should be requested from this interface.
interface Tunnel10
ip pim sparse-mode
ip pim nbma-mode

Procedure 6: Configure the EIGRP Routing Process
A separate EIGRP process is used over the DMVPN network. The primary reason for the additional process is to
ensure that routes learned from the WAN remotes appear as EIGRP external routes on the WAN distribution
switch. This helps in limiting the EIGRP query domain and increases the network scalability and stability.
Step 1. Enable an EIGRP process for DMVPN.
Configure EIGRP for the DMVPN mGRE interface. Routes from the other EIGRP process are redistributed.
Because the routing protocol is the same, no default metric is required.
The tunnel interface is the only EIGRP interface, and you need to explicitly list its network range.
router eigrp IWAN
!
address-family ipv4 unicast autonomous-system 200
network 10.4.34.0 0.0.1.255
eigrp router-id 10.4.32.243
exit-address-family
!
Step 2. Configure EIGRP properties for the tunnel interface.
Spoke-to-spoke in DMVPN phase 2 networks present a unique challenge because the spokes cannot directly
exchange information with one another, even though they are on the same logical network. This limitation requires
that the DMVPN hub router advertise routes from other spokes on the same network. This advertisement of these
routes would normally be prevented by split horizon, and can be overridden by the no ip split-horizon command
under the EIGRP name mode. In addition, the no next-hop-self command preserves the next hop routing
information for the dynamic spoke-to-spoke tunnel setup.
The EIGRP hello interval is increased to 20 seconds and the EIGRP hold time is increased to 60 seconds to
accommodate up to 500 remote sites on a single DMVPN cloud.
router eigrp IWAN
!
af-interface Tunnel10
hello-interval 20
hold-time 60
no next-hop-self
no split-horizon
exit-af-interface
exit-address-family
!

Step 3. Tag and redistribute the routes.
This design uses route tag to prevent routing loop between the two DMVPN networks. Routes learned over the
MPLS DMVPN network are tagged with 200 while routes learned over the Internet DMVPN are tagged with 100.
The routes are shared between the MPLS DMVPN and Internet DMVPN hub routers to ensure reachability.
However, the tagging scheme ensures the routes learned over MPLS DMVPN path are not readvertised out to
Internet DMVPN spokes and vice versa to prevent routing loops.
It is important to tightly control how routing information is shared between different routing domains; otherwise, it is
possible to experience route flapping, where certain routes are repeatedly installed and withdrawn from the device
routing tables. Proper route control ensures the stability of the routing table.
An inbound distribute list is used on the WAN aggregation routers to tag the routes so they can be differentiated
later on. An outbound distribute list is used to select which routes are allowed to be advertised out. The specific
route tags in use are shown in Table 7. In the route map, a prefix list is used to define what routes are originated
from the data center. In this case, we are allowing default route, the summary route for data center, and the
loopback addresses with 32 network masks.
Table 7. Route Tag Information for DMVPN Hub Router
Tag Route Source Route Map Name Tag Method Action
100 DMVPN over Internet INTERNET-DC1-IN
INTERNET-DC1-OUT
Explicit Tag
200 DMVPN over MPLS MPLS-DC1-IN
MPLS-DC1-OUT
Explicit Tag
This example includes all WAN route sources in the reference design. Depending on the actual design of your
network, you might need to use more tags.
router eigrp IWAN
!
!
topology base
distribute-list route-map INTERNET-DC1-IN in Tunnel10
distribute-list route-map INTERNET-DC1-OUT out Tunnel10
redistribute static
exit-af-topology
exit-address-family
!
ip prefix-list DC1-LOCAL-ROUTES seq 5 permit 0.0.0.0/0
!
route-map INTERNET-DC1-IN deny 10
match ip address prefix-list DC1-LOCAL-ROUTES
!
route-map INTERNET-DC1-IN permit 20
set tag 100
!

route-map INTERNET-DC1-OUT permit 10
match ip address prefix-list DC1-LOCAL-ROUTES
set tag 100
!
route-map INTERNET-DC1-OUT permit 20
description Advertise routes learned from MPLS DMVPN cloud
match tag 100
!
Procedure 7: Configure the MPLS DMVPN Hub
Repeat Procedures 1 - 6 in the DMVPN Hub Router Configuration section to provision the MPLS DMVPN hub
router. Refer to Table 6 and replace the attributes used for Internet with attributes for MPLS as listed in the table.
Remote-Site Dual DMVPN Spokes Configuration

Use this set of procedures for remote site with dual router, dual link design.
This set of procedures is for the initial configuration of a remote site DMVPN spoke router. You can use these
procedures when you configure each router of the dual-router, dual-link design.
Procedure 1: Configure the WAN Remote Router
Within this design, there are features and services that are common across all WAN remote site routers. These are
system settings that simplify and secure the management of the solution.
The local login account and password to provide basic access
!

hostname [hostname]
!
aaa new-model
!
ntp update-calendar
!
Step 2. Configure device management protocols.
CLI commands
Enable SNMPv2c for read-only and read-write community strings
!
ip ssh version 2
no ip http server
!
line vty 0 15
transport input ssh
line con 0
logging synchronous
line vty 0 15
access-class 55 in
!
Tech Tip
If you configure an access list on the vty interface you may lose the ability to use SSH to log in from one router to
the next for hop-by-hop troubleshooting.

As networks scale in the number of devices to maintain, it poses an operational burden to maintain local user
accounts on every device. A centralized AAA service reduces operational tasks per device and provides an audit
log of user access for security compliance and root-cause analysis. When AAA is enabled for access control, all
management access to the network infrastructure devices (SSH and HTTPS) is controlled by AAA.
key SecretKey
!
!
manage the switch in-band. Layer 3 processes and features are also bound to the loopback interface to ensure
process resiliency.
a unique network range that is not part of any other internal network summary range.
ip address [ip address] 255.255.255.255
ip pim sparse-mode
The ip pim sparse-mode command will be explained in the next step.
Bind the device processes for SNMP, SSH, PIM, TACACS+, and NTP to the loopback interface address for optimal
resiliency.

Step 5. Configure IP Multicast routing.
from a single source to multiple receivers. Using IP Multicast is much more efficient than using multiple individual
unicast streams or a broadcast stream that would propagate everywhere. IP telephony MOH and IP video
To receive a particular IP Multicast data stream, end hosts must join a multicast group by sending an IGMP
message to their local multicast router. In a traditional IP Multicast design, the local router consults another router
in the network that is acting as a rendevous point to map the receivers to active sources so they can join their
streams.
In this design, which is based on sparse mode multicast operation, auto-rendevous point is used to provide a
simple yet scalable way to provide a highly resilient rendevous point environment.
ip pim sparse-mode
Table 8. VRF and Crypto Parameters Example for Dual Routers Branch
rd 65512:1 65512:2
DMVPN-MPLS cloud respectively. The VRF name is arbitrary, but it is useful to select a name that describes the
to use the same VRF/ route distinguisher combination across multiple devices when using VRFs in a similar
ip vrf INET-PUBLIC
rd 65512:1

Tech Tip
Command Reference:
A route distinguisher is either ASN-related (composed of an ASN and an arbitrary number) or IP-address-related
(composed of an IP address and an arbitrary number).
For example, 65512:1
For example, 192.168.122.15:1
Procedure 3: Connecting to the Service Provider
Option 1: Receive a DHCP address from the Internet Service Provider
The remote sites that are using DMVPN can use either static or dynamically assigned IP addresses. Cisco tested
the design with a DHCP assigned external address, which also provides a dynamically configured default route.
The DMVPN spoke router connects directly to the Internet without a separate firewall. This connection is secured in
two ways. Because the Internet interface is in a separate VRF, no traffic can access the global VRF except traffic
sourced through the DMVPN tunnel. This design provides implicit security. Additionally, an IP access list permits
only the traffic required for an encrypted tunnel, as well as DHCP and various Internet Control Message Protocols
(ICMPs) for troubleshooting.
Step 1. Enable the interface, select VRF, and enable DHCP.
The DMVPN design uses FVRF so this interface must be placed into the VRF configured in the previous
procedure.
interface GigabitEthernet0/0
ip address dhcp
no cdp enable
no shutdown
Do not enable PIM on this interface because no multicast traffic should be requested from this interface.
Step 2. Configure and apply the access list.
To secure the connectivity and prevent unauthorized access to the router, an access list must be applied to the
Internet-facing interface. The IP access list must permit the protocols specified in Table 9. The access list is
applied inbound on the WAN interface so filtering is done on traffic destined to the router.
Table 9. Required DMVPN Protocols
Name Protocol Usage
non500-isakmp UDP 4500 IPsec via NAT-T
isakmp UDP 500 ISAKMP
esp IP 50 IPsec
bootpc UDP 68 DHCP

Example
ip access-group ACL-INET-PUBLIC in
ip access-list extended ACL-INET-PUBLIC
permit udp any any eq non500-isakmp
permit udp any any eq isakmp
permit esp any any
permit udp any any eq bootpc
The additional protocols listed in Table 10 may assist in troubleshooting, but are not explicitly required to allow
DMVPN to function properly.
Table 10. Optional Protocols DMVPN Spoke Router
Name Protocol Usage
icmp echo ICMP Type 0, Code 0 Allow remote pings
icmp echo-reply ICMP Type 8, Code 0 Allow ping replies (from our requests)
icmp ttl-exceeded ICMP Type 11, Code 0 Allow traceroute replies (from our requests)
icmp port-unreachable ICMP Type 3, Code 3 Allow traceroute replies (from our requests)
UDP high ports UDP > 1023, TTL=1 Allow remote traceroute
The additional optional entries for an access list to support ping are:
permit icmp any any echo
permit icmp any any echo-reply
The additional optional entries for an access list to support traceroute are:
permit icmp any any ttl-exceeded ! for traceroute (sourced)
permit icmp any any port-unreachable ! for traceroute (sourced)
permit udp any any gt 1023 ttl eq 1 ! for traceroute (destination)
Option 2: Static Address and Static Routing with Service Provider
A static IP address is assigned to the WAN interface connecting to the service provider. The interface is associated
with a VRF. This simplifies the routing and allows the use of a default route for the FVRF for the WAN interface.
Step 1. Enable the interface, select VRF, and assign an IP address.
ip address 192.168.3.129 255.255.25.252
no cdp enable
no shutdown
Step 2. Configure static default route in the FVRF to the service provider.
ip route vrf INET-MPLS 0.0.0.0 0.0.0.0 192.168.3.130

The crypto keyring defines a PSK (or password) valid for IP sources reachable within a particular VRF. This key is
a wildcard pre-shared key if it applies to any IP source. A wildcard key is configured using the 0.0.0.0 0.0.0.0
network/mask combination.
crypto keyring DMVPN-KEYRING1 vrf INET-PUBLIC
Step 2. Configure the ISAKMP policy and dead peer detection.
AES with a 256-bit key
Authentication by PSK
DPD is enabled with keepalive intervals sent at 40-second intervals with a five-second retry interval, which is
considered to be a reasonable setting to detect a failed hub.
encr aes 256
hash sha
group 2
!
crypto isakmp keepalive 40 5
keyring DMVPN-KEYRING1
The IPsec transform set for DMVPN uses:
ESP with the 256-bit AES encryption algorithm
ESP with the SHA (HMAC variant) authentication algorithm
Since the DMVPN hub router is behind a NAT device, the IPsec transform must be configured for transport mode.
mode transport

Step 5. Create the IPsec profile.
Step 1. Configure basic DMVPN tunnel interface settings.
numbering at 10 or above because other features deployed in this design may also require tunnels and they may
The bandwidth setting should be set to match the Internet bandwidth.
Configure the IP MTU to 1400 and the ip tcp adjust-mss to 1360. There is a 40-byte difference, which
corresponds to the combined IP and TCP header length.
interface Tunnel10
bandwidth [bandwidth (kbps)]
ip address [IP address] [netmask]
no ip redirects
ip mtu 1400
DMVPN uses multipoint GRE (mGRE) tunnels. This type of tunnel requires a source interface only. The source
interface should be the same interface used to connect to the Internet. The tunnel vrf command should be set to
the VRF defined previously for FVRF.
procedure.
interface Tunnel10
tunnel source GigabitEthernet0/0
NHRP is used by remote routers to determine the tunnel destinations for peers attached to the mGRE tunnel.
The spoke router requires several additional configuration statements to define the NHRP server (NHS) and NHRP
map statements for the DMVPN hub router mGRE tunnel IP address. EIGRP (configured in Procedure 6) relies on
a multicast transport. Spoke routers require the NHRP static multicast mapping.
The value used for the NHS is the mGRE tunnel address for the DMVPN hub router. The map entries must be set
to the outside NAT value of the DMVPN hub, as configured on the Cisco ASA 5500. This design uses the values
shown in Table 11.

Parameter Value Parameter Value
DMVPN cloud INTERNET MPLS
VRF INET-PUBLIC INET-MPLS
DMVPN hub public address (actual) 192.168.18.10 192.168.3.1
DMVPN hub public address (externally routable after NAT) 172.16.130.1 N/A
Tunnel IP address (NHS) 10.4.34.1 10.4.36.1
Tunnel number 10 20
NHRP network ID 101 102
cache holdtime should be configured to 600 seconds.
This design supports DMVPN spoke routers that receive their external IP addresses through DHCP. It is possible
for these routers to acquire different IP addresses after a reload. When the router attempts to register with the
NHRP server, it may appear as a duplicate to an entry already in the cache and be rejected. The registration no-
unique option allows you to overwrite existing cache entries. This feature is only required on NHRP clients
(DMVPN spoke routers).
interface Tunnel10
ip nhrp map 10.4.34.1 172.16.130.1
ip nhrp map multicast 172.16.130.1
ip nhrp nhs 10.4.34.1
ip nhrp registration no-unique
Step 3. (Optional) Configure the backup NHS feature for a redundant hub.
When there is more than one DMVPN hub that a DMVPN spoke is peering to, a backup NHS feature must be
used. The backup NHS feature allows the spoke router to limit the number of active DMVPN hub connections to
one while providing failover capability to an alternate hub in case of a connection failure to the primary hub. This is
needed for PfR to provide intelligent path control later on in the procedure.
interface Tunnel10
ip nhrp nhs 10.4.34.1 priority 1
ip nhrp nhs [Secondary Hub address] priority 2
ip nhrp nhs cluster 0 max-connections 1
Procedure 6: Configure IP Multicast Routing
This procedure includes additional steps for configuring IP Multicast for a DMVPN tunnel on a router with IP
Multicast already enabled.
Step 1. Enable PIM non-broadcast multiple access (NBMA) mode for the DMVPN tunnel.

To resolve the NBMA issue, you need to implement a method where each remote PIM neighbor has its join
messages tracked separately. A router in PIM NBMA mode treats each remote PIM neighbor as if it were
connected to the router through a point-to-point link.
interface Tunnel10
ip pim sparse-mode
ip pim nbma-mode
Step 2. Configure the designated router priority for the DMVPN spoke router.
Proper multicast operation across a DMVPN cloud requires that the hub router assumes the role of a PIM-
designated router. Spoke routers should never become the designated router. You can prevent that by setting the
designated router priority to 0 for the spokes.
interface Tunnel10
ip pim dr-priority 0
Procedure 7: Configure the EIGRP Routing Process
Step 1. Enable an EIGRP process on the spoke router.
A single EIGRP-IWAN process runs on the DMVPN spoke router. All interfaces on the router are EIGRP interfaces,
but only the DMVPN tunnel interface is non-passive. The network range must include all interface IP addresses,
either in a single network statement or in multiple network statements. This design uses a best practice of
assigning the router ID to a loopback address.
router eigrp IWAN
!
!
af-interface default
passive-interface
exit-af-interface
!
no passive-interface
exit-af-interface
!
af-interface GigabitEthernet0/0/0
exit-af-interface
!
network 10.4.34.0 0.0.1.255
network 10.5.0.0 0.0.255.255
network 10.255.0.0 0.0.255.255
eigrp router-id [IP address of Loopback0]
exit-address-family
!

router eigrp IWAN
!
!
hello-interval 20
hold-time 60
exit-af-interface
exit-address-family
!
Step 3. Configure the summary address for the remote-site LAN networks must be advertised.
The IP assignment for the remote sites was designed so that all of the networks in use can be summarized within a
single aggregate route. The summary address as configured below suppresses the more specific routes. If any
network within the summary is present in the route table, the summary is advertised to the DMVPN hub, which
offers a measure of resiliency. If the various LAN networks cannot be summarized, then EIGRP continues to
advertise the specific routes.
router eigrp IWAN
!
!
summary-address [summary network] [summary mask]
exit-af-interface
exit-address-family
!
Step 4. Configure and apply the outbound distribute list.
This design uses route tag to prevent routing loop between the two DMVPN networks. Routes learned over the
The routes are shared between the MPLS DMVPN and Internet DMVPN spoke routers to ensure reachability.
An outbound distribute list is used on the branch spoke routers to select which routes are allowed to be advertised
out. The goal is that routes coming from the hub will have a tag of either 100 or 200 and the route-map BRANCH-
OUT will prevent these routes from being advertised back out to the hubs. This ensures the routes learned over the
MPLS DMVPN path are not re-advertised out to Internet DMVPN spokes and vice versa to prevent routing loops.
In the route map, a prefix list is used to define the address block assigned to the tunnels. This ensures the tunnel
addresses are not advertised across the two networks, thus the second deny statement. The third and last permit
statement ensures that everything else that is not matching the explicitly denied statements will be advertised out
to the hubs.

router eigrp IWAN
!
!
topology base
distribute-list route-map BRANCH-OUT out Tunnel10
exit-af-topology
exit-address-family
!
ip prefix-list TUNNEL-ROUTES seq 5 permit 10.0.224.0/19 le 32
!
route-map BRANCH-OUT deny 10
match tag 100 200
match ip address prefix-list TUNNEL-ROUTES
route-map BRANCH-OUT permit 1000
!
Step 5. Enable EIGRP stub configuration and configure a leak map.
All DMVPN spoke routers should run EIGRP stub routing to improve network stability and reduce resource
utilization. To allow for routing exchange between two stub routers, leak map is needed. The example below allows
a spoke router to share all routes learned from a hub router with its peer.
router eigrp IWAN
!
eigrp stub connected summary redistributed leak-map LEAK-EIGRP
exit-address-family
!
route-map LEAK-EIGRP permit 1000
Procedure 8: Configure the MPLS DMVPN Spoke
Repeat Procedures 1 8 in the Remote-Site Dual DMVPN Spokes Configuration section to provision the second
DMVPN spoke router. Refer to Table 8 and replace the attributes used for Internet with attributes for MPLS as
listed in the table.

Remote-Site Single DMVPN Spoke Dual Links Configuration

Use this set of procedures for remote site with single router and dual link design.
Procedure 1: Configure the WAN Remote Router
Within this design, there are features and services that are common across all WAN remote site routers. These are
system settings that simplify and secure the management of the solution.
Device host name, for ease of identifying the device
The local login account and password, to provide basic access
!
hostname [hostname]
!
aaa new-model
!
ntp update-calendar
!
Step 2. Configure device management protocols.
Configure Secure Shell (SSH) for secure management of the network device

CLI commands
!
ip ssh version 2
no ip http server
!
line vty 0 15
transport input ssh
line con 0
logging synchronous
line vty 0 15
access-class 55 in
!
Tech Tip
As networks scale in the number of devices to maintain, it poses an operational burden to maintain local user
accounts on every device. A centralized AAA service reduces operational tasks per device and provides an audit
log of user access for security compliance and root-cause analysis. When AAA is enabled for access control, all
management access to the network infrastructure devices (SSH and HTTPS) is controlled by AAA.
key SecretKey
!

!
process resiliency.
a unique network range that is not part of any other internal network summary range.
ip pim sparse-mode
The ip pim sparse-mode command will be explained in the next step.
Bind the device processes for SNMP, SSH, PIM, TACACS+, and NTP to the loopback interface address for optimal
resiliency.
Step 5. Configure IP Multicast routing.
from a single source to multiple receivers. Using IP Multicast is much more efficient than multiple individual unicast
streams or a broadcast stream that would propagate everywhere. IP telephony MOH and IP video broadcast
streaming are two examples of IP Multicast applications.
To receive a particular IP Multicast data stream, end hosts must join a multicast group by sending an IGMP
message to their local multicast router. In a traditional IP Multicast design, the local router consults another router
in the network that is acting as a rendevous point to map the receivers to active sources so they can join their
streams.
In this design, which is based on sparse mode multicast operation, auto- rendevous point is used to provide a
simple, yet scalable, way to provide a highly resilient rendevous point environment.

ip pim sparse-mode
Table 12. VRF and Crypto Parameters Example for Single Router Dual Link Branch
rd 65512:1 65512:2
DMVPN-MPLS cloud, respectively. The VRF name is arbitrary, but it is useful to select a name that describes the
to use the same VRF/route distinguisher combination across multiple devices when using VRFs in a similar
ip vrf INET-PUBLIC
rd 65512:1
Tech Tip
Command Reference:
A route distinguisher is either ASN-related (composed of an ASN and an arbitrary number) or IP-address-related
(composed of an IP address and an arbitrary number).
For example, 65512:1.
For example, 192.168.122.15:1
Procedure 3: Connecting to the Service Provider
The remote sites that are using DMVPN can use either static or dynamically assigned IP addresses. Cisco tested
the design with a DHCP-assigned external address, which also provides a dynamically configured default route.
Option 1: Receive a DHCP Address from the Internet Service Provider
The DMVPN spoke router connects directly to the Internet without a separate firewall. This connection is secured in
two ways. Because the Internet interface is in a separate VRF, no traffic can access the global VRF, except traffic
sourced through the DMVPN tunnel. This design provides implicit security. Additionally, an IP access list permits
only the traffic required for an encrypted tunnel, as well as DHCP and various ICMP protocols for troubleshooting.
Step 1. Enable the interface, select VRF, and enable DHCP.
The DMVPN design uses FVRF so this interface must be placed into the VRF configured in the previous
procedure.

ip address dhcp
no cdp enable
no shutdown
Step 2. Configure and apply the access list.
The IP access list must permit the protocols specified in Table 13. The access list is applied inbound on the WAN
interface so filtering is done on traffic destined to the router.
Table 13. Required DMVPN Protocols
Name Protocol Usage
non500-isakmp UDP 4500 IPsec via NAT-T
Isakmp UDP 500 ISAKMP
Esp IP 50 IPsec
bootpc UDP 68 DHCP
Example
ip access-group ACL-INET-PUBLIC in
ip access-list extended ACL-INET-PUBLIC
permit udp any any eq non500-isakmp
permit udp any any eq isakmp
permit esp any any
permit udp any any eq bootpc
The additional protocols listed in Table 14 may assist in troubleshooting, but are not explicitly required to allow
DMVPN to function properly.
Table 14. Optional Protocols DMVPN Spoke Router
Name Protocol Usage
icmp echo ICMP Type 0, Code 0 Allow remote pings
icmp echo-reply ICMP Type 8, Code 0 Allow ping replies (from our requests)
icmp ttl-exceeded ICMP Type 11, Code 0 Allow traceroute replies (from our requests)
icmp port-unreachable ICMP Type 3, Code 3 Allow traceroute replies (from our requests)
UDP high ports UDP > 1023, TTL=1 Allow remote traceroute
The additional optional entries for an access list to support ping are:
permit icmp any any echo
permit icmp any any echo-reply
The additional optional entries for an access list to support traceroute are:
permit icmp any any ttl-exceeded ! for traceroute (sourced)
permit icmp any any port-unreachable ! for traceroute (sourced)
permit udp any any gt 1023 ttl eq 1 ! for traceroute (destination)
Option 2: Static Address and Static Routing with Service Provider
A static IP address is assigned to the WAN interface connecting to the service provider. The interface is associated
with a VRF. This simplifies the routing and allows the use of a default route for the FVRF for the WAN interface.

Step 1. Enable the interface, select VRF, and assign an IP address.
no cdp enable
no shutdown
Step 2. Configure a static default route in the FVRF to the service provider.
ip route vrf INET-MPLS 0.0.0.0 0.0.0.0 [Gateway Address]
The crypto keyring defines a PSK (or password) valid for IP sources reachable within a particular VRF. This key is
a wildcard PSK if it applies to any IP source. A wildcard key is configured using the 0.0.0.0 0.0.0.0 network/mask
combination.
crypto keyring DMVPN-KEYRING1 vrf INET-PUBLIC
Step 2. Configure the ISAKMP policy and dead peer detection.
AES with a 256-bit key
Authentication by PSK
DPD is enabled with keepalive intervals sent at 40-second intervals with a five-second retry interval, which is
considered to be a reasonable setting to detect a failed hub.
encr aes 256
hash sha
group 2
!
crypto isakmp keepalive 40 5
keyring DMVPN-KEYRING1

The IPsec transform set for DMVPN uses:
ESP with the 256-bit AES encryption algorithm
ESP with the SHA (HMAC variant) authentication algorithm
Since the DMVPN hub router is behind a NAT device, the IPsec transform must be configured for transport mode.
mode transport
Step 5. Create the IPsec profile.
Step 1. Configure basic interface settings.
numbering at 10 or above because other features deployed in this design may also require tunnels and they may
The bandwidth setting should be set to match the Internet bandwidth.
Configure the IP MTU to 1400 and the ip tcp adjust-mss to 1360. There is a 40-byte difference, which
corresponds to the combined IP and TCP header length.
interface Tunnel10
bandwidth [bandwidth (kbps)]
no ip redirects
ip mtu 1400
Step 2. Configure the tunnel.
DMVPN uses multipoint GRE (mGRE) tunnels. This type of tunnel requires a source interface only. Use the same
source interface that you use to connect to the Internet. Set the tunnel vrf command to the VRF defined previously
for FVRF.
procedure.
interface Tunnel10
tunnel source GigabitEthernet0/1

The DMVPN hub router is the NHRP server for all of the spokes. NHRP is used by remote routers to determine the
tunnel destinations for peers attached to the mGRE tunnel.
The spoke router requires several additional configuration statements to define the NHRP server (NHS) and NHRP
map statements for the DMVPN hub router mGRE tunnel IP address. EIGRP relies on a multicast transport. Spoke
routers require the NHRP static multicast mapping.
The value used for the NHS is the mGRE tunnel address for the DMVPN hub router. The map entries must be set
to the outside NAT value of the DMVPN hub, as configured on the Cisco ASA 5500. This design uses the values
shown in Table 15.
Parameter Values
DMVPN cloud INTERNET MPLS
VRF INET-PUBLIC INET-MPLS
DMVPN hub public address (actual) 192.168.18.10 192.168.3.1
DMVPN hub public address (externally routable after NAT) 172.16.130.1 N/A
Tunnel IP address (NHS) 10.4.34.1 10.4.36.1
Tunnel number 10 20
NHRP network ID 101 102
cache hold time should be configured to 600 seconds.
This design supports DMVPN spoke routers that receive their external IP addresses through DHCP. It is possible
for these routers to acquire different IP addresses after a reload. When the router attempts to register with the
NHRP server, it may appear as a duplicate to an entry already in the cache and be rejected. The registration no-
unique option allows you to overwrite existing cache entries. This feature is only required on NHRP clients
(DMVPN spoke routers).
interface Tunnel10
ip nhrp map 10.4.34.1 172.16.130.1
ip nhrp map multicast 172.16.130.1
ip nhrp nhs 10.4.34.1
ip nhrp registration no-unique

Step 4. (Optional) Configure the backup NHS feature for a redundant hub.
When there are more than one DMVPN hubs that a DMVPN spoke is peering to, the backup NHS feature must be
used. The backup NHS feature allows the spoke router to limit the number of active DMVPN hub connections to
one, while providing failover capability to an alternate hub in case of a connection failure to the primary hub. This is
needed for PfR to provide intelligent path control later on in the procedure.
interface Tunnel10
ip nhrp nhs 10.4.34.1 priority 1
ip nhrp nhs [Secondary Hub address] priority 2
ip nhrp nhs cluster 0 max-connections 1
Procedure 6: Configure IP Multicast Routing
This procedure includes additional steps for configuring IP Multicast for a DMVPN tunnel on a router with IP
Multicast already enabled.
Step 1. Configure PIM on the DMVPN tunnel interface.
Enable IP PIM sparse mode on the DMVPN tunnel interface.
interface Tunnel10
ip pim sparse-mode
Step 2. Enable PIM NBMA mode for the DMVPN tunnel.
To resolve the nonbroadcast multiaccess (NBMA) issue, you need to implement a method where each remote PIM
neighbor has its join messages tracked separately. A router in PIM NBMA mode treats each remote PIM neighbor
as if it were connected to the router through a point-to-point link.
interface Tunnel10
ip pim nbma-mode
Step 3. Configure the designated router priority for the DMVPN spoke router.
Proper multicast operation across a DMVPN cloud requires that the hub router assumes the role of PIM designated
router. Spoke routers should never become the designated router. You can prevent that by setting the designated
router priority to 0 for the spokes.
interface Tunnel10
ip pim dr-priority 0

Procedure 7: Configure EIGRP
Step 1. Enable the EIGRP process on the spoke router.
A single EIGRP-IWAN process runs on the DMVPN spoke router. All interfaces on the router are EIGRP interfaces,
but only the DMVPN tunnel interface is non-passive. The network range must include all interface IP addresses,
either in a single network statement or in multiple network statements. This design uses a best practice of
assigning the router ID to a loopback address.
router eigrp IWAN
!
!
af-interface default
passive-interface
exit-af-interface
!
exit-af-interface
!
exit-af-interface
!
af-interface GigabitEthernet0/0/0
exit-af-interface
!
network 10.4.34.0 0.0.1.255
network 10.5.0.0 0.0.255.255
network 10.255.0.0 0.0.255.255
eigrp router-id [IP address of Loopback0]
no auto-summary
router eigrp IWAN
!
!
hello-interval 20
hold-time 60
exit-af-interface
!

hello-interval 20
hold-time 60
exit-af-interface
exit-address-family
Step 3. Configure the summary address for the remote-site LAN networks must be advertised
The IP assignment for the remote sites was designed so that all of the networks in use can be summarized within a
single aggregate route. The summary address as configured below suppresses the more specific routes. If any
network within the summary is present in the route table, the summary is advertised to the DMVPN hub, which
offers a measure of resiliency. If the various LAN networks cannot be summarized, then EIGRP continues to
advertise the specific routes.
router eigrp IWAN
!
!
summary-address [summary network] [summary mask]
exit-af-interface
exit-address-family
!
Step 4. Configure and apply the outbound distribute list.
This design uses route tags to prevent routing loops between the two DMVPN networks. Routes learned over the
The routes are shared between the MPLS DMVPN and Internet DMVPN spoke routers to ensure reachability.
An outbound distribute list is used on the branch spoke routers to select which routes are allowed to be advertised
out. The goal is for routes coming from the hub to have a tag of either 100 or 200 and the route-map BRANCH-
OUT will prevent these routes from being advertised back out to the hubs. This ensures the routes learned over
MPLS DMVPN path are not readvertised out to Internet DMVPN spokes and vice versa to prevent routing loops.
In the route map, a prefix list is used to define the address block assigned to the tunnels. This ensures the tunnel
addresses are not advertised across the two networks, thus the second deny statement. The thrid and last permit
statement ensures everything else that is not matching the explicitly denied statements will be advertised out to the
hubs.
router eigrp IWAN
!
!
topology base
exit-af-topology
exit-address-family
!

ip prefix-list TUNNEL-ROUTES seq 5 permit 10.0.224.0/19 le 32
!
match tag 100 200
match ip address prefix-list TUNNEL-ROUTES
route-map BRANCH-OUT permit 1000
!
Step 5. Enable an EIGRP stub configuration and configure leak map.
All DMVPN spoke routers should run EIGRP stub routing to improve network stability and reduce resource
utilization. To allow routing exchange between two stub routers, leak map is needed. The example below allows a
spoke router to share all routes learned from a hub router with its peer.
router eigrp IWAN
!
eigrp stub connected summary redistributed leak-map LEAK-EIGRP
exit-address-family
!
route-map LEAK-EIGRP permit 1000
Deploying Intelligent Path Control
Design Overview
Intelligent Path Control using Cisco Performance Routing (PfR) improves application delivery and WAN efficiency.
PfR enables intelligence of Cisco IOS routers to improve application performance and availability. PfR allows
customers to protect critical applications from fluctuating WAN performance while intelligently load balancing traffic
over all WAN paths.
PfR monitors network performance and selects the best path for each application based upon advanced criteria
such as reachability, delay, jitter, loss, and mean opinion scores (MOS). PfR can evenly distribute traffic to
maintain equivalent link utilization levels using an advanced load balancing technique - even over links with
differing bandwidth capacities. IWAN Intelligent Path Control is key to providing a business-class WAN over
Internet transports.
PfR Components
PfR is comprised of two major components, a master controller and a border router. The master controller is a
policy decision point at which policies are defined and applied to various traffic classes that traverse the border
router systems. The master controller can be configured to learn and control traffic classes on the network.
Border routers are in the data forwarding path. Border routers collect data from their Netflow cache and from the
IP SLA probe results, provide a degree of aggregation of this information, and influence the packet switching path
to manage user traffic.

Master controller is the policy decision maker. At a large site, such as data center or campus, the master
controller is a standalone chassis. For smaller branch locations, the master controller is typically collocated
(configured) on the same platform as the border router. As a general rule, the large locations manage more
network prefixes and applications than a branch deployment, thus consuming more CPU and memory resources
for the master controller function. Therefore, it makes a good design practice to dedicate a chassis for the master
controller at large sites.
The branch typically manages fewer active network prefixes and applications. And due to the costs associated with
dedicating a chassis at each branch, the network manager can co-locate the master controller and border router on
the same platform. CPU and memory utilization should be monitored on platforms operating as master controllers,
and if utilization is high the network manager should consider a master controller platform with a higher capacity
CPU and memory.
The master controller communicates with the border routers over an authenticated TCP socket, but has no
requirement for populating its own IP routing table with anything more than a route to reach the border routers.
Because PfR is an intelligent path selection technology, there must be at least two external interfaces under the
control of PfR and at least one internal interface. There must be at least one border router configured. If only one
border router is configured, then both external interfaces are attached to the single border router. If more than one
border router is configured, then the two or more external interfaces are configured across these border routers.
External links, or exit points, are therefore owned by the border router; they may be logical (tunnel interfaces) or
physical links (serial, Ethernet, etc.).
Tech Tip
Cisco IOS Software Release 15.2(3)T and IOS-XE 3.6S and later support significant simplification enhancements,
including target discovery and better default behavior. This guide will only focus on the latest implementation.
PfR Classes of Service
The enterprise traffic is divided into three main application service classes that have their own policies.
Voice and Video traffic VOICE_VIDEO Service Class
Voice and video traffic is running between the central site and the branches. This traffic is already marked
with Differentiated Services Code Point Expedited Forwarding (DSCP EF) and the transport is Rapid
Transport Protocol (RTP). To capture video we also match DSCP AF41 and CS4 which is used by Cisco
TelePresence
endpoints.
We would like to track delay, jitter, and loss.
We want to have deterministic path selection, and therefore use the primary service provider called MPLS.
If the primary WAN experiences brownouts or a blackout, we want PfR to failover this traffic to the
secondary path.

Critical Application - CRITICAL Service Class
There is a highly important application running between branches and the central site network, and running
over TCP.
This traffic is marked with DSCP AF31.
This application is interactive so it is sensitive to delay. In addition, it does not respond well to packet loss.
The servers being used for this important application are also responsible for other applications that are not
only less important, but that have different requirements of the network.
We want to have deterministic path selection, and therefore use the primary SP MPLS.
If the primary WAN experiences brownouts or a blackout, we want PfR to failover this traffic to the
secondary path.
Best Effort Applications BEST_EFFORT Service Class
The rest of the traffic is just HTTP or email and should be load balanced over all exit interfaces, MPLS, and
Internet.
QoS is already in place with classification and marking procedures. Therefore, packets entering on the border
router are already classified and marked directly on the access switch connecting the station, IP phone, or
multimedia terminal. That means PfR can use the DSCP field as an efficient way for the traffic profiling phase. This
guide will use the DSCP values as the classification criteria.
You could also use a combination of destination prefixes and DSCP, or even NBAR. If the traffic is not marked
when entering the border router, then a good option is to classify on ingress, mark the DSCP, and then use the
DSCP as the classification criteria within PfR. Classification can be done with the usual use of access lists.
Design Details
In this guide we want to protect voice, video, and critical applications against soft errors, brownouts, and blackouts.
We want to be able to track jitter, delay, and loss, and we also want to have a fast failover. Therefore, performance
measurement will be in active mode and will use jitter probes. Jitter probe configuration for all the remote sites is a
painful process. Cisco PfR is now using a new feature called Target Discovery to help simplify the configuration
and deployment in such cases.
A peering session is configured between the master controller on the central site and the master controllers on the
remote sites. The master controllers which participate in this peering will exchange the list of inside prefixes and IP
SLA probe responder addresses that belong to the sites, along with the site name (configurable).
Deployment Details
This section covers:
Deployment details for master controllers
Deployment details for border routers
Implementing Dedicated Master Controller
Procedure 1: Configure the Master Controller Router
This section describes configuring a dedicated master controller for a large site such as a campus or data center.
Only the core relevant features are described. Figure 11 illustrates a typical hub technology, along with the IP
addresses. Table 16 outlines the parameters used in our deployment examples.

Figure 11. Hub Topology and Addresses

Table 16. Parameters Used in the Deployment Examples
Host Name Loopback IP Address Port Channel IP Address
VPN-ASR1002-1 10.4.32.241/32 10.4.32.2/28
VPN-ASR1002-2 10.4.32.243/32 10.4.32.3/28
MC-ASR1002 10.4.32.251/32 10.4.32.38/30
Within this design, there are features and services that are common across all WAN aggregation routers. These
are system settings that simplify and secure the management of the solution.
The local login account and password to provide basic access

!
hostname [hostname]
!
aaa new-model
!
Step 2. Configure device management protocols
CLI commands
!
ip ssh version 2
no ip http server
!
line vty 0 15
transport input ssh
line con 0
logging synchronous
line vty 0 15
access-class 55 in
!

Tech Tip
As networks scale in the number of devices, Cisco recommends the best practice to use a centralized AAA service
to reduce operational tasks per device and provides an audit log of user access for security compliance and root-
cause analysis. When AAA is enabled for access control, all management access to the network infrastructure
devices (SSH and HTTPS) is controlled by AAA.
key SecretKey
!
!
The loopback interface is a logical interface that is always reachable, as long as the device is powered on and any
process resiliency.
the IP address block that the distribution switch summarizes to the rest of the network.
Bind the device processes for SNMP, SSH, TACACS+, and NTP to the loopback interface address for optimal
resiliency.

Procedure 2: Configure Connectivity to the LAN
The master controller is not in line with the data flow and should only need one connection to the distribution
switches. In this guide two interfaces are used to configure EtherChannel connection for link redundancy.
Step 1. Configure a layer 3 interface.
interface Port-channel21
ip address 10.4.32.151 255.255.255.192
no shutdown
Step 2. Configure EtherChannel member interfaces.
Configure the physical interfaces to tie to the logical port channel by using the channel-group command. The
number for the port channel and channel group must match. Not all router platforms can support Link Aggregation
Control Protocol (LACP) to negotiate with the switch, so EtherChannel is configured statically.
description WAN-D3750X Gig1/0/9
!
description WAN-D3750X Gig2/0/9
!
interface range GigabitEthernet0/0, GigabitEthernet0/1
no ip address
channel-group 21
no shutdown
Step 3. Configure a default route. This provides reachability information for the Key Server to reach border routers
by using a default route.
ip route 0.0.0.0 0.0.0.0 10.4.32.129
Procedure 3: Configure Master Controller Policies
This section provides the foundation configuration required on the master controller. The configuration enables
automatic learning of traffic flowing through border routers using Netflow. In addition, PfR route control is enabled
to allow load balancing across all configured external interfaces.
Step 1. Configure the key chain.
The key chain is used to authenticate the connection with the border router.
key chain pfr
key 0
key-string cisco
!
Step 2. Configure the master controller.
Enable PfR master controller and designate the border routers' IP addresses and internal/external interfaces on the
border routers. Link group is used to specify the carrier and color the exit interface. The key chain is used to
authenticate the connection with border routers.

pfr master
!
border 10.4.32.241 key-chain pfr
interface Tunnel20 external
link-group MPLS
interface Port-channel1 internal
!
link-group Internet
interface Port-channel3 internal
!
Step 3. Configure PfR policies.
Set the load-balancing range to 30 percent. This setting is conservative and helps reduce the churn result
from movement of traffic classes across the links.
Disable auto tunnels as border routers are layer 2-adjacent.
Enable the use of policy-based routing (PBR) for finer application control
Maximum transmit utilization is 90 percent
pfr master
max-range-utilization percent 30
!
no mode auto-tunnels
periodic 90
probe packets 20
!
!
Tech Tip
Note the following will be enabled by default:
Automatic Learning
Mode route control
Maximum transmit utilization at 90%
Load balancing is used as the last resolver and cannot be disabled
Step 4. Enable NetFlow version 9 export (optional).
The following configuration defines the flow exporter. The data that is collected by NetFlow version 9 is sent to a
NetFlow collector for further analysis and storage. Flow exporter uses User Datagram Protocol (UDP) as transport
protocol.
flow exporter MYEXPORTER
destination 10.151.1.131
source Loopback0
transport udp 9991
option interface-table timeout 300
option sampler-table timeout 300

option application-table timeout 300
!
! Add NetFlow export to PfR
!
pfr master
exporter MYEXPORTER
!
The following configuration enables logging to syslog. This will log PfR-related messages on the console.
pfr master
logging
!
Implementing Dedicated Border Routers
This section describes adding dedicated border router functionality to an already configured WAN router with a
DMVPN tunnel at a large site such as a campus or data center. It includes only the additional steps required to
enable the border router capabilities.
Procedure 1: Configure Branch Router Functionality
The PfR configuration is centralized on the master controller. The border router configuration only includes:
The source address used to peer with the master controller
The master controller IP address
The key chain used to authenticate with the master controller
Step 1. Configure the key chain.
The key chain is used to authenticate the peering connection with the master controller.
key chain pfr
key 0
key-string cisco
!
Step 2. Configure the border router.
pfr border
logging
local Loopback0
master 10.4.32.251 key-chain pfr
Implementing Combined Master Controller and Border Router for a Branch
The following section shows configuration steps needed for enabling a branch router with combined master
controller and branch router functionality.

Procedure 1: Configure Combined Master Controller and Branch Router on a Router
Step 1. Configure the key chain
The key chain is used to authenticate the connection with the border router.
key chain pfr
key 0
key-string cisco
!
Step 2. Configure the master controller.
Enable the PfR master controller and designate the border router's local loopback IP addresses and
internal/external interfaces on the border routers. Link group is used to specify the carrier and color the exit
interface. The key chain is used to authenticate the connection with border routers.
pfr master
logging
!
interface Ethernet0/0 internal
link-group MPLS
link-group Internet
!
Step 3. Configure border router functionality.
pfr border
logging
local Loopback0
master 10.2.10.10 key-chain pfr
!
Implementing Target Discovery on Master Controller
This section applies to the master controller in either dedicated or shared mode. Target Discovery is applied for the
hub-and-spoke traffic model.
The PfR Target Discovery feature introduces a scalable solution for managing the performance of video, voice, and
critical applications across large, enterprise branch networks by automating the identification and configuration of
IP SLA responders. To optimize media applications using voice and video traffic, PfR uses jitter, loss, and delay
measurements. The IP SLA udp-jitter probe provides these measurements but requires an IP SLA responder. The
initial PfR solution was to manually configure the probes for all remote sites. PfR Target Discovery uses the PfR
Domain Peering.
Tech Tip
Target Discovery feature requires the router run IOS Software Release 15.2(3)T and later releases.

Procedure 1: Configure Target Discovery on the Hub Master Controller
Step 1. Define the prefix of interest.
We want to define the IP address of the shadow router used as a responder on the hub. We also want to control
the list of prefixes advertised to the spokes. Therefore, we use the responder list and inside prefix options.
ip prefix-list HQ_PREFIX seq 5 permit 10.10.1.0/24
ip prefix-list RESPONDER_PREFIX seq 5 permit 10.10.10.2/32
!
Step 2. Enable Target Discovery on the hub master controller.
pfr master
mc-peer domain 200 eigrp Loopback0
target-discovery responder-list RESPONDER_PREFIX inside-prefixes HQ_PREFIX
!
Procedure 2: Enable Target Discover on the Branch Master Controller
Step 1. Enable Target Discovery on the spoke master controller.
On the entire spoke master controllers, configure the IP address of the hub master controller. To simplify the
configuration on the branch devices, we do not want to manually define the target and prefix list. PfR will
automatically generate what is needed.
pfr master
mc-peer domain 200 eigrp Loopback0
target-discovery
!
Procedure 3: Tuning Service Advertisement Framework (SAF) Parameters
Step 1. Configure the hub master controller.
The PfR Target Discovery introduces master controller peering and uses service routing through the EIGRP
Service Advertisement Framework (SAF) to advertise, discover, and auto-configure IP SLA responders and
associated destination IP prefixes. The following procedure tunes the SAF parameters to increase PfR stability and
scalability.
The master controller peering is enabled and PfR Target Discovery is configured in dynamic mode. The SAF
configuration is shown here under the service-family command section, and this configuration is assumed to exist
before the PfR master controller peering and Target Discovery overlay configuration is added.
Increase SAF Hello timer to 200 seconds.
Increase SAF Hold timer to 600 seconds
Disable split horizon on the loopback interface
Enable dynamic peering with unicast listen
router eigrp IWAN
!
service-family ipv4 autonomous-system 200

!
sf-interface default
shutdown
exit-sf-interface
!
sf-interface Loopback0
no shutdown
hello-interval 200
hold-time 600
no split-horizon
exit-sf-interface
!
topology base
exit-sf-topology
remote-neighbors source Loopback0 unicast-listen
exit-service-family
!
Step 2. Configure the branch location master controller.
At the branch location the master controller peering is enabled and PfR target discovery is configured in dynamic
mode.
Increase SAF Hello timer to 200 seconds.
Increase SAF Hold timer to 600 seconds
Define Hub MC address through unicast neighbor statement
router eigrp IWAN
!
service-family ipv4 autonomous-system 200
eigrp stub connected
!
sf-interface default
shutdown
exit-sf-interface
!
sf-interface Loopback0
no shutdown
hello-interval 200
hold-time 600
exit-sf-interface
!
neighbor 10.4.32.251 Loopback0
exit-service-family
!

Tuning Automatic Learning on PfR
This process tunes the automatic learning capability for the PfR and optimizes the traffic going to the branches.
PfR will optimize mission-critical traffic based on policy parameters while sending the rest of the traffic based on a
different set of policy parameters.
Procedure 1: Tuning Learning Configuration
Step 1. Define the access list for deny global learning.
ip access-list extended DENY_GLOBAL_LEARN_LIST
deny ip any any
!
Step 2. Define the access list for voice and video traffic.
The classification of the traffic is based on DSCP marking. A common DSCP value for voice is EF, and AF41 and
CS4 are for video.
ip access-list extended VOICE_VIDEO
permit ip any any dscp ef
permit ip any any dscp af41
permit ip any any dscp cs4
!
Step 3. Define the access list for critical business applications.
This shows the classification of business applications based on DSCP AF31. The DSCP value can be modified to
match the specific business app you want to guarantee service.
ip access-list extended CRITICAL
permit ip any any dscp af31
!
Step 4. Define the access list for best-effort traffic. All other traffic will be classified as best effort.
ip access-list extended BEST_EFFORT
permit ip any any dscp default
!
Step 5. Define the IP prefix list to track traffic going to the branches only.
ip prefix-list BRANCH_PREFIX seq 10 permit 10.1.0.0/16 ge 24
!
Step 6. Applying the learning parameter to the PfR master controller.
Assume the traffic is already marked when it enters the border routers on all sites. This step defines learning based
on DSCP values while filtering based on the branch prefix. The filtering, based on the prefix, will make sure PfR
only learns traffic destined to the branch locations while the DSCP based access list will help categorize the
interesting traffic into different classes of service such as voice and video, critical, and best effort. The classes of
service are then associated with different optimization policies.
pfr master
!
learn
throughput

traffic-class filter access-list DENY_GLOBAL_LEARN_LIST
!
list seq 10 refname LEARN_VOICE_VIDEO
traffic-class access-list VOICE_VIDEO filter BRANCH_PREFIX
aggregation-type prefix-length 24
count 2000 max 10000
throughput
!
list seq 20 refname LEARN_CRITICAL
traffic-class access-list CRITICAL filter BRANCH_PREFIX
throughput
!
list seq 30 refname LEARN_BEST_EFFORT
traffic-class access-list BEST_EFFORT filter BRANCH_PREFIX
throughput
!
!
Procedure 2: Define Advanced Policies Per Group
Step 1. Define service level policies for voice and video.
For voice and video traffic, network characteristics such as latency, jitter, and loss are important to the voice and
video quality and user experience. Mode monitor is set to fast to allow quick decision-making of alternate paths
when an out-of-policy condition is detected.
Match command is to specify that this policy should be applied to all the traffic classes learned under list
LEARN_VOICE_VIDEO
Delay threshold is configured as 200 msec. The delay measured by PfR is round-trip time
Loss threshold is configured as five percent
Jitter threshold is configured as 30 ms
Probe frequency is set to eight seconds
Probes are automatically configured and generated by Target Discovery
pfr-map PFRMAP 10
match PfR learn list LEARN_VOICE_VIDEO
set periodic 90
set delay threshold 200
set loss threshold 50000
set jitter threshold 30
set mode monitor fast
set resolve delay priority 1 variance 5
set resolve loss priority 2 variance 5

set resolve jitter priority 3 variance 5
set probe frequency 8
set link-group MPLS fallback Internet
!
Step 2. Define service policies for critical data.
LEARN_CRITICAL
Re-evaluate the alternate path every 90 seconds (periodic 90)
Delay threshold is configured as 250 msec. The delay measured by PfR is round-trip time
Loss threshold is configured as five percent
Probe frequency is set to eight seconds and can be changed to a lesser value if the application being
controlled is critical
Probes are automatically configured and generated by Target Discovery
pfr-map PFRMAP 20
match pfr learn list LEARN_CRITICAL
set periodic 90
set delay threshold 250
set mode monitor fast
set resolve delay priority 1 variance 20
set resolve loss priority 5 variance 10
set probe frequency 8
set link-group MPLS fallback Internet
!
Step 3. Define policy for best-effort traffic.
LEARN_BEST_EFFORT
Monitor mode is set to both. This is the default mode. Monitoring is passive and echo probes are used to
help find a new exit when a traffic class is out of policy
Default policies used: link utilization first, then load balancing
pfr-map PFRMAP 30
match pfr learn list LEARN_BEST_EFFORT
set periodic 90
set mode monitor both
!
Step 4. Apply the policies on the PfR master controller for them to take effect.
pfr master
policy-rules PFRMAP
!


Printed in USA C07-731952-00 06/14

Guide IWAN Design

Uploaded by

Copyright:

Available Formats

Guide IWAN Design

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Guide IWAN Design

Uploaded by

Copyright:

Available Formats

2014 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.

You might also like