Routing Operation
Routing Operation
Routing Operation
IOS Routers
BRKRST-2350
Scott Custer
Team Lead – RTP TAC Routing Protocols
2011
BRKARC-2350 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 1
Routing Operation in Cisco IOS Routers
Topics Covered:
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 3
The Routing Table
• Basic Structure
• Route Selection
• Interface Down Events
• Backup Routes
• Static Routes
• Discard Routes
The Routing Table
Basic Structure
55
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
BRKARC-2350 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public
• The Routing Information Routing Protocols
Base or RIB
Interfaces
• Routing Protocols*
Other Sources
Install routes into the RIB
• Interfaces
Install routes into the RIB
RIB
• Other Sources
Install routes into the RIB
-Performance Routing (PFR)
-Reverse-Route Injection (RRI)
Basic Structure
FIB Adj
Hardware
Forwarding
Tables
Basic Structure
router#show ip route
Network
Route
Network + Route
router#show ip route
Connected interface 0
Static route 1
Internal EIGRP 90
IGRP 100
OSPF 110
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 15
Route Selection
How does the RIB decide which route is best among
various sources?
EIGRP 1
(a) EIGRP 1 (c) The RIB (d) EIGRP
installs a sends the decides if
Actually, it doesn‘t. route into new route the new
the RIB to EIGRP 1 route should
replace the
existing one
D EX 192.168.254.0/24 ....
EIGRP internal AD = 90
RIP AD = 120
EIGRP internal = 90
RIPv2 = Configured to 90
OSPF wins
EIGRP wins
EIGRP 1 EIGRP 1
AD=90 AD=90 OSPF
Metric=500 Metric=500 AD=110
EIGRP wins
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 27
BRKARC-2350 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
Interface Down Events
How does the RIB interact with the routing protocol when an interface
fails?
interface Ethernet1/0
ip address 10.1.20.1 255.255.255.128
router eigrp 1
network 10.0.0.0
A
E1/0
E1/0
B
interface Ethernet1/0
ip address 10.1.20.2 255.255.255.128
router eigrp 1
network 10.0.0.0
Interface Down Events
A#show ip route
The RIB tells EIGRP the interface is down 20:46:58.151: EIGRP-IPv4(1): Callback: route_adjust
Ethernet1/0
20:46:58.151: RT: interface Ethernet1/0 removed from routing
10.1.20.0, connected, is removed from the table
RIB 20:46:58.151: RT: del 10.1.20.0 via 0.0.0.0, connected metric
[0/0]
20:46:58.151: RT: delete subnet route to 10.1.20.0/25
172.16.24.0, learned through EIGRP, is 20:46:58.151: RT: delete route to 10.1.20.1 via 0.0.0.0,
removed from the RIB (before EIGRP Ethernet1/0
takes the neighbor down) 20:46:58.151: RT: no routes to 10.1.20.1, flushing
20:46:58.151: RT: delete route to 172.16.24.0 via 10.1.20.2,
Ethernet1/0
20:46:58.151: RT: no routes to 172.16.24.0, flushing
The EIGRP neighbor goes down
20:46:58.151: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1:
Neighbor 10.1.20.2 (Ethernet1/0) is down: interface down
20:47:00.139: %LINK-5-CHANGED: Interface Ethernet1/0,
The interface changes to down state changed state to administratively down
20:47:01.139: %LINEPROTO-5-UPDOWN: Line protocol on
Interface Ethernet1/0, changed state to down
The Routing Table:
Backup, Static, and Discard Routes
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 30
BRKARC-2350 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 30
Backup Routes
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 31
BRKARC-2350 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 31
Backup Routes
EIGRP internal AD = 90
RIP AD = 120
EIGRP
RIPinternal installed
route installed
router-b#debug ip routing
router-b#debug ip eigrp notifications The OSPF route fails…
....
RT: delete route to 10.0.16.0/24 EIGRP gets a callback for
10.0.16.0/24, which is the
IP-EIGRP: Callback: callbackup_routes 10.0.16.0/24
OSPF route that failed
IP-EIGRP: Callback: reload_iptable
RT: add 10.0.16.0/24 via 10.0.12.10, eigrp metric [170/3072256] EIGRP installs the
existing 10.0.16.0/24 route
from its topology table
router-b#show ip route
Codes: D - EIGRP, EX - EIGRP external, O – OSPF....
.... The route is now installed
D EX 10.0.16.0/24 [170/3072256] via 10.0.12.10, Serial0/3
.... by EIGRP
Static Routes
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 35
Static Routes
Static Routes Can Have a Next Hop of an IP Address
• ip route 10.1.1.0 255.255.255.0 10.1.2.1
• This causes the RIB and CEF to do a recursive lookup to find the
correct Layer 2 header to rewrite onto the packet
• Recursive lookup: For each packet destined to 10.1.1.0/24:
1) Look up the destination (10.1.1.0/24)
2) Find the next hop is 10.1.2.1
3) Look up how to get to 10.1.2.1
4) Find 10.1.2.1 is via connected interface
5) Look up the layer 2 header out connected interface to next-hop 10.1.2.1
• As long as the next hop is reachable, the router assumes the destination
through that next hop is reachable
Static Routes
Static Routes Can Have a Next Hop of a Point-to-Point
interface
• ip route 10.1.1.0 255.255.255.0 serial0
• The RIB and forwarding tables point the route directly out the
point-to-point interface
- No need to do a recursive lookup
• The RIB and CEF will point this route directly to the broadcast
interface
router#show ip route
....
10.0.0.0/16 is subnetted, 1 subnets
S 10.1.0.0 is directly connected, FastEthernet0/0
Static Routes
ip route 0.0.0.0 0.0.0.0 FastEthernet 0/0
10.2.1.0/24 via internetwork
A B
10.2.1.25
Packet to ARP for
10.2.1.25 10.2.1.25
Best route is Route to Proxy ARP must be enabled
through broadcast 10.2.1.25 through
interface FE0/0 internetwork
ARP reply
10.2.1.25 via B This happens for each
destination in which host
Send packets to
10.2.1.25 via B’s
A does not have a route
MAC address
Note: Proxy ARP disabled by default
Static Routes
• For a default route (0.0.0.0/0), this could result
in 232 ARP entries in A‘s local tables
This would overflow the ARP cache, and crash A
router#show ip route
....
10.0.0.0/16 is subnetted, 1 subnets
S 10.1.0.0 is directly connected, FastEthernet0/1
show ip route
....
D EX 10.1.1.0/24 via <C>
EIGRP
A learned route
No
dynamic
routing C
10.1.1.0/24
Static Routes
• The concepts of administrative distance and backup routes are
used to create floating static routes
• Configuring a static route with a very high administrative
distance ensures it won‘t be installed as long as there is a
dynamically learned route installed in the RIB using the default
AD
• 255 = unreachable
Static Routes
Floating Static Routes
ip route 10.1.1.0 255.255.255.0 <B> 250
show ip ip
route
route
....
....
S10.1.1.0
10.1.1.0
via
via
<C>
<B>
EIGRP
A learned route
No
dynamic
routing C
10.1.1.0/24
Static Routes
• When the dynamically learned route fails, the RIB calls the processes
looking for a backup route
• Since no other processes have routes to install, the static route with
an administrative distance of 250 wins
• This assumes that the primary route will be removed from the table in
a failure event. If a failure event will not remove the primary route from
the RIB then the floating static backup will not be installed.
Static Routing with Object Tracking
How can we get dynamic failover with no dynamic routing?
ip route 10.1.1.0 255.255.255.0 <C>
ip route 10.1.1.0 255.255.255.0 <B> 10
show ip route
....
10.1.1.0 via <C>
A 172.16.12.0/24
172.16.13.0/24
C
.2
B 172.16.24.0/24
.3
10.1.1.0/24
Static Routing with Object Tracking
ip route 10.1.1.0 255.255.255.0 172.16.12.2 track 1
ip route 10.1.1.0 255.255.255.0 172.16.13.3 10
ip sla 1
icmp-echo 172.16.24.2 source-interface Ethernet1/0
frequency 5
ip sla schedule 1 life forever start-time now
Probes are being sent to 172.16.24.2 02:34:12.106: ICMP: echo reply rcvd, src 172.16.24.2, dst
172.16.12.1, topology BASE, dscp 0 topoid 0
02:34:17.114: ICMP: dst (172.16.12.1) host unreachable rcv
from 172.16.12.2
The track object goes down 02:34:17.306: Track: 1 Change #9 ip sla 1, reachability Up-
when reachability fails >Down
02:34:17.306: %TRACKING-5-STATE: 1 ip sla 1 reachability
The routing table is updated to remove Up->Down
02:34:17.306: RT: del 10.1.1.0 via 172.16.12.2, static metric
the route to the destination through the
[1/0]
tracked path
02:34:17.306: RT: delete subnet route to 10.1.1.0/24
02:34:17.306: RT: updating static 10.1.1.0/24 (0x0) via
The floating static route is installed into 172.16.13.3
the routing table 02:34:17.306: RT: add 10.1.1.0/24 via 172.16.13.3, static
metric [10/0]
02:34:17.310: RT: updating static 10.1.1.0/24 (0x0) via
Probes are still sent to determine when 172.16.13.3
02:34:22.114: ICMP: dst (172.16.12.1) host unreachable rcv
this path is available again
from 172.16.12.2
Object Tracking Tip - Use EEM
event manager applet static_tracking
event syslog pattern "%TRACKING-5-STATE: 1 ip sla 1 reachability Up->Down"
action 1 wait 3
action 2 cli command "enable"
action 3 cli command "term len 0"
action 4 cli command "term exec prompt timestamp"
action 5 cli command "show log | append flash:log_output"
action 6 mail server "<mail_server_ip>" to "<email_address>" from "<sender>" subject “Link C-D is down."
end
A
Connectivity is unstable.
Any helpful tools?
C
www.cisco.com/web/go/eem 10.1.1.0/24
Static Routing with BFD
Interface GigabitEthernet0
ip address 2.2.2.1 255.255.255.252
bfd interval 500 min_rx 500 multiplier 5
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 51
Discard Routes
A
• Suppose
10.1.0.0/16
A is advertising a default
route to B
B is advertising the summary
10.1.0.0/16 to A
B is not building a discard route 0.0.0.0/0
10.1.3.0/24
for this summary
A
10.1.3.1 via 10.1.0.0/16
10.1.0.0/16
0.0.0.0/0
B 10.1.3.0/24
10.1.3.1 via 0.0.0.0/0
10.1.1.0/24
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 58
BRKARC-2350 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 58
Overriding the Routing Table
What if I do not want to route my traffic based only on destination?
• Policy-Based Routing (PBR)
Route-map
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 60
BRKARC-2350 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 60
Route Maps
• Route maps allow you to:
Combine more than one type of filter into a single phrase
Use some rudimentary forms of logical ―AND‖ and ―O
R‖
to filter routes
Set some route attributes, rather than just permitting or
denying routes
Set Description
ip next-hop Set the next hop in the routing table or transmitted route
ip next-hop recursive Set the next hop to a subnet which is not directly connected
ip next-hop verify Set the next hop and verify availability using tracking
interface Set the output interface
metric Set the metric of the redistributed or transmitted route
metric-type Set the type of external route
External type 1 or type 2 for OSPF
tag Sets the route tag
Route Map Logic (AND)
Logical AND
Logical OR
Route Map Logic (NOT)
Logical NOT
Route Maps
If this match
succeeds
10.1.3.0/24
Policy Based Routing
• PBR applied to an interface only affects traffic that comes in that
interface
• Can configure a PBR policy local to the router
Router#show run
• Local PBR policy only affects traffic generated from the router itself
Policy Based Routing
• Can PBR be Dynamic?
You can use Object Tracking with PBR to track the availability of the next-hop
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 79
Performance Routing
• Traditional routing based on destination of packet
• Policy-based routing allows routing based on more information about
the packet
Source IP, Protocol, Ports Used, QoS markings, etc
www.cisco.com/go/pfr
Performance Routing
Learn traffic and applications
Discovers traffic going through network via Netflow
Reroute traffic
Dynamically alters path of application traffic if current service not in
line with specified policy to sustain performance
www.cisco.com/go/pfr
Performance Routing
Enterprise
Information
ISP1
BR Internet
Command
MC Voice
Apps
BR
Information ISP2
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 83
BRKARC-2350 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 83
Load Sharing
• Assume the same routing process attempts to install two routes for
the same destination in the RIB
• The routing process may allow the second route to be installed based
on its own rules
• Per-session (Per-destination)
Eth1 –
Eth2 –
Flow 1 -
Flow 2 -
• Per-packet
Eth1 –
Eth2 –
Load Sharing
Per-Session Load Sharing
• Often referred to as per-destination load sharing, even within Cisco
IOS
• This method is the default behavior and does not require any
additional configuration
• A session is a flow that shares the same source and destination.
Traffic with different source to destination pairs tend to take different
paths
• This method ensures that traffic for a given session arrive in order
• Has the potential for traffic polarization and is more effective as the
number of source to destination pairs increase
Load Sharing
Per-Packet Load Sharing
• To utilize this method, configure ―ipcef load-sharing per-packet‖
in interface configuration mode. Each outgoing interface must
have this command configured
• Uses a round-robin method to determine which path each
packet takes to the destination without consideration of source
to destination sessions
• Ensures traffic is more evenly distributed over
multiple paths
• Packets for a given source to destination session may take
different paths, introducing a greater potential for packets to
arrive out of sequence. Not advisable for all types of traffic
• Method used when process-switching
Load Sharing
3072256/3072256 == 1
and assigns hash buckets to nexthop 10.3.3.2 Ethernet1/0, adjacency IP adj out of Ethernet1/0, addr 10.3.3.2
Routing entry for 1.1.1.3/32 < 0 > IP adj out of Ethernet1/0, addr 10.3.3.1 044C4608
< 1 > IP adj out of Ethernet1/0, addr 10.3.3.2 044C44E8
Each
Known via "ospf 10", distance 110, metric 20, type extern 2,
forward metric 30 < 2 > IP adj out of Ethernet1/0, addr 10.3.3.1 044C4608 next-hop
Last update from 10.3.3.2 on Ethernet1/0, 00:01:04 ago < 3 > IP adj out of Ethernet1/0, addr 10.3.3.2 044C44E8 has 8 hash
Routing Descriptor Blocks: < 4 > IP adj out of Ethernet1/0, addr 10.3.3.1 044C4608 buckets
10.3.3.2, from 70.70.70.70, 00:01:04 ago, via Ethernet1/0 < 5 > IP adj out of Ethernet1/0, addr 10.3.3.2 044C44E8
Route metric is 20, traffic share count is 1 < 6 > IP adj out of Ethernet1/0, addr 10.3.3.1 044C4608
* 10.3.3.1, from 70.70.70.70, 00:01:24 ago, via Ethernet1/0 < 7 > IP adj out of Ethernet1/0, addr 10.3.3.2 044C44E8
The result is
Route metric is 20, traffic share count is 1 < 8 > IP adj out of Ethernet1/0, addr 10.3.3.1 044C4608 a 50/50
< 9 > IP adj out of Ethernet1/0, addr 10.3.3.2 044C44E8 chance of
Each packet that comes in gets
<10 > IP adj out of Ethernet1/0, addr 10.3.3.1 044C4608 getting each
<11 > IP adj out of Ethernet1/0, addr 10.3.3.2 044C44E8
measured against the HASH, and the next-hop
<12 > IP adj out of Ethernet1/0, addr 10.3.3.1 044C4608
HASH result determines which hash <13 > IP adj out of Ethernet1/0, addr 10.3.3.2 044C44E8
bucket the packet uses <14 > IP adj out of Ethernet1/0, addr 10.3.3.1 044C4608 1:1
<15 > IP adj out of Ethernet1/0, addr 10.3.3.2 044C44E8 load-sharing
Load Sharing
Hash
Hash Inputs
1
2
3 via Serial3/0
4
5 via Serial3/1
Hash Result 6
.
Source 10.1.1.1
Destination 192.168.239.1 Hash
1
2
3 via Serial3/0
4
5 via Serial3/1
6
CEF hashes the source =5
.
and destination Hash Result
addresses, and chooses
a bucket from the load The load share table
share table points to an adjacency
corresponding to one of
the next hops in the
routing table
Load Sharing
1 B
2 E
Then all routers will make the same
A
path decision and as a result the
2 1 links between B=>E and C=>F will
F
C never be used!
2
G
Load Sharing
Polarization
We can fix this if we change the inputs on each router by looking at
something else besides just the src/dst IP
-But this extra input would need to be unique per router, otherwise
every router will pick the same path again
1 D Universal Algorithm
• Tunnel algorithm—
-designed to balance the per-packet load when only a few source and destination
pairs are involved.
Enterprise 20%
Routing Updates
ISP1
GW1
Internet
CORE Email
Voice
Routing Updates GW2
ISP2 Apps
60%
www.cisco.com/go/pfr
Load Sharing
Performance Routing
Enterprise 39%
Information
ISP1
BR
Command
MC
Internet
Email
Voice
BR
Information ISP2 Apps
41%
CE2 link was 60% utilized and CE1 only 20%. PfR can identify
this and move traffic to better balance out the egress link
utilizations.
Done by configuring a policy to keep link utilization within a % of
each other, so one link isn’t utilized more than the other
www.cisco.com/go/pfr
Routing Segmentation and Separation
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 101
Routing Segmentation and Separation
• When would you want to separate routing operations?
Prevent any potential exchange of data or routing information.
• A VRF can help prevent the exchange of routes as well as data and
does not have to be constantly updated.
Routing Segmentation and Separation
What is a VRF?
• A VPN Routing and Forwarding (VRF) is an IOS routing instance.
All tables (routing/cef) maintained in routing instance (vrf)
All protocols/features run independently in each VRF instance
Allows for logical separation at Layer-3
Router#show ip vrf
Name Default RD Interfaces
blue <not set> Et0/1
Lo1
red <not set> Et0/2
Lo2
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 107
CPU
• Central Processing Unit responsible for carrying out
instructions.
• IOS uses a priority run-to-completion model for executing
processes.
• The task scheduler is responsible for scheduling and executing
kernel processes on the CPU
• Process Priorities:
Critical
High
Medium
Low
• There is no preemption but higher priority processes have
more opportunity to access the CPU.
CPU
CPU utilization for five seconds: 5%/2%; one minute: 3%; five minutes: 2%
PID Runtime (ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
[snip]
2 68 585 116 1.00% 1.00% 0% 0 IP Input
17 88 4232 20 0.20% 1.00% 0% 0 BGP Router
18 152 14650 10 0% 0% 0% 0 BGP Scanner
CPU
www.cisco.com/web/go/eem
Routing and Router Resources:
Memory
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 114
Memory
• Managed in 2 pools: Processor and I/O
• The processor memory pool is the general memory pool common to
all IOS systems including storage for routing information.
• The I/O pool or packet memory manages memory for interface packet
buffers.
Memory
How does routing information affect memory resources?
• Most common example of where we see this is the storing of BGP
prefixes.
• BGP generally carries the largest number of prefixes as well as the
potential to store multiple tables.
Memory
• How much memory do I need to store my routing information?
Full BGP table
Multiple feeds
Route filtering
Soft reconfiguration inbound
Default route
Memory Usage Example
• BGP Profile 1
Baseline memory usage with no BGP peers
• BGP Profile 2
1 BGP peer sending 300,000 routes
• BGP Profile 3
2 BGP peers both sending the exact same 300,000 prefixes
Unique AS path and next-hop IP information from each peer
• BGP Profile 4
2 BGP peers both sending 300,000 unique prefixes with zero overlap
Unique AS path and next-hop IP information from each peer
Memory Usage Example
BGP Routes vs. Memory Consumption
1000000000
900000000
800000000
Total Memory (Bytes)
700000000
600000000
Free
500000000
Used
400000000
300000000
600000
200000000
300000 600000 Routes
No Routes
100000000 Routes Routes
0
1 2 3 4
BGP Profile #Peers #Routes Memory
1 0 0 27.5MB
The amount of memory used BGP Profile
to store prefixes also depends 2 1 300,000 221.2MB
on the amount of overlap 3 2 600,000 245.1MB
between peers.
4 2 600,000 416.1MB
Memory
Monitoring memory availability
• Show commands
show process memory sorted
show memory statistics history
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 121
Recommended Reading