TSHOOT Exam Guide v3.2
TSHOOT Exam Guide v3.2
TSHOOT Exam Guide v3.2
The Online
Introduction
I started www.ccnpguide.com as a way for me to capture technical notes as I prepared for the three major CCNP Exams SWITCH, ROUTE, & TSHOOT. As I began sharing my notes with the world, I immediately started to receive feedback on the three exams focus areas and how difficult they had become. What I realized was that the exam prep resources available (read: Cisco Press Books) were not even covering all of the exam topics, including some that you were required to configure in live simulation scenarios. First-time fail rates seemed normal and a big part of that was because the some of the simulation scenarios required you to know some extremely specific protocol configuration details that most network professionals just wouldnt know off the top of their heads. I began to tailor my notes to include topics that were not being covered in official exam guides and trimmed down those that just were not necessary. The feedback was overwhelmingly positive from the online community! The problem is, of course, that the notes were not formatted well for off-line consumption and didnt include enough lab/scenario-based examples. This guide is an answer to the countless requests to create a portable, comprehensive, and exam-focused TSHOOT prep guide. Ive refined the online notes even more to focus exclusively on exactly what you Cisco expects you to know on exam day. If you have questions, exam feedback, or want to reach out to me directly - shoot me an email at [email protected]. I promise youll get a response.
Best of luck.
2|Page
Network Maintenance The Art of Troubleshooting Layer 2 Troubleshooting Layer 3 Troubleshooting TSHOOT Exam Methodology
4 12 15 22 31
Shortcuts.
3|Page
Cisco Chapter 1:
Maintenance is no doubt an important component to network stability and that's why we see it covered on the troubleshooting exam. This is perfect multiple-choice style content as it is difficult to test your understanding of maintenance concepts and methodologies on a simulation-focused exam like TSHOOT. Keep that in mind as you walk through the following topics.
Good troubleshooting reduces the time an outage lasts, good maintenance minimizes outages themselves.
Maintenance Methodologies
Several well known maintenance models have been defined by a number of organizations. Many organizations use parts of several instead of adopting one method completely, but it is important as a network engineer to understand what models exist and how they translate into improving your organization. A documented maintenance strategy is worth its weight in gold.
5|Page
FCAPS
FCAPS is an IT maintenance model created by ISO that categorizes network management into five parts. FCAPS is an acronym using the first letters of the five categories it includes.
Fault management
Preventive maintenance
Configuration management
Both hardware and software installation and configuration Change control Inventory management
Accounting management
Capacity planning Cost efficiency
Performance management
Maximize performance on existing network investments
Security management
Confidentiality, integrity, availability (CIA) Authentication, authorization, accounting (AAA) Encryption Intrusion detection/prevention
6|Page
7|Page
Documentation
Up-to-date, clear, and complete infrastructure documentation is crucial to reduce recovery times and maintain a robust networked environment. Different levels of detail are appropriate for different audiences, but some common details that should be documented include: Production configurations Inventory (including serial numbers, support info, etc.) Circuit information Network drawings IP address assignments Another important component to network documentation is a performance baseline, or snapshot. It captures the expected performance of your network systems like link bandwidth, WAN jitter and delay, and port status. This is a tremendous help during troubleshooting efforts because without knowing what normal levels are, detecting abnormal traffic behavior becomes very subjective.
IOS Tools
Configuration
Configurations should be backed up periodically or after changes are made. One of the simplest methods is to save the configuration as a text file on a remote TFTP or FTP server. TFTP and FTP servers are available on all modern operating systems and free, open source offerings are widely available. Adding the date to the saved configuration can make rolling back changes easier in the future. Here's an example of a router saving it's configuration to a local TFTP server:
8|Page
RouterA# copy run tftp Address of name of remote host []? 10.10.1.35 Destination filename [routera-config]? Routera_config_02082012
Syslog
Syslog is a tool that collects alerts from network devices and stores them on a common log. Obviously, this can be very handy when you need to troubleshoot an issue across many devices. Know that every syslog message contains two parts, a severity level and a facility. The severity level goes from 0 to 7 with 0 being the most severe to 7 being simply informational.
9|Page
NTP
Alerting is important, but if the timestamps that are included are off then the alerts are unreliable (and next to useless). NTP stands for Network Time Protocol and is used to keep accurate and consistent time on all network devices. NTP works by pulling the current time from a time server, which is assigned by stratum. Stratum 1 clocks are synchronized directly with an atomic clock; stratum 2 clocks get their time from stratum 1 clocks, etc. Configuring NTP is easy - just point the device to the proper time server: Switch(config)# ntp server ip_address_of_ntp_server To verify: Switch# show ntp status
One last note for NTP, it is important to consider the time zone that each device is set to. Make sure you have it consistent (ex. local time zones, GMT, HQ time zones, etc)
Archive
Cisco has developed a built-in configuration backup and restore feature, called archive. The archive function maintains a copy of the current configuration as well as a set of past configurations. If a configuration change is made with unpleasant results, the switch or router can roll back to a previous configuration relatively easily. There are several keywords available inside archive configuration mode. Here is a list of some of the most common:
10 | P a g e
Path
Specifies where you want the backup configuration stored (ex. flash, tftp server, etc.). Examples: archive path flash://routerc archive path tftp://192.168.1.22/routerc.txt
Write-memory
When the write-memory keyword is configured, a backup of the configuration will be automatically saved every time the configuration is manually saved.
Time-period
Sets the maximum time allowed before another backup is automatically saved. When the archive function backs up a configuration, it appends a -1, -2, -3, etc. to the end of the file name depending on how many have already been saved. It will count up to 14 (represented as filename-14) and then cycle back to 1. If your time-period is set too frequently, then you're backups may be written over too often.
11 | P a g e
Cisco Chapter 3:
642 832
12 | P a g e
There are two elements to good troubleshooting - preparation and technique. Preparation comes in the form of documentation, change control, and understanding of the environment among other things. The second part, technique, is just as important. There are a number of methods to tackle the same problem. To be honest, Cisco doesn't promote a specific approach for the CCNP TSHOOT exam. The important part is that you are consistent and your troubleshooting methodology follows a structured approach.
Structured Troubleshooting
What Cisco calls structured troubleshooting simply means you use a system to solve a problem by collecting information about the problem, forming a hypothesis, and then testing it. The structured approach also is helpful when the hypothesis you create fails. It may rule out many more scenarios and likely leads to the next hypothesis to test. The recovery time for a structured troubleshooting approach is usually much less than randomly changing configurations or settings in a hurry to try and get things working. There are several common structured troubleshooting approaches, with these being the most common:
Bottom-Up
Start with the OSI physical layer and work your way up.
Top-Down
Start with the OSI application layer and work your way down.
Follow-the-Path
Consider the path a packet would take from source to destination, checking each node/device/configuration along the way.
Spot-the-Difference
This is where configurations are compared between what is currently running and what the expected configurations should be.
Move-the-Problem
Move a device to see if the problem moves with it.
13 | P a g e
If a configuration change was the culprit, think about your current change control policy and ask if it needs to be updated.
14 | P a g e
Cisco Chapter 3:
642 832
Layer 2 Troubleshooting
15 | P a g e
FCS-Err
Usually a cabling issue.
Xmit-Err
The transmission buffers are full. This is sometimes seen when switching from a fast link to a slower one.
Undersize, Giants
The transmitting NIC may have problems.
16 | P a g e
Spanning Tree
Spanning Tree Protocol is a loop prevention mechanism to allow redundant Ethernet network connections. Here is an important summary of how each switch determines Spanning Tree port roles: 1. Each switch periodically transmits BPDUs that include its bridge ID, current root bridge, and cost to that root bridge. Additionally, each switch starts by assuming it is the root bridge. 2. If a switch receives a BPDU from another switch with a different root, it does a comparison. If the BPDU has a lower advertised root, the switch changes its root to match and recalculates the cost to the new root. The port that received the BPDU is now the root port - all others become designated ports. 3. If a switch receives two BPDUs with the same root, it then compares costs and uses the port with the lowest cost. The port with the higher cost is blocked - also called a non-designated port.
To quickly review STP costs, below is a list of link costs based on interface speed. Link Speed Ethernet Fast Ethernet Gigabit Ethernet Ten Gigabit Ethernet STP Cost 100 19 4 2
After the whole process, there will be only one root bridge - with each non-root switch having only one root port. To see the status of spanning tree: # show spanning-tree vlan vlan-id To view sent/received BPDU information for a switch: #show spanning-tree interface interface detail
17 | P a g e
Broadcast Storms
Broadcasts storms can occur due to Spanning Tree misconfigurations and/or rogue switches being added which closes a loop. Regardless, a broadcast storm will be obvious when the switch slows way down, becomes unresponsive, and all the links light up solid green. The CLI may be very slow to respond if you still have remote access to it, so often times to fastest way to fix the problem is to physically begin pulling redundant links.
Troubleshooting EtherChannels
EtherChannel issues usually fall into one of three categories:
1. Every port participating in an EtherChannel must have identical speed, duplex, access or trunk settings. If an EtherChannel isn't forming, check each port configuration. 2. Both sides of the EtherChannel must be configured as a bundle directly or be using a link aggregation protocol (LACP or PAgP). If one side is configured as an EtherChannel and the other side is not, look for error-disabled EtherChannel ports on the EtherChannel-enabled switch. 3. If traffic is only flowing over a single link in a bundle, it is likely that the hash algorithm should be adjusted to use different seed values. Also note that link bundles should be used in even numbered pairs like 2, 4, 8, etc.
18 | P a g e
VLANs
When troubleshooting issues that you suspect are related to VLAN logic, you should first make sure you have tested for physical layer issues like bad cabling, a power failure, or bad switch ports. Also, check that you are not dealing with an issue with the switch itself - things like software bugs, loops, or ARP problems. VLAN issues usually come in the form of misconfigured VLANs, improper VTP mode, trunk issues, and native VLAN mismatches.
Switch Tables
It is important that you understand what show commands display information on what switch tables. These will come in handy when you are isolating a switching issue.
# show mac-address
# show vlan
Trunk Assignments
19 | P a g e
20 | P a g e
HSRP
HSRP is the primary FHRP covered on the TSHOOT exam, so let's go through the basics one more time. HSRP is configured using the standby command under interface configuration mode. Routers in the same HSRP group share a common MAC and virtual IP address. The standby configuration statements define the HSRP group as well as the virtual IP in use. Each HSRP-enabled router has a default HSRP priority of 100 (remember, highest wins). If another router joins the group with a higher priority it will still not become the active router unless the preempt command is applied.
An example HSRP configuration could look something like: Router(config)# interface gig1/1 Router(config-if)# ip address 192.168.1.2 Router(config-if)# standby 4 ip 192.168.1.1 Router(config-if)# standby 4 priority 200 Router(config-if)# standby 4 preempt
To show the current HSRP status, issue: # show standby # show standby brief
21 | P a g e
Cisco Chapter 4:
642 832
Layer 3 Troubleshooting
22 | P a g e
Routing Basics
Before we get into the layer 3 troubleshooting methods, we first need to make sure we have a basic understanding of how routers and multilayer switches route traffic. Three tables are used: the routing table, ARP table, and CEF mappings. The routing table pairs network prefixes with the router's preferred next hop address or interface. Packets are routed based on the output of the routing table by first matching the longest prefix and then using other IGP-specific metrics. The show ip route command displays the contents of the routing table. After the router has determined what the next-hop address is, the router then needs to translate that into a layer 2 MAC address. The ARP table is exactly what this is for. The show ip arp command will display the current ARP pairings. Lastly, CEF is used in layer 3 switches to optimize routing and layer 2 headers. To view the CEF entries, use the show ip cef command.
23 | P a g e
Step 1
Step 2
Step 3
Is there a more desirable route being used? (longer prefix or lower administrative distance)
Let's dissect each of these for the major routing protocols one at a time.
EIGRP
First, verify connectivity to the remote networks using pings and by taking a look at the local routing table. As a reminder, EIGRP stores its information in three different tables: the EIGRP interface table, neighbor table, and topology table.
24 | P a g e
25 | P a g e
Next, issue a show ip eigrp interface to make sure the interfaces you expect are participating in the EIGRP process. Lastly, route maps or distribution lists could be blocking routing traffic. Do a show ip protocols to display any distribute lists.
OSPF
These steps for troubleshooting OSPF are very similar to EIGRP. First, verify that there is a problem using pings and by taking a look at the routing table. OSPF stores its information in three different tables: the OSPF interface table, neighbor table, and link-state database.
26 | P a g e
The OSPF protocol values can be seen using the show ip ospf interfaces command. Lastly, route maps or distribution lists could be blocking routing traffic. Do a show ip protocols to display any distribute lists.
27 | P a g e
BGP
BGP stores its information in two tables: the BGP neighbor table and the BGP table.
BGP Table
This table contains all the received BGP prefixes as well as their associated attributes lists. Perhaps most importantly, it also shows the BGP best path to each destination. Use the show ip bgp command to display its contents.
28 | P a g e
Remember that BGP requires bidirectional communication as well as matching AS numbers and authentication. The show run or show ip bgp command will display that information. Also, consider that route maps or distribution lists could be blocking routing traffic. Do a show ip protocols to display any distribute lists.
Route Redistribution
Route redistribution can be a tricky situation to troubleshoot, but understanding the following concepts should be helpful. 1. Redistributed routes require an existing entry in the routing table. If the redistributing router does not have a routing table entry for the route being redistributed, it will not work. Seems simple, but it should checked right away. 2. Routing loops are a common problem with multi-router routing redistribution. Use a single router to perform the redistribution if possible. 3. Understand that redistributed routes lose their native metric information. When redistributing into EIGRP, a default metric MUST be set or no route will be imported. When redistributing into OSPF, all routes will be imported as classful unless the subnets keyword is appended to the end of the redistribution statement.
29 | P a g e
30 | P a g e
Heres the trick: Start each ticket by perform an ipconfig inside client 1s terminal. The IP address of Client 1 will very quickly narrow close to half of the tickets for you. All of the problem descriptions begin with Client X cannot connect to Destination Y. Knowing this, open ticket #1 and jump onto the clients terminal. Run the command ipconfig. The output will either give you a usable IP address, like 10.2.1.4 or give you an unusable IP address in the 169.x.x.x range. If the client has a 169.x.x.x address, then it is not getting a proper IP address from the local DHCP server. That means that it is likely a layer 2 connectivity issue on the connected access switch ASW1, the distribution switch DSW1, or a DHCP-related issue on the local router (R4 in this case).
31 | P a g e
1. Verify that the interface connected to the client (fa1/0/1) is up, running in access mode (not a trunk), and is assigned to the proper VLAN.
To check that the interface is in up/up status ASW1#show int fa1/0/1 To check that it configured for access mode ASW1#show run interface fa1/0/1 To check the ports VLAN assignment - ASW1#show run interface fa1/0/1 (Look for switchport access vlan 10)
2. Check for port security applied to the interface connected to the client (fa1/0/1) ASW1#show int fa1/0/1 (FastEthernet1/0/1 is down, line protocol is down (err-disabled) will show up in the first line) To resolve this, the port security configuration would need to be removed from ASW1 and fa 1/0/1 would require a shut/no shut.
3. Verify that the PortChannel interfaces in PortChannel are up and participating in the bundle. ASW1#show etherchannel summary (Look for po13) DSW1#show etherchannel summary (Look for po13)
32 | P a g e
4. Check the trunk link between the access switch, ASW1 and the local distribution switch, DSW1 to make sure the production VLANs are not being filtered on the trunk interface. ASW1#show ASW1#show ASW1#show ASW1#show DSW1#show DSW1#show DSW1#show DSW1#show int run run run int run run run po13 trunk interface fa1/0/19 interface fa1/0/20 interface po13 po13 trunk interface fa1/0/19 interface fa1/0/20 interface po13
First do a show interface trunk to see what VLANs are active on the trunk. If VLAN 10 is not active based on the output, use the following sh run commands to pinpoint the configuration error.
5. Check any DHCP-related issues on the distribution switch. Some production networks run DHCP servers on their local distribution switches to avoid the headaches of a dedicated, physical DHCP server. Doing so requires few devices and keeps DHCP services active at remote sites when the WAN goes down. Take a look at DSW1 for any DHCP server configurations. If one exists, the normal dhcp pool and network statements should be verified. Another very important statement to look out for is a ip dhcp excluded-address x.x.x.x x.x.x.x statement. In the scenario that Cisco provides here, only the 10.2.1.1 and 10.2.1.2 addresses should be included in any excluded-address statements. Both addresses are the VLAN 10 interface IPs, so they should not be included in any DHCP leases. If any other excluded-address range is configured on DSW1 and contains 10.2.1.4 (Client 1s IP) it is a problem and will block Client 1 from obtaining an IP address.
33 | P a g e
Ping the VLAN 10 default gateway address, 10.2.1.254 to check L3 connectivity to the distribution switch, DSW1 From the client, ping the DSW1s VLAN 10 address to verify the client has layer three connectivity to the switch. If the ping fails, there is likely IP address filtering going in somewhere on the local segment. Because the client is getting a valid DHCP IP address from DSW1, there is L2 connectivity but something else is blocking (or dropping) the IP traffic from the client. Remember those VLAN Access-maps you learned about in the SWITCH exam? Well it may be the case that a VLAN access-map is dropping traffic on DSW1. Access-maps are generally configured like this: vlan access-map EXAMPLE 10 drop match ip address 5 ! vlan filter EXAMPLE vlan-list 10 ! ip access-list standard 5 permit 10.2.0.0 0.0.255.255 ! interface VLAN10 ip address 10.2.1.1 255.255.255.0
In this example, ACL 5 and VLAN 10 access-map EXAMPLE are dropping all traffic from the entire 10.2.x.x/8 subnet. If you see any VLAN 10 access-maps filtering the same subnet as Client 1, the solution would be to remove the access-map completely.
34 | P a g e
If Client 1 has an IP address of 10.x.x.x AND can ping the gateway on DSW1 BUT cannot ping the fa0/1 interface of R4
The neighbor relationship between DSW1 and R4 needs to be checked. R4 and DSW1 share routing information via EIGRP Autonomous System 10. Do a show run on both devices and take a peek at the EIGRP configurations. Make sure the proper network statements are applied and that the correct AS number (10) is configured. Check for any EIGRP passive interface commands applied to the configurations. Remember that if either device is running an interface as passive, EIGRP hellos will not be sent and routes will not be shared.
If Client 1 has an IP address of 10.x.x.x AND can ping the gateway on DSW1 BUT cannot ping the serial0/0/0/0.34 interface of R4
Notice that R4 is the redistribution point between EIGRP and OSPF. If you cannot connect to the other side of R4, the redistribution statements need to be looked at.
35 | P a g e
Make sure a metric is set on the redistribute ospf x metric statement and that the subnets keyword id used on the redistribute eigrp 10 statement. If any route maps are applied to the redistribution configuration, make sure the route map names are configured correctly.
If Client 1 has an IP address of 10.x.x.x AND can ping the gateway AND can ping R2 [10.1.1.2] BUT cannot ping R1 [10.1.1.1]
If we can ping all the way from Client 1 to R2 but not to R1, then there is an issue between routers R1 and R2.
R1 and R2 share routes using OSPF, so we need to verify they are exchanging OSPF route information. The OSPF configuration should be looked at carefully on both R1 and R2. Based on the diagram Cisco has made available, R1s OSPF configuration should look something similar to: interface Serial0/0/0/0.12 point-to-point ip address 10.1.1.1 255.255.255.252 ip nat inside ! router ospf 1 network 10.1.1.0 0.0.0.3 area 12 default-information originate If you see any OSPF authentication-related lines, make sure they are applied exactly the same on both R1 and R2! An example interface authentication configuration: ip ospf authentication message-digest ip ospf message-digest-key 1 md5 EXAM
36 | P a g e
If Client 1 has an IP address of 10.x.x.x AND can ping the gateway AND can ping R1 [10.1.1.1]
Successful.
Fails.
If Client 1 can ping R1 and R1 can ping the web server, then R1, R2, R3, R4 and DSW1 and DSW2 can also ping web server. This means that the issue is likely a NAT translation misconfiguration on R1. Check that the inside and outside NAT statements are applied to the correct interfaces on R1. Also, check for any ACLs that NAT is using. If there are any, make sure 10.2.x.x is included in the permit statements.
37 | P a g e
At this point weve established that Client 1 has connectivity to R1 BUT R1 does not have connectivity to the web server. Again, this means we need to take a closer look at R1s configuration.
1. The first item that needs to be verified is R1s BGP configuration. R1 is using BGP to connect to the web servers remote network, so that is the first protocol to check. Comb through the BGP configuration line-by-line. R1# show run Make sure the network, neighbor, and AS numbers are correct.
2. Make sure there are no ACLs blocking the remote 209.65.200.224/30 network on R1. If you run into any inbound ACLs on R1, make sure they permit 209.65.200.224 0.0.0.3.
38 | P a g e
Best of luck.
39 | P a g e
40 | P a g e
41 | P a g e
42 | P a g e