Quantcast
Channel: Bang Bang Sounds Like Machinery
Viewing all articles
Browse latest Browse all 13

When RPF Breaks Traceroute

0
0
I came across an interesting little problem recently which was quite fun to unravel....

I work on a hub and spoke network, where most traffic from the spokes follows the default route back to the hubs, except for a few specific destinations which must be reached through a public-network-facing interface.

One day I tried to run a traceroute from one of our site routers, towards one of these specific destinations and found that I couldn't get past the first hop (provider router).

fr-rt01#traceroute 155.231.48.203

Type escape sequence to abort.
Tracing the route to 155.231.48.203

1 10.197.251.1 4 msec 0 msec 0 msec
2 * * *
3 * * *


Here's the interface config:

interface GigabitEthernet0/0.801
description Exit to N3 gateway
encapsulation dot1Q 801
ip address 10.197.251.4 255.255.255.0
ip access-group N3-ACCESS-IN in
ip verify unicast reverse-path
no ip unreachables
ip inspect SDM_LOW in
ip inspect SDM_LOW out
ip nat outside
ip virtual-reassembly
no cdp enable
crypto map N3-CM
end


So I removed the ACL, then the inspect statements and then the "no ip unreachables", but I still got the same result.

The next hop looked good but I checked my route just to make sure:

fr-rt01#sh ip ro 155.231.48.203
Routing entry for 155.231.48.0/24
Known via "bgp 65139", distance 200, metric 1, type internal
Last update from 10.139.202.11 18:44:40 ago
Routing Descriptor Blocks:
* 10.139.202.11, from 10.139.202.11, 18:44:40 ago
Route metric is 1, traffic share count is 1
AS Hops 0

fr-rt01#sh ip ro 10.139.202.11
Routing entry for 10.139.200.0/22
Known via "static", distance 1, metric 0
Routing Descriptor Blocks:
* 10.197.251.1
Route metric is 0, traffic share count is 1


Then clutching at straws, I removed my "ip verify unicast reverse-path", and traceroute worked just like it should...

fr-rt01#traceroute 155.231.48.203

Type escape sequence to abort.
Tracing the route to 155.231.48.203

1 10.197.251.1 4 msec 0 msec 0 msec
2 81.147.220.170 8 msec 8 msec 8 msec
3 172.16.216.193 8 msec 8 msec 8 msec
4 217.36.152.160 8 msec 8 msec 8 msec
5 217.36.152.64 8 msec 8 msec 8 msec
6 10.100.100.67 12 msec 8 msec 8 msec


And then I went and read up on traceroute and RPF to try to understand what was going on

When traceroute sends each UDP packet out, it expects to get an ICMP type 11 code 0 (time exceeded) back from each intermediate hop/router, with a source IP address of its exit interface (towards me).

RPF does a reverse lookup of the source address against CEF to check that the receiving interface is one of the best return paths to that address.

Taking the first few addresses in the working traceroute:

fr-rt01#sh ip cef 81.147.220.170 
0.0.0.0/0, version 191479, epoch 0
0 packets, 0 bytes
via 192.168.13.196, Tunnel13196, 0 dependencies
next hop 192.168.13.196, Tunnel13196
valid adjacency

fr-rt01#sh ip cef 172.16.216.193
0.0.0.0/0, version 191479, epoch 0
0 packets, 0 bytes
via 192.168.13.196, Tunnel13196, 0 dependencies
next hop 192.168.13.196, Tunnel13196


And there's the problem: without a route to the intermediate addresses, I only have a default route, which leaves by a different interface. So RPF is doing its job properly and breaking my traceroute.

In this case, it seems that the best solution is to tell RPF to ignore my traceroute replies by adding an ACL defining them as exceptions:

fr-rt01#conf t
Enter configuration commands, one per line. End with CNTL/Z.
fr-rt01(config)#access-list 101 permit icmp any host 10.197.251.4 11 0
fr-rt01(config)#int gi0/0.801
fr-rt01(config-subif)#ip verify unicast reverse-path 101
fr-rt01(config-subif)#^Z
fr-rt01#


Which allows both features to work without side effects:

fr-rt01#traceroute 155.231.48.203

Type escape sequence to abort.
Tracing the route to 155.231.48.203

1 10.197.251.1 4 msec 0 msec 0 msec
2 81.147.220.170 8 msec 8 msec 8 msec
3 172.16.216.193 8 msec 8 msec 8 msec
4 217.36.152.160 8 msec 8 msec 8 msec
5 217.36.152.64 8 msec 8 msec 8 msec
6 10.100.100.67 12 msec 8 msec 8 msec
7 * * *

fr-rt01#sh access-list 101
Extended IP access list 101
10 permit icmp any host 10.197.251.4 ttl-exceeded (15 matches)





Viewing all articles
Browse latest Browse all 13

Latest Images

Trending Articles





Latest Images