DMVPN with VRF on the NBMA Interface
Use DMVPN Phase 3 with OSPF, NOT use IPsec protection.
R1
interface Tunnel0 ip address 10.0.0.1 255.255.255.0 ip nhrp map multicast dynamic ip nhrp network-id 123 ip nhrp redirect ip ospf network point-to-multipoint
tunnel mode gre multipoint
tunnel source FastEthernet0/0
interface FastEthernet0/0 ip address 155.0.0.1 255.255.255.0 interface FastEthernet0/1 ip address 192.168.1.1 255.255.255.0 router ospf 123 router-id 1.1.1.1 network 10.0.0.1 0.0.0.0 area 0 network 192.168.1.1 0.0.0.0 area 0
R2
interface Tunnel0 ip address 10.0.0.2 255.255.255.0 no ip redirects ip nhrp map multicast 155.0.0.1 ip nhrp map 10.0.0.1 155.0.0.1 ip nhrp network-id 123 ip nhrp nhs 10.0.0.1
ip nhrp shortcut ip ospf network point-to-multipoint tunnel mode gre multipoint
tunnel source FastEthernet0/0
interface FastEthernet0/0 ip address 155.0.0.2 255.255.255.0
interface FastEthernet0/1 ip address 192.168.2.2 255.255.255.0
router ospf 123 router-id 2.2.2.2 network 10.0.0.2 0.0.0.0 area 0 network 192.168.2.2 0.0.0.0 area 0
R3 interface Tunnel0 ip address 10.0.0.3 255.255.255.0 no ip redirects ip nhrp map multicast 155.0.0.1 ip nhrp map 10.0.0.1 155.0.0.1 ip nhrp network-id 123 ip nhrp nhs 10.0.0.1
ip nhrp shortcut ip ospf network point-to-multipoint tunnel mode gre multipoint
tunnel source FastEthernet0/0
interface FastEthernet0/0 ip address 155.0.0.3 255.255.255.0
interface FastEthernet0/1 ip address 192.168.3.3 255.255.255.0
router ospf 123 router-id 3.3.3.3 network 10.0.0.3 0.0.0.0 area 0 network 192.168.3.3 0.0.0.0 area 0
Let WAN interface belong to VRF.
R1/R2/R3 ip vrf DMVPN_VRF rd 99:999
R1
interface FastEthernet0/0 ip vrf forwarding DMVPN_VRF ip address 155.0.0.1 255.255.255.0
R2 interface FastEthernet0/0 ip vrf forwarding DMVPN_VRF ip address 155.0.0.2 255.255.255.0
R3 interface FastEthernet0/0 ip vrf forwarding DMVPN_VRF ip address 155.0.0.3 255.255.255.0
Verify
R1#sh ip vrf Name Default RD Interfaces DMVPN_VRF 99:999 Fa0/0
R1#sh ip ospf nei (none)
R1#sh ip ospf int bri Interface PID Area IP Address/Mask Cost State Nbrs F/C Fa0/1 123 0 192.168.1.1/24 1 DR 0/0 Tu0 123 0 10.0.0.1/24 1000 P2MP 0/0
R1#sh ip nhrp (none)
R2#sh ip nhrp 10.0.0.1/32 via 10.0.0.1 Tunnel0 created 00:16:43, never expire Type: static, Flags: used NBMA address: 155.0.0.1
OSPF neighbors will not be established and NHRP packets will not be sent.
This is because 155.0.0.1 specified as the NHS address does not exist on the global routing table.
R2#sh ip route
Gateway of last resort is not set
10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks C 10.0.0.0/24 is directly connected, Tunnel0 L 10.0.0.2/32 is directly connected, Tunnel0 192.168.2.0/24 is variably subnetted, 2 subnets, 2 masks C 192.168.2.0/24 is directly connected, FastEthernet0/1 L 192.168.2.2/32 is directly connected, FastEthernet0/1
R2#sh ip route vrf DMVPN_VRF
Routing Table: DMVPN_VRF (omit) Gateway of last resort is not set
155.0.0.0/16 is variably subnetted, 2 subnets, 2 masks C 155.0.0.0/24 is directly connected, FastEthernet0/0 L 155.0.0.2/32 is directly connected, FastEthernet0/0
Configure the tunnel vrf <VRF-name> command to use the VRF routing table
for address resolution.
R1/R2/R3 int tun0 tunnel vrf DMVPN_VRF
R1# *Jun 16 16:03:15.639: %OSPF-5-ADJCHG: Process 123, Nbr 2.2.2.2 on Tunnel0
from LOADING to FULL, Loading Done *Jun 16 16:03:16.143: %OSPF-5-ADJCHG: Process 123, Nbr 3.3.3.3 on Tunnel0
from LOADING to FULL, Loading Done
R1#sh ip nhrp 10.0.0.2/32 via 10.0.0.2 Tunnel0 created 00:00:14, expire 01:59:56 Type: dynamic, Flags: unique registered used NBMA address: 155.0.0.2
10.0.0.3/32 via 10.0.0.3 Tunnel0 created 00:00:14, expire 01:59:56 Type: dynamic, Flags: unique registered used NBMA address: 155.0.0.3
R1#sh ip ospf nei
Neighbor ID Pri State Dead Time Address Interface 3.3.3.3 0 FULL/ - 00:01:43 10.0.0.3 Tunnel0 2.2.2.2 0 FULL/ - 00:01:43 10.0.0.2 Tunnel0
R2#sh ip route ospf (omit) Gateway of last resort is not set
10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks O 10.0.0.1/32 [110/1000] via 10.0.0.1, 00:00:53, Tunnel0 O 10.0.0.3/32 [110/2000] via 10.0.0.1, 00:00:53, Tunnel0 O 192.168.1.0/24 [110/1001] via 10.0.0.1, 00:00:53, Tunnel0 O 192.168.3.0/24 [110/2001] via 10.0.0.1, 00:00:53, Tunnel0
R2#ping 192.168.3.3 source f0/1 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 192.168.3.3, timeout is 2 seconds: Packet sent with a source address of 192.168.2.2 !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 100/276/440
R2#sh ip nhrp 10.0.0.1/32 via 10.0.0.1 Tunnel0 created 00:02:26, never expire Type: static, Flags: used NBMA address: 155.0.0.1
10.0.0.3/32 via 10.0.0.3 Tunnel0 created 00:00:04, expire 01:59:55 Type: dynamic, Flags: router implicit NBMA address: 155.0.0.3
192.168.2.0/24 via 10.0.0.2 Tunnel0 created 00:00:04, expire 01:59:55 Type: dynamic, Flags: router unique local NBMA address: 155.0.0.2 (no-socket)
192.168.3.0/24 via 10.0.0.3 Tunnel0 created 00:00:04, expire 01:59:55 Type: dynamic, Flags: router used rib nho NBMA address: 155.0.0.3
R2#traceroute 192.168.3.3 source f0/1 Type escape sequence to abort. Tracing the route to 192.168.3.3 VRF info: (vrf in name/id, vrf out name/id) 1 10.0.0.3 172 msec 224 msec 184 msec
SPOKE can communicate directly with each other.
Let LAN interface belong to VRF.
R1 interface FastEthernet0/1 ip vrf forwarding DMVPN_VRF ip address 192.168.1.1 255.255.255.0
R2 interface FastEthernet0/1 ip vrf forwarding DMVPN_VRF ip address 192.168.2.2 255.255.255.0
R3 interface FastEthernet0/1 ip vrf forwarding DMVPN_VRF ip address 192.168.3.3 255.255.255.0
R1#sh ip vrf Name Default RD Interfaces DMVPN_VRF 99:999 Fa0/0 Fa0/1
R2#sh ip vrf Name Default RD Interfaces DMVPN_VRF 99:999 Fa0/0
Fa0/1
R3#sh ip vrf Name Default RD Interfaces DMVPN_VRF 99:999 Fa0/0 Fa0/1
R1#sh ip ospf nei
Neighbor ID Pri State Dead Time Address Interface 3.3.3.3 0 FULL/ - 00:01:41 10.0.0.3 Tunnel0 2.2.2.2 0 FULL/ - 00:01:59 10.0.0.2 Tunnel0
R1#sh ip route ospf (omit) Gateway of last resort is not set
10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks O 10.0.0.2/32 [110/1000] via 10.0.0.2, 00:04:57, Tunnel0 O 10.0.0.3/32 [110/1000] via 10.0.0.3, 00:04:47, Tunnel0
R2#sh ip route (omit) Gateway of last resort is not set
10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks C 10.0.0.0/24 is directly connected, Tunnel0 O 10.0.0.1/32 [110/1000] via 10.0.0.1, 00:05:22, Tunnel0 L 10.0.0.2/32 is directly connected, Tunnel0 O 10.0.0.3/32 [110/2000] via 10.0.0.1, 00:05:22, Tunnel0
R2#sh ip route vrf DMVPN_VRF
Routing Table: DMVPN_VRF (omit) Gateway of last resort is not set
155.0.0.0/16 is variably subnetted, 2 subnets, 2 masks C 155.0.0.0/24 is directly connected, FastEthernet0/0 L 155.0.0.2/32 is directly connected, FastEthernet0/0 192.168.2.0/24 is variably subnetted, 2 subnets, 2 masks C 192.168.2.0/24 is directly connected, FastEthernet0/1 L 192.168.2.2/32 is directly connected, FastEthernet0/1
R2#sh ip ospf int bri Interface PID Area IP Address/Mask Cost State Nbrs F/C Tu0 123 0 10.0.0.2/24 1000 P2MP 1/1
Since OSPF is enabled only with the tunnel interface, can not advertise the LAN side segment.
Configure the VRF on the tunnel interface and enable OSPF on the VRF.
R1 interface tunnel0 ip vrf forwarding DMVPN_VRF ip address 10.0.0.1 255.255.255.0
no router ospf 123
router ospf 123 vrf DMVPN_VRF router-id 1.1.1.1 network 10.0.0.1 0.0.0.0 area 0 network 192.168.1.1 0.0.0.0 area 0
R2 interface tunnel0 ip vrf forwarding DMVPN_VRF ip address 10.0.0.2 255.255.255.0
no router ospf 123
router ospf 123 vrf DMVPN_VRF router-id 2.2.2.2 network 10.0.0.2 0.0.0.0 area 0 network 192.168.2.2 0.0.0.0 area 0
R3 interface tunnel0 ip vrf forwarding DMVPN_VRF ip address 10.0.0.3 255.255.255.0
no router ospf 123
router ospf 123 vrf DMVPN_VRF router-id 3.3.3.3 network 10.0.0.3 0.0.0.0 area 0 network 192.168.3.3 0.0.0.0 area 0
R1#sh ip vrf Name Default RD Interfaces DMVPN_VRF 99:999 Fa0/0 Fa0/1 Tu0
R2#sh ip vrf Name Default RD Interfaces DMVPN_VRF 99:999 Fa0/0 Fa0/1 Tu0
R3#sh ip vrf Name Default RD Interfaces DMVPN_VRF 99:999 Fa0/0 Fa0/1 Tu0
R1#sh ip ospf int bri Interface PID Area IP Address/Mask Cost State Nbrs F/C Fa0/1 123 0 192.168.1.1/24 1 DR 0/0 Tu0 123 0 10.0.0.1/24 1000 P2MP 2/2
R1#sh ip ospf nei
Neighbor ID Pri State Dead Time Address Interface 3.3.3.3 0 FULL/ - 00:01:35 10.0.0.3 Tunnel0 2.2.2.2 0 FULL/ - 00:01:39 10.0.0.2 Tunnel
R1#sh ip route (none)
R1#sh ip route vrf DMVPN_VRF ospf
Routing Table: DMVPN_VRF (omit) Gateway of last resort is not set
10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks O 10.0.0.2/32 [110/1000] via 10.0.0.2, 00:01:31, Tunnel0 O 10.0.0.3/32 [110/1000] via 10.0.0.3, 00:01:15, Tunnel0 O 192.168.2.0/24 [110/1001] via 10.0.0.2, 00:01:31, Tunnel0 O 192.168.3.0/24 [110/1001] via 10.0.0.3, 00:01:15, Tunnel0
R2#sh ip route vrf DMVPN_VRF ospf
Routing Table: DMVPN_VRF (omit) Gateway of last resort is not set
10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks O 10.0.0.1/32 [110/1000] via 10.0.0.1, 00:02:19, Tunnel0 O 10.0.0.3/32 [110/2000] via 10.0.0.1, 00:01:57, Tunnel0 O 192.168.1.0/24 [110/1001] via 10.0.0.1, 00:02:19, Tunnel0 O 192.168.3.0/24 [110/2001] via 10.0.0.1, 00:01:57, Tunnel0
R2#ping 192.168.3.3 source f0/1 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 192.168.3.3, timeout is 2 seconds: Packet sent with a source address of 192.168.2.2 ..... Success rate is 0 percent (0/5)
R2#ping vrf DMVPN_VRF 192.168.3.3 source f0/1 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 192.168.3.3, timeout is 2 seconds: Packet sent with a source address of 192.168.2.2 !!!!! uccess rate is 100 percent (5/5), round-trip min/avg/max = 136/186/212 ms
R2#sh ip nhrp 10.0.0.1/32 via 10.0.0.1 Tunnel0 created 00:04:07, never expire Type: static, Flags: used NBMA address: 155.0.0.1
10.0.0.3/32 via 10.0.0.3 Tunnel0 created 00:00:05, expire 01:59:54 Type: dynamic, Flags: router implicit NBMA address: 155.0.0.3
192.168.2.0/24 via 10.0.0.2 Tunnel0 created 00:00:05, expire 01:59:54 Type: dynamic, Flags: router unique local NBMA address: 155.0.0.2 (no-socket)
192.168.3.0/24 via 10.0.0.3 Tunnel0 created 00:00:05, expire 01:59:53 Type: dynamic, Flags: router used rib nho NBMA address: 155.0.0.3
R2#traceroute vrf DMVPN_VRF 192.168.3.3 source f0/1 Type escape sequence to abort. Tracing the route to 192.168.3.3 VRF info: (vrf in name/id, vrf out name/id) 1 10.0.0.3 160 msec 156 msec 160 msec
Spokes can transit traffic to other spoke directly using VRF on tunnel interface.
Eneble IPsec Protection.
R1/R2/R3 crypto isakmp policy 123 encr 3des hash sha256 authentication pre-share group 2
crypto isakmp key CCIE address 0.0.0.0
crypto ipsec transform-set TRANS esp-3des esp-sha256-hmac mode transport
crypto ipsec profile PROFILE set transform-set TRANS
interface Tunnel0 tunnel protection ipsec profile PROFILE
R1/R2/R3 clear ip nhrp clear ip route *
R1#sh ip ospf nei (none)
R1#sh ip nhrp (none)
R1#debug ip ospf hello OSPF hello debugging is on
*Jun 15 17:34:37.199: OSPF-123 HELLO Fa0/1: Send hello to 224.0.0.5 area 0 from 192.168.1.1 *Jun 15 17:34:46.303: OSPF-123 HELLO Fa0/1: Send hello to 224.0.0.5 area 0 from 192.168.1.1 *Jun 15 17:34:55.339: OSPF-123 HELLO Fa0/1: Send hello to 224.0.0.5 area 0 from 192.168.1.1 *Jun 15 17:35:02.959: OSPF-123 HELLO Tu0: Send hello to 224.0.0.5 area 0 from 10.0.0.1 *Jun 15 17:35:04.487: OSPF-123 HELLO Fa0/1: Send hello to 224.0.0.5 area 0 from 192.168.1.1 *Jun 15 17:35:13.807: OSPF-123 HELLO Fa0/1: Send hello to 224.0.0.5 area 0 from 192.168.1.1 *Jun 15 17:35:23.043: OSPF-123 HELLO Fa0/1: Send hello to 224.0.0.5 area 0 from 192.168.1.1
Do packet capture, but OSPF Hello has not appeared.
If VRF is used for NBMA address, use keyring in encryption setting.
R1/R2/R3 no crypto isakmp key CCIE address 0.0.0.0
crypto keyring cisco vrf DMVPN_VRF pre-shared-key address 0.0.0.0 0.0.0.0 key CCIE
R1#sh ip ospf nei
Neighbor ID Pri State Dead Time Address Interface 3.3.3.3 0 FULL/ - 00:01:54 10.0.0.3 Tunnel0 2.2.2.2 0 FULL/ - 00:01:44 10.0.0.2 Tunnel0
R2#sh ip route vrf DMVPN_VRF ospf
Routing Table: DMVPN_VRF (omit) Gateway of last resort is not set
10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks O 10.0.0.1/32 [110/1000] via 10.0.0.1, 00:00:45, Tunnel0 O 10.0.0.3/32 [110/2000] via 10.0.0.1, 00:00:12, Tunnel0 O 192.168.1.0/24 [110/1001] via 10.0.0.1, 00:00:45, Tunnel0 O 192.168.3.0/24 [110/2001] via 10.0.0.1, 00:00:12, Tunnel0
R2#ping vrf DMVPN_VRF 192.168.3.3 source f0/1 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 192.168.3.3, timeout is 2 seconds: Packet sent with a source address of 192.168.2.2 !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 288/503/764 ms
R2#sh ip nhrp 10.0.0.1/32 via 10.0.0.1 Tunnel0 created 00:14:07, never expire Type: static, Flags: used NBMA address: 155.0.0.1
10.0.0.3/32 via 10.0.0.3 Tunnel0 created 00:02:42, expire 01:57:19 Type: dynamic, Flags: router implicit NBMA address: 155.0.0.3
192.168.2.0/24 via 10.0.0.2 Tunnel0 created 00:02:40, expire 01:57:19 Type: dynamic, Flags: router unique local NBMA address: 155.0.0.2 (no-socket)
192.168.3.0/24 via 10.0.0.3 Tunnel0 created 00:02:39, expire 01:57:20 Type: dynamic, Flags: router rib nho NBMA address: 155.0.0.3
R2#sh crypto engine connections active Crypto Engine Connections
ID Type Algorithm Encrypt Decrypt LastSeqN IP-Address 3 IPsec 3DES+SHA256 0 0 0 155.0.0.2 4 IPsec 3DES+SHA256 2 0 0 155.0.0.2 5 IPsec 3DES+SHA256 0 34 34 155.0.0.2 6 IPsec 3DES+SHA256 29 0 0 155.0.0.2 7 IPsec 3DES+SHA256 0 0 0 155.0.0.2 8 IPsec 3DES+SHA256 0 0 0 155.0.0.2 1002 IKE SHA256+3DES 0 0 0 155.0.0.2 1003 IKE SHA256+3DES 0 0 0 155.0.0.2
R2#sh crypto isakmp sa IPv4 Crypto ISAKMP SA dst src state conn-id status 155.0.0.3 155.0.0.2 QM_IDLE 1003 ACTIVE 155.0.0.1 155.0.0.2 QM_IDLE 1002 ACTIVE
IPv6 Crypto ISAKMP SA
R2#sh crypto session Crypto session current status
Interface: Tunnel0 Session status: UP-ACTIVE Peer: 155.0.0.3 port 500 IKEv1 SA: local 155.0.0.2/500 remote 155.0.0.3/500 Active IPSEC FLOW: permit 47 host 155.0.0.2 host 155.0.0.3 Active SAs: 2, origin: crypto map
Interface: Tunnel0 Session status: UP-ACTIVE Peer: 155.0.0.1 port 500 IKEv1 SA: local 155.0.0.2/500 remote 155.0.0.1/500 Active IPSEC FLOW: permit 47 host 155.0.0.2 host 155.0.0.1 Active SAs: 4, origin: crypto map
R2#sh crypto ipsec sa
interface: Tunnel0 Crypto map tag: Tunnel0-head-0, local addr 155.0.0.2
protected vrf: DMVPN_VRF local ident (addr/mask/prot/port): (155.0.0.2/255.255.255.255/47/0) remote ident (addr/mask/prot/port): (155.0.0.3/255.255.255.255/47/0) current_peer 155.0.0.3 port 500 PERMIT, flags={origin_is_acl,} #pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0 #pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0 #pkts compressed: 0, #pkts decompressed: 0 #pkts not compressed: 0, #pkts compr. failed: 0 #pkts not decompressed: 0, #pkts decompress failed: 0 #send errors 0, #recv errors 0
local crypto endpt.: 155.0.0.2, remote crypto endpt.: 155.0.0.3 path mtu 1500, ip mtu 1500, ip mtu idb (none) current outbound spi: 0x4F1621AE(1326850478) PFS (Y/N): N, DH group: none
inbound esp sas: spi: 0xEADE6212(3940442642) transform: esp-3des esp-sha256-hmac , in use settings ={Transport, } conn id: 7, flow_id: SW:7, sibling_flags 80004000, crypto map: Tunnel0-head-0 sa timing: remaining key lifetime (k/sec): (4608000/3377) IV size: 8 bytes replay detection support: Y Status: ACTIVE(ACTIVE)
inbound ah sas:
inbound pcp sas:
outbound esp sas: spi: 0x4F1621AE(1326850478) transform: esp-3des esp-sha256-hmac , in use settings ={Transport, } conn id: 8, flow_id: SW:8, sibling_flags 80004000, crypto map: Tunnel0-head-0 sa timing: remaining key lifetime (k/sec): (4608000/3377) IV size: 8 bytes replay detection support: Y Status: ACTIVE(ACTIVE)
outbound ah sas:
outbound pcp sas:
protected vrf: DMVPN_VRF local ident (addr/mask/prot/port): (155.0.0.2/255.255.255.255/47/0) remote ident (addr/mask/prot/port): (155.0.0.1/255.255.255.255/47/0)
(omit)
R2#sh ip route vrf DMVPN_VRF ospf
Routing Table: DMVPN_VRF (omit) + - replicated route, % - next hop override
Gateway of last resort is not set
10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks O 10.0.0.1/32 [110/1000] via 10.0.0.1, 00:05:48, Tunnel0 O 10.0.0.3/32 [110/2000] via 10.0.0.1, 00:05:15, Tunnel0 O 192.168.1.0/24 [110/1001] via 10.0.0.1, 00:05:48, Tunnel0 O % 192.168.3.0/24 [110/2001] via 10.0.0.1, 00:05:15, Tunnel0
R2#traceroute vrf DMVPN_VRF 192.168.3.3 source f0/1 Type escape sequence to abort. Tracing the route to 192.168.3.3 VRF info: (vrf in name/id, vrf out name/id) 1 10.0.0.3 256 msec 328 msec 220 msec
Care must be taken when combining DMVPN and VRF interface.
～Useful Sites～
■Dive into CCIE：Tunnel VRF and Crypto Keyring https://www.sash.jp/cciers/tunnel-vrf-and-crypto-keyring
■Networking with FISH：Tunnels and the Use of Front Door VRFs http://www.networkingwithfish.com/tunnels-and-the-use-of-front-door-vrfs/
