Welcome to the Network Engineering Domain
Pape O. Fall's Blog

DMVPN Dual Hub/Dual Cloud – ASA – IPSec Encryption

In this specific use case, I will show you how to enable dual DMVPN clouds going over ASA firewalls as well as IPSec encryption. We will also take it up a notch and demonstrate DMVPN Phase 3 (Spoke to Spoke Communication), VRF configuration (We will be placing the source of the tunnels into their own VRFs), NAT (We will be natting the source of the tunnels to an unused public IP), PAT (We will translate Spoke 1 tunnel source to the outside IP address of the ASA).

DMVPN stands for Dynamic Multipoint VPN and it is a dynamic tunneling form of a virtual private network (VPN). At this point, if you are unfamiliar with DMVPN, I would suggest to revisit the following post first: DMVPN

Below is the high level diagram referencing our network to be built.

Here, we have distinct geophysical Data Centers, Hub1 located in Houston and Hub 2 located in Sydney, Australia. One remote site in Seville Spain (Spoke 1) and another in Venice, Italy.

Note that in a DMVPN network, the geophysical location here does not matter as it is agnostic to the solution. We will use EIGRP to advertise our networks across and BGP to receive a default route from the ISPs. We will try and deploy the entire solution without the use of any static routes.

Let me show you the physical diagram I’m working with today.

 

I’ll use a single router to mimic the ISPs and each eBGP uplink will go through a switch at each location to peer with the ASAs. We’ll VLAN off the outside and inside traffic at the switch level and we will use sub-interfaces at the ASA level to stitch out networks together.

Let’s dive right in…

Here, all IPs have been preconfigured and I have reachability on all connected networks. I also have the following VLANs defined in every switch:

VLAN 100: ASA to ISP
VLAN 200: DMVPN Router to ASA

Ethernet0/2 on every switch is trunked up because it points to the firewall which house our sub-interface pointing to the WAN as well as the one pointing to the LAN.

This is what it looks like on every switch:

HUB1-SW1#sh vlan br | exc 1002|1003|1004|1005

VLAN Name                             Status    Ports
---- -------------------------------- --------- -------------------------------
1    default                          active    Et0/3, Et1/0, Et1/1, Et1/2
                                                Et1/3, Et2/0, Et2/1, Et2/2
                                                Et2/3, Et3/0, Et3/1, Et3/2
                                                Et3/3
100  OUTSIDE                          active    Et0/0
200  WAN                              active    Et0/1

HUB1-SW1#
HUB1-SW1#sh int trunk                        

Port        Mode             Encapsulation  Status        Native vlan
Et0/2       on               802.1q         trunking      1

Port        Vlans allowed on trunk
Et0/2       1-4094

Port        Vlans allowed and active in management domain
Et0/2       1,100,200

Port        Vlans in spanning tree forwarding state and not pruned
Et0/2       1,100,200
HUB1-SW1#
HUB2-SW#sh vlan br | exc 1002|1003|1004|1005

VLAN Name                             Status    Ports
---- -------------------------------- --------- -------------------------------
1    default                          active    Et0/3
100  WAN                              active    Et0/0
200  LAN                              active    Et0/1

HUB2-SW#
HUB2-SW#sh int trunk

Port        Mode             Encapsulation  Status        Native vlan
Et0/2       on               802.1q         trunking      1

Port        Vlans allowed on trunk
Et0/2       1-4094

Port        Vlans allowed and active in management domain
Et0/2       1,100,200

Port        Vlans in spanning tree forwarding state and not pruned
Et0/2       1,100,200
HUB2-SW#
SPOKE1-SW#sh vlan br | exc 1002|1003|1004|1005

VLAN Name                             Status    Ports
---- -------------------------------- --------- -------------------------------
1    default                          active    Et0/3, Et1/0, Et1/1, Et1/2
                                                Et1/3, Et2/0, Et2/1, Et2/2
                                                Et2/3, Et3/0, Et3/1, Et3/2
                                                Et3/3
100  WAN                              active    Et0/0
200  LAN                              active    Et0/1

SPOKE1-SW#
SPOKE1-SW#sh int trunk

Port        Mode             Encapsulation  Status        Native vlan
Et0/2       on               802.1q         trunking      1

Port        Vlans allowed on trunk
Et0/2       1-4094

Port        Vlans allowed and active in management domain
Et0/2       1,100,200

Port        Vlans in spanning tree forwarding state and not pruned
Et0/2       1,100,200
SPOKE1-SW#
SPOKE2-SW#sh vlan br | exc 1002|1003|1004|1005

VLAN Name                             Status    Ports
---- -------------------------------- --------- -------------------------------
1    default                          active    Et0/3, Et1/0, Et1/1, Et1/2
                                                Et1/3, Et2/0, Et2/1, Et2/2
                                                Et2/3, Et3/0, Et3/1, Et3/2
                                                Et3/3
100  WAN                              active    Et0/0
200  LAN                              active    Et0/1

SPOKE2-SW#
SPOKE2-SW#sh int trunk

Port        Mode             Encapsulation  Status        Native vlan
Et0/2       on               802.1q         trunking      1

Port        Vlans allowed on trunk
Et0/2       1-4094

Port        Vlans allowed and active in management domain
Et0/2       1,100,200

Port        Vlans in spanning tree forwarding state and not pruned
Et0/2       1,100,200
SPOKE2-SW#

Great ! Now let me show you the IP configuration on the routers, ASAs and ISP router.

HUB1-RTR#sh ip int br | exc una
Interface                  IP-Address      OK? Method Status                Protocol
Ethernet0/0                10.100.10.1     YES TFTP   up                    up      
Loopback0                  10.10.10.10     YES manual up                    up
HUB2-RTR#sh ip int br | exc una
Interface                  IP-Address      OK? Method Status                Protocol
Ethernet0/0                10.100.20.1     YES manual up                    up      
Loopback0                  20.20.20.20     YES manual up                    up      
SPOKE1-RTR#sh ip int br | exc una
Interface                  IP-Address      OK? Method Status                Protocol
Ethernet0/0                10.100.30.1     YES manual up                    up      
Loopback0                  30.30.30.30     YES manual up                    up
SPOKE2-RTR#sh ip int br | exc una
Interface                  IP-Address      OK? Method Status                Protocol
Ethernet0/0                10.100.40.1     YES manual up                    up      
Loopback0                  40.40.40.40     YES manual up                    up
ISP-RTR#sh ip int br | exc una
Interface                  IP-Address      OK? Method Status                Protocol
Ethernet0/0                1.1.1.1         YES TFTP   up                    up      
Ethernet0/1                2.2.2.1         YES TFTP   up                    up      
Ethernet0/2                3.3.3.1         YES TFTP   up                    up      
Ethernet0/3                4.4.4.1         YES manual up                    up
HUB1-ASA# sh ip add
System IP Addresses:
Interface                Name                   IP address      Subnet mask     Method 
GigabitEthernet0/0.100   OUTSIDE                1.1.1.254       255.255.255.0   manual
GigabitEthernet0/0.200   DMVPN                  10.100.10.254   255.255.255.0   manual
Current IP Addresses:
Interface                Name                   IP address      Subnet mask     Method 
GigabitEthernet0/0.100   OUTSIDE                1.1.1.254       255.255.255.0   manual
GigabitEthernet0/0.200   DMVPN                  10.100.10.254   255.255.255.0   manual
HUB1-ASA# 
HUB1-ASA# sh run int Gig0/0.100
!
interface GigabitEthernet0/0.100
 vlan 100
 nameif OUTSIDE
 security-level 0
 ip address 1.1.1.254 255.255.255.0 
HUB1-ASA# 
HUB1-ASA# sh run int Gig0/0.200
!
interface GigabitEthernet0/0.200
 vlan 200
 nameif DMVPN
 security-level 100
 ip address 10.100.10.254 255.255.255.0 
HUB1-ASA#
HUB2-ASA# sh ip add
System IP Addresses:
Interface                Name                   IP address      Subnet mask     Method 
GigabitEthernet0/0.100   OUTSIDE                2.2.2.254       255.255.255.0   manual
GigabitEthernet0/0.200   DMVPN                  10.100.20.254   255.255.255.0   manual
Current IP Addresses:
Interface                Name                   IP address      Subnet mask     Method 
GigabitEthernet0/0.100   OUTSIDE                2.2.2.254       255.255.255.0   manual
GigabitEthernet0/0.200   DMVPN                  10.100.20.254   255.255.255.0   manual
HUB2-ASA# 
HUB2-ASA# sh run int Gig0/0.100
!
interface GigabitEthernet0/0.100
 vlan 100
 nameif OUTSIDE
 security-level 0
 ip address 2.2.2.254 255.255.255.0 
HUB2-ASA# 
HUB2-ASA# sh run int Gig0/0.200
!
interface GigabitEthernet0/0.200
 vlan 200
 nameif DMVPN
 security-level 100
 ip address 10.100.20.254 255.255.255.0 
HUB2-ASA#
SPOKE1-ASA# sh int ip br       
Interface                  IP-Address      OK? Method Status                Protocol
GigabitEthernet0/0         unassigned      YES unset  up                    up  
GigabitEthernet0/0.100     3.3.3.254       YES manual up                    up  
GigabitEthernet0/0.200     10.100.30.254   YES manual up                    up  
GigabitEthernet0/1         unassigned      YES unset  administratively down up  
GigabitEthernet0/2         unassigned      YES unset  administratively down up  
GigabitEthernet0/3         unassigned      YES unset  administratively down up  
GigabitEthernet0/4         unassigned      YES unset  administratively down up  
GigabitEthernet0/5         unassigned      YES unset  administratively down up  
GigabitEthernet0/6         unassigned      YES unset  administratively down up  
Management0/0              unassigned      YES unset  administratively down up  
SPOKE1-ASA# 
SPOKE1-ASA# sh run int Gig0/0.100
!
interface GigabitEthernet0/0.100
 vlan 100
 nameif OUTSIDE
 security-level 0
 ip address 3.3.3.254 255.255.255.0 
SPOKE1-ASA# 
SPOKE1-ASA# sh run int Gig0/0.200
!
interface GigabitEthernet0/0.200
 vlan 200
 nameif DMVPN
 security-level 100
 ip address 10.100.30.254 255.255.255.0 
SPOKE1-ASA#
SPOKE2-ASA# sh int ip br
Interface                  IP-Address      OK? Method Status                Protocol
GigabitEthernet0/0         unassigned      YES unset  up                    up  
GigabitEthernet0/0.100     4.4.4.254       YES manual up                    up  
GigabitEthernet0/0.200     10.100.40.254   YES manual up                    up  
GigabitEthernet0/1         unassigned      YES unset  administratively down up  
GigabitEthernet0/2         unassigned      YES unset  administratively down up  
GigabitEthernet0/3         unassigned      YES unset  administratively down up  
GigabitEthernet0/4         unassigned      YES unset  administratively down up  
GigabitEthernet0/5         unassigned      YES unset  administratively down up  
GigabitEthernet0/6         unassigned      YES unset  administratively down up  
Management0/0              unassigned      YES unset  administratively down up  
SPOKE2-ASA# 
SPOKE2-ASA# sh run int Gig0/0.100
!
interface GigabitEthernet0/0.100
 vlan 100
 nameif OUTSIDE
 security-level 0
 ip address 4.4.4.254 255.255.255.0 
SPOKE2-ASA# 
SPOKE2-ASA# sh run int Gig0/0.200
!
interface GigabitEthernet0/0.200
 vlan 200
 nameif DMVPN
 security-level 100
 ip address 10.100.40.254 255.255.255.0 
SPOKE2-ASA#

Very well ! Let’s start by setting up the eBGP sessions between the ISP and the remote sites. Here, the ISP router plays the role of the Internet cloud where we use our Internet circuits to build our encrypted mGRE tunnels to interconnect our sites.

Let me show you what the configuration looks like here.

ISP-RTR#sh run | s bgp       
router bgp 65100
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 1.1.1.254 remote-as 65101
 neighbor 2.2.2.254 remote-as 65102
 neighbor 3.3.3.254 remote-as 65103
 neighbor 4.4.4.254 remote-as 65104
 !
 address-family ipv4
  neighbor 1.1.1.254 activate
  neighbor 1.1.1.254 default-originate
  neighbor 2.2.2.254 activate
  neighbor 2.2.2.254 default-originate
  neighbor 3.3.3.254 activate
  neighbor 3.3.3.254 default-originate
  neighbor 4.4.4.254 activate
  neighbor 4.4.4.254 default-originate
 exit-address-family
ISP-RTR#
ISP-RTR#sh ip bgp sum | b Neighbor
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
1.1.1.254       4        65101       6       7        3    0    0 00:03:34        0
2.2.2.254       4        65102       6       7        3    0    0 00:03:21        0
3.3.3.254       4        65103       6       7        3    0    0 00:03:33        0
4.4.4.254       4        65104       6       7        3    0    0 00:03:35        0
ISP-RTR#
HUB1-ASA# sh run router
router bgp 65101
 bgp log-neighbor-changes
 address-family ipv4 unicast
  neighbor 1.1.1.1 remote-as 65100
  neighbor 1.1.1.1 activate
  no auto-summary
  no synchronization
 exit-address-family
!
HUB1-ASA# 
HUB1-ASA# sh bgp    

BGP table version is 2, local router ID is 10.100.10.254
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop        Metric LocPrf Weight  Path
*> 0.0.0.0          1.1.1.1                            0  65100 i
HUB1-ASA#
HUB2-ASA# sh run router
router bgp 65102
 bgp log-neighbor-changes
 address-family ipv4 unicast
  neighbor 2.2.2.1 remote-as 65100
  neighbor 2.2.2.1 activate
  no auto-summary
  no synchronization
 exit-address-family
!
HUB2-ASA# 
HUB2-ASA# sh bgp

BGP table version is 2, local router ID is 10.100.20.254
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop        Metric LocPrf Weight  Path
*> 0.0.0.0          2.2.2.1                            0  65100 i
HUB2-ASA# 
SPOKE1-ASA# sh run router
router bgp 65103
 bgp log-neighbor-changes
 address-family ipv4 unicast
  neighbor 3.3.3.1 remote-as 65100
  neighbor 3.3.3.1 activate
  no auto-summary
  no synchronization
 exit-address-family
!
SPOKE1-ASA# 
SPOKE1-ASA# sh bgp

BGP table version is 2, local router ID is 10.100.30.254
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop        Metric LocPrf Weight  Path
*> 0.0.0.0          3.3.3.1                            0  65100 i
SPOKE1-ASA#
SPOKE2-ASA# sh run router
router bgp 65104
 bgp log-neighbor-changes
 address-family ipv4 unicast
  neighbor 4.4.4.1 remote-as 65100
  neighbor 4.4.4.1 activate
  no auto-summary
  no synchronization
 exit-address-family
!
SPOKE2-ASA# 
SPOKE2-ASA# sh bgp

BGP table version is 2, local router ID is 10.100.40.254
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop        Metric LocPrf Weight  Path
*> 0.0.0.0          4.4.4.1                            0  65100 i
SPOKE2-ASA# 

Clearly here, we’ve successfully established eBGP sessions between the ISP and our ASAs. We also requested a quad zero from the Provider. You can see here under the BGP tables of the ASA that we’ve successfully received and installed a default route from the Provider. Note that this is technically not necessary but since the idea to build this lab without the use of static routes, then we’ll need to propagate that default route advertisement all the way to our WAN routers.

Note also that Eth0/0 on the WAN routers will be used to source our mGRE tunnels. In that sense, we can’t advertise them via our IGP which is EIGRP here. This is a typical case of a chicken and egg scenario where the notorious question is which one comes first. In our case, we can’t advertise the source of the tunnel through a tunnel that’s built from the same source interface. We’ll be getting bunch of tunnel flaps. However, we still need access to the outside from the WAN routers in order to build the mGRE tunnels.

So what we will be doing is establish iBGP sessions between the WAN routers and the ASAs in order to receive the same default route advertised by the provider. Furthermore, we’ll even ensure that Eth0/0 does not share the same routing domain as the connected routes in the main routing table. We’ll place each source interface into its own VRF called DMVPN.

Let’s do that.

HUB1-RTR#sh ip vrf
  Name                             Default RD            Interfaces
  DMVPN                            65101:10              Et0/0
HUB1-RTR#
HUB1-RTR#sh run int eth0/0
Building configuration...

Current configuration : 125 bytes
!
interface Ethernet0/0
 description **** DMVPN OUT ****
 ip vrf forwarding DMVPN
 ip address 10.100.10.1 255.255.255.0
end

HUB1-RTR#
HUB1-RTR#sh run | s bgp
router bgp 65101
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 10.100.10.254 remote-as 65101
 !
 address-family ipv4
 exit-address-family
 !
 address-family ipv4 vrf DMVPN
  neighbor 10.100.10.254 remote-as 65101
  neighbor 10.100.10.254 activate
 exit-address-family
HUB1-RTR#
HUB1-RTR#sh ip bgp vpnv4 vrf DMVPN sum | b Neighbor
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.100.10.254   4        65101       9       9        1    0    0 00:05:01        1
HUB1-RTR#
HUB1-ASA# sh run router
router bgp 65101
 bgp log-neighbor-changes
 address-family ipv4 unicast
  neighbor 1.1.1.1 remote-as 65100
  neighbor 1.1.1.1 activate
  neighbor 10.100.10.1 remote-as 65101
  neighbor 10.100.10.1 activate
  no auto-summary
  no synchronization
 exit-address-family
!
HUB1-ASA#

Great ! We can see here Eth0/0 is placed into a VRF called DMVPN and we’ve established a neighbor relationship with the local ASA.

I’ve only shown RT-WAN1 and HUB1-ASA outputs as they’re identical expect the IPs and the rd under the VRF stanza.

Let’s now see if we have received and installed the default route in our BGP table.

HUB1-RTR#sh ip bgp vpnv4 vrf DMVPN | b Network
     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65101:10 (default for vrf DMVPN)
 * i 0.0.0.0          1.1.1.1                  0    100      0 65100 i
HUB1-RTR#
HUB2-RTR#sh ip bgp vpnv4 vrf DMVPN | b Network
     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65102:20 (default for vrf DMVPN)
 * i 0.0.0.0          2.2.2.1                  0    100      0 65100 i
HUB2-RTR#
SPOKE1-RTR#sh ip bgp vpnv4 vrf DMVPN | b Network
     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65103:30 (default for vrf DMVPN)
 * i 0.0.0.0          3.3.3.1                  0    100      0 65100 i
SPOKE1-RTR#
SPOKE2-RTR#sh ip bgp vpnv4 vrf DMVPN | b Network
     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65104:40 (default for vrf DMVPN)
 * i 0.0.0.0          4.4.4.1                  0    100      0 65100 i
SPOKE2-RTR#

We can see here that we’ve received the quad zero from the ASA but we currently do not have the “greater than sign” next to the route. This is as good as none to us. Let’s take a closer look here.

The next hop in that BGP advertisement is the ISP in our case. This is a default behavior as the next-hop attribute is set only by the last eBGP speaker in the AS the advertisement originated from. The reason why this not a valid route is because our WAN routers do not know how to get to the ISP router.

Let’s check the routing table of one of the router.

HUB1-RTR#sh ip route | b Gateway
Gateway of last resort is not set

10.0.0.0/32 is subnetted, 1 subnets
C 10.10.10.10 is directly connected, Loopback0
HUB1-RTR#
HUB1-RTR#sh ip route vrf DMVPN | b Gateway
Gateway of last resort is not set

10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C 10.100.10.0/24 is directly connected, Ethernet0/0
L 10.100.10.1/32 is directly connected, Ethernet0/0
HUB1-RTR#

This confirms that our WAN routers do not know how to get to the ISP but they do know how to get to the ASAs. What we’ll essentially need to do is to have the ASAs change the next hop attribute to be the local IP peering with the routers via iBGP.

Let’s do that.

HUB1-ASA# sh run router
router bgp 65101
 bgp log-neighbor-changes
 address-family ipv4 unicast
  neighbor 1.1.1.1 remote-as 65100
  neighbor 1.1.1.1 activate
  neighbor 10.100.10.1 remote-as 65101
  neighbor 10.100.10.1 activate
  neighbor 10.100.10.1 next-hop-self
  no auto-summary
  no synchronization
 exit-address-family

Here, the ASA has changed the next hop attribute for prefixes announced to 10.100.10.1 which is the router. Note that the same command is done on all ASAs. Let’s now confirm that the WAN routers have successfully installed the default route into their BGP tables.

HUB1-RTR#sh ip bgp vpnv4 vrf DMVPN
BGP table version is 2, local router ID is 10.10.10.10
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65101:10 (default for vrf DMVPN)
 *>i 0.0.0.0          10.100.10.254            0    100      0 65100 i

Fantastic ! The WAN routers now know how to get out. We’re in good shape so far !

Let’s now prep the ASAs for the DMVPN piece, we’ll need to do the following:

  1. NAT tunnel source to either outside interface of the ASA or to an unused public IP
  2. Create ACLs to allow UDP/isakmp, UDP/4500 and ESP

Let’s tackle step 1. I wanted to combine different scenarios in this lab so here’s what we’re going to do:

HUB1, HUB2 and Spoke 2 will NAT the source of their mGRE tunnels to an unused public IP. However, let’s assume that Spoke 1 does not have any public IP available, in that sense the only choice we have would be to PAT the source of the mGRE tunnel to the actual outside interface of the ASA.

Let me show you what it looks like.

HUB1-ASA# sh run object
object network DMVPN-IN
 host 10.100.10.1
object network DMVPN-OUT
 host 1.1.1.100
HUB1-ASA# 
HUB1-ASA# sh run nat
nat (DMVPN,OUTSIDE) source static DMVPN-IN DMVPN-OUT
object network DMVPN-IN
 host 10.100.20.1
object network DMVPN-OUT
 host 2.2.2.100
HUB2-ASA# 
HUB2-ASA# sh run nat
nat (DMVPN,OUTSIDE) source static DMVPN-IN DMVPN-OUT
SPOKE2-ASA# sh run object
object network DMVPN-IN
 host 10.100.40.1
object network DMVPN-OUT
 host 4.4.4.100
SPOKE2-ASA# 
SPOKE2-ASA# sh run nat
nat (DMVPN,OUTSIDE) source static DMVPN-IN DMVPN-OUT

All right ! Natting the source of the tunnel to an unused IP is pretty straight forward here. We created object groups that referenced the real and mapped IP and we’re using source NAT to say: Translate DMVPN-IN to DMVPN-OUT.

Let me now show you what PAT looks like on Spoke1-ASA.

object network DMVPN-IN
 host 10.100.30.1
object network HUB1
 host 1.1.1.100
object network HUB2
 host 2.2.2.100
SPOKE1-ASA# 
SPOKE1-ASA# sh run nat
nat (DMVPN,OUTSIDE) source static DMVPN-IN interface destination static HUB1 HUB1
nat (DMVPN,OUTSIDE) source static DMVPN-IN interface destination static HUB2 HUB2

Here we’ve created 3 object referencing the real IP of the tunnel source as well as the NBMA addresses of both Hubs. Under the NAT statement, we’re saying statically map the source IP of the tunnel to the outside interface of the firewall if traffic is destined to HUB1 and/or HUB2. The reason why we have HUB1 and HUB2 twice is because those destination IPs are not mapped to any other IPs.

In some cases, we would be using the “dynamic” keyword instead of “static” as such…

Let’s say I have a VPN tunnel with a 3rd party company using UDP/4500 already. The ASA would already have UDP/4500 socket opened for a different traffic stream. The dynamic keyword comes in handy as it would instruct the ASA to use a different port and allow our DMVPN traffic in..

Great ! Let’s now configure our ACLs.

HUB1-ASA# sh run access-list  
access-list outside-access-in extended permit udp any object DMVPN-IN eq isakmp 
access-list outside-access-in extended permit udp any object DMVPN-IN eq 4500 
access-list outside-access-in extended permit esp any object DMVPN-IN 
HUB1-ASA# 
HUB1-ASA# sh run access-group 
access-group outside-access-in in interface OUTSIDE

Here, we’ve allowed inbound access from any host to the real mGRE tunnel source on UDP/500, UDP/4500 and ESP.

UDP/500: This is needed for the IKE Phase 1 and Phase 2
UDP/4500: This is also needed as it encapsulates ESP traffic into UDP/4500. This is done if and only if a device along the path performs a NAT functions prompting the embedded address of the tunnel in the IP payload to be different from the source address of the IKE packet as it’s being natted by the ASA in our case. The encapsulation to overcome this issue is called NAT Traversal. In our case here, we’ll most likely be using UDP/4500 as our ASAs are performing NAT and we will see it in a few via Wireshark.
ESP: This is essentially used if there’s no NAT devices along the path. I wanted to add it here so you know it’s needed in some cases.

Great ! Let’s now head to our WAN routers and configure our tunnels.

HUB1-RTR#sh run int tu100
Building configuration...

Current configuration : 366 bytes
!
interface Tunnel100
 description ***** HUB1 *****
 ip address 172.16.100.1 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip nhrp authentication RT-L3AK
 ip nhrp map multicast dynamic
 ip nhrp network-id 100
 ip nhrp holdtime 600
 ip nhrp redirect
 ip tcp adjust-mss 1360
 tunnel source Ethernet0/0
 tunnel mode gre multipoint
 tunnel key 100
 tunnel vrf DMVPN
end
HUB2-RTR#sh run int tu200
Building configuration...

Current configuration : 366 bytes
!
interface Tunnel200
 description ***** HUB2 *****
 ip address 172.16.200.1 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip nhrp authentication RT-LE@K
 ip nhrp map multicast dynamic
 ip nhrp network-id 200
 ip nhrp holdtime 600
 ip nhrp redirect
 ip tcp adjust-mss 1360
 tunnel source Ethernet0/0
 tunnel mode gre multipoint
 tunnel key 200
 tunnel vrf DMVPN
end
SPOKE1-RTR#sh run int tu100
Building configuration...

Current configuration : 425 bytes
!
interface Tunnel100
 description **** DMVPN SPOKE1 - HUB1 ****
 ip address 172.16.100.2 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip nhrp authentication RT-L3AK
 ip nhrp map multicast 1.1.1.100
 ip nhrp map 172.16.100.1 1.1.1.100
 ip nhrp network-id 100
 ip nhrp holdtime 600
 ip nhrp nhs 172.16.100.1
 ip tcp adjust-mss 1360
 tunnel source Ethernet0/0
 tunnel mode gre multipoint
 tunnel key 100
 tunnel vrf DMVPN
end

SPOKE1-RTR#sh run int tu200
Building configuration...

Current configuration : 425 bytes
!
interface Tunnel200
 description **** DMVPN SPOKE1 - HUB2 ****
 ip address 172.16.200.2 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip nhrp authentication RT-LE@K
 ip nhrp map multicast 2.2.2.100
 ip nhrp map 172.16.200.1 2.2.2.100
 ip nhrp network-id 200
 ip nhrp holdtime 600
 ip nhrp nhs 172.16.200.1
 ip tcp adjust-mss 1360
 tunnel source Ethernet0/0
 tunnel mode gre multipoint
 tunnel key 200
 tunnel vrf DMVPN
end
SPOKE2-RTR#sh run int tu100
Building configuration...

Current configuration : 425 bytes
!
interface Tunnel100
 description **** DMVPN SPOKE1 - HUB1 ****
 ip address 172.16.100.3 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip nhrp authentication RT-L3AK
 ip nhrp map multicast 1.1.1.100
 ip nhrp map 172.16.100.1 1.1.1.100
 ip nhrp network-id 100
 ip nhrp holdtime 600
 ip nhrp nhs 172.16.100.1
 ip tcp adjust-mss 1360
 tunnel source Ethernet0/0
 tunnel mode gre multipoint
 tunnel key 100
 tunnel vrf DMVPN
end

SPOKE2-RTR#sh run int tu200
Building configuration...

Current configuration : 425 bytes
!
interface Tunnel200
 description **** DMVPN SPOKE2 - HUB2 ****
 ip address 172.16.200.3 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip nhrp authentication RT-LE@K
 ip nhrp map multicast 2.2.2.100
 ip nhrp map 172.16.200.1 2.2.2.100
 ip nhrp network-id 200
 ip nhrp holdtime 600
 ip nhrp nhs 172.16.200.1
 ip tcp adjust-mss 1360
 tunnel source Ethernet0/0
 tunnel mode gre multipoint
 tunnel key 200
 tunnel vrf DMVPN
end

All right, we won’t spend much time here as this has been covered here.

Noticed how we have 2 tunnels on the spoke sides which reference both DMVPN clouds per the diagram above. Each tunnel IP is going to be in the same broadcast domain as its respective hub. Also notice that the NHRP authentication key is different as well as the network ID and the tunnel key. We finished off the tunnel configuration here by specifying the VRF membership since the source of the tunnels itself reside in its own VRF.

Great, let’s move forward with configuring the DCI (Data Center Interconnect) with the use of basic GRE tunnel between the HUBs.

HUB1-RTR#sh run int tu300
Building configuration...

Current configuration : 208 bytes
!
interface Tunnel300
 description ***** DCI *****
 ip address 192.168.1.1 255.255.255.252
 ip mtu 1400
 ip tcp adjust-mss 1360
 tunnel source Ethernet0/0
 tunnel destination 2.2.2.100
 tunnel vrf DMVPN
end
HUB2-RTR#sh run int tu300
Building configuration...

Current configuration : 208 bytes
!
interface Tunnel300
 description ***** DCI *****
 ip address 192.168.1.2 255.255.255.252
 ip mtu 1400
 ip tcp adjust-mss 1360
 tunnel source Ethernet0/0
 tunnel destination 1.1.1.100
 tunnel vrf DMVPN
end

Here, we have a basic GRE tunnel. You can read more about it here.

Awesome, let’s now encrypt our traffic and check the state of the DMVPN sessions.

Here is the configuration.

crypto keyring DMVPNSPOKES vrf DMVPN 
  pre-shared-key address 0.0.0.0 0.0.0.0 key RouteLeak
!
!
crypto isakmp policy 10
 encr aes 256
 authentication pre-share
 group 19
crypto isakmp keepalive 30 periodic
crypto isakmp nat keepalive 300
crypto isakmp profile DMVPN_PROFILE
   vrf DMVPN
   keyring DMVPNSPOKES
   match identity address 0.0.0.0 DMVPN
!
!
crypto ipsec transform-set TSET-DMVPN esp-aes 
 mode transport
!
crypto ipsec profile DMVPN_PROFILE
 set transform-set TSET-DMVPN 
 set isakmp-profile DMVPN_PROFILE
!
!
int tu1XX
tunnel protection ipsec profile DMVPN_PROFILE

PHASE 1
First, we start off by defining the pre-shared key. In this case I’ve used a keyring tied to our VRF. This is essentially optional in our case but best practice because moving forward, if I needed to create additional VRFs, I’d create different keyrings and stitch them to the distinct VRFs so that they have the same isakmp policies, same pre-shared keys, similar crypto ACLs… This is called VRF-aware IPsec.

The “address 0.0.0.0 0.0.0.0” instruct the router to use a single key for any router. This scale well since I have additional authentication at the tunnel level. It’s also important to note that encryption occurs before GRE and NHRP.

Next, we definied our ISAKMP policy. This is standard isakmp policy. I like to use the above settings as they’re deemed to be one of the strongest parameters at the time of this post. Note that the crypto keyring is also tied to an isakmp profile along with the VRF name.

 

PHASE 2
Next we defined a transform set to instruct the router which encryption to use. Also notice that to avoid additive overhead, we’re using “transport mode” instead of “tunnel mode”.

The next step is to create a ipsec profile and chain our transform set and isakmp profile to it.

The last step is to now apply the profile to our tunnel interfaces. Note that the “shared” keyword at the end is mandatory here since we’re using the same profile for multiple tunnels.

This is applied to all tunnel interfaces. Let’s now check if we have the DMVPN sessions up.

HUB1-RTR#sh ip nhrp brief       
   Target             Via            NBMA           Mode   Intfc   Claimed 
     172.16.100.2/32 172.16.100.2    3.3.3.254       dynamic  Tu100   10.100.30.1
     172.16.100.3/32 172.16.100.3    4.4.4.100       dynamic  Tu100   10.100.40.1
HUB1-RTR#
HUB1-RTR#show dmvpn | begin Peer
Type:Hub, NHRP Peers:2, 

 # Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
     1 3.3.3.254          172.16.100.2    UP 01:59:00    DN
     1 4.4.4.100          172.16.100.3    UP 02:08:32    DN
HUB2-RTR#sh ip nhrp brief       
   Target             Via            NBMA           Mode   Intfc   Claimed 
     172.16.200.2/32 172.16.200.2    3.3.3.254       dynamic  Tu200   10.100.30.1
     172.16.200.3/32 172.16.200.3    4.4.4.100       dynamic  Tu200   10.100.40.1
HUB2-RTR#
HUB2-RTR#show dmvpn | begin Peer
Type:Hub, NHRP Peers:2, 

 # Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
     1 3.3.3.254          172.16.200.2    UP 01:57:04    DN
     1 4.4.4.100          172.16.200.3    UP 02:06:15    DN
SPOKE1-RTR#sh ip nhrp brief       
   Target             Via            NBMA           Mode   Intfc   Claimed 
     172.16.100.1/32 172.16.100.1    1.1.1.100       static   Tu100   < >
     172.16.200.1/32 172.16.200.1    2.2.2.100       static   Tu200   < >
SPOKE1-RTR#
SPOKE1-RTR#show dmvpn | begin Peer
Type:Spoke, NHRP Peers:1, 

 # Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
     1 1.1.1.100          172.16.100.1    UP 02:01:36     S

Interface: Tunnel200, IPv4 NHRP Details 
Type:Spoke, NHRP Peers:1, 

 # Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
     1 2.2.2.100          172.16.200.1    UP 02:01:36     S
SPOKE2-RTR#show dmvpn | begin Peer
Type:Spoke, NHRP Peers:1, 

 # Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
     1 1.1.1.100          172.16.100.1    UP 02:06:38     S

Interface: Tunnel200, IPv4 NHRP Details 
Type:Spoke, NHRP Peers:1, 

 # Ent  Peer NBMA Addr Peer Tunnel Add State  UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
     1 2.2.2.100          172.16.200.1    UP 02:06:38     S

Very well ! Looks like we’re solid so far. Earlier we’ve talked about NAT-T and how our ESP traffic are encapsulated in UDP/4500. Let me give you a quick snapshot of what it looks like.

 

Here we can clearly the source and destination of this stream which is Spoke1 to Hub1. We can also see the L4 information which is UDP/4500.

Let’s check the hit counts on our ACL.

HUB1-ASA# sh access-list outside-access-in | inc 4500|esp
access-list outside-access-in line 2 extended permit udp any object DMVPN-IN eq 4500 (hitcnt=3) 0x513f3701 
  access-list outside-access-in line 2 extended permit udp any host 10.100.10.1 eq 4500 (hitcnt=3) 0x513f3701 
access-list outside-access-in line 3 extended permit esp any object DMVPN-IN (hitcnt=0) 0x9ed3cf1f 
  access-list outside-access-in line 3 extended permit esp any host 10.100.10.1 (hitcnt=0) 0x9ed3cf1f

One thing to highlight here is that there’s no hit counts on the ACL matching against ESP traffic. This is expected in our scenario here.

Let’s now enable EIGRP across our DMVPN clouds.

SPOKE1-RTR#sh run | s eigrp
router eigrp 1001
 network 30.30.30.30 0.0.0.0
 network 172.16.100.2 0.0.0.0
 network 172.16.200.2 0.0.0.0
SPOKE2-RTR#sh run | s eigrp
router eigrp 1001
 network 40.40.40.40 0.0.0.0
 network 172.16.100.3 0.0.0.0
 network 172.16.200.3 0.0.0.0
HUB1-RTR#sh run int tu100
Building configuration...

Current configuration : 452 bytes
!
interface Tunnel100
 description ***** HUB1 *****
 ip address 172.16.100.1 255.255.255.0
 no ip redirects
 ip mtu 1400
 no ip split-horizon eigrp 1001
 ip nhrp authentication RT-L3AK
 ip nhrp map multicast dynamic
 ip nhrp network-id 100
 ip nhrp holdtime 600
 ip nhrp redirect
 ip tcp adjust-mss 1360
 tunnel source Ethernet0/0
 tunnel mode gre multipoint
 tunnel key 100
 tunnel vrf DMVPN
 tunnel protection ipsec profile DMVPN_PROFILE shared
end

HUB1-RTR#sh run | s eigrp
 no ip split-horizon eigrp 1001
router eigrp 1001
 network 10.10.10.10 0.0.0.0
 network 172.16.100.1 0.0.0.0
 network 192.168.1.1 0.0.0.0
HUB2-RTR#sh run int tu200
Building configuration...

Current configuration : 452 bytes
!
interface Tunnel200
 description ***** HUB2 *****
 ip address 172.16.200.1 255.255.255.0
 no ip redirects
 ip mtu 1400
 no ip split-horizon eigrp 1001
 ip nhrp authentication RT-LE@K
 ip nhrp map multicast dynamic
 ip nhrp network-id 200
 ip nhrp holdtime 600
 ip nhrp redirect
 ip tcp adjust-mss 1360
 tunnel source Ethernet0/0
 tunnel mode gre multipoint
 tunnel key 200
 tunnel vrf DMVPN
 tunnel protection ipsec profile DMVPN_PROFILE shared
end

HUB2-RTR#sh run | s eigrp
 no ip split-horizon eigrp 1001
router eigrp 1001
 network 20.20.20.20 0.0.0.0
 network 172.16.0.0
 network 172.16.200.1 0.0.0.0
 network 192.168.1.2 0.0.0.0

The command “no ip split-horizon eigrp 100” is needed under the tunnel interfaces at the HUB side as EIGRP is a distance vector protocol. EIGRP by default has “Split-Horizon” enabled and what it means is simply not to relay advertisement back out of the same interface in which it was received from. This a routing loop mechanism and a default behavior of the protocol. In our case here, we will need the Spokes to query the hub and the hub to respond to the queries through the same interface.

Let’s check if we have adjacency and if we can ping the loopback addresses advertised.

HUB1-RTR#sh ip ei nei
EIGRP-IPv4 Neighbors for AS(1001)
H   Address                 Interface              Hold Uptime   SRTT   RTO  Q  Seq
                                                   (sec)         (ms)       Cnt Num
1   172.16.100.3            Tu100                    14 00:12:45   15  1398  0  7
2   172.16.100.2            Tu100                    11 00:18:07   20  1398  0  37
0   192.168.1.2             Tu300                    12 00:25:08   13  1398  0  32
HUB1-RTR#
HUB1-RTR#sh ip route eigrp | b Gateway
Gateway of last resort is not set

      20.0.0.0/32 is subnetted, 1 subnets
D        20.20.20.20 [90/27008000] via 192.168.1.2, 00:12:42, Tunnel300
      30.0.0.0/32 is subnetted, 1 subnets
D        30.30.30.30 [90/27008000] via 172.16.100.2, 00:18:10, Tunnel100
      40.0.0.0/32 is subnetted, 1 subnets
D        40.40.40.40 [90/27008000] via 172.16.100.3, 00:12:41, Tunnel100
      172.16.0.0/16 is variably subnetted, 3 subnets, 2 masks
D        172.16.200.0/24 [90/28160000] via 192.168.1.2, 00:12:42, Tunnel300
                         [90/28160000] via 172.16.100.3, 00:12:42, Tunnel100
                         [90/28160000] via 172.16.100.2, 00:12:42, Tunnel100
HUB1-RTR#
HUB1-RTR#ping 20.20.20.20             
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 20.20.20.20, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 8/9/11 ms
HUB1-RTR#ping 30.30.30.30             
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 30.30.30.30, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 9/10/11 ms
HUB1-RTR#ping 40.40.40.40             
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 40.40.40.40, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 8/10/15 ms
HUB2-RTR#sh ip ei nei
EIGRP-IPv4 Neighbors for AS(1001)
H   Address                 Interface              Hold Uptime   SRTT   RTO  Q  Seq
                                                   (sec)         (ms)       Cnt Num
2   172.16.200.3            Tu200                    12 00:13:19   16  1362  0  8
1   172.16.200.2            Tu200                    12 00:24:31    8  1362  0  36
0   192.168.1.1             Tu300                    13 00:25:48   15  1398  0  39
HUB2-RTR#
HUB2-RTR#sh ip route eigrp | b Gateway
Gateway of last resort is not set

      10.0.0.0/32 is subnetted, 1 subnets
D        10.10.10.10 [90/27008000] via 192.168.1.1, 00:13:25, Tunnel300
      30.0.0.0/32 is subnetted, 1 subnets
D        30.30.30.30 [90/27008000] via 172.16.200.2, 00:13:25, Tunnel200
      40.0.0.0/32 is subnetted, 1 subnets
D        40.40.40.40 [90/27008000] via 172.16.200.3, 00:13:25, Tunnel200
      172.16.0.0/16 is variably subnetted, 3 subnets, 2 masks
D        172.16.100.0/24 [90/28160000] via 192.168.1.1, 00:13:25, Tunnel300
                         [90/28160000] via 172.16.200.3, 00:13:25, Tunnel200
                         [90/28160000] via 172.16.200.2, 00:13:25, Tunnel200
HUB2-RTR#ping 10.10.10.10
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.10.10, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 9/10/14 ms
HUB2-RTR#ping 30.30.30.30     
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 30.30.30.30, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 9/10/12 ms
HUB2-RTR#ping 40.40.40.40 
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 40.40.40.40, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 7/8/10 ms
SPOKE1-RTR#sh ip ei nei
EIGRP-IPv4 Neighbors for AS(1001)
H   Address                 Interface              Hold Uptime   SRTT   RTO  Q  Seq
                                                   (sec)         (ms)       Cnt Num
0   172.16.100.1            Tu100                    14 00:20:47   18  1398  0  40
1   172.16.200.1            Tu200                    14 00:26:31   18  1398  0  33
SPOKE1-RTR#
SPOKE1-RTR#sh ip route eigrp | b Gateway
Gateway of last resort is not set

      10.0.0.0/32 is subnetted, 1 subnets
D        10.10.10.10 [90/27008000] via 172.16.100.1, 00:20:54, Tunnel100
      20.0.0.0/32 is subnetted, 1 subnets
D        20.20.20.20 [90/27008000] via 172.16.200.1, 00:20:54, Tunnel200
      40.0.0.0/32 is subnetted, 1 subnets
D        40.40.40.40 [90/28288000] via 172.16.200.1, 00:15:25, Tunnel200
                     [90/28288000] via 172.16.100.1, 00:15:25, Tunnel100
      192.168.1.0/30 is subnetted, 1 subnets
D        192.168.1.0 [90/28160000] via 172.16.200.1, 00:20:54, Tunnel200
                     [90/28160000] via 172.16.100.1, 00:20:54, Tunnel100
SPOKE1-RTR#ping 10.10.10.10
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.10.10, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 8/9/11 ms
SPOKE1-RTR#ping 20.20.20.20
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 20.20.20.20, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/7/9 ms
SPOKE1-RTR#ping 40.40.40.40 
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 40.40.40.40, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 10/13/15 ms
SPOKE2-RTR#sh ip ei nei
EIGRP-IPv4 Neighbors for AS(1001)
H   Address                 Interface              Hold Uptime   SRTT   RTO  Q  Seq
                                                   (sec)         (ms)       Cnt Num
1   172.16.200.1            Tu200                    14 00:17:15   32  1398  0  33
0   172.16.100.1            Tu100                    13 00:17:21   21  1398  0  40
SPOKE2-RTR#
SPOKE2-RTR#sh ip route eigrp | b Gateway
Gateway of last resort is not set

      10.0.0.0/32 is subnetted, 1 subnets
D        10.10.10.10 [90/27008000] via 172.16.100.1, 00:17:20, Tunnel100
      20.0.0.0/32 is subnetted, 1 subnets
D        20.20.20.20 [90/27008000] via 172.16.200.1, 00:17:20, Tunnel200
      30.0.0.0/32 is subnetted, 1 subnets
D        30.30.30.30 [90/28288000] via 172.16.200.1, 00:17:20, Tunnel200
                     [90/28288000] via 172.16.100.1, 00:17:20, Tunnel100
      192.168.1.0/30 is subnetted, 1 subnets
D        192.168.1.0 [90/28160000] via 172.16.200.1, 00:17:20, Tunnel200
                     [90/28160000] via 172.16.100.1, 00:17:20, Tunnel100
SPOKE2-RTR#ping 10.10.10.10
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.10.10, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 9/10/12 ms
SPOKE2-RTR#ping 20.20.20.20
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 20.20.20.20, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 8/9/11 ms
SPOKE2-RTR#ping 30.30.30.30
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 30.30.30.30, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 13/14/17 ms

This is good ! We have reachability across both DMVPN networks. The idea now is to prefer the path through DMVPN Cloud 1 to reach internal services. We can easily achieve that by lowering the delay metric on HUB1-RTR.

Let’s do that.

HUB1-RTR#sh int tu100 | inc  DLY
  MTU 17912 bytes, BW 100 Kbit/sec, DLY 50000 usec, 
HUB1-RTR#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
HUB1-RTR(config)#int tu100
HUB1-RTR(config-if)#delay 4000
HUB1-RTR(config-if)#do sh int tu100 | inc DLY
  MTU 17912 bytes, BW 100 Kbit/sec, DLY 40000 usec, 
HUB1-RTR(config-if)#
HUB1-RTR(config-if)#
HUB1-RTR(config-if)#do sh int tu300 | inc DLY
  MTU 17874 bytes, BW 100 Kbit/sec, DLY 50000 usec, 
HUB1-RTR(config-if)#      
HUB1-RTR(config-if)#int tu300
HUB1-RTR(config-if)#delay 4000               
HUB1-RTR(config-if)#
HUB1-RTR(config-if)#do sh int tu300 | inc DLY
  MTU 17874 bytes, BW 100 Kbit/sec, DLY 40000 usec, 
HUB1-RTR(config-if)#end
HUB2-RTR#sh int tu200 | inc  DLY
  MTU 17912 bytes, BW 100 Kbit/sec, DLY 50000 usec,

Now that we have HUB1 delay metric lower that Hub 2, the spoke routers should now prefer Hub1 over Hub2.

Let’s confirm.

SPOKE1-RTR#sh ip route eigrp | b Gateway
Gateway of last resort is not set

      10.0.0.0/32 is subnetted, 1 subnets
D        10.10.10.10 [90/27008000] via 172.16.100.1, 00:34:38, Tunnel100
      20.0.0.0/32 is subnetted, 1 subnets
D        20.20.20.20 [90/27008000] via 172.16.200.1, 00:01:14, Tunnel200
      40.0.0.0/32 is subnetted, 1 subnets
D        40.40.40.40 [90/28032000] via 172.16.100.1, 00:05:35, Tunnel100
      192.168.1.0/30 is subnetted, 1 subnets
D        192.168.1.0 [90/27904000] via 172.16.100.1, 00:01:14, Tunnel100
SPOKE2-RTR#sh ip route eigrp | b Gateway
Gateway of last resort is not set

      10.0.0.0/32 is subnetted, 1 subnets
D        10.10.10.10 [90/27008000] via 172.16.100.1, 00:29:12, Tunnel100
      20.0.0.0/32 is subnetted, 1 subnets
D        20.20.20.20 [90/27008000] via 172.16.200.1, 00:01:16, Tunnel200
      30.0.0.0/32 is subnetted, 1 subnets
D        30.30.30.30 [90/28032000] via 172.16.100.1, 00:05:38, Tunnel100
      192.168.1.0/30 is subnetted, 1 subnets
D        192.168.1.0 [90/27904000] via 172.16.100.1, 00:01:16, Tunnel100

Awesome ! We’re in good shape so far. Let’s now talk about DMVPN Phase 3.

As of right now, if Spoke 1 needed to communicate to to Spoke 2, traffic would route through the Hub. The ideal setup would have been if Spoke 1 can dynamically build a tunnel with Spoke 2 without the hub intervention. That would be DMVPN Phase 3.

Let’s see how traffic is routed from Spoke 1 to Spoke 2.

SPOKE1-RTR#traceroute 40.40.40.40 num
Type escape sequence to abort.
Tracing the route to 40.40.40.40
VRF info: (vrf in name/id, vrf out name/id)
  1 172.16.100.1 8 msec 8 msec 9 msec
  2 172.16.100.3 11 msec *  14 msec
SPOKE1-RTR#
SPOKE1-RTR#traceroute 40.40.40.40 num
Type escape sequence to abort.
Tracing the route to 40.40.40.40
VRF info: (vrf in name/id, vrf out name/id)
  1 172.16.100.1 10 msec 8 msec 9 msec
  2 172.16.100.3 12 msec *  13 msec

Here, we can clearly see that we have to go through the hub to reach the remote spoke. Let’s fix that with the following commands.

HUB1-RTR#sh run int tu100 
Building configuration...

Current configuration : 496 bytes
!
interface Tunnel100
 description ***** HUB1 *****
 ip address 172.16.100.1 255.255.255.0
 no ip redirects
 ip mtu 1400
 no ip next-hop-self eigrp 1001
 no ip split-horizon eigrp 1001
 ip nhrp authentication RT-L3AK
 ip nhrp map multicast dynamic
 ip nhrp network-id 100
 ip nhrp holdtime 600
 ip nhrp redirect
 ip tcp adjust-mss 1360
 delay 4000
 tunnel source Ethernet0/0
 tunnel mode gre multipoint
 tunnel key 100
 tunnel vrf DMVPN
 tunnel protection ipsec profile DMVPN_PROFILE shared
HUB2-RTR#sh run int tu200
Building configuration...

Current configuration : 484 bytes
!
interface Tunnel200
 description ***** HUB2 *****
 ip address 172.16.200.1 255.255.255.0
 no ip redirects
 ip mtu 1400
 no ip next-hop-self eigrp 1001
 no ip split-horizon eigrp 1001
 ip nhrp authentication RT-LE@K
 ip nhrp map multicast dynamic
 ip nhrp network-id 200
 ip nhrp holdtime 600
 ip nhrp redirect
 ip tcp adjust-mss 1360
 tunnel source Ethernet0/0
 tunnel mode gre multipoint
 tunnel key 200
 tunnel vrf DMVPN
 tunnel protection ipsec profile DMVPN_PROFILE shared
SPOKE1-RTR#sh run int tu100
Building configuration...

Current configuration : 497 bytes
!
interface Tunnel100
 description **** DMVPN SPOKE1 - HUB1 ****
 ip address 172.16.100.2 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip nhrp authentication RT-L3AK
 ip nhrp map multicast 1.1.1.100
 ip nhrp map 172.16.100.1 1.1.1.100
 ip nhrp network-id 100
 ip nhrp holdtime 600
 ip nhrp nhs 172.16.100.1
 ip nhrp shortcut
 ip tcp adjust-mss 1360
 tunnel source Ethernet0/0
 tunnel mode gre multipoint
 tunnel key 100
 tunnel vrf DMVPN
 tunnel protection ipsec profile DMVPN_PROFILE shared
end

SPOKE1-RTR#sh run int tu200
Building configuration...

Current configuration : 497 bytes
!
interface Tunnel200
 description **** DMVPN SPOKE1 - HUB2 ****
 ip address 172.16.200.2 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip nhrp authentication RT-LE@K
 ip nhrp map multicast 2.2.2.100
 ip nhrp map 172.16.200.1 2.2.2.100
 ip nhrp network-id 200
 ip nhrp holdtime 600
 ip nhrp nhs 172.16.200.1
 ip nhrp shortcut
 ip tcp adjust-mss 1360
 tunnel source Ethernet0/0
 tunnel mode gre multipoint
 tunnel key 200
 tunnel vrf DMVPN
 tunnel protection ipsec profile DMVPN_PROFILE shared
SPOKE2-RTR#sh run int tu100
Building configuration...

Current configuration : 497 bytes
!
interface Tunnel100
 description **** DMVPN SPOKE1 - HUB1 ****
 ip address 172.16.100.3 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip nhrp authentication RT-L3AK
 ip nhrp map multicast 1.1.1.100
 ip nhrp map 172.16.100.1 1.1.1.100
 ip nhrp network-id 100
 ip nhrp holdtime 600
 ip nhrp nhs 172.16.100.1
 ip nhrp shortcut
 ip tcp adjust-mss 1360
 tunnel source Ethernet0/0
 tunnel mode gre multipoint
 tunnel key 100
 tunnel vrf DMVPN
 tunnel protection ipsec profile DMVPN_PROFILE shared
end

SPOKE2-RTR#sh run int tu200
Building configuration...

Current configuration : 497 bytes
!
interface Tunnel200
 description **** DMVPN SPOKE2 - HUB2 ****
 ip address 172.16.200.3 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip nhrp authentication RT-LE@K
 ip nhrp map multicast 2.2.2.100
 ip nhrp map 172.16.200.1 2.2.2.100
 ip nhrp network-id 200
 ip nhrp holdtime 600
 ip nhrp nhs 172.16.200.1
 ip nhrp shortcut
 ip tcp adjust-mss 1360
 tunnel source Ethernet0/0
 tunnel mode gre multipoint
 tunnel key 200
 tunnel vrf DMVPN
 tunnel protection ipsec profile DMVPN_PROFILE shared

The “no ip next-hop-self eigrp 1001” is the only command needed for Phase 2. In this phase, Spoke to Spoke tunneling is allowed. That essentially means all spokes need to receive full routing information from its peer but the problem is that by default, EIGRP sends updates with the Next Hop Address embedded in the header. Let’s say the Hub router sends an update to Spoke 1; Spoke 1 will then receive the update and populate its routing table with the Hub router as the next hop for the addresses it received. Then in that case, Spoke to Spoke tunneling would not be possible because the next hop would be the Hub ! That’s why we have to specify the “no ip next-hop-self eigrp” command to send those updates with the next-hop unchanged. Note that this is only true for EIGRP.

Phase 3 and Phase 2 are somehow similar in a sense where Spoke to Spoke tunneling is allowed but there are much more to this phase than usual.

The command “ip nhrp redirect” is more or so similar to IP ICMP redirect messages. When a IP Packet is received inbound on the mGRE tunnel interface and switched out of the same interface, a “nhrp redirect message” is embeded in the frame with the goal of informing the source of the packet that it is currently using a suboptimal path to reach the destination and that it should look for a better path.

The originator now receives the message and queries the destination IP by traversing the hub. The Spoke now responds to the NHRP querie and forward the NBMA address back to the originator but this time, traffic flow does not include the hub as a hop; it rather establish a session directly with the originator. Remember this could not be done with the previous phases as the hub is no longer the only source of NHRP information. Note that in this phase, NHRP replies carry routing information rather than just next hop information. Summarization is also possible at this level !

That’s all I have for today.

Comments

  1. damian says:

    Hello, i test your LAB at home and i cant see where the 1.1.1.100, 2.2.2.100, 3.3.3.100 and 4.4.4.100 are configured. Can you share all configs?

    best regards
    damian

  2. Pape says:

    Hello Damian. Those are the mapped addresses for the source of the tunnels. Those are configured on the ASAs. It is actually written in the blog. Look for the section where I referenced the following:

    HUB1-ASA# sh run object
    object network DMVPN-IN
    host 10.100.10.1
    object network DMVPN-OUT
    host 1.1.1.100
    HUB1-ASA#
    HUB1-ASA# sh run nat
    nat (DMVPN,OUTSIDE) source static DMVPN-IN DMVPN-OUT

    That’s the same approach for the other sites. Please let me know if it works for you.

  3. damian says:

    Hello,
    i make your lab only with 1 router. From hub-rtr i cant reach via icmp the ip address 1.1.1.100. From isp-rtr i cant reach 1.1.1.100 and 3.3.3.100.

    here are spoke1-rtr config (cisco 1841)

    ip vrf DMVPN
    rd 65103:30
    interface Tunnel100
    description **** DMVPN SPOKE1 – HUB1 ****
    ip address 172.16.100.2 255.255.255.0
    no ip redirects
    ip mtu 1400
    ip nhrp authentication RT-L3AK
    ip nhrp map 172.16.100.1 1.1.1.100
    ip nhrp map multicast 1.1.1.100
    ip nhrp network-id 100
    ip nhrp holdtime 600
    ip nhrp nhs 172.16.100.1
    ip tcp adjust-mss 1360
    tunnel source FastEthernet0/1/0
    tunnel mode gre multipoint
    tunnel key 100
    tunnel vrf DMVPN
    interface FastEthernet0/1/0
    description WAN
    ip vrf forwarding DMVPN
    ip address 10.100.30.1 255.255.255.0
    duplex auto
    speed auto
    !
    !
    router eigrp 1001
    network 30.30.30.30 0.0.0.0
    network 172.16.100.2 0.0.0.0
    !
    router bgp 65103
    bgp log-neighbor-changes
    no bgp default ipv4-unicast
    neighbor 10.100.30.254 remote-as 65103
    !
    address-family ipv4
    exit-address-family
    !
    address-family ipv4 vrf DMVPN
    neighbor 10.100.30.254 remote-as 65103
    neighbor 10.100.30.254 activate
    exit-address-family

    here the hub-rtr (Cisco 887VG-W)

    ip vrf DMVPN
    rd 65101:10
    interface Loopback0
    ip address 10.10.10.10 255.255.255.255
    !
    interface Tunnel100
    description ***** HUB1 *****
    ip address 172.16.100.1 255.255.255.0
    no ip redirects
    ip mtu 1400
    ip nhrp authentication RT-L3AK
    ip nhrp map multicast dynamic
    ip nhrp network-id 100
    ip nhrp holdtime 600
    ip nhrp redirect
    ip tcp adjust-mss 1360
    tunnel source Vlan1
    tunnel mode gre multipoint
    tunnel key 100
    tunnel vrf DMVPN
    interface Vlan1
    description DMVPN OUT
    ip vrf forwarding DMVPN
    ip address 10.100.10.1 255.255.255.0
    no autostate
    !
    router bgp 65101
    bgp log-neighbor-changes
    no bgp default ipv4-unicast
    neighbor 10.100.10.254 remote-as 65101
    !
    address-family ipv4
    exit-address-family
    !
    address-family ipv4 vrf DMVPN
    neighbor 10.100.10.254 remote-as 65101
    neighbor 10.100.10.254 activate
    exit-address-family

    here the config from hub-asa (cisco asa5505)

    interface Vlan1
    description outside
    nameif outside
    security-level 0
    ip address 1.1.1.254 255.255.255.0
    !
    interface Vlan2
    description inside
    nameif DMVPN
    security-level 100
    ip address 10.100.10.254 255.255.255.0
    !
    ftp mode passive
    object network DMVPN-IN
    host 10.100.10.1
    object network DMVPN-OUT
    host 1.1.1.100
    access-list outside-access-in extended permit udp any object DMVPN-IN eq isakmp
    access-list outside-access-in extended permit udp any object DMVPN-IN eq 4500
    access-list outside-access-in extended permit esp any object DMVPN-IN
    nat (DMVPN,outside) source static DMVPN-IN DMVPN-OUT
    access-group outside-access-in in interface outside
    router bgp 65101
    bgp log-neighbor-changes
    address-family ipv4 unicast
    neighbor 1.1.1.1 remote-as 65100
    neighbor 1.1.1.1 activate
    neighbor 10.100.10.1 remote-as 65101
    neighbor 10.100.10.1 activate
    neighbor 10.100.10.1 next-hop-self
    no auto-summary
    no synchronization
    exit-address-family

    here the config from spoke1-asa (cisco asa5505)

    interface Vlan1
    description outside
    nameif OUTSIDE
    security-level 0
    ip address 3.3.3.254 255.255.255.0
    !
    interface Vlan2
    description DMVPN
    nameif DMVPN
    security-level 100
    ip address 10.100.30.254 255.255.255.0
    object network DMVPN-IN
    host 10.100.30.1
    object network HUB1
    host 1.1.1.100
    object network HUB2
    host 2.2.2.100
    object network DMVPN-OUT
    host 3.3.3.100
    access-list outside-access-in extended permit udp any object DMVPN-IN eq isakmp
    access-list outside-access-in extended permit udp any object DMVPN-IN eq 4500
    access-list outside-access-in extended permit esp any object DMVPN-IN
    nat (DMVPN,OUTSIDE) source static DMVPN-IN DMVPN-OUT
    access-group outside-access-in in interface OUTSIDE
    router bgp 65103
    bgp log-neighbor-changes
    address-family ipv4 unicast
    neighbor 3.3.3.1 remote-as 65100
    neighbor 3.3.3.1 activate
    neighbor 10.100.30.1 remote-as 65103
    neighbor 10.100.30.1 activate
    neighbor 10.100.30.1 next-hop-self
    no auto-summary
    no synchronization
    exit-address-family

    and at the last config from isp-rtr (cisco 1802)

    interface FastEthernet1
    description 1.1.1.1
    no ip address
    !
    interface FastEthernet2
    description 2.2.2.1
    switchport access vlan 2
    no ip address
    !
    interface FastEthernet3
    description 3.3.3.1
    switchport access vlan 3
    no ip address
    !
    interface FastEthernet4
    description 4.4.4.1
    switchport access vlan 4
    no ip address
    interface Vlan1
    ip address 1.1.1.1 255.255.255.0
    no autostate
    !
    interface Vlan2
    ip address 2.2.2.1 255.255.255.0
    no autostate
    !
    interface Vlan3
    ip address 3.3.3.1 255.255.255.0
    no autostate
    !
    interface Vlan4
    ip address 4.4.4.1 255.255.255.0
    no autostate
    router bgp 65100
    bgp log-neighbor-changes
    no bgp default ipv4-unicast
    neighbor 1.1.1.254 remote-as 65101
    neighbor 2.2.2.254 remote-as 65102
    neighbor 3.3.3.254 remote-as 65103
    neighbor 4.4.4.254 remote-as 65104
    !
    address-family ipv4
    neighbor 1.1.1.254 activate
    neighbor 1.1.1.254 default-originate
    neighbor 2.2.2.254 activate
    neighbor 2.2.2.254 default-originate
    neighbor 3.3.3.254 activate
    neighbor 3.3.3.254 default-originate
    neighbor 4.4.4.254 activate
    neighbor 4.4.4.254 default-originate
    exit-address-family

    why i cant connect from spoke1-rtr to hub-rtr ?
    thanks for help

    best regards
    damian

  4. Pape says:

    Hello Damian,

    From the looks of it, I’m not seeing that you have configured IPSec for the DMVPN tunnels. You have only enabled the following on the outside interfaces of the firewall:

    access-list outside-access-in extended permit udp any object DMVPN-IN eq isakmp
    access-list outside-access-in extended permit udp any object DMVPN-IN eq 4500
    access-list outside-access-in extended permit esp any object DMVPN-IN

    This essentially means that unless your ICMP packets are encapsulated in ESP, the firewall would drop them. You can allow ICMP and try again before you encrypt the tunnels. Also, you can run a packet capture on the firewall to ensure your packets are getting across.

    Use this command on both sides:

    access-list outside-access-in extended permit icmp any any echo-reply

    Let me know how it goes.

Leave a Reply

Your email address will not be published. Required fields are marked *

A Little About Myself

Hello I'm Pape. My friends call me Pop. I'm CCIE #48357. I enjoy my field and love to share it with others. I love to write so I'm sharing my blog with you.

Sign up to receive notifications and updates whenever new topics or videos are uploaded!

RouteLeak Calendar

September 2024
M T W T F S S
 1
2345678
9101112131415
16171819202122
23242526272829
30