Dual IKE gateway with OSPF

I would like to share some of my IPSEC testings which I hope very close to a real life example.  Below is the topology of this lab. J41 device is the IPSEC HUB and J23 and J21 devices are spokes. What I wanted to achieve is if J23 loses connectivity with the primary IKE end point (212.45.64.2) it should fail over to the other interface i.e 212.45.65.2 and OSPF neighborship should be established again. This is of course based on a scenario that BGP link between J41 and J36 goes down and network isn’t reachable via that path any more. I am not going to give whole configuration but some configuration which are the basis for this setup and the issues I have experienced so far including BGP.

 BGP

I have set up the BGP configuration first to be sure routing works properly and J41 exchanges routes between its peers J36 and J37. Then to reduce the failover time, I also configured BFD as you can see below.

[edit protocols bgp]
root@j41# show 
group 8501 {
    type external;
    import myImport;
    export myExportPri;
    peer-as 8501;
    bfd-liveness-detection {
        minimum-interval 1000;
    }
    neighbor 212.45.64.1;
}
group 8503 {
    type external;
    export myExportSec;
    peer-as 8503;
    bfd-liveness-detection {
        minimum-interval 1000;
    }
    neighbor 212.45.65.1;
}

I wanted to announce only one network which is 212.45.63.0/24 to outside network and prevent my AS being a transit AS plus make the J36 path more preferred with the AS path prepend. If you don’t put the reject rule at the end, you end up announcing everything which makes you a transit AS.  (myExportPri policy is the same only without the as-path-prepend statement)

[edit protocols bgp]
root@j41# top show policy-options policy-statement myExportSec    
term 10 {
    from {
        route-filter 212.45.63.0/24 exact;
    }
    then {
        as-path-prepend "7000 7000";
        accept;
    }
}
then reject;

IPSEC

Then IPSEC setup comes. I configured IKE gateway on J23 spoke as below;

gateway gw-hub {
    ike-policy ike-pol-1;
    address [ 212.45.64.2 212.45.65.2 ];
    dead-peer-detection {
        interval 10;
        threshold 3;
    }
    external-interface ge-0/0/0.0;
}

This config means: Probe the IKE gateway (e.g 64.2)  three times in 10 seconds interval and if there is no response, failover to the other IKE gateway. This also tears down the IPSEC tunnel. I must note that DPD is a requirement if you have more than one gateway in the list. Junos shouldn’t even let you commit if you haven’t configured DPD.

What about HUB configuration? Actually this is something I could think of so far which I am not sure if it is the best practice or not.

[edit protocols bgp]
root@j41# top show security ike 

gateway gw-23 {
    ike-policy ike-pol-1;
    address 192.168.179.2;
    dead-peer-detection {
        interval 10;
        threshold 3;
    }
    external-interface ge-0/0/0.64;
}

gateway gw-23-sec {
    ike-policy ike-pol-1;
    address 192.168.179.2;
    dead-peer-detection {
        interval 10;
        threshold 3;
    }
    external-interface ge-0/0/0.65;
}

I have configured the same destination in two gateway configuration block in which only external-interface is different without “establish immediately” statements.

OSPF

I have configured OSPF on hub and spokes and involved st0.0 interface in the neighborship i.e

[edit]
root@jgw23# show protocols ospf 
export vpn-routes;
area 0.0.0.0 {
    interface st0.0;
}

SCENARIO

The scenario I have tested is this;

I have initiated an HTTP session from Ubuntu3 with the following command;

ubuntu3#wget --limit-rate=100K http://212.45.63.2/file.iso

Then I shutdown the interface of J41 facing J36 to trigger the BGP failure and the followings happened in sequence;

• wget displayed stalled warning that it can no longer fetch the file.
• BFD detected the change very quickly and tore down the BGP session after which routes are learnt via the backup link from J37 
• During this link failure OSPF dead timer expired and route towards the source Ubuntu3 IP 192.168.181.2 disappeared in the routing table which made
  the flow session route invalid during which I observed that “show security flow … extensive” for this particular HTTP session shows that “Route: 0x0“
• AS BGP link came up in the mean time, J23 ipsec client failed over to the second IKE gateway 65.2 and ipsec tunnel is established
• This event is followed by the OSPF neighborship to be established. Once OSPF is established, 192.168.181.2 route became available again after which
  current HTTP flow session’s route entry became also valid.
• and finally wget resumed its download

On top of this I have also tested what happens to a session which has the incoming interface ge-0/0/0.64 initially (not from the VPN spoke)  ie. you initiate a traffic which is flowing through J41 device and it enters the firewall via ge-0/0/0.64 interface first. Then BGP goes down and traffic enters the box via another interface ge-0/0/0.65. What I have seen is that SRX firewall detects this route change and update the session table to reflect with the interface change. i.e in “security flow session” you see the new interface. This requires some seconds to update though, one requirement which I hadn’t thought was for this to work both of these interfaces must be in the same zone.

Please do share your experience or topology ideas with me such as what can be done to have a more robust failover etc.