I am working on a failover concept for a customer and we need the following:

1. The main connection will be a MPLS connection (point to point) 10mbps between the customer and the data centre.
2. The 10mbps connection comes with a 2mbps backup connection, which should take over if the 10mbps goes down.
3. If both MPLS connections die, a normal DSL connection has to take over and open a VPN tunnel to the data centre.
4. If the DSL connection is also not available, the router needs to drop back to a UMTS connection.

These connections need to take over automatically, when the previous connection dies and when the previous connection re-appears, the router needs to automatically re-establish contact with the previous / main connection.

Data must not "leak" out onto the DSL or 3G connections, unless they are actually needed, at which time the VPN tunnels will come into play and all traffic will go over the tunnel (the users are not allowed access to the Internet from the network).

Does anybody have any experience with such scenarios and the associated hardware? I am looking at the Cisco 880G series and Lancom routers at the moment. Does anybody have any suggestions on other hardware?

Oh, and I'm having to document it all in German!

Edit: Oh, and before anybody points out that the router is the weak point, we are planning on either HA or hot-standby, depending on the capabilities of the router.

what if someone hasn't paid the mobile phone bill

And what do you do after the UMTS isn't available? Although if land lines and mobile are not available, you probably have bigger problems than worrying about packing consignments...

As the the 3G, 70€ for 30GB LTE data per month. That should hopefully be enough data, should the main lines fail for a few hours.

I wish our customers took redundancy that seriously! Most of them depend on a 5.99 broadband with no SLA, and then go ballistic when it's down...

The nearest we have out there is done on a combination of routers. Specifically, a Funkwerk Bintec and a Draytek.

I design a lot of the systems at our place. We have two datacentres - not massively far apart but probably 'enough'. I once was in a meeting explaining how the failover on one system worked and I said something like 'If this building gets hit by a meteor or something, it'll fail over to the other one' and someone (a manager, unsurprisingly) asked

"So if this building gets hit by a meteor, we'll be OK?"

My response was

"well, since you and I both work in this building, I'd have to say from our perspective the answer is 'no'."

Is this completely separate from your internet connections, i.e. you are buying all these lines in order to service point to point for one remote network only?

Is the mpls supplied as a fully managed service in which case Global Crossing or somebody would be managing the routers for the 10 and 2mb lines, or are you integrating the mpls with your own routers?

Yes, MPLS is a point-to-point leased line - although it effectively runs on a dedicated cable to the Telekom backbone, then it runs over the backbone to the other end, then has a dedicated line into the datacentre. They guarantee privacy and speed. If the 10mbps line fails, you also receive a 2mbps backup line.

The DSL and 3G/UMTS lines would then be used for backup, if the MPLS fails.

Ok, I am going to assume that the DSL and 3G lines are for failover of the point to point connection only. If your main internet line goes down you won't be doing anything foolish like suddenly trying to use them as failover for that without adequate advance planning I hope.

Likewise, for mpls, your service provider could be selling you a router for you to manage, or you might even be buying your own, but in nearly all cases it is supplied as a manged black-box service. So I will assume you won't have any direct administrative control over the routers for that. Do you at least know how many such routers there will be? Some ISPs supply it with one router aggregating both links, others use a pair running HSRP. If you want to use a redundant router protocol (HSRP or VRRP), have you asked them what their feeling is on the matter? They may support only one option which will impact your range of options if that is HSRP. They might insist on managing the other routers, which would add a pile to your bill, ort he very notion might impact your SLA, making the whole idea bad.

Have you placed your mpls order yet? If so, what information have you supplied the ISP that manages mpls routers in your order?
If they are going to be dropping by next Thursday to install a pair of Cisco 2800 series boxes in HSRP with your internal IP range already configured on the interfaces and in the routing tables, then that will adversely impact your range of options here and we can start discarding things like failover firewalls from consideration unless they are willing to consider a major configuration change at the last minute.

If you are still at the planning stage and haven't even purchased the line yet, have you considered buying a layer2 vlan connection instead? mpls for a point to point is a bit excessive really.

Correct, the network is not "on the Internet" as such. The DSL line would be used by 2 other machines on a separate network segment on another router during normal times. When the MPLS line fails, they would be cut-off and the traffic that was on the MPLS line would be routed over VPN to the data centre.

The 3G line is purely for failover.


AFAIK, the client will receive a basic router with the connection. This will then be fed into our "failover" router, which will have 3 or 4 WAN ports + 3G and an internal link to the main switch.


It is the customer site and they have placed the order, but I haven't received any information yet.


I'm hoping it will be like our leased line, the Telekom provide a 4 port Cisco router, which we feed into a pair of Juniper firewalls running in HA mode.


The MPLS is a requirement of the company's main customer.

They do logistics for a large concern and the large concern dictates what they can and can't do with their network.

Ah, so this is an extranet connection, where the large company has a multi-site mpls vpn network and your guys are getting a link into that?

If that's the case then you will presumably need to fill in a detailed site overview document and they will provide you with a detailed requirements schedule. So you will be ok to plan for this in whichever way you like - your HA Juniper option shouldn't be an issue, but if the other guys do object they will tell you whether they need you to use ospf/bgp/vrrp/hsrp (at least two of which will still be options with your planned equipment).

That DSL failover thing sounds a bit odd. Is the idea to manually move stuff about in the event of an emergency, or does it need to be designed to fail over automatically - cutting off the other guys when your VPN is brought up?
Hopefully the former option; as the latter wouldn't get security approval from the other customer, and I wouldn't have the faintest idea how to set it up.

No. This is a failover design for a customer. We don't get access to the MPLS (well, we get VPN access to the server in the datacentre for maintenance).

I had to come up with a failover plan for the customer's network.


The idea is for the VPN to automatically be started over DSL when the main connection fails. At that time, the users of the internet connected machines will be told to stop using their internet based software - which isn't mission critical - until the main connection can be restored.

Whether their connection will be stopped or they are manually requested to reduce their throughput is not clear at this stage and will probably depend on workload, something that will become clear during testing.

The DSL connection would have its own router and the failover router would have a WAN port to the DSL router, which would only be brought up in emergencies.

MPLS is not one thing. It can come as a Layer2 connection that you have the ability to tag for VLANs, or L2 with no tagging, L3 with a router that you aren't allowed to configure, or one that you are. You could be putting IP packets over it or ethernet segments, or any number of other things.

It's not clear either what's at the other end of the link. Is this your customer's datacentre rack or is it their customer's? Are you providing a solution that includes active components in that DC that participate in the failover routing, or is a third party with closed kit involved?

If the MPLS is on L2 (with or without tagging) and you have both ends of the link, this stuff is very easy. If you are going to use a Cisco 880 on your DSL make sure you get one with a software image that supports managed switching though. Or make your life that much easier still by telling them to get a dedicated DSL line for failing over to.

If the DC is third party, and the MPLS is managed at L3 by Global Crossing or MDNX or somebody, and the third party has lots of extranets to support, and your network segment is 192.168.0.x or some other obvious choice; then budget about 10 extra days of engineering to cover the massive pain you are about to experience. I had one of those that was so bad I had to request a new TAC engineer with Cisco because the guy I was dealing with had a nervous breakdown. It took two months 6 way conference call to fix because 6 companies were involved and the Cisco guy also had a big argument with the other firm's Juniper JTAC support agent.


The JTAC lady was very nice though.

The data centre is run by a hosting company, who have a dedicated pair of machines for us (async-mirrored in another off-site pair of machines in their other data centre). Their end has the Telekom MPLS router going into their local firewall (Cisco ASA).

In the DC, the machines run Oracle and DCOM at the moment, should be migrated to SAP in the long run.

Our end will have the dedicated Telekom router, attached to our Firewall / Router (probably Cisco GM800) and the local network will have industry touch terminals running Windows 7 Embedded running custom apps to start with, then later RDP into a terminal server.

Not familiar with the GM800, what would that be?

It doesn't look like you would have much luck using either a routing or a first hop redundancy protocol, so your original plan you go with policy based routing seems like your best bet. I never use Juniper, but I hear it is easily done on their screenOS boxes.

Bah, sorry 880G.