[ILUG] redundant db driven web serving

[Nick Murtagh]

Gary Pigott wrote:
> Round robin DNS is the way to go for load balancing, but it's not a way 
> of implementing redundancy. If one of your servers goes down, the DNS 
> server will still hand out it's IP as before. TTLs & DNS caching, and 
> propagation delays ensure that there's no way you can change the DNS 
> records fast enough in the event of an outage to redirect traffic only 
> to the live server.

It was sort of implied, but I guess not obvious, that the monitoring
system would update the DNS in the case of a switchover.

Obviously a DNS based switchover is not instantaneous, but it is
pretty easy to implement compared to multihoming.

> Multihoming may be the way to go, although I'm not sure if you can 
> multihome an IP address to two sites. We multihome 1 IP address down 
> multiple pipes (from multiple providers) to the same location using BGP. 
> That way our clients never notice a link go down and always get the 
> lowest hop count when connecting. I suppose there's nothing stopping one 
> of the links pointing to a separate site, giving you multi-site load 
> balancing. There are others on this list that know this kind of thing 
> way better than I do (and do contract work for a reasonable fee).

I don't see why it shouldn't be possible to set things up this way. How
quickly does a BGP routing change propagate in the case of a switchover?

> The one tricky thing is reconciling transactions after an outage..... 
> it's entirely possible that the link between both sites can go down, and 
> both accept traffic assuming the other is dead.... this can lead to 
> duplicate transaction IDs etc. You need to have some mechanism in place 
> to either tolerate this within the application or to reconcile them later.

That's why the monitoring + switchover system really needs to live in a
third datacentre.