Recently, I rewrote a quote service, and being paranoid about uptime (and possible lurking bugs in my code), I also wrote some code to add redundancy which switches quote providers when a failure occurs. I did this on 2 levels… switching between the new and old quote service (these were written as providers using the factory pattern), and also switching between my new services (the same service running on 2 different servers).
Since the switching code seemed like a common element, and needed to error-free, it was a good candidate for abstraction. For illustration, I’ll use multiplication as redundant service. (By the way, I didn’t end up using this code). I’ll work backwards from implementation toward the abstract class.
First, the interface for the 2 functions I want to switch between:
interface Multiplier
{
int Multiply(int x, int y);
}
The 2 functions which implement the interface:
class SimpleMultiplier : Multiplier
{
public int Multiply(int x, int y)
{
return x * y;
}
}class SlowMultiplier : Multiplier
{
public int Multiply(int x, int y)
{
int output = 0;
for (int i = 0; i < x; i++)
{
output += y;
}
return output;
}
}
Now here is the code for FailoverMultiplier. I’ll show this code before the abstract class to illustrate how the abstract class is used.
class FailoverMultiplier : Failover<int, Multiplier>
{
public FailoverMultiplier()
{
Initialize(“Multiply”);
members.Add(new SimpleMultiplier());
members.Add(new SlowMultiplier());
}
public int Multiply(int x, int y)
{
return RunFailover(x, y);
}
}
Now for the abstract Failover class. The interesting part of the code was wrapping the individual Multiply functions within the try/catch of another function, and doing it in an easily reusable way. (fun activity: guess how it’s implemented first). Notice that I also have functionality to optionally prefer the 1st member, and revert back to it after a certain amount of time.
abstract class Failover<TReturns, TBase>
{
protected object lockObject = new object();
protected int memberIndex = 0;
protected List<TBase> members = new List<TBase>();
protected bool preferFirst = false;
protected DateTime primaryFailTime = DateTime.MinValue;
private System.Reflection.MethodInfo method;protected void Initialize(string functionName)
{
Type t = typeof(TBase);
method = t.GetMethod(functionName);
}protected TReturns RunFailover(params object[] parameters)
{
int iterations = 0;
bool failedState = true;
TReturns output = default(TReturns);if (members.Count == 0)
throw new Exception(“members list is empty”);
if (method == null)
throw new Exception(“‘method’ was not initialized”);lock (lockObject)
{
do
{
iterations++;
try
{
output = (TReturns)method.Invoke(ActiveMember, parameters);
failedState = false;
}
catch
{
// do logging
}if (failedState)
{
if (memberIndex == 0)
primaryFailTime = DateTime.Now;
memberIndex = (memberIndex + 1) % members.Count;
}
} while (failedState && iterations < members.Count);return output;
}
}protected TBase ActiveMember
{
get
{
if (preferFirst && DateTime.Now.Subtract(primaryFailTime).Hours >= 1)
{
memberIndex = 0;
}
return members[memberIndex];
}
}
}
An earlier version of this class required the user to define a function in the FailoverMultiplier class which was to to be called by the abstract class’s function “RunFailover”. This is necessary if you don’t use reflection or if you need more performance. In my tests, it took 7 seconds to do 1M iterations with the invoke, but .02 seconds to call the function directly.
