Thursday, March 31, 2011

Monads! European Virtual ALT.NET. Video is now available

Thanks to the excellent Jan Van Ryswyck, the video of my EVAN talk is now on Vimeo here:

http://vimeo.com/21705972

It’s very similar to the DDD talk video that I posted before, but I think a little more polished. It’s much easier to follow in this format too.

I start off with a brief introduction and history of Monads, but spend most of the talk deriving a Monad by attempting to do function composition with two Maybe returning functions. I briefly overview the differences between C#, F# and Haskell Monad handling at the end.

Check out my series on Monads here:

http://mikehadlow.blogspot.com/2011/01/monads-in-c1-introduction.html

Sunday, March 27, 2011

Monads! European Virtual ALT.NET Tuesday 29th March

Jan Van Ryswyck has very kindly invited me to rant at length on the subject of Monads. I’m going to be giving an hour long presentation for the European Virtual ALT.NET this Tuesday. This will be a first for me, I haven’t attempted an on-line presentation before, so I’m very interested to see how it goes.

The talk is going to be very similar to the Monad talk I gave at DDD9, but hopefully I’ve improved the flow and code examples, so it might make more sense. The full announcement (and details about how to join in) are here:

http://europevan.blogspot.com/2011/03/mike-hadlow-on-monads-on-29-march-2011.html

See you on Tuesday!

Wednesday, March 16, 2011

7000 Concurrent Connections With Asynchronous WCF

It’s rare that a web service has some intensive processor bound computation to execute. Far more common for business applications, is a web service that executes one or more IO intensive operations. Typically our web service would access a database over the network, read and write files, or maybe call another web service. If we execute these operations synchronously, the thread that processes the web service request will spend most of its time waiting on IO. By executing IO operations asynchronously we can free the thread processing the request to process other requests while waiting for the IO operation to complete.

In my experiments with a simple self-hosted WCF service, I’ve been able to demonstrate up to 7000 concurrent connections handled by just 12 threads.

Before I show you how to write an asynchronous WCF service, I want to clear up the commonly held misconception (yes, by me too until a year or so ago), that asynchronous IO operations spawn threads. Many of the APIs in the .NET BCL (Base Class Library) provide asynchronous versions of their methods. So, for example, HttpWebRequest has a BeginGetResponse / EndGetResponse method pair alongside the synchronous method GetResponse. This pattern is called the Asynchronous Programming Model (APM). When the APM supports IO operations, they are implemented using an operating system service called IO Completion Ports (IOCP). IOCP provides a queue where IO operations can be parked while the OS waits for them to complete, and provides a thread pool to handle the completed operations. This means that in-progress IO operations do not consume threads.

The WCF infrastructure allows you to define your operation contracts using APM. Here’s a contract for a GetCustomer operation:

[ServiceContract(SessionMode = SessionMode.NotAllowed)]
public interface ICustomerService
{
[OperationContract(AsyncPattern = true)]
IAsyncResult BeginGetCustomerDetails(int customerId, AsyncCallback callback, object state);
Customer EndGetCustomerDetails(IAsyncResult asyncResult);
}

Essentially ‘GetCustomerDetails’ takes a customerId and returns a Customer. In order to create an asynchronous version of the contract I’ve simply followed the APM pattern and created a BeginGetCustomerDetails and an EndGetCustomerDetails. You tell WCF that you are implementing APM by setting AsyncPattern to true on the operation contract.

The IAsyncResult that’s returned from the ‘begin’ method and passed as an argument to the ‘end’ method links the two together. Here’s a simple implementation of IAsyncResult that I’ve used for these experiments, you should be able to use it for any asynchronous WCF service:

public class SimpleAsyncResult<T> : IAsyncResult
{
private readonly object accessLock = new object();
private bool isCompleted = false;
private T result;

public SimpleAsyncResult(object asyncState)
{
AsyncState = asyncState;
}

public T Result
{
get
{
lock (accessLock)
{
return result;
}
}
set
{
lock (accessLock)
{
result = value;
}
}
}

public bool IsCompleted
{
get
{
lock (accessLock)
{
return isCompleted;
}
}
set
{
lock (accessLock)
{
isCompleted = value;
}
}
}

// WCF seems to use the async callback rather than checking the wait handle
// so we can safely return null here.
public WaitHandle AsyncWaitHandle { get { return null; } }

// We will always be doing an async operation so completed synchronously should always
// be false.
public bool CompletedSynchronously { get { return false; } }

public object AsyncState { get; private set; }
}

Now we’ve got an AsyncResult we can implement our APM based service by implementing ICustomerService:

[ServiceBehavior(
InstanceContextMode = InstanceContextMode.PerCall,
ConcurrencyMode = ConcurrencyMode.Multiple)]
public class CustomerService : ICustomerService
{
public const int DelayMilliseconds = 10000;

public IAsyncResult BeginGetCustomerDetails(int customerId, AsyncCallback callback, object state)
{
var asyncResult = new SimpleAsyncResult<Customer>(state);

// mimic a long running operation
var timer = new System.Timers.Timer(DelayMilliseconds);
timer.Elapsed += (_, args) =>
{
asyncResult.Result = GetCustomer(customerId);
asyncResult.IsCompleted = true;
callback(asyncResult);
timer.Enabled = false;
timer.Close();
};
timer.Enabled = true;
return asyncResult;
}

public Customer EndGetCustomerDetails(IAsyncResult asyncResult)
{
return ((SimpleAsyncResult<Customer>) asyncResult).Result;
}

private static Customer GetCustomer(int customerId)
{
return new Customer(customerId, "Mike_" + customerId);
}
}

We’re mimicking a long running IO operation by using a timer. I believe that the Timer also uses an IO completion port, but don’t quote me on that. When WCF invokes BeginGetCustomerDetails we first create a new SimpleAsyncResult with WCF’s state object. WCF will pass the AsyncResult to the EndGetCustomerDetails method when the timer completes, so we can use it to pass any response state. In our case this is an instance of Customer.

Next we set up a Timer and attach a closure to the Elapsed event. When the timer’s Elapsed event fires we create a customer instance, pass it to our AsyncResult and then pass the AsyncResult to WCF’s callback function.

After the BeginGetCustomerDetails method completes, WCF returns its thread back to the WCF thread pool so that it can service other requests. Ten seconds later, the operating system posts an item on to the IOCP queue, a thread from the pool picks up the item and executes the continuation. This in turn calls WCF’s callback which in turn executes EndGetCustomerDetails. WCF then packages up the Customer as a SOAP response and returns it to the client.

Only a tiny fraction of that 10 seconds has actually occupied a thread, so we should be able to make thousands of concurrent calls to this service.

In my tests, this service has been able to handle a little over 7000 concurrent connections.

The code is here: https://github.com/mikehadlow/Mike.AsyncWcf

Just follow the README instructions to run the tests and try it out for yourself.

Wednesday, March 02, 2011

WebStresser: A very simple web load testing tool

I’ve been doing some experiments recently with WCF, seeing how much load I can throw at different configurations. I’m especially interested in seeing what difference async web services can make with IO intensive operations. One problem I had was finding a good HTTP load tool. I tried to get WCAT working, but found it complex and hard to understand. It also gives almost no feedback at all when you get something wrong.

In the end I wrote my own little load tool around WebRequest. It’s almost trivial, but there are few little hoops you need to jump through in order to make it work. I’ve wrapped it up as a little console application, WebStresser. You can get the source from github here:

https://github.com/mikehadlow/WebStresser

Or download a precompiled binary which requires .NET 3.5 or greater to work. It’s also reported to work fine on Mono:

https://github.com/downloads/mikehadlow/WebStresser/WebStresser.zip

To use it, just run it with a minimum of a URL:

WebStresser -u=http://localhost:5401/

Which outputs:

Starting test...
Completed: 0 Faulted: 0 Connections: 1
Completed All 1
Faulted 0
Elapsed ms 2,001
Calls per second 0
Avergate call duration ms 1,127

By default it just executes a single GET request to the given URI. If you want to see the response, add the –r option:

webstresser -u=http://localhost:5401/ -r

Which outputs:

Starting test...
Completed: 0 Faulted: 0 Connections: 1

http://localhost:5401/
Status: 200, OK
X-AspNetMvc-Version: 3.0
Connection: Close
Content-Length: 7259
Cache-Control: private
Content-Type: text/html; charset=utf-8
Date: Wed, 02 Mar 2011 21:49:37 GMT
Server: ASP.NET Development Server/10.0.0.0
X-AspNet-Version: 4.0.30319

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3. .... etc

To run a load test, just tell it how many requests to make with the –i option, and the delay, in milliseconds, between each request with the –t option. Here I’m running 100 requests 20 ms apart:

webstresser -u=http://localhost:5401/ -i=100 -t=20

Finally here’s an example calling a SOAP web service. Note the SOAPAction header and the –p option that points to a file with the postdata:

webstresser -u=http://mike-2008r2:8123/proxy -m=POST -i=100 -t=10 -p=postdata.txt 
-H=SOAPAction:http://tempuri.org/ICustomerService/GetCustomerDetails

For help:
webstresser -h

Options
-?, -h, --help
-u, --uri=VALUE REQUIRED: The URI you want to call
-m, --method=VALUE The HTTP method. Default is GET
-i, --iterations=VALUE Number of iterations to run, default is 1
-t, --interval=VALUE Iterval between each call in milliseconds, default
is 10000
-p, --postdata=VALUE Path to file containing post data
-r, --responses Print responses
-k, --keepalive KeepAlive header value (true or false), default is
true
-a, --accept=VALUE Accept header value, default is 'text/xml'
-c, --contenttype=VALUE ContentType header value, default is 'text/xml;
charset="utf-8"'
-z, --timeout=VALUE Timeout in milliseconds, default is 10000
-H[=VALUE1:VALUE2] Add a header to the request. e.g: -H MyHeader=
MyValue

I’m using the excellent Mono.Options library to do command line processing. It makes putting this kind of tool together very easy.

Happy stressing!