Async Programming and Bottlenecks

An attendant of Devscovery asked Richter what are the gotchas when using the AsyncEnumerator. Jeff gave an example specific to the async enumerator, but here are a couple more that are applicable to multi-context programming. That’s a better name than multi-threaded since the CLR could execute everything on a single thread and these problems would still exist.

To use Jeff’s example,

If it takes five seconds to fetch an image, and you fetch two synchronously it’ll take ten seconds. But if you fetch them asynchronously it will only take five seconds. In fact if you fetch 1,000 asynchronously it will still only take five seconds.

There are two, perhaps unobvious, gotchas with that scenario. By removing the request-generation bottleneck by quickly initiating requests you can create new problems.

If you were creating a photo gallery listing the image sizes you might run out of memory downloading the images before you can start freeing the memory by de-queuing the IAsyncResult. I imagine their are other finite resources that could be exhausted this way, network connections, file handles perhaps.

The other potential problem, depending on your goal. What happens to the image server? It might have enjoyed having a request-generation bottleneck!

So watch out for the implications of solving a request generation bottleneck, unless your job is writing the stress-tester.