Parallel Computing in .NET

It was a little surprise but not shocked when I heard that INTEL are talking to Microsoft for parallel computing programming because in a couple of years, there will be CPUs with 16 cores to 256 cores.

As we have already seen many new PCs come with quad-core (four core processing units), it is not a surprise to hear 16 cores PC will come out soon. But for 256 cores? I am a little surprised. As in 1960's, those old slow computer on APOLLO could send human to Moon, now equipped with 256 cores computers, what will human beings do with the computing power? Yeh, of course, someone will use the computing power to play games!

The demonstration of parallel computing programming was run on a laptop with duo-core processor.

The first time, the program ran a million times of heavy math calculation. It took about 10 seconds to complete with one CPU running nearly 100% and another one idle.

The second demonstration was to separate the calculation into 10 blocks. This time, both CPU ran at high usage and the total time was 5.3 seconds.

By looking at the time, we noticed that the parallel computing is not exactly half of the one thread computing. This was because of the “context switch”, the extra work involved in parallel computing.

Since this is a duo-core laptop, there are only two working threads at a time. When one thread is processed, the CPU will have to clean up the context of the thread and load the context for a new thread. It is a like a hotel with two rooms. If a customer checked out, we need to clean up the room for new customer. This is the overhead of “context switch”.

This session was given by Keith Rome. He said the so-called “Hyper-threading” label on some PCs was not good. For a single core CPU, hyper-threading just added more context space in process memory. If you have only two threads running on PC, it can improve the speed by pre-load the next thread context. But if you truly believe the “hyper-threading” and began to “multi-tasking” on such old PC, there will be a lot of context switching and the actual speed will be very slow.

You may have already experienced it.

Warning: ONLY when CPU one runs at high and others idle, you may try multi-threading programming. Usually, the optimization in operating system is better than you can design in your code.

There is a situation when your computer runs slow but when you look at the performance, the CPU is running at low usage, 10 to 15% maybe.

This is not issue of threading or parallel computing. Most times, the bottleneck is at the I/O. Reading and writing to hard drive is slow but reading and writing to a network drive can be even slower. In this case, APM model is introduced. APM is Asynchronous Programming Model.

Asynchronous programming means you make an asynchronous I/O call, switch off the thread and let the CPU to notify you when the Asynchronous call returned a value. So the thread of CPU can be used for other computing tasks while your task is waiting on a reply from I/O.

The benefit is this can increase the scalability of a system, operating system threads are used more often, more efficient use of available resources.

The difficulties are that debugging will be harder, code can be more difficult to read, difficult to maintain.

In ASP.NET, to deal with I/O latency problem, one can consider using APM pattern to write asynchronous page, call asynchronous web services.

My question: what about the traditional ASP pages? Will multi-cores server handle the threading properly?