Saturday, December 13, 2008

Google's approaches to concurrency

I have a bit of a thing for concurrency. It is one of those challenges where in the answer actually lies in choosing the right collection of technologies such that programmers can achieve what they want to, safely and effectively.

The traditional approach, using locks et al, requires a level of mathematical proof that is infeasible. If we are going to use locks, we need to do it in a way that the compilers and run times will be able to verify that we aren't being stupid, otherwise it becomes too hard.

Google's various approaches to utilising concurrency safely are intriguing. The first, MapReduce, is beautiful in the way that it elides concurrency. Suddenly programmers can write programs that will be run across thousands of machines, in a way that is both safe and manageable.

Then there is the message passing paradigm that has surfaced in both Google Gears and in Android. This can be looked at as another implementation of the Actor model, but in both cases using languages that are mutable and in widespread use. It is the fact that messages are serialised across the communication boundary that gives safety, but the rest of the language environment isn't impacted. It only requires programmers to treat talking to another thread the same way that they talk to another process across the internet, a widely understood abstraction.

Where this gets interesting for me is the Chrome multi-process architecture. Here is an architecture that facilitates traditional programmers moving into concurrent code using message passing, but also in a way that is compatible with all the current libraries.

Actually, where it gets really interesting is Google's native x86 web plugin. This technique could be used to embed plugins as completely managed threads inside of an application. Near as I can tell this is bringing Chrome's multi-process architecture into other web browsers, by the back door.

What I want to know is what would happen if someone took Google's x86 plugin architecture and bolted it into something like Qt, where the thread responsible for the screen paint, the thread holding the application code, and various other threads (network io, database io, et al) were each one of these managed x86 processes? It would allow Qt applications to dynamically download code into a running application, and spin them up as additional sandboxed services.

The possibilities...