Monday, March 23, 2009

Brain melt

I'm trying to implement a webapp api in Clojure inspired by the elegant model of Seaside. There are two important shifts in this port, firstly migrating from SmallTalk's object model to lisp's macro based approach, and secondly taking advantage of Clojure's concurrency primitives.

My brain is gently melting out my ears as a result. On the upside, when I get this to work, I get a surprisingly easy way to implement co-operative web applications. A wonderful environment for trying out new approaches to user interface design. It's just the game of getting from vision to implementation.

Sometime next year, at this rate. =)

Thursday, March 19, 2009

So where now?

I have a question, but first I need to draw out some data points.

Firstly, we have Eucalyptus, a project that aims to replicate AWS's Elastic Computing Cloud APIs. This is being used by Sun and others to pitch data centers on the idea of being able to compete in the Cloud Computing game. In short, cloud computing is about to get comoditised.

Secondly, Erlang, Haskell, Scala, Clojure and various other languages are chasing how to best expose concurrency and network IO in a programmer friendly way. I believe we are on the cusp of beating this challenge. Actually, I think Clojure already has, but it's going to take a while for the reality to shake out. And for programmers to either accept they have to learn a new language, or go and open a cafe.

Given the two points above, the game change I see is that it now becomes pizza + beer pricing to host a webapp on a EC2 node that can service 2,000+ concurrent long poll connections. Development could be done on a $20/month node, early roll out on $50/month node, and when it gets really popular you can scale up to the 15.5G of ram beast for $800 a month. But all those prices are about to fall massively, thanks to the return of Moore's law, and the explosion of AWS EC2 wannabes.

So, what can you do with a server capable of mainting say 5K connections? Think FaceBook, but real time. Think livejournal meets twitter. Think corporate business workflow applications with the ability to co-operatively (aka, socially), work with co-workers.

In short, we are about to see a qualitative shift in the way people work together. And this will enable people to act together in ways that will make the 5k person parties organised over fb look positively pale in comparison. Yes, this is going to threaten pretty much everyone in a position of power, be that government, business, or religion. It's about to become affordable for everyone to host real time communication hubs.

On the upside, this means that we are about to start a process of learning how to group communications well. Honestly, everything we've done so far has sucked, and sucked badly. The biggest thing I'm getting out of Gladwell's Outliers is that up until now we have badly wasted the one reasource this planet that can make a difference. Us.

Friday, March 13, 2009

Rebuilding the world. Again.

I'm a little intimidated by the power of Clojure. It changes the dynamics of designing software. Traditionally it was important to get the overall design right because once coding started, changing the design suddenly became quite expensive. Changing the number of fields on a form has rippling changes right through the code base.

Not so in the world of properly factored Clojure development, because all the source code dependent on the shape of the forms is generated using macros. This changes design time from before coding to afterwards. This makes the design iterative, and done upon the working code.

I can see why lisp wizards were both prized and feared. They can build systems that cannot be built other ways, but at the same time it takes Gladwell's 10,000 hours to be ready for Lisp. As a lisp developer you need to understand the impacts of design changes right through the system, from the user interface to disk layouts, because the macros you write cover the whole solution space.

So I finally have an answer to why Lisp was never popular. It's not popular because it is genuinely hard to develop in. The repl is intimidating the same way a drag car is. Sure, it is a beast of enormous power, but it also has no time for formalities. If you don't know what you are doing, this beast will show you up, and probably hurt you in the process.

On the upside, I am now seriously looking at commercial software that costs thousands of dollars per year, per seat, and feeling that I can build it in a reasonably short time frame, once I have written a bunch of macros. Mmmm. Nothing like a challenge to get me going =)

Thursday, March 12, 2009

It's knowledge, not intelligence.

First off, an admission. I'm probably the dumbest of my clan. I have next to no artistic ability, i'm not musically inclined, and I don't have the razor sharp wit. In short, I'm the runt of the litter.

In some ways, knowing I'm not the smartest has been an advantage. I've kept at things long after my other siblings gave up. I'm nowhere near Gladwell's 10,000 hours for mastery of any craft, but I've kept at computers long enough to be able to get things done.

It's why I'm sad about my brother. He's as smart, if not smarter, than I am. Yet he has wasted his talents by being combative with people who were trying to help him. He was so busy trying to prove his smarts that he never had time enough to pick up the knowledge.

In short, his insecurity driven rampages damaged the very friendships upon which his success was reliant.

Tuesday, March 10, 2009

My head hurts.

I've just spent three days trying to understand Clojure's for macro. And I reckon I have another couple of days worth of work pulling it apart yet.

It's a bit of a disservice calling it for, as it is actually a list comprehension, but it is called for because this is the construct to use when a traditional programmer would reach for a for loop.

It has iteration, variable binding, and choice. It is actually a baby programming language in a 30 odd line macro. I can see the parallels between this macro and the de-sugaring that haskell does with the monadic do blocks.

But still, it is a mind warping exercise pulling apart this macro. Fun tho. =)

Monday, March 2, 2009


Clojure, a Lisp for the JVM with STM and Actors. Bringing the power of macros and functional programming to concurrency.

Terracotta, a network attached memory for clustering JVMs and persisting objects across JVM restarts.

jQuery, a JavaScript native api for manipulating the DOM using CSS3 selectors.

Raphael, a JavaScript library for creating vector graphics on the web, using SVG on Firefox, WebKit and Opera, and VML on Internet Explorer.