Friday, December 26, 2008

Antlr, a reflection

I worked with ANTLR, attempting to build a small code generator. ANTLR, along with TextTemplate includes a complete toolchain for building everything required for a code generator, ANTLR includes both a parser generator and a tree walker generator, and TextTemplate is a full templating solution.

However, working with the ANTLR suite turned out to be harder than I was expecting. Debugging the generated AST transformation code required a lot more spelunking through generated code than I was happy with. ANTLR has good support for debugging parsers over text, but I couldn't get debugging tree walkers in the ANTLR tools working to my satisfaction.

I can see two alternate paths going forward. Either I take the lisp approach and build out tree transformation code using something like clojure, and learn to love emacs and clojure's REPL. Or I take the path of writing code in a programming language written by compiler writers to write compilers in. Two obvious examples include Haskell and Scala.

The reason I am persisting with figuring this out is that we are on the cusp of one of those interesting times when we get to re-write the world again. We re-wrote everything when PCs appeared, then we re-wrote it again when Windows beat out MS-DOS, then we re-wrote everything again when the web first appeared.

And we are going to do it again to allow people to be able to use applications inside their web browsers, disconnected from the web. This is different from traditional desktop applications, even though from the users perspective they appear to be the same. The new functionality is that data is being kept both on the client, in a web browser hosted sql datastore, as well as on the server.

Which means we are about to re-write all of the applications that business users use. Oh goody =)

Saturday, December 13, 2008

Google's approaches to concurrency

I have a bit of a thing for concurrency. It is one of those challenges where in the answer actually lies in choosing the right collection of technologies such that programmers can achieve what they want to, safely and effectively.

The traditional approach, using locks et al, requires a level of mathematical proof that is infeasible. If we are going to use locks, we need to do it in a way that the compilers and run times will be able to verify that we aren't being stupid, otherwise it becomes too hard.

Google's various approaches to utilising concurrency safely are intriguing. The first, MapReduce, is beautiful in the way that it elides concurrency. Suddenly programmers can write programs that will be run across thousands of machines, in a way that is both safe and manageable.

Then there is the message passing paradigm that has surfaced in both Google Gears and in Android. This can be looked at as another implementation of the Actor model, but in both cases using languages that are mutable and in widespread use. It is the fact that messages are serialised across the communication boundary that gives safety, but the rest of the language environment isn't impacted. It only requires programmers to treat talking to another thread the same way that they talk to another process across the internet, a widely understood abstraction.

Where this gets interesting for me is the Chrome multi-process architecture. Here is an architecture that facilitates traditional programmers moving into concurrent code using message passing, but also in a way that is compatible with all the current libraries.

Actually, where it gets really interesting is Google's native x86 web plugin. This technique could be used to embed plugins as completely managed threads inside of an application. Near as I can tell this is bringing Chrome's multi-process architecture into other web browsers, by the back door.

What I want to know is what would happen if someone took Google's x86 plugin architecture and bolted it into something like Qt, where the thread responsible for the screen paint, the thread holding the application code, and various other threads (network io, database io, et al) were each one of these managed x86 processes? It would allow Qt applications to dynamically download code into a running application, and spin them up as additional sandboxed services.

The possibilities...

Sunday, December 7, 2008


Feeling the rush, and calm. I like.

Saturday, December 6, 2008

48 hours

That was a rush. Intense. Insane. I want more. I want time to think. I want to understand. I want to just feel. Body longing, mind screaming. Sanity yet prevails, almost.

Where to from here? I know not. It is in the hands of the fates.

*wolfish grin*

Wednesday, December 3, 2008

It's all about concurrency

Probably the most amusing, to me at least, comment after the AppEngine talks yesterday was "It's all about concurrency."

The thing with AppEngine is that it is factored in such a way as to allow insane amounts of concurrency in a way that doesn't break programmer's heads. The traditional programming by mutation based computation model is still present. There is no need for programmers to get side affect free religion to use AppEngine's massive scale out capacity.

The only downside of AppEngine is that it requires programmers to grok DataStore. DataStore is where the concurrency rubber hits the road, and it breaks programmers. Programmers are control freaks by nature, and being forced to give up control for MT safety is emotionally hard.

It's going to be an interesting decade. Just as the wide spread adoption of GUIs forced OOP on a generation of programmers, Multicores are going to force concurrency on this generation. I'm kinda expecting a lot of programmers not to make the transition...

Looking forward...

I'm looking forward to being closer to the sea.

Monday, December 1, 2008

Brain full now, check back later

Possibly the most amusing part of playing with technology is how routinely I wind up feeling completely and utterly stupid.

Sunday, November 30, 2008

Change is in the air...

(And not just the smells of summer)

When the luminaries of an industry are all out on the hustings evangelising change, and by change I'm talking about stuff that no one has tools or methodologies for yet, then it's fairly obvious things are in a state of flux.

This change can probably be best described as the son of Ajax. The amusing thing with the explosion of Ajax was that it wasn't anything particularly new, most of the underlying technologies had been in place for the best part of a decade. What Jesse James Garret did when he launched the Ajax meme was give us the lexicon to talk about the changes we were witnessing with Google Maps and GMail.

These new web application capabilities forever changed the perceived capabilities of web deployed applications. Suddenly, every application being launched onto the web had to have these capabilities. Most back end coders scoffed, Javascript has long been derided as the joke programming language. But, what customers want, customers get.

This new revolution is going to change everything because the web browser is going to become host to complete applications, and the back end is going to become nothing more than a persistent communications hub for people to interact through. This changes the performance requirements of the back end, away from relational databases and towards appengine style distributed datastores.

The new capability that this gives customers is the ability to use the webapp from a disconnected client. This might not seem like a big thing, until you take into consideration the explosion in sub notebook computing devices, netbooks, iphones and androids, all of which are partially connected to the net through the 3G telephone network and the occasional WiFi hotspot.

This new capability is going to be a huge driver, and it requires a complete rethink of how we build web applications.

Saturday, November 29, 2008

It's a tad freaky doing research, and getting results that include mailing list discussions in which I took part, all too long ago. It's almost like I'm retracing my steps, but I don't remember making them...


At the time, I knew I was fighting for a life. I didn't intuit that the life I was fighting for was mine.

In retrospect, I can see how I was controlling so many factors, it truly was an onerous and thankless task. In fact, after the event, I doubt any of the players will even think I had anything to do with what went down. But then, said actors aren't exactly known for their ability to step outside of the here and now, or even outside of themselves. The chance of any of them glimpsing the full picture really is nil.

I suppose that is why I feel alone a lot of the time, not for lack of company, but for lack of people who can see what I see. My father can, but mainly if i try to explain what i see all i get is blank of disbelieving looks. I learned long ago not to explain.

And now, finally, I have learned that I can walk away. I don't need to save people. In fact, they need to save themselves. I need all my power just to save me.

Tuesday, November 25, 2008

Dave Thomas inspired this one.

I lost faith in AppEngine, partly because of a pratt in England that annoyed the shit out of me, and partly because I couldn't see how to implement the applications I wanted using it. The DataStore, simply, wasn't enough.

Tonight I watched, horror struck, as Dave Thomas (Of OTI fame) told a room full of Java coders that Java is the new Cobol, and that interesting applications of the future are going to be written on the Cloud, and in JavaScript. Cue deafening silence as a hundred odd java programmers start burrowing under their chairs.

It inspired me. Instead of trying to use AppEngine's DataStore as a transactional store, which is something it sucks at, I can instead using it as a pub/sub notification hub.

This would move the query side out to the clients, with their local Gears datastores, and make updates the only part that had to be propagated from the client back up to GAE, and thence out to all the other clients.

It actually works. A complete application deployment platform built from Google's software. Kinda scared right now.

Saturday, November 22, 2008

Craig Mundie on the Future of MS

Wharton interviews MS's Craig Mundie

Craig is predicting a future where in we have full blown client side applications that interact with the cloud. Migrating applications out of the web browser, if you will. The destruction of the web browser has actually been a long term aim of MS, as made obvious in the their post IE6 destruction of the IE development team.

Obviously, migrating people from web browsers - which are inherently operating system agnostic - back to client applications which are tied to specific operating systems would be good for continuing MS's Operating System monopoly.

There are application domains that are ill suited to current web browsers, ones that are compute intensive or graphics intensive, for instance. Games, primarily. However, the text rendering engine inside of modern web browsers leaves the current generation of application toolkits (Swing, SWT, WF, MFC+, Qt) for dead.

The biggest disadvantage to me of a heavy client approach to interacting with the web is that suddenly which computer I am sitting at matters again. Having two laptops and a desktop, that hurts. Having an application that comes in both web and heavy versions, and stores all configuration in the web would reduce that burden.

Soon, I suspect, I will need an iPhone version as well...

Thursday, November 20, 2008

Right. That's it.

I'm taking a holiday. I'm stuffed. I've had it. Worn out. Tired. Roll me over, i'm friggen done.

Just looked at some positions that a recruiter sent me. I should have been jumping up and down, there is some cool stuff in there. But all I can think to do is to get some grub, drink some beer, and sleep. For about 48 hours.

Wednesday, November 19, 2008

Odd realisation of the day

I've just come to the realisation that my issue with learning new stuff is dealing with the emotional distress of going through the acknowledgement of incompetence. It's yet another grieving process.

I've seen the emotional response in others when attempting to grapple with something new, I saw it all over the AppEngine list, the Clojure list, and various other places. What is new is the realisation that my own aversion to study is linked to this emotional reaction as well.

So being able to manage the grieving means I can start learning properly. It's been a while...

Monday, November 17, 2008

I feel like i've fallen back in time...

Playing around with MS's Visual Studio Express editions is a bit of a time warp. It reminds me of when I was programming against win16, and I use the word against advisedly, back probably the better part of two decades ago.

MS obviously builds some of the best tooling for first use. Click here, fill in code there, and voila, one working web application. I'd like to say that the traditional J2EE stack approach is inherently better than this, but it aint. From a pure get stuff done, the bang per buck measure, C# ASP.Net with it's databound controls and integrated JQuery has the Java world up against a wall. Sure, VS licenses aren't cheap, but what is the cost of a team of Java developers?

This is going to be an interesting adventure.

Saturday, November 15, 2008

An experience like no other...

I must say, today was an experience in overload. In many different ways.

It was the Sydney Bar Camp, and it was fun up to a point. When people started getting tired, man. Ranting about how Google is evil, then about how to verify who should have the power to verify if companies handling of data is valid, and then finally the whole australian net censorship discussion. Gar. Raw emotions on display.

My head hurts.

On the upside, I can see a bunch of new technologies stepping up to allow us to step through the multi-core insanity. F# on .Net is going to be interesting, to say the least. =)

Thursday, November 13, 2008

Thoughts on Remix culture

One of the things that I like about developing for the web is that there is a culture of openness about web development techniques. There has to be, all the source is out in the open already. In fact, there is significant pressure on web developers to write good code because it's all hanging in the breeze.

One of the impacts of this is rapid migration of new ideas. Part of this is down to the ease with which one can pop open firebug and have a poke around inside someone's html and css. It makes for easy remixing.

I suspect this is probably the hardest part for adoption of Flash and Flex - all the really cool apps are not open for review and learning. Actually, the same can easily be said of back end development. *cough* Google *cough*

Wednesday, November 12, 2008

Realisation of the day

(Although, I'm sure I've had this one before.)

Application development is broken. This much is obvious to most folk. The reason is interesting in that it reflects something that is true of all design.

There is no right time to design the application's interface.

During initial specification, when most people spec an interface, most people can't visualise using the application. Thus you get underspecified and inconsistently specified interfaces. By the time an application is built, it is too late to specify, as change at this point usually requires complete rewrites of the majority of the code base.

In short, by the time people can actually use an application, their reaction is always "That's not what I want!"

There has to be a better way. And I intend to build it.

Tuesday, November 11, 2008

Change in perspective...

Possibly the biggest change after coming to grips with ANTLR is now no system is "too big" to tackle. If I can design a DSL that encompasses the design, then I can implement the whole thing. Sure, there is a whole bunch of work writing the various traversals and the code generation templates.

But there is suddenly no boring trudge work left. =)

Sunday, November 9, 2008

Random musings

For an industry whose primary sales schtick is "money savings through automation" we are remarkably attached to doing things manually.

Heretical stance of the day.

I love of all things Google - I use GMail, Google Docs, Blogger, Analytics, AppEngine, and GWT. But that doesn't stop me thinking things are wrong with the big G.

One of the important things about capitalism is it's darwinian nature. One of the things wrong with most companies is that they are, for all intents and purposes, command economies. Google tries very hard to allow darwinian nature in, but there is a serious problem.

Google is the TRS-80 of the cloud era. It is amazing that it works, and it is creating a new world, but that doesn't mean it is of the new world. The TRS-80 was no Apple PowerBook.

Google is the leading edge, and thus being technically right is important, but for what comes next, design is more important than algorithms. So, who is going to be the Apple of the Cloud Computing era?

Saturday, November 8, 2008

My head, it hurts...

CSS, JavaScript, Yahoo UI, ANTLR...

One of these days I might learn to not try to boil the ocean.


ANTLR as a gateway drug to lisp?

I'm reviewing the code I wrote for my code generator, and it wound up looking a lot like lisp. Actually, a lisp with destructing bind. Like Clojure, say.

The biggest advantage for me with using ANTLR over the lisps is that I can, and do, look at the generated parser and tree parser code. I understand the underlying implementation, which is essential for being able to debug. Clojure is a step into the wilds for me, I don't really understand what it is doing, and there is only so much macro-expand tells me.

Hopefully playing more with ANTLR will allow me to understand how Clojure works...

Friday, November 7, 2008

Drunk, reflective, and in need of a holiday

I'm making bad decisions, being pig headed when i should roll over and admit i'm wrong.

A fair bit of it of late.

On the up side, i'm having a holiday soon. It'll being a juggle financially, but on the upside I'll focus on my projects instead of trying to force my work into being interesting.

And if my projects work out, work will get a whole lot easier, and maybe even more interesting.

This has been brought to you by the letters b, e, e, and r. And the number 2. Yeah, time I slept. More beer tomorrow.

Tuesday, November 4, 2008

Another direction change

Now that my code gen is starting to spit out code, and actually usable code at that, I am starting to get larger aspirations.

One of the continuing bugbears of my professional existance is the creation of webapps. They have a bunch of inputs, there is some processing, and a bunch of outputs. In all honesty, it sounds, at the highest level, like something that should respond to automation.

In fact, a lot of people have tried. The biggest issue is that all the current code generators are finished products. They are designed from the ground up to produce finished output. I think this is the wrong approach - the resultant webapps are invariably broken by the inability of the user to express their specific requirements in the tool's input medium.

I think ANTLR's approach to building a DSL, and then allowing embedding of chunks of java through it is the correct approach. The DSL expands, and the embedded java is exposed in place. I think this is what is required. A DSL for capturing web applications, along with the ability to embed java in it as the "business logic".

Mmmm. Yea yea. I've sold my soul to ANTLR. =)

Monday, November 3, 2008

Thinking about libraries in lisp

One of the things that always confused me with lisp is that for a fifty year old language, there is bugger all in the way of battle hardened libraries. Take java, a mere puppy at ten, and yet it has more libraries than you can poke a stick at. And don't get me started about c. We'd be here until Christmas. The year after next.

Now that I am playing with ANTLR, I think I have understood why there are no libraries. You see, the desire for libraries is to reduce the programming effort to achieve a goal. The amount of code required to write all the database access layer in a reasonable sized webapp is considerable, so people use Hibernate instead. No one likes Hibernate, in fact there are rants upon rants about how bad it is, but all said and done, using hibernate is better than not shipping at all.

The problem with Hibernate and co is that the programmer winds up giving up control. He (or she) has no choice but to interact with the library on it's terms. If you don't like the terms, find another library or write your own. If you want to do something like what Hibernate does, but slightly different? Fork it, if you are brave. More likely, you will curb your tongue, and agree that Hibernate's way is the one and only way, and deal.

With a code generator, this is suddenly not a trade off I have to make. As long as I can write prototype code from the database up to the user interface layer, and I can express my design in some computable data format, then I can slice up the prototype code into templates, and generate out the whole application. Libraries suddenly are a source of headache, constriction, and bugs, instead of being a saviour.

In other news, I'm finding Google less and less useful as a source of knowledge. Thanks to an explosion of indexed content of varied value - i'm thinking of web forums full of questions - my keyword searches are returning pages full of my keywords, but no actual usable information.

I'm resorting to long form book purchases to get the knowledge I need, often in pdf format. A positive thing for artists and authors, but worrying for the big G.

I'm becoming a religious zealot



I wrote the guts of a Model Driven Engineering code generator over the weekend. I think I might be getting ANTLR finally.

Only downside? Everything I look at I start working out how to build a compiler and/or a generator for it. Gah.

Friday, October 31, 2008

Getting one of the Zens of Lisp

It's odd, I've always admired, and been scared of, lisp. It's one of those languages that everyone always talks about emotionally. There are the zealots, preaching macros and homoiconic data / code representation, and there are the rest who pick on it's parenthetically obtuse nature.

I think I am starting to understand why Lisp's macro system is such a big thing. The amusing thing is that I'm not using Lisp to do it - I'm using a code generator to write a code generator from a hand designed data language that looks suspiciously like JSON. But it's a tad more prescriptive than JSON. Honest.

I'm finally getting the religion on capturing requirements in computable format. Clients always change one of two things - the original spec, or how they want the whole product to look. The first requires only that the requirements be updated, while the later requires that the generator be updated. In neither case is a bulk change to the whole code base required.

Finally, the project implementation can be correctly factored. I like it, a lot.

Wednesday, October 29, 2008


The current webapp designs use a layered model, like all things in computer science. Each layer has it's primary concerns, the database layer is concerned with persistance, the client layer with presentation, the middle layer with everything else. The problem with this design is that abstractions leak, and constraints of each layer are visible in other layers.

My belief is that the correct way to engineer a webapp is to acknowledge the constraints of each part. The disks can deal with about 100 seeks per second per spindle, give or take. The channel between the web browser and the web app is high latency. And the most important is that any webapp design that requires disk seeks on either queries or updates is inherently DOSable.

This leads me towards a asynchronously updating network design, wherein browsers notify the webapp of changes, the webapp responds with changes it has seen, and the webapp streams updated state to disk. Think of a spider with it's eight legs spread on different strands, listening to it's web. The trick is going to be generating consistent code across the various nodes, the html, the javascript, the concurrent server code, the reporting code.
A new place to rant and ramble, a new face to see the world with, a new time to create new realities. As they say, growth is change, and change involves letting things go. 

And with this, I start anew.