New Technologies
February 15, 2009Andrew 3 Comments »I’ve always maintained that programming is an exercise in managing complexity. Once a project grows too large, it becomes impossible to maintain an image of the entire source in one’s head. Not only that, but today there are huge frameworks with thousands of objects and methods, like .NET or Java. Therefore, the biggest obstacle any programmer faces is trying to comprehend the complexity of a project when extending or modifying the code base. This is why it takes Microsoft’s large team of smart software engineers years to make what many would consider “superficial” changes to the windows code base.
Unfortunately, the challenge of managing complexity isn’t something that only effects large projects. I’m feeling the overload of trying to comprehend all the pieces that fit into inkblotter. On the surface, inkblotter is a simple web 2.0 app – we have user generated content that we must present – but, the way we implement the system has many pieces. Most of the pieces are 3rd party libraries, many of them not yet at version 1.0. The documentation is often light to non-existent.
For example, we have experimented with CouchDB, but ultimately decided against its use because it was not yet stable or tested enough. To connect CouchDB with python, we had to use a thin wrapper written by some guy on Google Code. It took valuable time to figure these systems out. So when we decided against using Couch, I had learned some APIs for nothing. The same thing happened when we experimented with Pinax.
Now we are using Git, Gitosis, Gitweb, GitPython, and Git-Django, Beautiful Soup, HTML Purifier, and a host of other markup language libraries. I have to learn how to use all these tools. Don’t get me wrong – I can’t complain about learning new technologies, but these programs and libraries add to the complexity of inkblotter immensely. We need to get each of them working together in a seamless, efficient, and bug free way.
So, would it be better to roll our own tools? Make our own version control system? Our own HTML validator? Our own markdown to HTML converter? Absolutely not. We don’t have the time, and I suspect the authors of these tools have spent more time and energy in creating these great tools than we even spend on inkblotter; given our school load.
We have no reasonable choice, but to use these tools to make inkblotter as good as it can be. It just means I’ll spend a lot more time thinking about the code I’m going to write than actually writing it.
February 15th, 2009 at 2:59 PM
How dare you describe inkblotter as a web2.0 site! It has *purpose*.
Besides, I wouldn’t say we dropped CouchDB for lack of stability. We first decided to try out CDB because it would solve our problem using MySQL to store a nested section scheme. However, CDB didn’t really solve this problem perfectly either. In addition, the final nail in the coffin was RDFA and the lack of requiring nested sections at all. Oh and git too.
February 15th, 2009 at 3:11 PM
@Karan Bhangui
I was trying to make a point man. Jeez.
February 15th, 2009 at 3:17 PM
I understand. I feel overwhelmed with the various technologies we have to deal with as per our meeting last night, but it’s important to give credit/demerit where due.