New Technologies

I’ve always maintained that programming is an exercise in managing complexity.  Once a project grows too large, it becomes impossible to maintain an image of the entire source in one’s head.  Not only that, but today there are huge frameworks with thousands of objects and methods, like .NET or Java.  Therefore, the biggest obstacle any programmer faces is trying to comprehend the complexity of a project when extending or modifying the code base.  This is why it takes Microsoft’s large team of smart software engineers years to make what many would consider “superficial” changes to the windows code base.

Unfortunately, the challenge of managing complexity isn’t something that only effects large projects.  I’m feeling the overload of trying to comprehend all the pieces that fit into inkblotter.  On the surface, inkblotter is a simple web 2.0 app – we have user generated content that we must present – but, the way we implement the system has many pieces.  Most of the pieces are 3rd party libraries, many of them not yet at version 1.0.  The documentation is often light to non-existent.

For example, we have experimented with CouchDB, but ultimately decided against its use because it was not yet stable or tested enough.  To connect CouchDB with python, we had to use a thin wrapper written by some guy on Google Code. It took valuable time to figure these systems out.  So when we decided against using Couch, I had learned some APIs for nothing.  The same thing happened when we experimented with Pinax.

Now we are using Git, Gitosis, Gitweb, GitPython, and Git-Django, Beautiful Soup, HTML Purifier, and a host of other markup  language libraries.  I have to learn how to use all these tools.  Don’t get me wrong – I can’t complain about learning new technologies, but these programs and libraries add to the complexity of inkblotter immensely.  We need to get each of them working together in a seamless, efficient, and bug free way.

So, would it be better to roll our own tools?  Make our own version control system? Our own HTML validator? Our own markdown to HTML converter?  Absolutely not.  We don’t have the time, and I suspect the authors of these tools have spent more time and energy in creating these great tools than we even spend on inkblotter; given our school load.

We have no reasonable choice, but to use these tools to make inkblotter as good as it can be.  It just means I’ll spend a lot more time thinking about the code I’m going to write than actually writing it.

Linked Lists

I had five interviews on Tuesday and one today (Wednesday).  I’ll refrain from talking about them at this point because I’ve decided that discussing the interviews before I’ve been ranked or I’ve made my rankings is probably unwise.  Despite this, I want to discuss a programming question I was asked:

Given a singly linked list, in sorted order, how do you remove duplicates?

During the interview I brain-farted because I forgot that the list was sorted, so I suggested a horrible solution involving keeping another linked list of all unique elements that had appeared and then checking each new element in the original list to see if it appeared in the second.  Clearly, this algorithm has a O(n^2) run time, and O(n) memory usage (I think?).  If I had remembered the list was sorted, I would have suggested simply keeping a copy of the previous element as you advance in the list, and removing the current element if it is the same as the previous.

Maybe it’s my inexperience with these kind of interview questions, but it seems like most of them have a little “trick” to them, and you either get it or you don’t.  If you do get it, great, but if not, well you might not get the job.  Once the interviewer points out the correct method, it often seems obvious.  I understand this is testing whether the candidate understands programming intuitively, but that doesn’t mean the candidate is actually a good software engineer.  I’d explain what I mean here, but I’m hungry… and I might be wrong.

In conclusion, I’m bitter that I fumbled at an interview

EDIT: Karan suggested this solution that removes the next element if it’s the same as the current one.

Macro Economics

My most fascinating class this semester is my first university level liberal arts elective, macroeconomics, taught by the infamous Larry Smith.  I have my first midterm in macroeconomics tomorrow, which is noteworthy because up until now, all my midterms have been on engineering or math subjects.  I’m interested in comparing the relative difficulty of the classes.  Up until this midterm, macroeco has been by far my easiest class because I had no homework.  Tonight I spent about three and half hours preparing, and I plan to spend a couple more tomorrow.  This is comparable to my weekly effort put into each calculus and algebra.

On an unrelated note, today I had an interview with Genesys Communications Laboratories, a part of Alcatel-Lucent.  The job is for a research position in mobile software.  It’s the first time the positions been offered, so my interviewer Adrian warned me that the daily job activity would be uncertain and in flux.  He didn’t know what specific platforms or technologies I would work with if chosen.  Still, it’s an exciting opportunity.

Finally, findthehotties.com has been hacked (again), but, have no fear, for at some future, undefined point in time, we’ll fix the exploit!

CUSEC Part Two: Karan’s Warning

I’ve returned to Waterloo from CUSEC.  On Friday, Noah Sugarman and I presented findthehotties.com to the CUSEC democamp.  We were a big hit, but unfortunately, we were hacked by three seperate groups within minutes of finishing the presentation.  Two of the groups personally contacted me with suggestions on how to improve security; of that I’m grateful.  I was surprised when I found out our game was hacked, although, I should have expected it. Firstly, we gave the presentation to a bunch of software engineers, and secondly, my dear friend Karan had warned me that we were doing server validation horribly wrong.  He was 100% correct.

Unfortunately, Noah and I could not fix the site because the hotel took away our internet access for going over our bandwidth limit. (Note: Never stay at a Best-Western).  Today, we finally got the site working again, this time with marginally better security.  To make the site properly secure, we need to do a major overhawl, but that will have to wait because we are both very busy making up school work, preparing for exams, and attending job interviews.

So far, I have five interviews lined up for the coming week.  Finally, I left my fridge open while I was at the conference.  This is most unfortuate because I’d just bought three bags of fresh milk.  Oh well, Murphy’s law, right?

CUSEC Part One

This week I’m at the Canadian Undergraduate Software Engineering Conference (CUSEC) in Montreal. So far, I’ve been to two of the opening day keynote speeches. First up was Leah Culver, a stunning software engineer from San Francisco, arguably most famous for founding the social networking site Pounce, which unfortunately, recently  shutdown.  Leah is an enthralling speaker, and I found her advice energizing.  She suggested that we need to take risks, get out there, push ourselves, and to use the tools we have available.  I’ve been a fan of Leah for a while, because of her use of Django, and success with her free laptop project.

It’s too bad Pounce wasn’t successful, but I’m confident she’ll have a lasting influence on the software industry, as evidenced by her involvement in open source projects, such as OAuth.  Not to mention, she knows quite a few other famous people (some more than merely “internet” famous), to whom I’m now only seperated by two degrees!  I was lucky enough to speak with her for a few minutes before her presentation, but hopefully I’ll have a chance to ask her advice about building a website before she leaves.

I’ll have more to report in the coming days.

Am I Sacrificing my Integrity?

A couple of days ago, my dear friend Karan shared with me an idea about making a CAPTCHA that forced the user to choose amongst the photos of several women and identify the attractive one. This is a decent CAPTCHA algorithm because bots would be hard-pressed to know what an attractive women looks like in order to defeat it. Karan was being facetious, but the idea intrigued me. It quickly became apparent that it could never be a particuarly effective CAPTCHA because of the limitation of the number of pictures that could be reasonably displayed at once. For example, if nine pictures are shown, a bot has a one-ninth chance of guessing the correct image, which is far higher than for traditional text-based CAPTCHAs.

But, I was not to be deterred. Instead of a CAPTCHA, I along with my friend Noah decided to make a high quality flash game based on the concept of picking out the one “hot chick” from a set of 9 images. The faster you pick the right image, the more points you get. I went and registered a domain, and setup a server. We are currently polishing and improving the game, but you can see our progress at findthehotties.com.

The question is, “Am I losing my integrity by creating the game?” I view it as harmless fun, but maybe others would view at sexist, and degrading. To counteract that, we’ve decided to make a male version as well. If people are really opposed to it, we’ll take it down. Have I gone to far?