As a fresh faced youngster majoring in Computer Science at the University of Essex (1984-1987) in the UK, I was wide eyed with all of the possibilities that computers would bring to all aspects of civilization as we knew it. The course provided a comprehensive coverage of languages and concepts ranging from Assembly (machine language) programming, Pascal, COBOL for business and my choice for my final year project, artificial intelligence and expert systems. My professor at the time was a young genius called Richard Bartle, known to the world as the co-author of MUD (Multi-User Dungeon), a text-based role playing game which was arguably the precursor for every major adventure known to teen.
As Wikipedia describes it “Traditional MUDs implement a fantasy world populated by fictional races and monsters, with players being able to choose from a number of classes in order to gain specific skills or powers. The object of this sort of game is to slay monsters, explore a fantasy world, complete quests, go on adventures, create a story by roleplaying, and advance the created character. Many MUDs were fashioned around the dice rolling rules of the Dungeons & Dragons series of games.”
Back then we were allocated limited timeshare hours on the DEC PDP-11’s to perform our programming assignments. Needless to say I burnt most of mine playing MUD, so much so that for a time there my friends nicknamed me MUDdy. Below is an example of the type of the text descriptions of the game and the commands (n-for north, w-for west etc). Exciting huh? MUD was written in Lisp (List programming language).
And therein lies the connection, as the terms Map and Reduce come from Lisp and functional programming. Click here for a MapReduce primer if you need it. There is also a very interesting presentation by a student at the University of Maryland on the evolution of MapReduce and its relationship to Lisp and the Google File System.
As a Richard Bartle disciple I decided that I would emulate my mentor and work on a final year Lisp project. I had a choice between creating a business style application requiring a relational like repository, or designing and coding a mini-MUD, which efficiently stored a large inventory of objects and game descriptions. More about which project I selected later.
After I graduated in 1987, and entered the workforce as a programmer, I found out that there wasn’t much opportunity for my Lisp skills in the commercial business world. So I started programming in languages such as C, Cobol and RPG on the business machine known as the AS/400. I would also later become adept at being a DBA for Oracle and Informix RDBMS on Unix systems. By 1994, I was a software architect, extremely frustrated at my company’s lack of ability to market and sell more of what I thought was the best product on the planet. That is when someone suggested I take matters into my own hands and become a product manager. Product management gave me the best of all worlds, I was able to get out from under my programming desk to develop relationships with people all across the organization. Influencing and pulling together teams for a common cause. Ultimately I was able to have a major influence in how product requirements were defined and prioritized. I also took on outbound product marketing and positioning to ensure that customers who had problems understood why our product was the best fit for them.
I had all but tucked away Lisp and my days at the University of Essex in my long term memory cache, when I attended Chris Wenzel’s MapReduce and Hadoop presentation last month. As he started to describe MapReduce concepts and showed some of the simple code examples on screen, it was as if I had a parallel query convergence of everything I had learned and done from my days at the University of Essex with Lisp; combined with my years toiling away tuning indices on Oracle and Informix.
This was it, I thought to myself, I had seen this before and maybe, just maybe, if I had created the business application with the relational like repository rather than the mini-MUD for my final project, I could have ……. then I woke up and got realistic.
So now you know why I feel I was early to the MapReduce and Hadoop party. We’ve all learned over the years however that there is no place for technology that is ahead of its time, just as the best technology doesn’t always win in the marketplace. The big data explosion we see today was not an issue 20 years ago. So early or not, business requirements, value and benefits need to align for the right technology solution to have mass market appeal. Which is why I hung up my coding gloves long ago and moved into product management and product marketing. It enables me to admire and enjoy the work that engineers from companies like Aster Data, Greenplum, Cloudera, Google, Yahoo and others are producing, while I focus on marketing solutions and helping drive the requirements for new applications that these technologies can and will solve.