Tuesday, April 14, 2009

Google App Engine and Java

On April 2nd, I was glancing through the list of RSS feeds I follow, and saw this one Brand New Language on Google App Engine. Well, it being the day after April Fools, I didn't suspect anything. While I was waiting for the page to load after clicking on the link I remember thinking to myself "Please, please, let it be Java". As I read that the "new" language they were supporting was Fortran 77, it dawned on me that it was a April Fools joke. As much as I enjoyed the effort that Google put into this years joke, I was quite disappointed that it wasn't Java.

Well, last week Google answered the most popular feature request by introducing Java support. Though I havn't had a chance yet to check it out, I am looking forward to a plethora of new tools to come out on the Google infrastructure.

Watch this space! I will try to keep you posted of my experiences as I try it out.

Tuesday, February 3, 2009

It's Easy!

I am becoming increasingly annoyed when I read articles celebrating some new programming language or toolkit as being "easy". It has gotten to the point now that for me, such statements count as a strike against the technology in question, rather that a point in favour.

To be fair, I suppose it's not a strike against the language or toolkit, but rather against the author of the statement. I have just gone through an exercise in aggravation that stemmed from this notion. I needed to develop a performance measurement tool for a client. After considering my options, I chose to develop it using the Google App Engine (GAE). My decision was based on 1) Time to market, 2) Cost of hosting, and 3) Reuse of existing infrastructure. GAE is built on the Python scripting language, and though I had never coded in Python, from what I had heard and read, it was supposed to be an easy language to pick up and work with. In truth, it is easy to learn Python. The language is quite well documented, and the syntax is fairly intuitive. It should have been, according to the proponents of scripting languages, really easy to punch out this application. Combine that with GAE, and not having to write a whole user administration component, and I should be laughing.

My mistake was in thinking that the once the Python application is delivered, it will be of the same caliber as its Java counterpart. This is not the case. Python may be a good prototyping language, and it may even serve for simple little scripts. It is not what I would call a carrier grade language. Which surprises me, since Google seems to use it so extensively, I would have expected more. Here are three reasons why I would stay away from the scripting languages, and stick to the more mature languages like Java or C++ for any application for a paying customer.
  1. With Java, when I refactor a method or class name or signature, I am not relying on a keyword search of the code base to find the instances that need to be changed. The IDE will locate all the references to the method or class in question and change them. Should the IDE miss something, the compiler will catch it. With a scripting language, you make your changes, and then have to run the code to see if you have missed anything.
  2. With Java, there are many tools for tracking your code coverage. And in fairness, there may be such tools in Python, but that would mean hunting them down and trying to find out which one are best, then learning how to use them, etc, etc, etc. When I make a change to my code in Java, I can find out almost instantly if it is sufficiently covered with unit tests.
  3. The type safety provided by java is second to none. These languages that allow you to change the type of the variable on the fly (like Python and Ruby) scare me. How can you code with any confidence knowing that the objects you are referencing may cease to be the same type of object the next time it is run? One of my colleagues used to say that "Software doesn't rust". Well, with these types of languages, I'm affraid that it does!
I was lured by easy development, and I am now paying the price for my laziness. It looks like I may be re-writing my application (at my cost) in a language that will last! I sure hope that GAE starts supporting Java soon, it's otherwise a great service.


Tuesday, January 20, 2009

Pay Me now, or pay me later...

Fram used to have a set of advertisements where they illustrated the importance of a good quality oil filter by having an auto mechanic explain that if you don't pay a little bit more now for the Fram oil filter, you will pay a lot more later for the repairs. He would say "Pay me now, or pay me later".

Many in the software industry have looked for a metaphor with similar "punch" to drive home the need for quality in the software industry. To date, I have always used the Fram example. Not any more.

One of my colleagues showed me a blog posting written on November 1st, 2007 by Steve McConnell called Technical Debt[1]. This is the best illustration I have ever read on the subject, and it is now my new favourite metaphor.

This illustrates in very clear language the importance of managing the quality of a software project. I strongly encourage all of you to read it, and look for ways to internalize it into your organizations.

  1. Sometimes, debt is a necessary evil. There are times when going into technical debt is ok, it just shouldn't be done on a whim.
  2. Just like monetary debt, Technical Debt must be paid off. The longer you wait with paying it off, the greater the debt.
  3. As with monetary debt, Technical Debt must be tracked. The old saying "You can't manage what you can't measure" comes to mind. Don't sweep your technical debt under the rug, it will just trip you up later.

Thanks Steve for making this subject so simple to understand and explain.
[1] http://blogs.construx.com/blogs/stevemcc/archive/2007/11/01/technical-debt-2.aspx

Tuesday, January 6, 2009

Agile Planning For Software Projects

(Removing Risk with Scope-based Planning)

Much has been written over the past few years about Agile methodologies for software development. To set the record straight, I am in favour of a more agile approach to software development. Unfortunately, many people erroneously assume that the various agile methodologies that have become so popular remove the need for planning. It is the intent of this article to show that increasing agility in one's approach to software development does not need to be at the expense of planning.

In this post I will propose an approach for planning software projects that will not only allow for agility in the project life cycle, but even encourage it. This approach to planning will reduce cost over-runs, feature creep, resource mis-allocation, and deadline slippage. All of these conditions are often seen as indicators of poor project management. Unfortunately, this often gets reflected on the project managers. The truth of the matter is that the whole team is responsible for correcting and avoiding these issues.

Scope-based Planning

The mantra of Scope-based planning is "Plan Early, and Plan Often". This is accomplished by setting aside ten percent of the scope of each stage to planning for that stage. It is important that this be done at each stage. It is equally important that the full allotment of planning time be used. The level of detail to which one will plan will be proportional to the length of time in the stage. A long stage will get macro (high) level planning, while a short stage will get micro (low) level planning. So if you are just starting the project, this would be ten percent of the expected total project time. If you are at a one week stage (an iteration) it would be ten percent of that week. It is also important to note that this is ten percent for each resource, not just the developers, but the testers, the interaction designers and the project managers.

Ten percent doesn't sound like a lot, and in fact it isn't. I've had some people suggest that this should be twenty percent, but I feel that is too much, especially when you consider that we are talking about ALL RESOURCES. What we are asking of the team isn't the creation of a full design document at each stage, just that we stop for a moment and think about the tasks that this stage will require. If the stage is only one week long, those tasks will by definition be fairly fine grained.

The planning needs to be repeated at each stage of the project. Initial planning will begin as soon as the project scope analysis has been completed. This planning phase may also include a more detailed planning phase for the first release. Then each subsequent release would see a planning phase for that release. In each iteration of the release, each resource would spend 10% of his or her time planning for that iteration.

For example...
The following image shows a dummy project. (Click on it to enlarge)
Let us assume that the initial scope analysis shows that the project is expected to take 2 months. The team will be made up of three Developers, one Interaction Designer (ID) and one Project Manager (PM). The developers will have 7 weeks or 35 days of work each (105 days total). The ID is expected to have 3 weeks (15 days). The PM is expected to have 2 weeks (10 Days). That's a total scope of 130 days. Therefore, the project warrants 13 days of planning up front. (This would be added to the scope.) Assuming that the PM spent 1 day meeting with the client and coming up with this initial scope, The ID, the PM, and the lead developer should spend 4 days each planning the project. During those 4 days, they will gather requirements, formalize the release and iteration plan, and validate the expected scope. The last step in the planning is a review of the scope. It is possible, even likely, that the scope will change already at this point. This is the agile part. The PM will need to make a "Go / No Go" decision. If he or she is not empowered to do so, this may require a meeting with the client. The point is that if the initial estimate was wrong, you want to know about it as soon as possible.

Now, assuming that the scope hasn't changed, lets say that we have planned for two releases of one month each. That means that the first iteration would be primarily executed by the ID. Before she starts working, she should spend half a day planning her work. During this week, she would likely involve the lead developer to discuss options, but it would mostly be just her. This would be the time for the lead developer to set up the database and server environments, and perhaps start some high level designs.

By the end of week one, if the ID has discovered any unexpected elements of the project, the team can re-assemble and determine the impact to the scope before continuing.

If no further changes are required, the developers will start working on the project on the second week. Once again, ten percent of the week would be spent (up front) planning. In our example, each developer would spend Monday morning planning the week. At some point in the week, the PM will want to meet with everyone to find out how things are going. Now your agile methodology might call for daily "stand-up" or "scrum" meetings, that is not what I'm referring to. I'm talking here about reviewing progress against the plan. Also, in many shops, the ID will also be tasked weekly with reviewing the tickets that are completed, so our example shows that as well. At about the mid way point, the ID and Lead Developer will have to take some time to plan the second release. Then, the second release continues much like the first.

Scope-based Planning has the following benefits:
  • No surprises. When surprises arise, they can be dealt with at once. Sharing bad news with the client early gives them the ability to deal with it before it becomes a crisis. Cost over-runs are not seen as negative as they might otherwise be, because the client had knowledge of them early and has the option to remove something else that wasn't yet started, or cancel the project all together, cutting losses early.
  • Feature Creep is eliminated, because the planning will show the impact of new features to the budget and the time lines.
  • Resource mis-allocation is also eliminated because the scope of the project is validated prior to a large team being assembled.
  • Deadline slippage is also controlled because early and often planning will expose any risks to the project early. The PM then has the opportunity to address those risks before the project gets too far.

The final Word...
Now many of you might say "All that planning is actually going to slow the project down". To that I say "Yes, it will"... and this is a good thing. We are talking here about doing what is best for the client, not the consultant. Your mother wouldn't let you go running through the house with scissors in your hands either. Sometimes the best thing you can do is slow down, and watch your step.

Others are no doubt saying "Planning doesn't prevent surprises". Again I say "Correct". But planning early and often, reduces the hit that those surprises will have on the overall project.

I am not advocating wasting time on filling binders with useless information that will never be looked at again. A colleague of mine puts it this way: "Plans are useless, Planning is priceless". If you are taking your plans and writing them up in some template, and printing three copies for the management team etc... you are a fool. Planning can be accomplished on a whiteboard, or a napkin, and the documenting of those plans can be captured with the camera in your cellphone. If it's a macro level plan, throw it up on a wiki or a quick email to the team.

I hope that this inspires you when you start your next project to take the time it warrants to think it through. I am positive you will reap the rewards of a more stress-free project life-cycle.