Wednesday, November 10, 2010

On promises made by marketing

In a large software organization, like in all large and complicated mechanism, friction always occurs at the interfaces.To overcome it, multiple communication lines are often erect between different functions. Take the army for example - while there's communication up-top, between the generals, often there is a lot of direct communication in the middle ranks and even more in the lower ranks. Doing otherwise would cause havoc on the battlefield - the air-force might bomb a location, but the message from the ground forces scout (that the target is long gone) is still en-route through the "official" channels.
In a software organization however, there's very little communication between the R&D department and the marketing department at the low ranks. Only the top managers - product level and upwards usually - meet with the people that actually meet the customers. This is usually excused by the need to "free the low-ranking R&D people (read-  the developers) from business concerns".
However, this lack of communication causes strange scenarios, where the marketing people don't really know what the product can and can't do and, even worse, don't know the future directions that can be traversed. Sometime only a tiny change in the code is required to enable a huge benefit (and a large increase in sales) for the customers. However, this request for change never comes to R&D because it's filtered by the marketing devision.
The same issue, only in reverse, occurs in the R&D department. Sometime we, developers, labor for days and months to enable a complex feature - that ultimately is used only by a small fraction the customers. Even more often, we over-develop - create a feature too wide in scope and too sophisticated - when the customer really just wants the simple thing done (usually over and over - and could we please provide some automation for that?).
Probably the worst sin of the marketing devision is committing to something that can't be delivered - or can be, but with great costs (development, architectural or performance). This is not done out of spite, though the complete effect does sometimes seem like the something done by an evil gremlin. The marketing devision truly wants to make the customers happy - but they lack the specialized knowledge the R&D personnel have - how exactly the product works. The worst sin R&D can make is barring (or even removing) features without really understanding the impact on the customer base. Again - not out of spite, but simply because no one has any idea.

The emerging pattern is of two disparate organizations - one dedicated for the "beauty and pureness of code", the other for "placating the customers" - while none is really dedicated for "create a product the customers want to use". This pattern especially visible the more "technological" the product is - the harder it is to actually understand the underlying logic.

Friday, November 5, 2010

On perspective (or "concentrate on what's important")

"We already have this great framework we use during the main data processing, why not for user management?"
Or
"There's already a bulk of code responsible for loading and saving millions of records - why should I write something new to load and save configuration records (all 10 of them)?"

I hear those (and similar) often. Or worse, I don't hear them, just see the commits in the version control. I think this is a price we pay for modularizing our code. The more functional and setup-free, the easier it is to reuse a piece of code in many contexts.

When a developer write a piece of code, he usually has a specific use-case in mind. This drives performance requirements, defaults, existing fixtures that need to be in place and other "non-functional" requirements. Innocently, that developer checks-in the code, usually along with it's dependent "client" code (and hopefully, some tests), and says to himself "Good job. Well done, me - time for some coffee".
Then, another developer appears. Let's call the first one "Fred", for being a Framework developer, and the newcomer "Cliff", for developing a new Client code.
So, anyway, Cliff comes in with a new requirement - create a client to change a specific configuration. Since Cliff is "constructively lazy", he asks around for some similar pieces of code to what he needs. Cliff finds Fred, still drinking his coffee and basking in the warmth that comes from a job well done. Fred tells Cliff - "Oh yes, I've just checked-in a very useful piece of code for the highly complex, extremely sensitive, super-important project from the high-lord boss. It's a beauty.". Cliff is happy now - all he needs to do for his tiny client part that, truly, is not that important to the main business, is link it with the shiny new framework Fred has written. He does so, and shortly joins Fred for some black tea (anything with caffeine goes...).
The result is that now both the "super important" code (main data processing, critical engine part, analysis engine...) and the relatively "less important" code (system configuration load and save) use the same underlying code - the framework code. However, all Cliff needed was a simple SQL statement - one for loading, one for writing.

Think about it like this - it's reasonable to hire a chef to cater for out important formal Dinner. However, you probably don't want the same chef, his sous-chefs, the kitchen boy and the delivery girl all running in your kitchen, just because you're peckish and want to grab a cold snack.
The smarter that framework is, the more sophisticated its analyzers are, the bigger the cost is when using it on a non-main path. Easy tasks should have easy solutions. The fact that the engine is there, ready to be used, doesn't mean it should be used - there are usually many hidden costs ("non functional" - remember?). Not to mention the extra coincidental dependency that now exists. Think about it the other way around - you wouldn't use the simple code for your main flow. If you try that, you'll hit problems very soon - performance, compatibility, corner cases. However, no one thinks twice about using the complex code in use cases analogous to opening the fridge and getting a chocolate bar.

To finish, a very known quote from an old Chinese master (paraphrased): "Don't use a cannon to kill a mosquito".