"That gives you 48 hours to develop a methodology – since we haven't done one of these before – prototype the model, build it, calibrate it to client expectations, and write the report. But don't worry, the client is paying a premium for the rush."
Every analyst has been in this sort of situation. The client wants something now, and your boss has agreed to deliver it. The trouble is that before that perfectly crafted 3-line email three is sent, accompanied by a hefty invoice, you actually have to build the thing. What is more, lack of time is no excuse for sloppy work. No mistakes, after all we're professionals.
Working to a deadline
The Mythical Man-Month is the best book about managing projects to a deadline that I have ever read. Although focused on software engineering, the book has lessons that apply to project management in any technical field. The author, Frederick P. Brooks, provides a mathematical argument for why adding manpower to a technical project that is running late makes it later (Brook's Law). The problem is that time spent getting new people up to speed (initiation) and coordinating additional manpower (co-ordination) exceeds the additional productivity. There is a fixed time-cost to incorporate a new worker onto a project. Unless the project has a long way to go, adding people just slows it down.
An obvious corollary to Brook's Law is that there is a minimum time required to complete any project that is directly proportional to its complexity. Moreover, this minimum time corresponds to a unique number of workers: at some point additional manpower has decreasing returns to scale in the form of co-ordination and initiation costs. What does this mean for your 48-hour rush-job? It is a one-man, or at most two-man, show. Looks like you will be staying back in the office on your own.
No mistakes...well sort of
The big consultancy houses and banks have built a powerful image for their industries over time. That image is in every logo, every website, every banner at a conference. The image is one of flawlessness, of crystalline perfection, of exacting standards.
This notion of a clinical, relentless pursuit of perfection has managed to convince the non-technical members of society that for the right price perfection is not only achievable, but the norm. On the first day of my first job, I remember my boss emphasising the many processes that were in place to prevent errors. My boss was, naturally, a manager. Managers do not run the numbers: if no-one finds an error then they assume there were none to find. I rapidly realised that although it was unacceptable for errors to be discovered, it was perfectly acceptable for errors to exist.
A one page document can contain perfect spelling; a single table of data can be copied exactly from another source. But the problem for the analyst is managing complexity. Large documents are more likely to house inconsistencies; models that attempt to capture more variables are more prone to calculation errors. The Law of Large Numbers sucks, doesn't it.
Of course, the notion of perfection is somewhat flexible for the rush-job. The odd inaccurate assumption is forgivable, as is a minor inconsistency in the results. But if there is a major mistake, don't dare blame the deadline. So how do you avoid mistakes during a rush-job? The answer is a variation of the "Keep It Simple Stupid" (KISS) rule.
Keep it Linear Stupid
Let's assume for a second that you are young and naive, some might even say a cock-eyed optimist who has gotten caught up in the world of international modelling intrigue. Despite the impending deadline, you nevertheless are determined to capture every aspect of the phenomenon at hand. One crucial variable is clearly quadratic, and to assume it is linear would be a travesty against all that mathematics stands for. Was it all for nothing, Archimedes?
You have a choice in front of you: you pick the blue line – the complexity ends, you wake up in your bed and believe whatever you want to believe. You pick the red line – you stay in mathematical wonderland and this model is going to show you just how deep the rabbit-hole goes. You are young, you are fearless, you are stupid – you choose the red line.
 |
| "I know what you're asking yourself: why oh why didn't I pick the blue line?" |
Down the rabbit-hole
Okay, so its not as bad as waking up in a petri dish to find that you have a massive fire-wire port in the back of your skull. But by 2 am in the morning of the second day, you are going to wish that you could go back and choose that blue line – I guarantee it.
You see the problem with a rush job is that you don't have time to go back. The client expects the results to make sense, and there is always one number that can't be explained by anything other than an error. But when you go to iron it out, that red line comes back to haunt you. Every tiny alteration to the inputs results in a non-linear change to the results.
Ordinarily, this is fine as you have plenty of time to get it right Given time you would be able to rework the model, but that is time you don't have. Every time you pin down one set of numbers, another one jumps out of its place. Non-linearity is hard to explain and even harder to defend, especially when you know that there may be errors. If you are lucky, you will be able to explain away the aberration. If not, you have a report that may as well be lit with a neon sign saying "Error: Division by zero".
The price of sanity
Okay, so you learned your lesson. Next time you get a rush job, you restrict your analysis to the one-dimensional case (and even that seems gutsy), tether your key variables to a stake in the numeric al equivalent of Alcatatraz, and assume that everything from the inflation rate to the diffusion of information throughout the economy can be approximated by a linear function. Nice job.
You have done well by your boss, you have done well by yourself. You have even done right be your client, as they have the report they wanted. But what good is it? The purpose of quantitative analysis is to enlighten decision making, to add something that could not have been uncovered by logic alone. Once you have removed all complexity, you have also removed all value. You have created consistency and clarity but at vulgar cost. In truth, you have obscured more than you have explained.
An analyst often has little choice in the matter at the time, but they can make clear how banal the results are that spring from these sorts of exercises. More time may be an option, absolute internal consistency may not be essential. If the alternative is simplification ad absurdum, then it may be better to pass on the job.
Keep it simple, keep it trivial. Keep it linear? Stupid.
No comments:
Post a Comment