Further to the perils of (quantitative) prediction
Yet another example of the sins of the analytic methodologists who overemphasize quantitative approaches to intelligence problems – or at least their academic counterparts – has crossed our desks thanks to Mapping Strategy (and Zenpundit).
We would strongly second these statements, especially when discussing the vital importance of distinguishing between the elements of the actuarial – essentially the use of historical analogies, albeit in numerically more precise form – versus the critical contributions of imaginative speculation grounded in the rigorous examination of competing qualitative factors in forecasting future events. The call for more creative and strategic intelligence is one driven by the recognition of the need for far more imagination and vision in the analytical process.
As we have said, quants have their place and quite often will have unique value. But intelligence failures are rarely the result of poor numbers. Rather too often we see the inability to construct coherent narratives from overwhelming flows of rapidly changing information, or the limitations on describing the gaps in our knowledge to permit the development of new collection approaches in order to begin to fill them. We see the lack of willingness to challenge conventional wisdom – not as the result of politicization, but out of the lack of desire to do the hard work of defining and defending alternative viewpoints; coupled with the unwillingness to face the potential that while the alternative might be equally wrong, you stand alone in that error (whereas the group’s error allows you to avoid personal blame.)
We would also venture that the most recent examples too often cited as high profile “intelligence failures” owe far less to errors in methodology as they do to a much deeper failing of the community’s processes for sensemaking and explanation. But these are far more difficult to teach and to manage than another numbers crunching apparat – though that is another discussion entirely.
We would strongly second these statements, especially when discussing the vital importance of distinguishing between the elements of the actuarial – essentially the use of historical analogies, albeit in numerically more precise form – versus the critical contributions of imaginative speculation grounded in the rigorous examination of competing qualitative factors in forecasting future events. The call for more creative and strategic intelligence is one driven by the recognition of the need for far more imagination and vision in the analytical process.
As we have said, quants have their place and quite often will have unique value. But intelligence failures are rarely the result of poor numbers. Rather too often we see the inability to construct coherent narratives from overwhelming flows of rapidly changing information, or the limitations on describing the gaps in our knowledge to permit the development of new collection approaches in order to begin to fill them. We see the lack of willingness to challenge conventional wisdom – not as the result of politicization, but out of the lack of desire to do the hard work of defining and defending alternative viewpoints; coupled with the unwillingness to face the potential that while the alternative might be equally wrong, you stand alone in that error (whereas the group’s error allows you to avoid personal blame.)
We would also venture that the most recent examples too often cited as high profile “intelligence failures” owe far less to errors in methodology as they do to a much deeper failing of the community’s processes for sensemaking and explanation. But these are far more difficult to teach and to manage than another numbers crunching apparat – though that is another discussion entirely.
Labels: analytic tradecraft, intelligence failure, teaching intelligence
posted by Kent's Imperative at 21:36
<< Home