Battling Bias
The danger of Selective Insights is not limited to the government sector; it can prove costly in the business world as well. A few years ago, my firm was hired by a large software company to conduct research on a new money-management product their key competitor was rumored to be developing. The client was worried because they believed the competitor was teaming up with another tech giant to create a low-priced, cloud-based version of the product – something that promised to disrupt my client’s entire business. My client was, to be blunt, scared as hell.
To address the research objective, we began collecting intelligence much in the same way we would for any project: we talked to the competitor’s vendors, partners, and contractors, as well as current and former employees including developers, software engineers, and product managers– anyone who would have knowledge about such a program.
During the first few weeks of research we interviewed dozens of people who would have been involved in such a project, if one existed. However, it quickly became apparent that there was in fact no competing product in the works. Sometimes the best threat intelligence is determining there is no threat at all! Figuring we had just saved our client a few sleepless nights, we typed up our report and sent it to the client.
The following day my client and I had a conference call to discuss the report and findings. To my surprise, she was not pleased. At all. Despite the fact that interviews with dozens of internal competitor sources had not yielded any evidence of the competitive threat, the client insisted that we had not been thorough enough; she demanded that we talk to more sources.
Frustrated but determined, we resumed our research – we spent the next two weeks conducting additional interviews, looking for anyone who might be connected to the project. After speaking to more than fifty people, including executives, engineers and project managers, it became clear that not one of them had any knowledge whatsoever about a product that would compete with my client’s. We dutifully updated our report and sent it back to the client.
Unbelievably, she was again disappointed. She insisted we were somehow missing something, despite all the hard work my team had done, despite the thoroughness of the research, despite the fifty plus sources we had spoken with. I finally asked the client why she was so convinced the company was working on a competitive product. I will never forget her response: “I have some data points that say they are.”
“Data?” I asked, “What kind of data?
“Lots of secondary indicators.” she said. “I got it from our business intelligence software.”
What she meant was that her company had an expensive piece of Big Data software that it used to keep an eye on the market. That’s a good thing. There were clues she needed to investigate and that’s why she hired me. But now, after we had done our best to provide her with HUMINT, she was still telling me we were missing the real story.
I’ll admit I almost lost my cool, but I felt the urge to set my client straight. “Well I am not talking about data here, I am talking about intelligence,” I told her. “They are not the same thing. I am talking about actual conversations with the people who write the software. The people who market and plan for it. The people that would certainly know if such a program existed. Data and intelligence are not equal.”
Unfortunately, my client couldn’t be convinced. What I eventually learned was that she was never interested in the “truth,” but rather in advancing her own agenda. As it turned out she was using the threat of a competitor to get additional funding for one of her pet projects, and no amount of contradictory intelligence was going to alter her strategy. Gathering intelligence solely to prove a theory, or support an agenda, is dangerous and can lead to a costly misinformed strategy.
Heath Gross