Call to market researchers:

Move data to the center and surveys to the edge!

Photo by Ivan Bertolazzi from Pexels

Amazon is an amazing company and deserves attention if not admiration on how it creates, shapes, and manages markets -A reason why I consider the annual letter of its founder and CEO, Jeff Bezos, a must read for everyone, be he or she a seasoned professional or a graduate student seeking to learn from one of the most powerful companies in the world.

In his 2018 letter to shareholders, Jeff Bezos wrote that market research doesn’t help, which stung me as I spent many years in this industry, first managing projects then selling projects. Once the surprise passed, I admitted that Jeff Bezos’ statement has triggered a desire to analyze the market research industry. A competitive analysis makes sense in the light of the current digital transformation and business datafication taking place everywhere and in every company.

The current paper is a call to the industry professionals to tell them it is time to think of market research in a different way now that the lines that used to separate research companies, consulting firms, and data agencies are blurring. Nowadays, every data-centered company is a consulting and implementation company. This explains the current frenzy of mergers and acquisitions in the sector of digital and data services.

After explaining that no industry can escape disruptive innovation, the paper describes market research as-is and suggests how it should be. The proposal of change is based on lessons from how strategy consultants solve complex problems and how data agencies leverages client’s legacy systems before considering surveys.

Disruptive innovation

Disruptive innovation rarely sparks within incumbents. They ignore it and once it really starts to eat in their market, they positioned it as a cheaper alternative to their premium solution. By cheaper, they hope clients would interpret it as inferior. A big mistake as clients often give cheaper alternatives a try on small scale. If the alternative delivers on promise, they switch to it full scale. Otherwise, they throw it away with no regret because the lessons learned outweigh the costs incurred.

And once the alternative starts to bite further in the market, incumbents tend to follow the trend but in an awkward way. Instead of totally endorsing the alternative, they are inclined to continue positioning their offering at the center and the alternative at the edge of the solution as if the latter is optional. And they continue doing so without really admitting that a deep transformation is taking place. This creative destruction process may take years but it always ends by the entrants becoming market leaders, the incumbents becoming market followers.

It happened to telecoms, retail, music, media, advertising, and many other sectors. And it’s happening now to the market research industry, where the mainspring is still to survey citizens, consumers, or customers on their intentions, attitudes, preferences, or actions.

Market research

Until recently, market research incumbents did not pay too much attention to consulting firms and data agencies who viewed the client problem in a different way. Instead of selling to the client the need to collect more data (through questionnaire design, survey sampling, and data collection; things that they are not familiar with) they valued the data that the client already has on one hand and the data that is already available, on the other hand, for almost free on the Internet through APIs to the major social networks.

The consequences were dramatic to market research incumbents. Clients now allocate almost all of their marketing budget to upgrade their customer relationship management systems and enterprise resource planning systems, and to a lesser extend to their websites, mobile apps, and social media pages. That is what digital transformation and business datafication is all about. Only a small portion of the budget is allocated to the market research.

In a few years, the market research as we knew it, moved from the center to the edge of the marketing intelligence system of clients in retail, luxury, banking, and almost in all the business-to-consumer markets. The era of data collection as the king is coming to its end, as the focus has shifted from how we collect data to how to learn from data we already have. To take back its crown, the market research needs to change according to this fact.

The traditional process that consists of client brief, questionnaire design, survey sampling, data collection, data statistics, and findings presentation should be revamped to give more importance to the problem formulation and structuring, data audit and mining, problem-solving, and solution selling. The proposal to the client brief should not be anymore “we have a panel to survey for you” or “we will set-up a survey to collect data for you” but it should be “give us access to your CRM data to analyze”, “if need be, we will complement it with more data.

Data in the center, surveys at the edge

In the traditional market research, the process is composed of five steps:

To rejuvenate, market research needs less emphasis on survey data collection and more emphasis on strategic problem-solving as follows:

Trouble: What makes the problem real and present?

Owner: Whose problem is this?

Success criteria: What will success look alike and when?

Constraints: What are the limits of resources and time-line?

Actors: Who has a say in the way the problem will be solved and what do they want?

Before starting anything, consulting firms try to understand the big picture to understand if the top-level goal is a transformation in client experience, internal operations, or business model. They also request to meet all the actors, those who initiated the project as well as those who will be impacted by the outputs of the project:

Wrap-up

For market research firms to win back the center of delivering insights and not being pushed to the edge of collecting data, they need to lower emphasis on data collection and raise emphasis on the larger problem at the origin of the request for proposals. To this end, they should learn from how consulting firms solve problems and sell solutions to the general management and how data agencies master data sourcing from internal databases and external suppliers. In short, now that clients want to leverage and value their data even if it resides in silos, surveys should be considered as the last resort and no more as the primary solution to client problems.

About the author and the paper

Strategy consultant and interim executive for complex technological transformations.

This paper benefits from strategy and implementation work done for top-tier consumer research firms, digital and data agencies, CPG/FMCG retailers, luxury and beauty brands, and a large mutual insurance.

Comments are welcome. Please, either post them on Medium or Linked, or email them to hlaasri at hbmjpartners dot com.

#marketresearch #innovation #strategy #businesstransformation #data

Data strategy consultant, now leading marketing for Sparkling Logic, Inc.