Conducting qualitative research in Asset Management

At Phronesis we pride ourselves on our ability to conduct full-service international research within the Asset Management sector, across both qualitative and quantitative methodologies. What’s interesting, however, is the extent to which the sector relies on quantitative (“quant”) research  as compared to qualitative (“qual”).

I led the Market Research Centre of Excellence for a large US asset manager for several years and I can count on one hand the number of qual projects I ran. This was a shame because good qual research can be incredibly powerful and more impactful than purely quant research. 

I get it, Asset Managers are typically numbers-driven organizations. Stakeholders understand numbers, they are used to seeing trended data and track records, and they understand concepts like statistical robustness. I get it, I really do.

But quantitative surveys tend to go broad and not very deep. Do numbers alone give you the same depth of understanding as a qual survey? 

There are many well-trodden debates about when and why to use qual vs quant surveys. I’m not going to repeat them here. I’m merely going to point out a couple of places where I think qual is undervalued and underused in our sector. 

In my blog Conducting International Market Research in Asset Management I wrote about fund selectors in Europe. In summary, there aren’t very many of them! And yes, they are busy people, but if an in-depth qual study is positioned correctly, and if it sounds interesting and offers them value in return, then they are often very willing to give up their time. The vast majority are highly articulate, intelligent individuals who enjoy talking about their job and the industry as a whole. 

Understanding the project’s objectives, knowing when to use standalone qual 

Let’s take something as simple as understanding content consumption among fund selectors. To gather meaningful insights, to find out how and why they consume content, and what content stands out - and why, a few in-depth discussions are likely to be more insightful than a broad quantitative survey. 

An in-depth interview allows the interviewer to share different pieces of content, to follow up on a new or interesting perspective, and to uncover real-life examples of what content is being read. An interviewer can probe for the “So What?” factor, to identify what actions people take after reading content, which is often the question from asset managers. They want to know if their content is “worth it”. Do they share it with their CIO, do they make a recommendation to their Investment Committee? This type of story-based feedback is gold dust. 

Although you can share stimulus and ask similar questions in quant surveys, it’s harder to control what a respondent reads and absorbs, and it’s harder to build a rapport where the respondent is willing to open up and give you more outcome-based feedback. Additionally, some quant surveys can veer towards reportage. I’ve often sat through presentations where the researcher has told me that “on average fund selectors go to the Financial Times 3-4 times per week” which is ok but not actually very insightful or actionable. 

Using qual data to support quant data 

Building on the theme of using qual to tell a story, I would always recommend including 2-3 open-ended free-form questions in any quant survey. Typically, these would be follow-up “Why?” type questions in response to a particular rating or score. These are invaluable for providing real-life evidence to back up quantitative data and to provide diagnostic feedback. They are also critical for gaining stakeholder buy-in. 

I could recite many examples from my time client-side, but one occasion stands out. I was sharing the results from a client survey with my European ExCo. The quantitative data showed a low score for one of our admin functions. Surprisingly, the ExCo was fairly relaxed about this particular low score, arguing that because it was purely an admin function, it wasn’t that important, and most clients wouldn’t be too bothered. 

However, I followed this up with another slide which simply showed a verbatim quote from a client about this admin function. The client said that they had walked away from awarding us a very significant new mandate because of our poor admin and instead awarded it to our nearest competitor. 

This real-life example about how our poor admin was having an impact on the bottom line changed the mood in the room instantly and the conversation quickly changed to “So how do we fix this?”. 

In summary the debate around qual vs quant will continue to run, and there are often other factors at play, such as budget and timings. 

My advice for client-side insight teams is simply: 

• Treat qualitative feedback as data too!  Just because it isn’t shown as a number doesn’t mean it isn’t “data”. 
  
• Think about creative ways of showing qualitative data. Even simple word clouds can be very impactful, and LLP models are changing the game rapidly. 
  
• Seek to include 2-3 well-crafted open-ended questions in quant surveys. They will provide color and provide evidence to support quant data.
  
• Don’t be afraid to push and test respondents. Open-ended questions should be designed to encourage respondents to go beyond stating the obvious. Comments like “it looks good” or “they are professional” are neither valuable nor actionable. Push for impact and outcomes, e.g. “What did you do with [content] after reading it?”, or “Give me an example of when Firm X went over and beyond?” People in this industry typically enjoy the cut and thrust of a well-constructed thoughtful survey. 

By Stephen Whitten, Practice Lead – Asset and Wealth Management