Message Testing for an AI Solution Provider

A major enterprise software provider sought to refine the marketing message for its AI-powered business solutions. To ensure the message resonated with key decision-makers across IT, HR, marketing, sales, supply chain, and logistics, the company engaged Phronesis Partners to conduct a comprehensive two-phase study—combining qualitative in-depth interviews and quantitative surveys. The research resulted in a sharpened, high-impact message with predictive insights into its effectiveness—empowering the client to launch a compelling marketing campaign with confidence.

Objective

The company had designed a preliminary marketing message to communicate the suite’s core value proposition and detailed benefits to users in functions including IT, HR, marketing, sales, supply chain, and logistics. It wanted feedback from potential recipients to refine the message, and also to test any modifications made as a result of the feedback received.  

The marketing message’s components were:  
  • A value proposition designed to communicate the core offering 
  • A set of three promises that further spelled out the value proposition 
  • A broad user benefit corresponding to each promise 
  • A set of detailed proof points describing specific benefits corresponding to each promise benefit pair 

Solution

Phronesis Partners designed a two-phase study with a qualitative study followed by quantitative research. The qualitative research was expected to provide inputs for fine-tuning the initial copy of the message. Based on the inputs from the qualitative phase, a near-final copy of the message would be developed and tested.  

Qualitative phase 

The qualitative phase of the study comprised 20 qualitative in-depth interviews (IDIs) across six countries — the US, the UK, Ireland, Germany, Italy, and the UAE, conducted via online conferencing on MS Teams. We developed a discussion guide for these interviews with inputs from the client, containing questions, probes and discussion points around all elements of the message framework – the Value Proposition, Promises and Proof Points. These were presented as stimuli and kept neutral in terms of stylistic visual elements like color, as the intention was to receive feedback on the textual content of the elements.

For each message element including the proof points, we asked questions around:
  • Relevance  
  • Clarity
  • What specific key words mean to the respondent and what first comes to mind when they think about that word in the context of Business AI. 
  • What are some alternative words or phrases that come to mind when thinking of specific key words in the message 
  • Appeal
  • How relevance, clarity and appeal could be improved. 
Respondents were also exposed to the entirety of the hero message – Value Proposition, Promises, and Benefits, and were asked questions around: 
  • Immediate emotive reaction to the message – affective response  
  • The respondent’s response as a professional in his line of work – cognitive response 
  • Distinctiveness of the message among other marketing messages around Business AI that the respondent may have received 
  • The respondent’s attitude toward the brand (undisclosed) and solution after receiving this message – behavioural response 
Text analysis was used to analyze the data from the qualitative interviews. Specific points where respondents thought the message elements could be improved were laid out, and common alternative phrasings were listed, along with the reasons for these alternative phrasings. The results, with our recommendations and an executive summary, were presented to the client in PowerPoint. 

Based on the findings from the qualitative phase, the client developed a near-final copy of the message with the same elements – a Value Proposition, Promises, Broad Benefits and Proof Points.  

Quantitative Phase 

The quantitative phase comprised 300 online surveys across the US, UK, and Germany. Respondents were mainly from finance, IT, HR, marketing, and procurement (67% from these functions). A smaller proportion from other functional areas including operations, logistics, supply chain, and sales were also included. The respondents were decision-makers and influencers in purchasing technology and enterprise solutions for their businesses. 

Phronesis Partners developed a quantitative questionnaire to evaluate the receptivity and potential response of the target audience to the message at the overall level and for the specific message elements. For the final message, the client had developed a narrative text that provided background to position the company’s Business AI offering (without naming the client in the quantitative surveys). Respondents were asked to summarize the narrative text’s message (open-ended) and highlight the parts of the narrative text they liked or disliked, using our online text highlighter tool.

We also asked a set of questions designed to evaluate the attractiveness and perceived stickiness of the message, and used a battery of scale statements to evaluate responses to the narrative text on: 

  • Perception of the message itself – affective and cognitive response items 
  • Perception of message effects – behavioral intentions battery (inspires curiosity to know more, would prompt the recipient of the message to visit the company website, prompt the recipient to consider the Business AI solution) 
These quantitative scale questions were also applied to different elements of the broader “hero” message. 

The client provided 27 proof points (reasons-to-believe statements) grouped by the three promise statements. Phronesis Partners then designed and implemented an online MaxDiff experiment to establish a reliable stacked ranking of these statements, which was also grouped by the different promise statements. The data from the online survey was analyzed at the country level and presented as a PowerPoint presentation. 

Results

Our research, and the deep insights gained from the qualitative interviews, helped the client generate ideas for creating its message framework: 

  • The quantitative research provided the data-based validation needed to use the message in the actual marketing campaign.  
  • The message evaluation provided predictive insights into how successful the message is likely to be against a range of common evaluation parameters contributing to the fine-tuning and refining of message elements.  
  • The MaxDiff exercise helped the client prioritize the order of the detailed reasons to believe statements, making the message more engaging and easier to understand. 

Related Solutions
Client Objectives
Related Industries

Connect with
Our Experts

Reach out today to speak with an expert who can provide the guidance you need to navigate your challenges and unlock new opportunities. Let us help you transform data into actionable strategies!

Contact us today

Related Work

View More Case Studies