Posted By Randi Stillman,
Thursday, June 6, 2019
Updated: Monday, June 3, 2019
Annual Conference Reporter on the Scene: Many Ways to Tell a Story: Exploring Different Approaches to Displaying Data
At the 2019 QRCA Annual Conference, Caroline Volpe, Compass Market Research LLC and Maria Virobik, ResearchScribe, presented on the “Many Ways to Tell a Story: Exploring Different Approaches to Displaying Data”. Through their hands-on presentation, the presenting team shared inspiration, ideas, and tools to communicate research data in text-based reports by the creative use of visual elements to tell a story with all in attendance.
QRCs can apply a few simple guidelines and have access to free or low-cost online tools if they want to transform the bulleted text of a traditional PPT report into a visually compelling story. The presenters recommend creating a desktop inspiration folder to save icons, vector graphics (for infographics), images, and report formats, such as the methodology section for depicting the who, what, when, where, and how of your project. Don't forget to use built-in visual design elements, such as size, color, spacing, etc. for clarity, direction, and importance when you put your report together to tell a story. Some of the resources mentioned include: The Noun Project, Flat Icon, Free Pick, CanStockPhoto, 123RF, Slideshop, SmartArt, and Inkscape.
Putting it into practice:
I will continue to think visually as a storyteller when doing my own reports and I'll explore some of the resources mentioned for inspiration and new ideas.
For best results, allow yourself some time to explore and be playful when getting ideas and searching images that will help you tell a compelling story.
Posted By Vidhika Bansal,
Thursday, May 30, 2019
Updated: Monday, May 20, 2019
Annual Conference Reporter on the Scene: Opening Closed Doors with Role Play
Although immersive ethnographic research is the gold standard for gathering real-world insights, there are situations when financial, regulatory, logistical, privacy, and ethical constraints make such contextual research extremely challenging, if not altogether unfeasible. Luckily, as Elizabeth George shared with attendees at the 2019 QRCA Annual Conference, role play is a fantastic alternative research method that can allow us to learn a lot about the nuanced conversations that occur “behind closed doors”.
By pairing a respondent immersed in a scenario with an actor who engages with the respondent to simulate a real-world situation, we can begin to better understand the natural language used and interactions characteristic of dialogue around a given product or topic. We can also gain deeper insight into the dynamics of complex decision-making, such as how potential customers may react to two different sales pitches. These insights are especially helpful in developing messaging strategy, educational content that addresses gaps in awareness, as well as gauging receptiveness to specific ideas or approaches.
When to use it: Role play research can be especially helpful when attempting to glean nuanced information from stakeholders in industries that are highly-regulated, prohibitively-costly, or ethically-challenging.
The process: There are several steps to the process of conducting role play research, namely: identifying the research objectives and scenario of interest, determining appropriate roles and recruiting actors for them, creating profiles and guidance for your actors and respondents, “setting the scene” for respondents and observing interactions play out, and finally debriefing both parties.
Valuable outcomes: Content and messaging strategy development is often a key outcome of such research, along with a better understanding of receptiveness (or lack thereof) to certain products or approaches.
Putting it into practice:
The next time I work with a client in the financial or healthcare space or am struggling to get “behind a closed door” of any sort for immersive research, I will keep this invaluable tool in my back pocket as a high-quality alternative approach.
Especially in salesperson-prospect and physician-patient scenarios, respondents often are able to act relatively natural despite being in a simulated setting with an actor because they tend to be accustomed to using role play when they are first being trained.
Using the physical space to your advantage, by walking the respondent over to the actor (and conducting setup and debriefs in a different space), can help limit the influence of the environment being simulated vs. real for the respondent.
Role play research can be a fantastic way to validate personas that you have already created, especially if they were proto-personas largely based on assumptions.
Liz was an absolutely fantastic presenter—engaging, knowledgeable, and able to explain her process in digestible and relatable terms—and this research method is a great one to add to our expanding toolkit.
Posted By Jeff Walkowski,
Tuesday, May 28, 2019
Updated: Friday, May 24, 2019
How To Create Effective Screeners
Whether you’re experienced or just breaking into qualitative research, it never hurts to review what makes a screener effective in finding just the right people for a research project. It is a questionnaire that recruiters will use to find qualified participants for the study. It is called a “screener” because it is like panning for gold—we have to sift through many people to find the nuggets (qualified people) to be invited to participate. Screeners are used by telephone recruiters, or they may be online surveys as a way to automate the recruitment process. Automation helps reduce expense by eliminating the human effort of dialing phones and talking to potential participants. Keep in mind that automated screeners still have costs associated with them – most notably programming costs which may include quota control, skip patterns, and conditional questions (all of which are typical of any online survey).
All the rules/guidelines about questionnaire construction apply to qualitative research screeners. The most effective screeners have the following characteristics:
They Are Short
If a screener is too long, participants may hang up the phone with a recruiter or simply decide to discontinue completing an online survey. Ideally, screeners have no more than 10-15 questions, or they take no longer than 5 minutes to administer (online or offline).
They Are Clear about the Purpose at the Beginning
Tell participants that it is not a sales call. Explain that we are looking for people to participate in a market research interview, but we must spend some time asking some questions to determine if they qualify.
They Do Not Provide Hints that Encourage Cheating
They include an intentionally general description of the nature of the research so as to not tip off participants to answer a particular way so that they can be invited. For example, say, “We are putting together a focus group on beverages,” instead of “We are putting together a focus group to determine what consumers think of Starbucks.”
They Include Questions Up Front that Are Easy to Answer and that Quickly Eliminate People Without Taking too Much Time
For example, if we are looking for millennial females, we will first ask about gender and age so that non-millennial males are quickly excused.
They Include Need-to-Know Questions – Not Nice-to-Know Questions
Asking nice-to-know questions lengthens the screener, can be frustrating to potential participants going through the screening process, and makes the recruitment process less efficient and possibly more expensive. Keeping the focus on questions that help determine whether a person should be invited or not is best.
They Include Intriguing Questions
Interesting questions keep survey-takers engaged. The objective is to not lose them along the way due to boredom.
They Feature Mostly Closed-End Questions
Again, this is designed to help the prospective recruit move through the process as quickly as possible. Closed-end responses also make the task easier for the recruiter (no judgment required).
They Often Include One or More of the Following Question Types
Product/service category use
If they are not users of a particular product or service, they are unlikely to be useful.
Brand(s) used more often and/or brands they would never use
If the project is about a particular brand, we probably do not want individuals who reject the brand outright (unless, of course, the purpose is to attract those who reject the brand).
Past participation in market research surveys, focus groups, and interviews
Preference is given to those who are not considered “professional” participants, so that they approach the research experience with a fresh attitude.
Employment in certain industries
We typically do not want those who are employed in advertising, public relations, or market research. In addition, we tend to rule out those who are employed in the industry that the project is about, because they may “know too much” and not represent the typical customer for the product/service.
They May Include an “Articulation” Question
Such open-end questions are used to help ensure that a participant will be able to make a meaningful contribution to the discussion. Sometimes questions are asked that pose a creativity challenge to the potential participant (e.g., “List 10 ways in which rubber bands might be used”). Ideally, however, a question that is related to the product category will be more relevant (e.g., in a study of high-end golfing equipment, potential participants might be asked to demonstrate some core knowledge of current equipment). In markets where participants may have differing levels of proficiency with the language to be used in the group (e.g., English), the recruiter may be asked to judge the ability of the potential participant to be clearly understood. This serves as an additional articulation assessment.
Jeff Walkowski is the principal of QualCore.com Inc., a consulting firm providing traditional and online qualitative research services to a wide range of industries including health care, financial services, automotive, and information services. He was schooled as a quantitative specialist and entered the industry in the 1980s as a statistician. He later discovered his talents as a moderator and evolved into a qualitative specialist by the mid-1990s.
Posted By Graciela Braniff,
Thursday, May 23, 2019
Updated: Monday, May 13, 2019
Annual Conference Reporter on the Scene: The Neuroscience of Memorable Messages
Dr. Carmen Simon presented during the 2019 QRCA Annual Conference on strategies for transforming yourself and your message into something worth noticing and remembering. According to Dr. Simon, one of the biggest problems with business content is that audiences forget 90% of what you share after two days. To help us understand how people pay attention, remember content, and ultimately act on it, we need to look at the field of neuroscience, which reveals insights on how the brain processes information and tends to remember it – or, more often – forget it. Throughout her session, Dr. Simon taught us how to convert neuroscience insights into practical guidelines you can use to craft content with lasting impact. This is critical because both your internal and external audiences make decisions in your favor based on what they remember, not on what they forget.
This was the most interesting presentation regarding neuroscience that I have heard in a long time. Instead of a explaining how neuroscience can help to analyze concepts and advertising and discover how emotionally interesting or engaging they are (which from what I have read is not possible) Dr. Simon explained what can help make our presentations and our reports more memorable in our spectators’ minds.
One remembers only maybe 10% of what is presented. So, it is extremely important that there is one main idea that unifies the whole presentation. It must be clear, and it has to convey a reward to the spectator. This idea must engage on mental models (existing associations) that the spectator knows because this helps them perceive it rapidly. In the presentation there has to be a balance between what is new and what is a mental model, so twist the familiar with what is new to consolidate memorability. These ideas can also be applied to the messages that our clients want to convey
Putting it into practice:
After hearing Dr. Simon’s presentation, I took the opportunity to renew my company's presentation and reports I had done to help them be more memorable!
There was a moment during the presentation that I realized how much something Dr. Simon presented resonated: “An idea that moves and motivates me thoroughly, will also move and motivate others.” It seems simple, but it really stuck with me. I even used the presentation information to redo two important upcoming presentations based on this information.
Posted By Peggy Moulton-Abbott,
Thursday, May 16, 2019
Updated: Monday, May 13, 2019
Annual Conference Reporter on the Scene: Better AND Faster? It can be Done!
In the dynamic, fast-paced environment in which we all operate, innovation is crucial. During the 2019 QRCA Annual Conference, Allison Rak of Vatoca Partners presented practical methods to take our businesses and offerings to the next level. Allison gave us Quallies so many methods to lighten our work loads and speed up our outputs including a method for report writing that will literally cut days off of your turnaround time, while taking your deliverables up a notch!
Follow these steps:
List everything you do in the course of your work.
Identify tasks you love.
Identify tasks ONLY you can do.
Find someone to whom you can outsource everything else.
Build the costs into your overall budget and do not break it out to the client.
There are many great, efficient and inexpensive resources are available to help us with our work, we need to tap into these resources!
Putting it into practice:
It’s a great self-examination exercise to identify everything we’re doing and concentrate on outsourcing the time/labor intensive tasks we could pay someone else to do so we can concentrate on what we love to do.
There are so many sites and apps that can provide qualified people to do our drudge work and even assist our clients with theirs. Everyone should look and listen to this sage advice!
Posted By Maria Virobik,
Tuesday, May 14, 2019
Updated: Monday, May 13, 2019
Data Visualization: 3 Ways to Make Your Qualitative Reports Pop
What Can Data Visualization Do for Us?
Data visualization—the graphical representation of information and data—can be a powerful tool in qualitative reporting. While we certainly can’t completely escape text-centric pages in our qualitative reports, graphics add visual interest and help break up the monotony of pages (or slides) of text. Done well, graphics help support qualitative findings and enable us to communicate in more interesting ways beyond words on paper (or a screen). Effective data visualization can also help readers understand concepts more quickly and easily and make information more memorable.
All the Cool Kids Are Doing it
Newspapers and other media outlets have jumped on board the data visualization bandwagon. Publications like The Washington Post, The New York Times and the Los Angeles Times employ full-time data journalists to augment their reporting. These folks take an enormous trove of data on a particular topic—for instance, the earlier start of spring in some parts of the U.S. or the confirmed U.S. measles cases by county in 2019 —and expertly slice, dice and manipulate the information into interactive graphics that communicate big ideas in an accessible and elegant way.
Data Visualization and Qual: Not a Linear Journey
Visualizing quantitative data is relatively easy. Hard numbers and percentages naturally lend themselves to visual representation. Charts, graphs and their modern equivalent—infographics—are easy to create from quant data.
Qualitative data can be harder to visualize; transforming qual data into graphics isn't as straightforward or simple. A search for “infographics and qualitative data” reveals that some people even argue that qual data can’t be turned into infographics. Take heart, however. An equal number argue that it can and provide examples to back this assertion.
But it’s not a linear journey from qualitative data to data visualization. Many of us have heard from end clients who want hard numbers or percentages included in a final report to quantify how various concepts or ideas stacked up against each other. We can explain that “qual isn't quant” until the cows come home, but clients persist in making such requests.
Instead of giving in to these requests (or refusing them outright), there is another option. We can take this as the opportunity to develop data visualization approaches that give our clients the detail they want and expect without compromising the qualitative nature of the report. A few examples follow.
Word Clouds – an Oldie but Goodie
Word clouds are a common data visualization technique in qualitative reports. Using font size (and often color), they convey magnitude of various responses, thoughts or ideas. Larger words=more popular/frequent/common. This approach works well because it’s a way to provide granular detail without showing the actual numbers behind the information.
While word clouds aren't the answer for every situation, they are a great tool, and websites for creating them abound. The PollEverywhere blog lists nine favorite word cloud generators, including Wordle and Tagxedo. A Google search for “word cloud generator” will point you to others.
Customer Journey Maps:Timelines in Disguise
Customer journey maps are another way to employ data visualization in qualitative reports. These maps are essentially timelines; a quick Google search on this term turns up many great examples that can be easily adapted to fit your particular purpose.
Here’s one example: a timeline detailing milestones in the 21st Century Conservation Service Corps history from 2010 to 2014.
The example above is organized by year, but the general format can be adapted to visualize a customer journey. Year markers become phases in the purchase journey: research, comparison, selection, purchase. The linear format allows room above and below the line for details on the individual steps consumers undertake in each phase.
Venngage is one great resource for infographic templates and tools, including many for timelines (such as the one below). They offer a couple different subscription plans. But you can peruse the templates for free, and that might be all the inspiration you need to create your own.
Bubble graphs are another idea we can borrow from data journalism. During the 2012 London Olympics, The New York Times kept a running medal count by country and visualized the data in a simple table (below). The information is clear, but the table doesn't do a great job conveying the magnitude of differences among countries.
The Times formatted the same information into a bubble graph. This approach does a much better job conveying magnitude. You can easily identify the countries that led the medal counts. Readers could hover over any circle for more detailed information, including a country’s medal count by type (gold/silver/bronze). (Visit the link below the graphic and try it for yourself!)
The same idea—sans numbers, of course—could be employed in qualitative reporting. For example, we could use a bubble graph to report the characteristics that participants want in a dog.
Readers can immediately see which characteristics were most important and which were mentioned by fewer participants. By keeping numbers out of it, the graphic remains faithful to the spirit of qualitative research.
Sky’s the Limit
These are just a few examples of how data visualization techniques can be employed to make qualitative reports more engaging and communicate findings and implications more effectively.
Here are several links to more examples; many additional resources can be found by searching data visualization:
What are your go-to data visualization techniques and tools? What works? What doesn't? If you have advice or a favorite resource to share, please leave a comment.
Maria Virobik joined QRCA in 2018 but has worked in qualitative research since 1997. After early dalliances in the advertising world, she came to her senses and has been devoted to qualitative analysis and reporting ever since. Originally from Southern California, she and her husband sold their house last year and now live a nomadic lifestyle with their two marginally obedient dogs, Lucy and Ginger Snap.
Annual Conference Reporter on the Scene: What Qual Can Learn from Coaching
In this session at the 2019 QRCA Annual Conference, Jay Zaltzman of Bureau West talked about his experience and training as a coach to demonstrate how coaching can provide a new perspective for qualitative work.
Coaching has gained in popularity in recent years. Coaches used to only be hired by athletes and other performers who wanted to go beyond average to excellent, but many have realized they don’t have to be performers to benefit from coaching and become more effective. Qualitative researchers can use coaching principles to stand out and provide even more value to clients.
As a qualitative practitioner, you already have skills that the coaching profession values, according to Zaltzman. You're comfortable speaking, you can listen without judgment and you know not to interrupt or impose your own point of view. Coaches are very good at helping clients look at issues in their own lives through different perspectives, and if quallies can bring multiple perspectives to their work, they will provide a lot of value in a lot less time.
Putting it into practice:
Having worked with a coach previously, I know that many coaches do initial exercises with their clients to help them clarify their values. Jay recommended starting with clarifying values to help inform various parts of qualitative research, including creating a discussion guide and further probing the values of the brand using a similar approach.
The best coaches know how to ask great questions. Jay introduced us to what he calls the miracle question during his session:
Imagine you go to bed tonight and sometime in the middle of the night, while you are sleeping, a miracle happens and the challenge that we are discussing is resolved. When you wake up in the morning, what’s the first small sign you’d see that would make you think, “Well, something must have happened” the problem is gone! When was the last time you saw a little bit of the miracle, even just for a short time?
Not only is this a useful question personally, but it's a question that delves into the heart of Zaltzman's belief that taking on a coaching attitude and asking better questions can help you go deeper and gain more insights for your clients.
If you're interested in more resources on coaching, Jay recommends checking out Co-Active Coaching textbooks or the work of Steve Chandler.
Posted By Kayte Hamilton,
Thursday, May 2, 2019
Updated: Wednesday, May 1, 2019
Annual Conference Reporter on the Scene: : UX Live! Revitalizing the Customer Experience
At the 2019 QRCA Annual Conference Shaili Bhatt and Nancy Baum, both from C+R Research, gave all of us quallies a live, interactive demo of mobile usability testing. Through small group work on writing effective mobile User Experience (UX) questions and a Q/A session that had the room buzzing, we gained helpful practices to help us execute digital user experience sessions.
I was thrilled to learn there are several easy-to-use applications that can be used to conduct digital usability testing on the market. While the available platforms range in pricing and features, many of them combine live video recordings, task based assignments, and real time updates that QRC’s can utilize to conduct more usability testing sessions nationwide while working remotely.
Putting it into practice:
I am excited to utilize the tools we tested in session to conduct remote usability testing sessions!
Being able to see the live video with screen animation was a revelation. It really does replicate what you see in person, but allows you to be in more than one place at a time as a moderator
The presented platforms and hands on application in session was extremely useful for many of us in the room to understand the impact these platforms could have on our work. The question and answer session was lively, with many questions that sparked insightful conversations, it’s clear these tools are going to make a big impact on how many of us work!
Posted By Lisa Horwich,
Tuesday, April 30, 2019
Updated: Tuesday, April 30, 2019
Why Quallies Should Care about Marketing Technology (MarTech)
The “Rise of the Machines” and how We Got Here.
When I graduated from business school back in the late ‘90s I never dreamed I would become a total tech geek…in fact, I really thought I was going to be a high-powered consultant (think McKinsey, Bain, BCG). Instead, somehow, I found myself implementing large-scale computer systems (fears of Y2K!) and then became a product manager for a small software company. My journey to tech geekdom had begun without me knowing it.
Fast forward to today. After spending much of my time working on quantitative and qualitative research for large tech companies, I can honestly say that I really love learning and studying technology.
With this in mind, about 2 years ago, a prediction from Gartner (the big technology industry research firm) caught my eye – their analyst Laura McLellan predicted that by 2017 CMOs will spend more on technology than CIOs. She was almost correct – it happened in 2016, a year ahead of schedule.
Think about it. Marketing departments are now spending more on information technology than the department that is responsible for a company’s technology infrastructure. Crazy, I know!
This has led to a proliferation of companies clamoring for a piece of this MarTech pie. From 2011 when 150 companies offered MarTech solutions, we are now in 2019 looking at over 7,000 companies competing in this space.
What is the aim of all these solutions? More importantly, what has changed with CMOs to prompt this massive investment in technology? It boils down to three main factors:
Most CMOs now share P&L responsibility. Instead of just being a “cost center,” marketing is looked on as fundamental part of revenue generation.
Marketing funds and designs the entire cross-functional customer experience (CX). If you think of CX holistically from generating awareness through post-sales feedback, it makes sense that marketing is in charge.
Finally – and arguably most importantly – with the soaring costs involved in attracting, maintaining, and growing the customer base, marketing now has to justify the ROI of their activities.
CMOs are turning to data-driven solutions that help them deeply understand every phase of the customer journey – tracking and quantifying the ROI of all marketing activities along this journey. They are also investing heavily into solutions that personalize the customer’s experience with the hope of converting these interactions into greater sales opportunities.
Technology Solutions and Their Uses
As researchers, we need to know the types of technologies where our clients are spending significant portions of their overall budgets (~30%) so we can recognize where we fit as human insight professionals. We don’t have to be experts in tech, just conversant — so when we walk in the door and our clients say they are using a new “Artificial Intelligence email optimization tool,” we understand what that is and can talk about how our services complement and augment this tool.
I’ve put together a few charts and tables outlining some of the fundamental building blocks of these solutions. Most MarTech offerings are powered by technologies such as Artificial Intelligence, Machine Learning, Business Intelligence, and Real-Time Analytics. I find it useful to see the interaction of these technologies with a chart:
To understand definitions of these technologies and common uses, this table is a quick reference (CAUTION: Tech speak ahead):
Unified customer data platforms, predictive analytics, and contextual customer journey interactions.
Any system that learns from past data to make judgments about previously unseen new data.
Optimize ad campaigns and other metrics, predict churn.
Opportunities for Quallies
Many of the technologies outlined above have inherent limitations – which I like to think of as “opportunities” for qualitative researchers. Most of the limitations center around the data – quality (how good is your data) and quantity (do you have enough of the right type of data). In addition, the other major limitation is having enough marketing content – a major bottleneck in the quest for personalized customer engagement.
Decisions are made solely on data – past and present.
Use the data as a launching point for deeper qualitative analysis.
Existing data is not predictive enough for decision-making.
Create and maintain communities focused on pinpointing predictive behavior.
Need exponentially more messaging content for personalization.
Assist in narrowing target messaging by identifying key characteristics valued by customers.
Insufficient data to train the machine/AI.
Provide personas and other descriptive metrics to help “train” algorithms.
Lack of “industry specific” attributes.
Create detailed feature lists to describe the unique features inherent to that industry.
While the ideas above are great tactical opportunities, strategically, our most important job as qualitative researchers to remind our clients how, in a world of automation, humanizing the experience of individual customers is key to authenticity.
Lisa Horwich is the founder of Pallas Research Associates, a B2B-focused research and consulting firm located in Seattle, WA. She is a self-ascribed tech geek and loves talking to developers, IT decision-makers, and CIOs. She also co-chairs the QRCA B2B SIG.
Posted By Peggy Moulton-Abbott,
Thursday, April 25, 2019
Updated: Tuesday, April 23, 2019
Annual Conference Reporter on the Scene: 20 lbs. of Potatoes in a 10 lb. bag; Managing Client Expectations
Kate Wagenlander Watson of KCW Global Research, LLC gave a super practical presentation at the 2019 QRCA Annual Conference that provided strategies and hacks to manage overly-ambitious client demands. Throughout her presentation, she covered tactics and tips to help all QRC’s become savvy consultants at all points in the process from the first client call to the creation of the moderator guide and finally the actual execution of the groups for application across a wide variety of research objectives.
Kate takes a realistic, but lighthearted approach to scenarios we can all relate to when clients continually over-stuff the study sack with more content than it can logically hold. With years of experience and an indefatigable spirit, Kate advises us to:
Set the Foundation so clients' expectations are aligned with reality. Do this starting with the proposal. Protect both parties' interests by clearly stating how much of the client's goals/objectives can be accomplished within the methodology, time frame, and budget they're offering. Offer alternatives, set boundaries and enforce them contractually (in writing). And don't be afraid to say no and save yourself if it becomes obvious that no reasonable detente can be achieved.
Manage the Stimuli and Guide Early and Often – continuously reinforce what is realistic within the amount of time available. Kate provided strategies for shrinking/refining concepts, both in size and number. She also demonstrated how to illustrate the time crunch in quantitative terms clients can understand, along with many other clever and insightful methods to manage scope-creep and over-stuffed guides.
Controlling Potential Fieldwork Issues – before, during, and as groups are ending.
Offering the "nuclear option" – Kate demonstrates how just "giving clients what they ask for" can be the magic bullet to making them truly understand what they're demanding is unreasonable and unrealistic.
Putting it into practice:
I want to tattoo this presentation on the inside of our eyelids and implement it EVERY time we bid, field, and report a project!
Make your clients participate in a mock-session BEFORE fielding so they will see for themselves exactly how much content fits in a given time period. This way they really comprehend the limitations of the "time-space continuum"!
This presentation should be required-reading and standard practice for all QRC's, as well as clients. A first-timer from the client-side told this reporter that Kate's presentation was the most eye-opening of the entire conference!