Posted By LaiYee Ho,
Tuesday, March 24, 2020
Remote Research in the Time of Coronavirus (COVID-19)
With offices mandating employees to work from home, and people across the world hunkering down, researchers everywhere are scrambling to figure out how to make it all work from their home offices.
I’ve been doing remote research out of my home for years, so I thought I’d share a few tips and tricks from my own remote-research arsenal!
In-office conversations with colleagues: Try Slack
For all conversations, whether it’s discussions about business strategy, sharing ideas on research plans, or sharing random funny articles online, we use Slack. Slack enables us to have many fluid conversations at once, and to organize conversations by topic (such as #customer-discussions, #finance-accounting, #strategy).
Communication with participant recruits: Try Intercom
Rather than using email or a group email, consider using a CRM to communicate with research participants. I have used Intercom as a CRM and have found it incredibly helpful as a way to keep track who we reached out to for which study, and to keep a log of all the communication we’ve had so far with each recruit. It allows my team members to see the previous conversations and take over for me if necessary (something that’s nearly impossible in email). If Intercom doesn’t work for your particular client, consider checking out other group CRMs to solve the same problem.
Scheduling interviews: Try Calendly
Calendly is an incredible streamlined way to have participants schedule time for studies. It syncs directly with your calendar (for us, it’s Google calendar), and you can set parameters for when someone can schedule time. Just send them a link and they can instantly book a time with you.
In-depth interviews: Try Zoom
Zoom is an online video chat service with great video quality that is super easy to use. It can also support video chats with large groups of people if you’re running a study with more than one participant. You can also record the session (make sure to always ask for consent first!)
Compensating participants: Send digital gift cards
Compensate participants by sending digital Amazon gift cards. These will get emailed straight to them. As an additional tip, if you have international recruits in other countries that want a gift card for their local Amazon branch, purchase the gift card from that country’s Amazon site. (For instance, go to https://www.amazon.co.uk/ to send someone an Amazon UK gift card. Amazon gift cards aren’t transferable between countries once purchased.
Writing documents: Try Google Docs
If you were using Microsoft Word and emailing them around, or printing them out for colleagues to review, it’s time to switch to Google Docs. Google Docs is online and fully collaborative. Colleagues can comment directly in your document and collaboratively write with you in real time.
Transcript analysis and coding: Try DelveTool
If you were printing out transcripts and highlighting them using Post-it notes or using desktop based tools like NVivo or ATLAS.ti without online capabilities, consider switching to DelveTool. (Full disclosure, I’m the co-founder so I designed and created this tool out of my own pain points). Now that your research team is working remotely from their homes, DelveTool offers a way for your team to code and analyze a single project together from wherever they are.
Creating presentation decks: Try Paste by WeTransfer
If you were using Microsoft PowerPoint, it’s time to consider using Paste. It’s online, fully collaborative, and makes your decks absolutely beautiful with very little effort. It doesn’t have all the power features of PowerPoint, but that’s precisely the benefit. You can create gorgeous slides with just a quote on a single page or drop in a video clip from a research session. You’ll spend significantly less time making the deck and it will look 1000% better than a standard PowerPoint.
Streamlining and automating your process: Try Zapier
Zapier takes a bit of tech tinkering, but is a great way to automate any repetitive, manual tasks that you’re already doing. For example, if you’re keeping track of participant recruitment status in a Google Sheet, you can use Zapier to automatically update that spreadsheet when participants schedule an interview using Calendly.
Best of luck setting up your remote research workspace. If you have any questions or have recommendations that you want to share, please reach out to me!
LaiYee Ho is the co-founder of DelveTool, where she pours her years of experience as a UX researcher and designer into creating tools for researchers. Before beginning her entrepreneurial journey, she was one of the first UX designers of the Amazon Fire TV, where she learned about the importance of simplicity in design. She then went on to build the first UX research team at a smart home automation startup, where she learned how to uncover human motivations. She has a degree in Information Science from Cornell and lives in New York City.
Market Research Technology
Remote Market Research
| Comments (6)
Posted By Ted Kendall,
Tuesday, February 11, 2020
Updated: Tuesday, February 11, 2020
Adapting Your Listening Skills to the Online World
By: Ted Kendall
As a successful qually, you intuitively know the importance of listening, how to listen well, and how to show participants that you are listening.
Listening is important because it engenders trust, creates rapport, and opens participants up.
In a physical setting, the key things we do to listen, and to show we are listening, include:
- Asking questions in response to participant’s thoughts
- Using verbal and non-verbal cues to show how you are listening
- Letting participants complete their own sentences
- Maintaining eye contact
- Acknowledging comments in specific ways like boarding or post-it notes
You will have noticed that most involve physicality—you have to be there in real life.
So, how do you listen, and just as importantly, show you are listening, in online qual?
Before we get into this, let me clarify that when I am talking about online qual in this context, I am referring to text-based online qual—primarily bulletin board style. While webcam interviews may be considered online, real life listening skills can be applied to the medium fairly easily.
Set Expectations to Counter Online Research Misperceptions
A unique challenge with online qual is that participants don’t necessarily know the difference between a survey and a qualitative discussion, so they often treat the study as if it were a survey. And they often believe that any interactions will be with a chatbot, not a real person.
It’s critical to counter these widely held beliefs and set the appropriate expectations up front. Tell participants you are listening to what they will say. And let them know it’s not a survey—it’s a conversation.
I can sometimes be pretty blunt about this—even going so far as to tell participants that if they just speed through the answers to my questions, they will not get the incentive. And then, when someone does that, I follow through on the promise and call them on it. Often it changes their interactions. Sometimes it doesn’t. But they definitely know you are listening. And, if the discussion is open to the whole group, others will see that you are listening as well.
Depending on the platform, you can use the messaging tools as well as the landing pages to accomplish this. And if the tools aren’t there, just use email or text, even phone, outside of the application.
I also make it a habit to reply to every participant post in the introductions—much like I do in a traditional focus group setting, or for that matter, in a conversation with a stranger. These replies can often reflect common ground, ” I love spending the day in the mountains with my dog too. What kind of dog do you have?” That’s not a question that will provide rich insights, but it will help open up the participant and really shows you are listening.
It’s critical to establish early in the conversation that you are a living, breathing, listening human being—not some chatbot or AI ghost in the machine. This has a huge impact on how participants approach your conversation.
Avoid AI Tools
Several online platform providers are touting AI generated responses to participants. All I can say is that this is what we get when we let the programmers drive development. Avoid this feature. Yes, it saves you time during the discussion. But it also removes you from the conversation—you are no longer actively listening. You wouldn’t let a robot take over your focus group session just to save time, would you?
Also, AI is not yet perfect. And it needs to be in this case. It’s not a life or death situation, unless you consider the life or death of the research conversation. Even if the AI gets 90% of the interactions correct, there is that 10% that will suck the air right out of your conversation with that participant. If you are using a group setting, other participants will see the mistake and the negative impact becomes exponential.
So just don’t do it. The potential losses greatly outweigh the potential time savings. Besides, actually responding manually forces you to listen and learn—which is what this is all about. Don’t let a robot take your job.
How to Digitally Use “Non-verbal” Cues and Maintain Eye Contact
In the online, text-based world, you certainly can’t maintain eye contact, nor can you provide non-verbal cues to show you are listening. So how do you employ those key principles of listening in an online, text-based world?
Probably the most obvious way is replying to participants’ posts with questions to better understand what they have said or get some clarification on their comment. Yes, I am talking about the same probing questions we lay on participants in focus groups and interviews. These probing questions work just as well online as they do in real life.
To replace those non-verbal cues, I have found it quite effective to comment or ask questions even when there is no need to do so. The idea is that by just saying something, participants recognize that you are there and you are reading what they are posting—you are listening.
Sometimes it is easy to just copy and paste the same general comment to several participants when you do this. If the participants can’t see one another, this is fine and saves you time. But if the participants can see each other, then it just makes you look like a robot.
It’s important when making comments just to show yourself to not require a reply—often this is an option. I like to just thank people for providing quality detail or thank them for an interesting take on the topic. The important thing is to personalize it a bit, to keep it from sounding generic.
Another way to show you are listening is to use the messaging app within the platform to hold meta conversations outside the actual discussion. I make it a point to send reminders at set times as well as thank-yous at the end of the day of discussion.
These messages don’t have to be just logistical in nature. You can also use them to show you are listening. Sometimes I will include a comment about some of the discussion—an insight that came through for the whole group of participants, or sometimes personalizing it to a specific participant.
In the end, listening is important to successful qual, whether you are in the same room as the participant or interacting digitally. It’s just how you listen, and how you show that you are listening, that can take a little adjustment in the digital qual world. But it’s no less important and no less doable.
Ted Kendall is the founder of TripleScoop, a boutique research agency that has a focus on online qualitative. Ted got to this place in his career by being in the right place at the right time to pioneer in early online methods. He was a co-founder of QualTalk that became 20/20 Research’s QualBoards. He learned how to moderate online qual through trial and error and has moderated hundreds of online qual discussions and interviews since that first one in 1997. And he is usually a good listener.
| Comments (0)
Posted By Jennifer Dale,
Tuesday, September 17, 2019
Qual or Quant? Choosing the best method for your research study
Quantitative and qualitative research are both scientific methods for data collection and analysis. They can be applied alone, or in combination, to maximize insights.
The Basic Difference: Going Beyond What vs. Why
Quantitative research relies on large sample sizes to collect numerical data that can be mathematically analyzed for statistically significantfindings. Surveys are structured, questions are typically closed-ended, and answer choices are fixed. However, quantitative research may also include a limited number of short-answer open-ended questions to help clarify why people responded the way they did to a closed-ended question. Eye tracking, facial coding, and even Big Data fall under the umbrella of quantitative research, with computers analyzing enormous volumes of data incredibly fast.
Quantitative studies produce numerical data, which allows for statistical analysis and ultimately precise findings. The US Census is a great example of a quantitative research study – fixed and close-ended questions, an enormous sample size, a collective review of many respondents, and measured population segments.
In contrast, qualitative research seeks to understand the reasons behind the numbers, as well as what is not yet known. Sample sizes are smaller, questions are unstructured, and results more subjective. Unlike quantitative research, qualitative studies insert the researcher into the data collection process. The researcher probes responses and participants provide more detail. Qualitative data is collected through interviews, group discussions, diaries, personal observations, and a variety of other creative and ever-expanding means.
Qual studies work with textual and visual data, interpreted and analyzed for directional findings. Qualitative research studies include fluid and open-ended questions, a smaller sample size, an in-depth review of each respondent, and emerging themes.
I like to think of the difference visually, where a quant study collects specific data from a large number of people, and a qual study goes deeper to collect greater insights from a small number of people.
How to Choose
The answer to whether you proceed with quantitative or qualitative research lies in your research objective and available resources.
- Why you’re doing the research
- What you need to know
- Your budget, staff, + schedule
- How the findings will be used
Consider these possible scenarios the next time you’re stuck and don’t know which way to go:
Quant + qual can come together in other ways. A questionnaire with open-ended questions, while ultimately coded numerically, can offer a window into the unknown. Focus groups that also include poll questions or surveys can produce hard data when analyzed in total, even if the results are not statistically significant.
With good planning, quantitative and qualitative research come together like a dance, guiding the marketer’s success with every step.
I Say Hybrid, You Say Multimethod
Combining quantitative and qualitative research approaches is an ancient strategy, but the names continue to change with the times. I did a bit of research and found the following terms being used to describe that ideal combination of quantitative and qualitative research. What term do you use? And why? ;)
Jennifer Dale, President + CEO Inside Heads, is a seasoned marketing professional and pioneer in online market research. Her passion for marketing, human behavior, and technology keep InsideHeads on the short list of research providers for some of the world’s most discriminating clients. Jennifer is co-author of Qual-Online, The Essential Guide and has published a number of articles in VIEWS, Alert! and Quirk’s Marketing Research Review.
| Comments (3)
Posted By Isabel Aneyba,
Tuesday, June 25, 2019
Updated: Monday, June 24, 2019
Let’s Work Together: The Consumer Co-Creation Camp
While focus groups have long been a part of the innovation process, many clients have voiced their frustration about the limitations of traditional focus groups. To respond to this and other client needs, we created a methodology called Consumer Co-creation Camp. It is designed to expedite the research process while making it fun and provide a more direct connection between the client and consumers.
We had a client that decided it was time for his company to start an innovative process. This is how he requested the research: “I do not want boring focus groups, I want a fun process like a reality show, where we are looking to discover new things. I do not want to listen to top-of-mind responses, I want a deeper understanding. We want to achieve a year’s worth of research in one comprehensive study: understand the target, create product/brand concepts and evaluate those concepts”
To address this client’s broad request, we facilitated three groups simultaneously in three days to create products and brands with consumers. This process involved multiple stakeholders: the client team, the advertising agency and the consumers. We called this engaging process: The Consumer Co-creation Camp.
At the end of the fieldwork, the client stated: “We clearly know what we need to know to make this product a success in the marketplace”. How did this project provide such clarity and confidence to the client team and agency? In my view, it was the co-creation of compelling consumer-ready ideas. Three successive stages lead them to:
We wanted the participants to get to know one another first, so we asked Millennial participants to introduce themselves using a collage they created prior to the Camp. This set the stage that this process was about the Millennials and about being together. They felt appreciated while they found new friends and were free to use their own colloquial language.
During this process, our clients moved from feeling “I want to hear this and that” to “These people are interesting”” to “This is going to be big”. There was a perception shift because it was the first-time clients had a chance to see how these Millennials saw themselves.
Millennials created new concepts after testing the product. Collages helped participants to articulate their feelings because many times participants do not know how to describe their feelings and emotions. Collages were a springboard to show their feelings and it was a great equalizer, giving them all the opportunity to adapt the product and the brand to themselves. Our clients witnessed how the brand concepts matched Millennials’ needs and personal styles.
This stage motivated the clients the most. The Millennials presented their ideas directly to them, in the same room. The client team and Millennial teams had a vigorous conversation. There was ‘one voice in the room’. Consumers and clients worked in tandem focused on the unifying goal, with no barriers, mirrors or attitudes. After the final presentation, all the clients knew what the final output of the research was!
At the end of the process, three key outcomes would significantly impact product management, the brand vision, and consumer engagement.
Product Management. The global R&D and Marketing team became aligned and felt empowered to make necessary product and packaging changes.
Brand Vision. The client and ad agency gained a deeper understanding of Millennials, their needs, and shared this with the entire corporation. This understanding inspired them to create a new brand vision.
Engagement. The marketing teams learned how Millennials made friends, and this insight helped them to better engage with this target – utilizing a relevant marketing platform.
Even after the camp, the participants’ ideas were referred to constantly by the clients and the agency. Their vivid experiences allowed for crisper memories. The co-creation experience anchored the clients’ understanding on this target audience through a human connection. It was clear how the Co-Creation Camp streamlined the research process, and in the end, saved the client money and time while enhancing their understanding.
Do you believe your corporate clients would value working together with the consumers in a fun, engaging process that yields high quality insights and speedier outcomes?
If so, how can you streamline your next research project to generate compelling consumer -ready ideas? Consumer Co-creation Camp is a great alternative. When empowered and enabled by the research process our experience has shown that Millennials and Clients are happy to embrace the challenge of creating new products and services.
Isabel Aneyba is president and chief insight generator of COMARKA, an Austin, Texas research firm. COMARKA empowers marketers to develop meaningful product and brand ideas with their customers through dialogue. www.comarka.com
| Comments (0)
Posted By Jeff Walkowski,
Tuesday, May 28, 2019
Updated: Friday, May 24, 2019
How To Create Effective Screeners
Whether you’re experienced or just breaking into qualitative research, it never hurts to review what makes a screener effective in finding just the right people for a research project. It is a questionnaire that recruiters will use to find qualified participants for the study. It is called a “screener” because it is like panning for gold—we have to sift through many people to find the nuggets (qualified people) to be invited to participate. Screeners are used by telephone recruiters, or they may be online surveys as a way to automate the recruitment process. Automation helps reduce expense by eliminating the human effort of dialing phones and talking to potential participants. Keep in mind that automated screeners still have costs associated with them – most notably programming costs which may include quota control, skip patterns, and conditional questions (all of which are typical of any online survey).
All the rules/guidelines about questionnaire construction apply to qualitative research screeners. The most effective screeners have the following characteristics:
They Are Short
If a screener is too long, participants may hang up the phone with a recruiter or simply decide to discontinue completing an online survey. Ideally, screeners have no more than 10-15 questions, or they take no longer than 5 minutes to administer (online or offline).
They Are Clear about the Purpose at the Beginning
Tell participants that it is not a sales call. Explain that we are looking for people to participate in a market research interview, but we must spend some time asking some questions to determine if they qualify.
They Do Not Provide Hints that Encourage Cheating
They include an intentionally general description of the nature of the research so as to not tip off participants to answer a particular way so that they can be invited. For example, say, “We are putting together a focus group on beverages,” instead of “We are putting together a focus group to determine what consumers think of Starbucks.”
They Include Questions Up Front that Are Easy to Answer and that Quickly Eliminate People Without Taking too Much Time
For example, if we are looking for millennial females, we will first ask about gender and age so that non-millennial males are quickly excused.
They Include Need-to-Know Questions – Not Nice-to-Know Questions
Asking nice-to-know questions lengthens the screener, can be frustrating to potential participants going through the screening process, and makes the recruitment process less efficient and possibly more expensive. Keeping the focus on questions that help determine whether a person should be invited or not is best.
They Include Intriguing Questions
Interesting questions keep survey-takers engaged. The objective is to not lose them along the way due to boredom.
They Feature Mostly Closed-End Questions
Again, this is designed to help the prospective recruit move through the process as quickly as possible. Closed-end responses also make the task easier for the recruiter (no judgment required).
They Often Include One or More of the Following Question Types
- Product/service category use
- If they are not users of a particular product or service, they are unlikely to be useful.
- Brand(s) used more often and/or brands they would never use
- If the project is about a particular brand, we probably do not want individuals who reject the brand outright (unless, of course, the purpose is to attract those who reject the brand).
- Past participation in market research surveys, focus groups, and interviews
- Preference is given to those who are not considered “professional” participants, so that they approach the research experience with a fresh attitude.
- Employment in certain industries
- We typically do not want those who are employed in advertising, public relations, or market research. In addition, we tend to rule out those who are employed in the industry that the project is about, because they may “know too much” and not represent the typical customer for the product/service.
They May Include an “Articulation” Question
Such open-end questions are used to help ensure that a participant will be able to make a meaningful contribution to the discussion. Sometimes questions are asked that pose a creativity challenge to the potential participant (e.g., “List 10 ways in which rubber bands might be used”). Ideally, however, a question that is related to the product category will be more relevant (e.g., in a study of high-end golfing equipment, potential participants might be asked to demonstrate some core knowledge of current equipment). In markets where participants may have differing levels of proficiency with the language to be used in the group (e.g., English), the recruiter may be asked to judge the ability of the potential participant to be clearly understood. This serves as an additional articulation assessment.
Jeff Walkowski is the principal of QualCore.com Inc., a consulting firm providing traditional and online qualitative research services to a wide range of industries including health care, financial services, automotive, and information services. He was schooled as a quantitative specialist and entered the industry in the 1980s as a statistician. He later discovered his talents as a moderator and evolved into a qualitative specialist by the mid-1990s.
| Comments (2)
Posted By Jay Zaltzman,
Tuesday, February 5, 2019
Updated: Monday, February 4, 2019
I know it’s tempting, when a client — whether it be a company or an internal client — says they want four focus groups in Chicago, to answer “coming right up!” But it’s important to remember that, as qualitative researchers, our job isn’t only to conduct discussions or interviews, but rather to design research that will be most effective in answering our clients’ research questions.
We need to remind those clients about the value we can bring to the table. Remember, they may be considering “John in Marketing — he’s good with people, let’s have him ask the questions in the focus group.” But “John in Marketing” doesn’t necessarily know about designing research methodology. And he doesn’t realize that there’s more to a discussion guide than just “asking the questions.”
So, when clients ask if I can do four focus groups in Chicago, I say I can; but then I ask them to give me the background to the project. What are their objectives? How will the results be used? Do they have a budget? Why were they thinking of four focus groups in Chicago, specifically?
In my proposal, I’ll provide a cost for the four focus groups they requested, but based on what I learned, I might also suggest some alternatives. Let’s say it turns out they wanted the four groups in Chicago because that’s local, and they don’t have the budget for executives to travel to view groups. I might suggest two mini-groups in Chicago and four online mini-groups via webcam. Or depending on the topic, perhaps to two groups in Chicago and some online journaling nationwide. And of course, I’ll explain the reasoning behind those suggestions. Even if the clients say “thanks, but we’ll stick with the four groups in Chicago,” they will be impressed by the thinking... and you’ll be more likely to be hired than the competitor who had not included those kinds of options!
And don’t forget to offer to help the clients digest the research results. I try to always include the option of running a workshop with the stakeholders after the research is completed, to discuss the findings and how they might be utilized. It’s fulfilling to see the impact of the research, and it provides more value to your clients... and that’s the name of the game!
By: Jay Zaltzman
Jay Zaltzman believes that qualitative researchers can offer true value to clients by combining empathy with creative methodology and analytical rigor. Jay has been president of Bureau West Market Research/Voice of the Customer for the past twenty years. He is an active member and past president of the QRCA.
| Comments (0)