connections committee news

October 2014
Vol. 13, Number 8


QRCA Management News
Conference News
Chapter News
Committee News
Member News
 

Tips from FieldCom—Algorithms: Love 'Em or Hate 'Em, They're Here to Stay

Merrill Shugoll, mshugoll@ShugollResearch.com

If you've ever ordered from Amazon (and just about everyone has), you're familiar with algorithms.  Based on your purchase, the online giant “recommends” other books, CDs and DVDs that you might like.  Many people find this helpful, or at least accept the process, and of course it contributes to the company’s sales.

But what is, and should be, the role of algorithms in qualitative research?  Over the last 20 years these formulas, or so-called "typing tools," have been integrated into qualitative recruitment screeners.  They are intended to help identify market segments based on a combination of behavioral, attitudinal, psychographic and demographic variables.

What are the benefits of using algorithms in qualitative research?
Now that we are in the age of niche rather than mass marketing, algorithms can be beneficial because they more specifically define market segments.  Algorithms help marketers better understand how to position their products, services and ideas to specific market segments; they also guide communications initiatives to be more precise.  Specifically, two primary uses for algorithms in qualitative research are persona builds and communications development.  Persona builds allow us to dig deeper and learn more about the personality of each niche segment, while communications development testing helps us explore which messages would be most effective in motivating each segment to act.  The results can produce a greater return on investment for marketing dollars spent.

What are the concerns about using algorithms in qualitative research?
Using algorithms for qualitative research often is the equivalent of trying to fit a square peg in a round hole.  Algorithms apply a mathematical formula derived from a combination of attitudinal and behavioral questions in a prior large-scale survey to small sample, non-quantitative methodologies (focus groups, in-depth interviews, ethnographies, etc.).  Further, these answers, collected at a single point in time, can easily change in subsequent interviews. 

For example, it is not unusual that someone categorized as a brand "enthusiast" based on her responses in an initial screening for a qualitative study will be reclassified only a week later in a confirmation call.  She may move down a category to "on the fence" simply because her memory or attitudes toward the product shifted slightly during that time; originally she rated it “9” on a 10-point scale and now she rated it “8.”  This is not a big difference to the respondent, perhaps, but a major one according to the screener.  The recruiter then has to cancel the respondent since she no longer qualifies for the "enthusiast" group, and she can't be recruited for the other groups whose quotas have already been filled.  Respondents get discouraged or angry when they no longer qualify for study participation, seeing no reason for the change.  This is not good for building respondent cooperation rates for qualitative research.  Further, there may not be any meaningful difference between the original and confirming answers. 

Because algorithms enable researchers to segment the marketplace so narrowly, the incidence of people who fall into the niche segments can be extremely low.  Researchers have to contact many more households to find these rare individuals, which makes recruiting increasingly expensive and time consuming.  The use of algorithms typically doesn't allow for conducting research at a reasonable cost and with a quick turnaround. 

Worse, use of algorithms, with their combining of multiple variables, can mask the essence of the type of respondent the client really seeks.  For example, in one recent low incidence study, the client wanted to find patients who suffered from a chronic disease and took specific prescription drugs to treat it.  That alone would be challenging enough, but it is doable.  On top of this, however, the client provided an algorithm that defined five market segments based on type of symptoms, level of symptoms, frequency of symptoms and attitudes toward the disease.  They only wanted to recruit people from two of those five segments.  After screening 2,300 people, only seven people fell into the two desired segments; almost everyone else fell into the three segments the client did not want recruited. 

Some time back, QRCA's FieldCom released a list of the top 12 recruiting Do's and Don'ts and number nine on the list was "avoid using algorithms," because they complicate recruiting, annoy respondents, significantly increase recruiting costs and rarely work the way they are intended.  This axiom still holds true today.  FieldCom recommends that QRCs try to convince clients to avoid using algorithms in qualitative research.
If an algorithm must be integrated into the recruitment phase of a qualitative project, QRCs are strongly encouraged to communicate the following to their clients:

  • Incidence rates for any market segment hovering at 5% or lower suggest that recruiting may be next to impossible.
  • Loosening of the "typing tool" or "specifications" may be necessary to get the job done; that is, simplify the formula or algorithm by focusing only on the key variables.
  • Algorithms will increase the cost of recruiting for the client. 
  • Algorithms will lengthen the time needed for recruitment.
  • Geographic markets vary, so it may be difficult to identify an adequate number of representatives from all the target segments in one particular market. Consider using multiple market locations instead. 
  • Algorithms may prevent the field service from achieving a full recruit in every group, which means smaller sample sizes.
There is no doubt that micro-marketing is alive and well, so the use of algorithms in qualitative research is undoubtedly here to stay.  We must work together to find more cost-effective and time-efficient methods for recruiting these low-incidence market segments.

Back to Top


FieldCom Announces the Results of a Major Industry Study of Problem Participants

Quality in qualitative research:  it’s a matter of how good the moderator is, of course, and of how trustworthy study participants are.  No matter how good the moderator, if the participants are questionable, the research will be doubted.  Participant quality has been a persistent concern for the research industry, as indeed it is for other types of market and public opinion research. 

To find out to what extent industry stakeholders believe problem research participants affect qualitative research (QLR), the QRCA Field Committee conducted an online survey with industry stakeholders.  We focused on Frequent Participants (those who participate too often) and Deceivers (those who purposefully do not tell the truth about their qualifications).  This is the largest ever industry survey on the issue, done in partnership with the Marketing Research Association and Quirk's Marketing Research Review.  The sample was 448 professionals actively involved in conducting QLR in the United States.  Respondents represented all facets of the qualitative research industry, including field managers, facility owners, moderators and end clients.  Over the past two years, 97% have used in-person/phone methods and 63% have used online/mobile methods.  

The good news is that, overall, most recruiting is seen positively by all survey respondent segments.  At the same time, problem participants are seen as worrisome.  Importantly, a number of end clients—the people who commission QLR—as well as some QRCs have had personal experiences or heard firsthand of problem participants.  While this may happen relatively rarely, the research professionals in our survey recognize that concerns about problem participants hurt QLR’s image in the industry and among the general public.   

The survey showed that support exists for an industry program to help reduce the incidence of problem participants.  A sizeable number of end clients and QRCs said they would consider funding the work; further research is needed to explore what these stakeholders would be willing to pay.

Recommended practices to ensure quality research participants are described in the full report, but strategies that QRCs can implement include:

  • Ask critical screener questions in unaided or open-ended formats to make it harder for respondents to guess what recruiters are looking for.
  • Describe desired respondents to the recruiting manager in a phone call or in writing, not just through a screener.
  • Ensure that recruiters use a trained, live person in screening and rescreening, rather than relying solely on online methods.
  • Ensure that topics are masked in all participant communications.
  • Request that participants show proof of product use and/or ownership when feasible.
  • Continue an open dialogue between QRCs and recruiters/facilities after the research is done, providing feedback on problems (along with praise for a job well done).
  • Work on developing a standard industry feedback form for QRCs to report problem participants to field services after interviews.  (Currently, some QRCs provide feedback, others do not.)   

For more information, please join FieldCom members Judy Langer and Dave Kains at the QRCA Conference in New Orleans for their roundtable discussion FieldCom Industry Survey Insights: Strategies to Reduce Cheaters/Repeaters on Thursday, October 16th from 5:00 to 6:00 p.m., and on Friday, October 17th from 4:15 to 5:15 p.m.

A full, detailed report is available upon request by contacting:

Shannon Thompson
Executive Director, QRCA
exdir@qrca.org
888-674-7722
651-290-7491

Back to Top


MemCom Happenings

Philip Smith, philip@environmetrics.com.au

This month has been a busy month for the MemCom team as we discussed and thought about methods to engage members.  We are pulling these together in a paper which will be presented to the Board.  One area that we discussed extensively is communication amongst QRCA members and I’d like you to reflect on how QRCA communicates with you and how, by responding to QRCA communications, you can get more from QRCA.

Connections
You are reading this, so I assume you read Connections! This monthly e-zine provides an in-depth look at what is happening in the association.  The Board details its thinking and direction, committees provide insight into their activities, and the latest initiatives are promoted to members. Special Interest Groups (SIGs) are a great way to learn more about particular areas, and local chapters also use Connections to highlight what’s happening.  Personal Connections is where we see what our colleagues and friends are doing and the other important role of Connections is to tell all members about our new members.  Please reach out to a new member if they are in your city or region—you know the power of a personal connection.

News Bites
I think of this e-newsletter as the QRCA value driver.  It is a must-read every two weeks.  This is where you can get real value! Partner conferences are promoted, as well as other discounts, and you can find info on the QRCA Conference, information flow, Q-Casts, and QRCA TV.

QRCA website
The QRCA website—www.qrca.org—is YOUR QRCA data bank and it is worthwhile to go to the website once or twice a week.  The Forum keeps you abreast of hot topics and is a great way to share and help others.  The website provides detail about all SIGs and chapters, promotes and details the conference, has back issues of Connections and VIEWS, provides professional development through the archives of Q-Casts and QRCA TV, and showcases the discounts that our partners provide related to training and education. And if you don’t want to go on daily basis, you can now get a weekly summary!

Social Media
QRCA is on Facebook, Twitter, Pinterest and LinkedIn!  Do you use those media? If so, please share your thoughts and communicate with other QRCA members.

Back to Top


Membership Expansion Task Force Report

Susan Sweet, susan@sweetinsightgroup.com

With gratitude and excitement, I have the honor of announcing that QRCA’s year-long investigation into the possibility of expanding our membership to other research practitioners is complete.  You can read the report now on our members-only forum! Click here to access.

The Board of Directors owes a huge debt of gratitude to the Membership Expansion Task Force (METF), for its exhaustive research among our members, and thoughtful synthesis of both qual and quant data on how our members feel about expanding membership beyond our existing criteria.  We were lucky to have a group filled with members of various tenures in the industry, years of membership in QRCA, and personal perspectives on the issue.  The METF was, and is, a stellar task force, comprised of Ava Lindberg, Judy Langer, Randi Stillman, Casey Bernard, Kea Wheeler, Holly Blankstein, and its outstanding chairperson, Rick Weitzer.

The Board is pleased to present the final report and recommendations for your review and comment.  We are eager to get your reactions, questions, and thoughts after you’ve had a chance to digest the report (don’t worry—it’s just 20 quick slides).

The METF will present this report at the QRCA Conference in New Orleans, but we wanted to let all members have a chance to review it in advance, and to stimulate discussion right away. Please post your comments on the forum! You’re also welcome to contact our President Mark Sumpter at mark@c-linkmoderating.com, if you prefer not to post comments and questions publicly.

Back to Top


VIEWS Welcomes New Tech Talk/Online Editors

Patrick Wooldridgekaren lynchThe VIEWS Committee would like to welcome Karen Lynch and Patrick Wooldridge as the new co-editors of the Tech Talk/Online section. Karen and Patrick are picking up the baton from this section’s previous co-editors, Ted Kendall and Mary Aviles. Thanks to Ted and Mary for their amazing contributions to the magazine—they have done a great job providing interesting and absorbable articles on key technologies for qualitative researchers.

Back to Top


TechCom's HOT TIPS for Cool QRCs CONTEST and GLOW FEST 2014

The QRCA Members Forum is sponsoring a contest for ALL members.

Post a hot tip and be entered in a random drawing for a $50 Amazon gift card (contest ends Friday, October 24).

Post a business tip, new technique or tool, or a helpful website.

FURTHER CONTEST RULES:

  1. Start your tip as a NEW thread in this part of the Forum (Business Issues and Opportunities). DO NOT POST IT AS A REPLY TO THIS THREAD (you'll get better long-term exposure having it as a separate thread).
  2. Start your subject line with HOT TIP: (followed by short description); this way, we can easily find the tips and enter you into the contest.
  3. REMEMBER to sign up for automatic alerts to your tip and any others you want to follow (including any new notices in this part of the Forum). https://qrca.site-ym.com/forums/Posts.aspx?topic=929096
  4. Winner will be announced on the Forum. One entry per person.

For those attending the conference, the Forum is sponsoring GLOW FEST 2014 at the Thursday party. All attendees will receive festive glow-ware to adorn themselves or collectively build something creative. We¹ll be there to document all the fun and post it to the Forum.

So, bring on the heat, great tips can¹t be beat!

Back to Top


QRCA Committee/Task Force Reports — September 2014

Darrin Hubbard, darrinh@ewald.com

Annual Conference

Staff:
  • Staff is working on onsite, signage and mobile app.
  • Staff is working on BEO’s with Hotel.  AV has been ordered.
  • Staff is still securing more exhibitors and sponsors this week.  Gathering logos and ads.
  • Staff is fulfilling all sponsorship requirements. Ordering all bags and Lanyards.
  • Staff is working on Room block and Board room needs.
  • Promotion emails went out September 9 and September 3.
  • Mobile app is being designed and populated.
Committee:
  • Sponsorship Committee continues to meet to update each other on progress.
  • Marketing Emails with Speaker interviews were sent.
  • Committee is working with individual committees to get all logistics in place.
Other: 2016 Conference bids have been gathered and will be shared soon with the Board. 

Connections

Staff: September Connections went out September 12.
Committee: Material for the October issue is due by September 26 and is scheduled to go out on October 10.

Field

Committee:
  • The committee last met on September 12.
  • An article on Algorithms was finalized and will be distributed in Connections for October. The September issue will feature a piece on Communication between the QRC and facility.
  • The committee has requested to distribute the Cheater/Repeater study to Quirks and MRA.
Other: The next meeting will be at the conference.

Industry Relations

Staff: Staff sent a conference banner ad to GreenBook per our agreement.
Committee:
  • The committee met on September 9 to discuss ongoing details of our industry partnerships.
  • Several committee members are working on proposals for qualitative tracks at industry events.
Other: Next meeting is on November 11.

International

Staff: Staff has arranged for the GOS recipients to be reimbursed for $1,000 of their expenses.
Other: The next meeting will take place at the conference.

Investment Advisory Group

Committee:

Investments have been made per Jim Berling’s recommendations to the Board on how to invest matured CDs.

Marketing Committee

Staff: Staff sent membership, communication and event metrics to the group per their request.
Committee: The Marketing Committee held its first meeting on August 14.
Other: The next committee meeting will be held on September 16.

Membership Expansion Task Force

Staff: The final deck has been revised and is ready for posting to the member forum.
Committee: The Board and METF will be presenting the findings and recommending next steps at the conference.

Membership

Staff:
  • New member lists were sent by Darrin to Board and Chapter leaders.
  • Shannon sent email welcomes to August new and returning members.
Committee:
  • The committee met on September 10 to discuss recruitment and engagement strategies.
  • The membership drawing will take place at the conference.
Other: The next meeting is October 8.

Nominating Committee

Committee:
  • The committee met on August 20 to make a recommendation for Officers.
  • A memo for the Board has been sent with NomCom recommendations for Officers.

Public Relations

Staff:
  • Laurie Pumper continues working with Co-Chairs and volunteer editors to confirm publication of articles with our content partners.
  • Laurie continues updating the News & Press page.
  • Laurie sent information about the Young Professional Grant winners to several media outlets and shared on Facebook and Twitter.
Committee:

Our Co-Chairs and the volunteer editors continue working with authors to submit material to our content partners.

Qcasts

Staff:
  • Have begun promotion for the September Qcast.
  • Jessica tested video streaming with the committee for the live Qcast at Annual Conference.
Committee: All Qcasts topics are finalized for 2014.
Other: September Qcast had registration at 158.  Live attendees were 97.

Technology

Staff:
  • The Forum Digest goes out every Wednesday.
  • UX report has been submitted to the Board. 
Committee: The committee met on September 11 to discuss how to promote Forum usage, SEO and other business.
Other:

Next meeting is at the conference.

VIEWS

Staff:
  • Laurie Pumper continues work on a variety of transition tasks; she assisted with proofreading the Fall issue.
  • Nicki Brunner completed house ads for the Fall issue.
  • Laurie will work with NEBM to announce the new digital edition once it launches.
Committee:
  • The Twitter Team re-posted links to Summer VIEWS.
  • Fall issue should launch the week of September 15 on the new digital platform.
Other:
  • The next committee meeting is scheduled for October 10.
  • Committee leadership continues working with NEBM to get the Fall issue launched.

Back to Top