Bad Actors in Research
If you’ve spent any time in the market research industry, you’ve almost certainly encountered fraudulent respondents before. For research respondent providers, working to find fraudulent respondents is like a game of cat and mouse; methods are in place to help prohibit it, but fraud is notoriously difficult to catch. It’s indisputable that fraud is an enemy of good research and that it needs to be eliminated from studies, quantitative and qualitative alike. Today, we will discuss the core components of the market research world, frauds place in it, and the ways that researchers can work to keep it from affecting their insights.
Understanding the market research industry
The market research industry relies on an ability to collect, analyze, and present accurate information to clients. This information is acquired in one of two ways, qualitative or quantitative research methods. In our article “Qualitative vs. Quantitative: Answering the Whats, Whys, and Whens,” we defined these as the practice of collecting insights directly from people through interviews and focus groups, and as a data collection process that makes it possible for researchers to inspect numerical patterns, respectively. While both of these methods are useful in different circumstances, it’s important to note that quantitative research is less protected against fraud.
Fraud in the industry
Fraud comes in many shapes and sizes; there are survey farms, repeat survey takers, and participants who embellish, manipulate details, and lie. The level of fraud in the industry continues to escalate, which is compromising the external validity, that is, how generalizable results are in other contexts, of the data that researchers obtain from participants. An article published on Greenbook suggests that, on average, fraud accounts for anywhere between 15 to 30% of data within the market research industry; in some cases, the percentages are estimated to be even higher. It’s generally believed quantitative research methods are more susceptible to fraud than qualitative ones. This is because, with quantitative research that most commonly takes the form of surveys, fraudulent participants have the luxury of hiding behind a screen, or better yet, behind automated programs. They can more easily be dishonest about their background and knowledge to exploit a system that relies on them being truthful. One way to think about this is that, with quantitative research, the level of accountability that researchers can hold participants to is relatively low, highlighting the importance of prioritizing methodologies that raise that level of accountability.
THE CONVENIENT FORGETTER: The respondent who can't quite remember exactly when they bought that car. Checks off "purchased car in last 12 months" but really it was 14 months ago.
THE YAPPER: The participant in an interview or focus group that talks way too much. They ramble, they go on tangents, and they make productive conversations sour. Do they like the sound of their own voice? Is it a lack of self awareness? Or is it that they just have a lot to say? Make sure to politely interrupt them to keep the conversation on track!
THE CLOCK RUNNER: The participant that knows they’re on the clock during an interview. Their only goal, have the call extend as long as possible and then collect the extra money. Watch as they take a question asked with two minutes to go and turn it into a 15 minute monologue. Of course they come knocking for their extra pay after the call.
THE PEOPLE PLEASER: The respondent that wants to be the best respondent possible (the researcher's favorite), and will answer accordingly, even if they need to twist the truth. Their motto is "Tell 'em what they want to hear and you'll be in the clear."
THE EMBELLISHER: The respondent that checks off they purchased a software tool because of it's enhanced security, it's AI capabilities, and it's flexible terms when in reality, it was the cheapest option.
THE VETERAN: The respondent who knows survey styles like the back of their hand. They know to be confident but cautious. "When did you last participate in a research call?" they're asked... "around three months ago" they answer as they reminisce on the lovely focus group from earlier in the week.
THE “EVERYTHING IS CONFIDENTIAL”: The participant that takes confidentiality seriously (as they should) but to a fault. When asked their opinion on an industry, the direction of a technology, or even which state they live in, their answer is always the same, "It's confidential".
THE CURT ONE: The opposite of "The Yapper". They answer with only a few words to the point that it's rude. Pulling teeth is an understatement. The call is choppy, awkward, and filled with pauses where a reasonable participant would still be talking. They may be a great fit for the project, but their behavior makes it impossible to get anywhere.
THE REVERSE ENGINEER: The respondent that must figure out the researchers goal before answering. They ask themselves "what does the researcher want me to answer?" They pick a narrative and stick with it.
THE AGGRESSIVE GOOGLE-ER: The participant that said they could discuss a topic in detail but boy were they wrong. Now every question they're asked has a 20 second pause, darting eyes, frantic typing, and a response that was clearly looked up moments ago. ChatGPT has given this group more confidence as of late.
THE ENABLER: The panel or recruiter that sacrifices integrity for money. They knowingly allow fake respondents to take surveys, lead the witness on screeners, manually change screening answers so respondents will pass, and sometimes even build their own bot farms. Any time they're called out on an issue, according to them, it was a one-time mistake...
THE SURVEY FARMER: Typically located in countries where the dollar goes far. This person or group takes and fakes surveys for a living. Their VPN usage and experiential knowledge allows them to be anyone and everyone, at scale. Make no mistake, this is a career for them and they take it very seriously.
THE ANARCHIST: The respondent that just wants to watch the world burn. No one knows what motivates these people to falsely answer screeners, use profanity in their answers, harass researchers, and more. Is it fun? Is it pure evil? We may never know.
THE TECH FRAUDSTER: This person chooses to use their skills for evil. They program bots and algorithms to take surveys without any human assistance. Their only goal, money money money. If only they’d use their programming skills for good instead.
Disclaimer: the rankings associated with each description on the spectrum do not represent the official opinion of Zintro or it's staff. Each score is an average obtained by surveying multiple individuals from Zintro and other market research companies.
Defending against fraud
Learning how to defend against fraud is the next important step. Unlike quantitative research, qualitative research has an added level of protection against fraud, suggesting that it may be a great tool for gathering truly insightful data. With qualitative studies, researchers have the opportunity to speak directly with people about their perspectives on a given topic, and it’s much easier to spot discrepancies in real time. With that in mind, it’s important to note that there is still a considerable amount of fraud in qualitative research, and sometimes things slip through the cracks. These instances of fraud tend to be less egregious and they are generally a combination of embellishment, exaggeration, and occasionally, pure dishonesty. At Zintro, we are dedicated to correcting for the unwanted instances of fraud that inevitably occur; if a client suspects that the validity of their data is being affected due to fraud, they will not pay for the fraudulent participant and their concerns will be investigated by a team of objective reviewers. Additionally, we’ve found that a combination of enhanced screening, photo ID matching, adding in trap questions, allowing for pre-calls, and providing clients with a grace period helps us to keep fraudsters from compromising our client’s research.
Written by: Sadie Estey
