Past Participation Requirements in Market Research

The basics of past participation requirements

Past participation requirements are regulations put in place to ensure that project participants in market research are not taking part in studies too often or of the same nature. Specifically, these requirements will ask potential participants whether they’ve participated in other studies within a certain amount of time or whether other studies they’ve participated in have been on particular topics. They are put in place by clients and their designated researchers and they are most commonly seen in the screeners that are distributed to potential participants prior to qualitative research studies such as focus groups and in-depth interviews. If a participant does not meet the requirements, they will be immediately removed from the study so as not to compromise it. Clients value original and honest insights because they give them a competitive edge, and if an individual is a frequent flier in the market research world they are often referred to as ‘repeats’, ‘regulars’, and ‘groupies’. So, why are past participation requirements controversial in the market research space?

Are past participation requirements a necessity?

Yes and no, because it depends who you ask. On the market researcher side, they always want to deliver alpha and unique insights to their client, so it’s logical that they would want participants that haven’t been doing interviews on similar topics. The end clients tend to have a similar perspective. They’re paying a lot of money for a particular study, so they want to avoid obtaining insights from the same people who provided their competitors with insights. 

On the participant side, they couldn’t dislike past participation requirements more. To them, the requirements are an inconvenience at best, and oftentimes, if a participant is told that they were the reason for their disqualification, it’s enough for them to never work with that vendor or panel again. Their thoughts are “why should it matter what other studies I’ve done if I’m qualified?”  This question tends to turn honest participants into liars, something we highlighted in our previous article, “Bad Actors in Market Research”.

Before we dissect the vendor side, be sure to take into consideration that this article is, of course, being written by an employee of a recruitment vendor. From the perspective of a vendor, the inclusion of past participation requirements on screeners is one of the worst things that they can see. When we surveyed 18 individuals in the qualitative recruitment space and expert network industry, 17 of the 18 listed past participation requirements as a top-3 worst question someone can ask. This placed past participation requirements right up there with multiple information-extracting open response questions, questions that seek confidential information, and questions about race, ethnicity, sexual orientation, gender, or personal identity in B2B research.

What are the negative implications of past participation requirements?

In the same way that there are instances when past participation requirements can be useful, there are instances where they are not. When these requirements are used in excess, they cause issues for the people at every level of the market research process. The best way to tackle this pyramid is from the top down, starting with how they can be a nuisance for the client.

For the clients:

The improper use of past participation requirements can negatively affect a client because it can completely derail the objective of their study. By eliminating strong candidates because of their past engagements, clients risk obtaining less meaningful insights. For example, some participants (i.e., CEOs, CTOs, CFOs) are highly sought after, meaning that they take part in lots of studies. If a client stipulates that the participants cannot have taken part in research within the last three, six, or even 12 months, it begs the question: are the participants that the vendor was able to retrieve somehow flawed? Why is it that they have not been taking part in research if people of their status are in such high demand? Are their responses still valuable? Are they lying? Should we have let the more qualified participants be the exceptions to our past participation requirement rules?

The Zintro team surveyed 400 people at random from its one million person network. They found that 88% of the people said they had participated in some sort of qualitative research activity in the previous 12 months, 70% in the previous six months, 52% in the last three months, and 42% in the previous month. This highlights the difficulty that comes with recruiting individuals that meet strict past participation requirements. That said, we acknowledge that there is bias in this study because anyone that is signed up to work with Zintro has the intention of doing research projects. Additionally, we acknowledge that certain audiences of B2B decision makers (e.g., IT, HR, and Finance) are likely to have higher rates of recent past participation solely due to demand for those roles. 

For the researchers:

On the researcher side of the equation, researchers must toe the line between appeasing their client and maintaining a strong relationship with their vendor. There is an understandable desire to obtain top tier participants with original insights, but the client's definition of ‘top tier’ is often too narrow. The researcher is expected to approach a vendor with these requirements, but vendors will not want to work with them if they repeatedly supply them with standards that are nearly impossible to reach. It also introduces two very real possibilities: the project could go poorly because no one will qualify or, something we see often, worse participants will be selected because they were the only ones who met the past participation timeline. This emphasizes the importance of a healthy researcher-vendor relationship; the researcher must be open to taking pushback on their requirements for the sake of the project's success and to maintain client and vendor relationships with longevity.

For the vendors:

Similar to the researcher, vendors have to balance a relationship. They are responsible for maintaining a strong relationship with the researcher while providing them with critique as well as ensuring that their participant pool has a good experience. That said, past participation requirements are generally terrible for vendors. In any given project, this requirement can make projects up to 400x harder (we know this from experience). More often it’s in the 2x-10x range, but the more ways a researcher cuts the available sample, the less likely they’ll get strong fitting participants. 

Companies providing participants have to get them from somewhere, generally from ads, cold recruitment, or referrals. Each of those methods has associated upfront costs that ideally get made up when that person sticks around and does future research projects. Companies spend fortunes on aggregating the best data so they can have higher conversion rates and improve both the client and participant experience, while still running a functioning, healthy-margin business. By telling a vendor they cannot use people that they’ve worked hard to get into their panel, it forces them to use expensive alternative methods. 

One detail that many researchers and clients don’t typically consider is the market share of their respective vendor. Qualitative research is happening at many levels and at many companies. Aside from market research, there’s also the adjacent expert network industry doing billions of dollars per year in revenue and millions of 1:1 interviews. So, to think that a small-business recruiter that may provide 0.0001% to 0.01% of all the interviews conducted annually can find multiple business decision makers that haven’t done any research for other companies in the last year, but will for this small recruiting vendor, doesn’t quite add up.

While it is of the utmost importance that the researcher gets the participants that they need, it is also important that the vendor is able to provide a good experience for participants. They rely on reputation, and if the vendor is regularly sending out projects where good fitting respondents don’t get selected, participants will be hesitant to work with them and oftentimes request to never be contacted again. Needless to say, it’s a balancing act for vendors. 

For the participants:

In an industry that relies on people being willing to participate in research, it is vitally important that those people do not get discouraged as a result of constant rejection. Individuals often choose their study participation wisely, and if they are qualified yet frequently turned away due to their participation in previous studies, they may choose not to work with particular vendors anymore or to not partake in any research whatsoever. Not only is this damaging to the participant who loses out on an opportunity to share their expertise, but it is also damaging to the market research industry as a whole because it may make the people that the industry revolves around uncertain about getting involved. Additionally, we often find that the otherwise-honest participants lie about their past participation, and because vendors only have records of when an individual last did a project for them, it becomes nearly impossible for them to identify these liars if they’ve been participating with other companies. It’s rare to see loyalty to one or even a few platforms, as people that are on one tend to be on others too. For example, 97.8% of surveyed Zintro participants said they had answered screeners from other companies or had accounts registered for other vendors. Moreover, many research “veterans” know about past participation requirements and know that answering truthfully will be the nail in their coffin. We’ve seen many instances of great participants lying and then during research, it turns out that not only had they done an interview on a similar topic, it was also for the same researcher or client.

What does this mean for the world of market research?

All in all, the inclusion of strict and sometimes unnecessary past participation requirements in market research projects negatively affects a whole slew of people. There are multiple ways to find middle ground though, so hope isn’t lost. A few ways include: asking about past participation and using it as a deciding factor but not terminating respondents because of it, asking respondents about the exact topic and determining if there was any conflict of interest, or shifting the industry perception entirely with data. How would researchers react if they knew just how many research calls Fortune 500 decision makers did per year? Two individuals that we surveyed noted that they do anywhere between 300-400 calls per year across 20 companies. Do you think a new baseline can be established? By being more selective about when to use such requirements in the future, we can make the industry more approachable as well as help research projects trend toward success rather than failure. Stay tuned as Zintro works hard to tackle this fundamental disconnect. 

Written by: Sadie Estey