Qualtrics defines sampling as “the process of using a subset of a population to represent a whole population,” a definition closely aligned with goals of generalizability in communication scholarship. While probability/random sampling is widely considered the gold standard for reducing the risk of bias, many social science researchers broadly utilize non-probability sampling methods (e.g., convenience sampling) due to time, financial, and other practical constraints of academic life. Unfortunately, these sampling methods are highly susceptible to biases and, according to Qualtrics, “the results are often lacking in their application to the real world.” One pervasive manifestation of such bias is the overrepresentation of Western, educated, industrialized, rich, and democratic (WEIRD) among social science research participants.
This is far from a novel insight for communication scholarship as our discipline has faced decades of criticism for our overreliance on WEIRD samples. However, the problem with WEIRD samples cannot be overstated. Joseph Heinrich and colleagues challenged the—often implicit—assumption by social scientists that these WEIRD research subjects are representative of the broader population that social scientists make broad claims about in their work with a review that demonstrates that these subjects are, in fact, frequent outliers in a variety of domains. They even go as far as to say that “members of WEIRD societies... are among the least [emphasis added] representative populations one could find for generalizing about humans.”
To further illustrate this, in his Southern Communication Journal editorial, former Editor-in-Chief Bejamin R. Bates noted that only about 14% of people live in Western societies, median worldwide education level is below secondary school level, and that less than 5% of the world’s population are governed under a full democracy with less than half even living in a flawed democracy. Yet, we continue to see this overreliance on WEIRD samples, not just in the social sciences, but in many fields including healthcare, where consequences range from development and use of broadly inapplicable interventions and recommendations, which are ineffective at best, to delayed or misdiagnosis of disease.
Many factors make certain populations hard-to-reach. Members of stigmatized communities are often reluctant to self-identify and have a strong desire for anonymity. Others may even fear legal repercussions because of their participation in research. Individuals with other marginalized statuses often hold a deep mistrust of researchers, especially for those who rarely see their own communities represented among those who study them. Such mistrust in the research process is highly warranted, given the extensive history of exploitation and harm done to marginalized communities by researchers, including the Tuskegee experiments, forced sterilization of women of color through racial eugenics campaigns, and the case of Henrietta Lacks, just to name a few. Thus, when researchers choose to engage with these communities in our work, it is essential to develop a deep understanding of our own relationship to their oppression.
As a white, cisgender scholar in a field where my interests and perspectives are more than well-represented, reflexivity on the influence of my positionality on my work is paramount. For example, although I am a woman identifying as both queer and a member of the consensually non-monogamous community, these factors alone do very little to make me qualified to study and make generalizable claims about such a diverse population. Constant confrontation and interrogation of both my whiteness and WEIRDness is both necessary and fundamentally limited by my privileged perspective.
Studying the work and recommendations passed down by decades of scholars of color was an essential first step for me to reckon with my own privileged identity. Some that I found particularly impactful were the Racial Healing Handbook by Anneliese A. Singh, which is full of practical exercises guiding the understanding of one’s place in systems of privilege and oppression, and Medical Apartheid by Harriet A. Washington, which provides a very sobering account of the historical and ongoing racism in medical research. Angie Beeman’s Liberal White Supremacy significantly aided my understanding of the many ways my white colleagues and I perpetuate racialized discursive violence within the academy. Megan E. Cardwell’s recent Critical Communication and Race Theory of Research and Practice is an excellent resource for interrogating our discipline’s investments in and ongoing reproduction of whiteness and provides a foundation for deeper understanding and actionable recommendations to move forward.
While learning from these resources has been essential to my development as an early-career scholar, implementing these recommendations in my own work is crucial to my scholarly integrity. This was imperative to me in my recent work collecting data from large samples and studying various aspects of identity and relational communication among those identifying as or participating in consensually non-monogamous relationships. The most effective means of recruiting hard-to-reach samples is to have a community member of the population join your research team in some capacity. Ideally, this is a scholar with subject matter expertise that strongly identifies with the population being studied who can lead research design and recruitment. However, given the WEIRDness of academically trained social science researchers themselves, this may be unfortunately difficult to obtain in one’s own network.
Fortunately, there are other ways to develop meaningful, collaborative relationships with communities who may be hesitant to, or have been systematically excluded from, participating in research. First, experts recommend developing preliminary knowledge about a target population prior to developing research questions. This can be accomplished through partnerships with community organizations, public/political figures, and other key influencers to provide feedback on various aspects of research design (e.g., survey design), facilitate sampling, and enhance the overall quality of the findings and their interpretation. However, considering mistrust and other barriers, it is critical to be mindful of both the timing and context in which researchers approach community members and organizations for such assistance. Langer and colleagues also recommend tailoring advertisements to the population, providing consent information in multiple modalities, and providing incentives for participation. Provision of incentives should be approached with caution to ensure fair compensation that is not coercive or exploitative in nature. As Gallegos and colleagues note, these relationships are best shaped by reciprocal trust and rapport and thus necessarily ongoing. Researchers should donate time, expertise, and service in some capacity early in their engagement with these communities, without the expectation of research participation in exchange, and return to these communities to translate and share any findings after data collection and analysis are complete. Researchers must also engage in empathetic listening and be receptive to any feedback from the community about all stages of their research process. Raifman and colleagues also recommend, and provide detailed implementation instructions, for time-location sampling, respondent-driven sampling, and starfish sampling methods for recruiting hard-to-reach populations. Tim Huffman’s Qualitative Inquiry for Social Justice also provides guidelines for community-based research projects and establishing meaningful collaborations with diverse partners.
Online tools have also proven useful for researchers recruiting from hard-to-reach populations. In my own dissertation research, I recruited participants in committed consensually non-monogamous relationships (with other inclusion criteria) using snowball sampling via my own personal network, on social media (e.g., Facebook, Twitter, Reddit, and Instagram), listservs and other professional resources for researchers interested in consensual non-monogamy, and correspondence with self-identified polyamory-friendly professionals. Upon entering these digital spaces, I directly messaged any group moderators to ask for permission to post my recruitment materials in their community spaces. In many cases, these same moderators publicly endorsed my research and disclosed to me that they even participated themselves. By recruiting in these online spaces, I was also able to provide information to participants about community-relevant resources and engage in an ongoing dialogue with participants to disseminate findings and invite them to participate in future studies. However, upon reflecting on feedback from trusted mentors, I realize that my use of these online methods perpetuated social class bias, and potentially age bias, due to my implicit assumption of certain levels of computer/digital literacy among the population I wished to study.
As so many have said before me, recognizing and addressing the biases so deeply embedded in our scholarly inquiry is crucial for the advancement of the communication discipline. While I am not an expert, I hope these reflections prove helpful to researchers aiming to produce accurate, representative, and impactful scholarship. Overcoming the limitations of WEIRDness and whiteness in communication requires meaningful reflexivity and collaboration with marginalized communities and for each of us to walk a unique path of understanding our own relationships with privilege and oppression. I welcome any feedback on these reflections on my own journey thus far and am inspired by an emerging generation of communication scholars committed to change in our discipline and fostering change beyond.