As the communication- across- the- curriculum director at a liberal arts university, enthusiastic messaging about communication skills forms the heart of my daily work. In addition to student-facing workshops, I guide faculty members across campus on the best practices of integrating speaking assignments into their courses. Several faculty members have asked my take on AI communication coaches, the web-based applications that provide users individualized feedback on their speaking performances. The AI communication coach industry has exploded in the past five years to include dozens of companies such as Yoodli, Getmee, Orai, and Poised AI. While these platforms differ in features and design, they all claim to offer accessible, actionable, personalized, judgment-free feedback on speaking performance.
Understanding the social and rhetorical effects of algorithms is a central learning goal of my Rhetoric of Algorithms course, so I enlisted the undergraduate students of my Fall 2023 class to critically analyze several AI communication coaches. In my experience, students possess uncanny noses for sniffing out busywork from learning opportunities. They approached the task with gusto.
We held several questions in mind as we explored these programs: What nascent theory or theories of communication underlies the product? From where do each product’s ideas about effective and competent communication come? And for fun: Could we reverse engineer an AI communication coach by delivering a nonsense speech that still hit the algorithmic markers and earned high scores? (Spoiler alert: We sure can!)
My students were so alarmed by the potential harms of AI speech coaches that they co-authored this open letter to university administrators. The academic honesty and student privacy conversations ensuing from generative AI have been important, but have occluded some of the other ways that AI appears in educational contexts, such as AI-driven responsive personalized tutoring. In regard to our future career stability, communication faculty—especially those of us who teach public speaking—should be extremely concerned about the advent of AI communication coaches.
My students and I acknowledge our relative privilege teaching and learning about communication skills at a small liberal arts university that features discussion-based, face-to-face classes. Our public speaking course, for instance, caps at sixteen, a number that allows students more speaking time and more personalized feedback. As many universities experience crisis-level financial pressures, we predict the false promises of AI communication coaches will tempt some institutions to further deplete and casualize communication faculty members. While there may be some limited reasonable uses for them in higher education, AI communication coaches must be approached with caution and should never supplant traditional communication instruction.
Argument 1: AI communication coaches do not reliably improve communication skills.
If you were a public speaking instructor, how would you assess the following?
Good morning. For my first point, apples, bananas, grapes. 42 apricots, 17 watermelons. Therefore, I think you can see 14 pineapples. Strawberries, peaches, pears, grapes, bananas, apples, peaches, pineapples. Pineapples again. 42% pineapples compared to 17 strawberries.
Most humans would rightfully assess this monologue as complete gibberish. A leading speech coach, however, scored it at 85% overall–and gave it perfect scores in the areas of sentiment, empathy, and filler words. (You’ll pardon us if we find the latter, well, bananas.) As a class, we discussed the scary possibility of instructors using scores generated by AI speech coaches as the “official” grades for a student’s assignment. AI generated scores are not only faulty, but students would attempt to game the algorithms, resulting in a classic perverse incentive.
When we scored higher, it was not because we became better speakers–rather, we became better at calibrating our performance to the AI speech coaches algorithms. The way we learned to speak to maximize our AI speech coach scores recalls a famous 1931 study involving a monkey and a baby. In an era before Institutional Review Board protocols, comparative psychologist Winthrop Niles Kellogg adopted chimpanzee Gua and raised him alongside baby Donald. In the early stages of the experiment, Gua excelled at his cognitive and mobility tests compared to his human counterpart. However, Gua’s chimpanzee vocal abilities could not compete with Donald’s. The Kellogg family abruptly ended the experiment because “while Gua showed no signs of learning human languages, her brother Donald had begun imitating Gua’s chimp noises.” Like baby Donald “learning” from Gua, when we “learn” from AI speech coaches, we are not learning how to speak with humans.
The idea that technology can quantify something as complex as human communication is akin to tech writer Meredith Broussard’s notion of technochauvinism, or the assumption that technological solutions are superior to other solutions. Adding to the problem is the subjective nature of algorithms and the AI software as they are created by a small group of individuals deciding what qualifies as “good” communication. Tarleton Gillespie highlights the fact that many digital platforms reflect the biases of their (typically white male) creators, narrowing the complexity of human communication down to a handful of algorithmic quantifications. AI communication coaches reduce communication to the outdated transmission model and do not account for contextual complexity. In a critique of the rise of therapeutic chatbots, Misti Yang states: “When practiced outside a human-centered apparatus, pain and conversation risk becoming unmoored from political futures and reduced to a mechanism for gathering objectified and commodified information offering only one script for pain.” Whether in a therapy session or delivering an informative speech,
We will pause our critique to assert that we are not anti-technology, though many attempts to integrate technology in higher education rarely fulfill the promises of improved access, decreased costs, and better learning outcomes. We are concerned that AI speech coaches may be on the brink of what educational technologist Justin Reich cautions is the next “learning-at-scale hype cycle.” We agree with edtech expert Justin Reich:
Autograders are unevenly useful across the curriculum. They are most useful in the fields in which desired human performance is sufficiently routine for algorithms to reliably identify the features of high-quality and low-quality performance and to assign grades and scores accordingly. [...] Much of what we want students to learn, however, cannot be demonstrated through performances that adhere to these kinds of rigid structures.
Reich’s last statement above should refocus us on why we teach communication skills. Even components of communication that appear relatively quantifiable by an AI speech coach, such as eye contact or speaking rate (often measured in words per minute) are subject to context collapse. (Notably, the AI speaking coaches that we looked at are capable only of measuring eye contact with a camera and not eye contact with a human.) The myriad factors that determine something like appropriate eye contact or speaking rate (audience, occasion, purpose, public address versus interpersonal conversation, etc.) will always exceed what Reich refers to as the “rigid structures” of autograders—and rightfully so. AI speech coaches will never fully account for the breadth of human communication because it is rarely “sufficiently routine.” The rhetorically sensitive interpretation of contextual complexity may be among the most foundational of communication skills. Even the best AI communication coaches, by design, reduce or ignore contextual nuance.
When AI speech coaches attempt to quantify unquantifiable components of communication, the results are much worse than just a glitchy product–the results can be quite harmful. Take, for instance, one student’s experience with the sentiment assessment algorithm. In the context of a speech on algorithmic racism, the student stated that content moderation practices on major platforms often lead to an increase in “harmful ideas like white supremacy.” Because of this and similar statements, the student scored 20% on sentiment and was informed: “Good effort. However, your choice of words created a largely negative tone.” Condemning white supremacy is not cheerful enough for the AI speech coach. Because proprietary algorithms are black boxes, we do not know the precise terms that trigger low sentiment scores. However, we do know that many aspects of the world are harmful, terrible, tragic, awful, devastating (these are all words we suspect the coach would flag for “negative sentiment”) -- and failing to name them as such only perpetuates these conditions. Rewarding a positive, happy affect recalls what Salzano and Yang refer to as technoliberal managerialism, “the use of the connection, quantification, control, tracking, and optimization capacities of technology to manage everyday interactions.” To be clear, the harm here is not just that the AI coach failed to measure what it claimed to measure. The larger harm is that it teaches people to stop telling the truth about the world.
Similarly to sentiment, AI speech coaches disregard the human experience of empathy, framing it instead as a strategic method of inducing compliance. We suspect that the popularity of Brene Brown’s work drives this focus on empathy, so we will use her definition here: Empathy is connecting with the emotion that someone else is feeling. Some AI speech coaches refer repeatedly to empathy in their marketing materials but then do not attempt to measure or provide feedback for empathy. For the AI speech coaches that evaluate empathy, as far as we can tell, they rely on key phrases such as “I heard you say...” while paying lip service to the idea that empathy can be communicated through visual, verbal, and paralanguage channels. Rather than fostering genuine empathy, AI speech coaches reward shortcuts to fake a compelling empathy performance.
Argument 2: Not only do AI communication coaches fail to scale communication instruction, but they also introduce more potential harms for the most vulnerable among us.
In their marketing materials, AI communication coach companies position themselves as accessible and equitable. Nothing could be further from the truth. Like most programs that rely on facial recognition technology, speech-to-text software, and algorithmic reduction of complex practices, AI speech coaches are racist, ableist, classist and further inculcate students into regimes of digital surveillance. The Matthew effect, a principle that asserts that the most advantaged tend to accumulate more advantage, is applicable in educational technology contexts: “[W]hen researchers evaluate how learners from different backgrounds access and use new technologies, it is common to find that the benefits of new technologies–even free technologies–accrue most rapidly to the already advantaged.” For example, the students who tend to benefit the most from auto-grading tutors are students who have already developed self-regulation and higher-order executive skills in traditional educational settings.
AI communication coaches do not simply “know” what counts as good communication; rather, they must be programmed to value some communication practices over other communication practices. We are curious about the extent to which AI communication coach companies spend time thinking about the links between Western colonialism and what counts as competent communication. Have they read about white language supremacy? Have they programmed their algorithms to respect and honor African American Vernacular English, or do their algorithms think of AAVE as “standard English with mistakes”? (To be fair, we could reasonably ask these questions of all communication instructors, human and non-human alike.)
Internal data on the extent to which AI speech coaches are biased is closely guarded; however, it is well documented that the error rates for facial recognition technology (on which AI speech coaches rely) are higher for people with darker skin and women. In her body of work that advocates for algorithmic justice, Joy Buolamwini describes the coded gaze as the way that racial biases are programmed into algorithmic systems. We follow Ruha Benjamin’s assertion that algorithmic racism is a feature, not a bug. In other words, large digital companies would like us to believe that their products are only incidentally racist and that they should not be held accountable for discriminatory results.
We also take issue with the growing surveillance capacities of digital products such as AI speech coaches. When students use an AI communication coach, they may not realize that several companies will own their video data forever. Almost all digital products conform to the principles of surveillance capitalism, an economic system built on the secret extraction and manipulation of human data. In the “Terms of Service” fine print for nearly all AI communication coaches, video data is collected, stored, and used to retrain its models. In other words, the AI communication coach company now owns your data. However, it would be a mistake to consider this a closed system. Rather than write established code from scratch, many AI speech coaches outsource to big companies. For example, one leading AI speech coach uses Amazon’s speech-to-text service and likely stores this data in Amazon’s data lake. Privacy at the Amazon data lake is ambiguous: “We will not access or use Your Content except as necessary to maintain or provide the Services.” When AI communication coaches are inevitably integrated into learning management systems such as Canvas, yet another company whose surveillant data privacy practices warrant scrutiny is introduced into the mix. Following the work of Simone Browne and Khiara Bridges, we know that surveillance practices disproportionately harm Black and Brown people and poor people.
In sum, we warn against the integration of AI communication coaches into higher education communication courses. Administrator interest will undoubtedly be piqued by AI speech coach campaigns promoting their products as effective, efficient, and equitable. We caution against this rise with deep skepticism regarding the utility of AI speech coaches and mounting concern regarding their capacities to harm. We urge university administrators to invest in human-centered communication instruction.
Dr. Rowland is the Maurer Professor of Performance & Communication Arts at St. Lawrence University and runs an instructor-facing website about integrating communication skills into courses across the curriculum called Speak to Engage. For a list of the students who collaborated on this essay, please refer to our open letter.