Category: Information Ethics

The Chatbot Will See You Now: Concerns about Artificial Intelligence and Medicine

Editor’s note: We welcome a guest blog post from Rebecca Kyser, Research and Instruction Librarian at Himmelfarb Health Sciences Library, The George Washington University.

Disclaimer: All opinions expressed in this post are the author’s own and do not represent her place of work.

As a medical librarian who specializes in medical misinformation and disinformation, I tend to have a healthy skepticism for cure-alls. So when I first heard about AI’s promises of automating workflows, eliminating repetitive tasks and curing disease, my first thought was to investigate exactly how true they were. It’s been around four years since then but I’ve emerged from this process to find myself firmly an AI skeptic, especially in regards to large language models (LLMs). And to my disappointment, I’ve found LLMs frequently cited as a solution to solve countless problems of medicine. 

With that in mind, I wanted to share some of my concerns about AI being integrated into medical technology. Not all of these issues are exclusive to medicine – many are found in any field that AI touches – but some of the consequences of AI behaving badly get far worse when people’s health is involved. 

AI is Inaccurate

At this point, almost everyone has heard about AI hallucinations: instances where AI makes up false information. Hallucinations are baked in the technology: there is no way of getting rid of them entirely 1. Rates of false answers vary from model to model 2, and the rate is often published by independent assessors, rather than the companies themselves. The best response to this problem is to lower the rate as much as possible.

However, when we’re talking about wrong answers regarding medical information, what threshold of inaccuracy do we find acceptable? Hallucination rates of 39.6% 2 may be acceptable when asking a bot about the weather, but would we consider that rate acceptable, or even a 10% hallucination rate, for a doctor prescribing you medicine? This isn’t to say humans don’t make medical errors – there’s plenty of scholarship on the topic and how to prevent it – but are their error rates comparable with a bot?

The error rate isn’t great at the moment. Multiple studies have shown chatbots make mistakes regarding medical queries: surgical errors leading to patient injury 3, undertriaging cases 4 and providing faulty medical advice to the public 5. Those mistakes aren’t small outliers either: ChatGPT undertriaged cases 52% of the time 4 and provided correct relevant conditions in less than 34.5% of cases5. 

There is the argument that as AI gets better (which is an assumption worth investigating), this will cease to be an issue. However, even if AI had an accuracy rate equal or less than humans, another problem arises: who can be held liable for medical error when the error comes from a machine? Is it the person operating the machine or the person who built it? Without these kinds of clearly defined policies, addressing mistakes with AI in medicine becomes a thorny business. 

Garbage in, Garbage Out

Most people know that LLM are trained off broad swaths of information online, but I don’t think we always consider exactly what that entails. Without having access to these models’ training data directly, we can only make educational guesses on what the training data entails, usually by looking at outputs and trying to reverse engineer their source. This is something I’ve done myself: if you have access to an AI that provides sources, ask it about a headline from Alex Jones’ infamous platform Infowars and watch it regurgitate it back to you.

A picture of a headline from Alex Jones Infowars website, showing Terminator-style robots with the headline Humanoid Soldiers Tested in Ukraine; Founder Eyes Contract To Patrol US Border
A picture of a headline from Alex Jones Infowars website, showing Terminator-style robots with the headline “Humanoid Soldiers Tested in Ukraine; Founder Eyes Contract To Patrol US Border”

There is an argument this is remedied by ensuring the model only works off high quality training data. The problem is, we often don’t know exactly what was in the training data to begin with. Even if a model claims to be based on entirely scholarly sources, we should understand all the nuances of what that means: are all the sources peer-reviewed? Does the training set have safeguards against poor-quality sources from predatory publishers? Are retracted publications excluded from the training data? Is the data set monitored for works that are retracted after they enter the training data?

I think the best way to demonstrate this problem is to examine one of the few cases we’ve seen under the hood for a LLM: Claude. Due to a lawsuit, a database was published containing books Claude was trained off of, which the public can access. 

One of the names that comes up in the training set is Graham Hancock. For those who don’t recognize the name, Graham Hancock is a pseudohistorian, who often argues that the work of ancient civilizations was actually built by aliens. His inclusion isn’t surprising – his work is incredibly popular and Claude debunks his theories if asked – but it does raise concerns about the training data for these models. Especially when there are more concerning inclusions, such as the work of Richard Lynn, who can also be found in the data set. 

Richard Lynn was a well known scientific racist, known for promoting pro-eugenic ideas up until his death in 2023. Why is a scientific racist’s work fed into this model?  If this model is being trained to answer questions, why does it have disproven scientific racism in it? You can’t argue its inclusion is due to popularity; unlike Hancock, only one of these publications is well known enough to even have its own Wikipedia page, and that page is mostly about its racist background 6.

I use Lynn as an example because of the history of scientific racism in medicine. Many medical AI attest to training only off medical publications or scholarly journals, but that does not mean those are free of false or harmful ideas. Lynn is an example of this: he published multiple works promoting scientific racism 7. How do we ensure this kind of work doesn’t end up in these models? And is anyone actually looking out for them?

Bias

Many people like to think machines aren’t biased, but that simply isn’t true. Machines are built by humans who contain a wide variety of biases, and those biases get filtered down into both our creations and the data we feed them. A really good example of this is from ProPublica’s investigation of recidivism calculators, which declared Black people at higher risk of offending than white people, even when the white person had a lengthier criminal record. 8 

This is also an issue in medicine. One of the most common mistakes I see in these machines when it comes to medicine is anchoring bias: the bias to assume your first assumption is right. For example, when I asked an AI model for medicine for a differential diagnosis for a patient experiencing chest pain, it suggested cardiac causes. However, when I asked the same question but stated the patient had a history of anxiety, the differential shifted entirely to psychological causes. 

What’s likely happening here has to do with the training data: these models are being fed on content like textbooks, case reports and other educational content. A classic case presentation often gives the reader the information they need to make a diagnosis but might leave out superfluous details that a patient might give in real life. Since the bot is unused to seeing those details outside of cases where they are important, it sometimes anchors on the wrong idea. This may explain why the accuracy rate fell so drastically when it came to doctors inputting information over patients: the doctors knew what information was clinically relevant, the patients did not. 5

No Free Lunch

While many LLMs are free to access at the moment, it seems unlikely that they will remain so. The power and processing power required to run these LLMs is significant and these companies are not non-profits. Which raises a question: how will these companies make money anyway? As the old economic saying goes, there is no free lunch. 

Currently, that’s a question without an answer: OpenAI, the owner of ChatGPT, is still running at a loss and isn’t expected to turn a profit until 2029. 9 They have announced planned revenue streams involving integrating their product into their software (think of an LLM embedded into Word or another consumer product)10, but I want to focus on three potential answers to this revenue problem in particular.

The first revenue option is subscriptions: most AI companies already offer fee-based options such as early access to newer versions of the model or custom-built models with training data specified by the user. Which raises an issue common in subscription models: the cost of access. The inflationary costs of scholarly journals and publications have long posed a problem for library budgets; what happens when we add AI subscriptions to the mix? Will access to the latest AI models widen inequities in education?

Another likely answer to the revenue problem is advertisements. There is a long history of web platforms turning to advertisers to generate revenue when users are reluctant to pay for a service. This might seem more obnoxious than harmful, but the stakes when advertising medical interventions are much higher than those in marketing new shoes. Marketing to doctors fertilized the seeds of the Opioid Crisis, and we should consider the ramifications of Open Evidence- an AI platform that advertises itself to medical professionals- potentially advertising some drugs over others (this idea is explored fully in an excellent post from the Krafty Librarian)

We also have to consider a revenue stream most commonly used by social media sites: the sale of user data to advertisers. Of course, there is a hiccup here: individually identifiable health information is protected under HIPAA and cannot be disclosed. But that doesn’t mean data can’t be used if it is anonymized.

Let’s try an example. In this hypothetical, let’s say there’s a small town in Southern Colorado that suffers from a pollution issue that leads to interstitial respiratory disease. Medical professionals in the area are shown to ask one AI company queries regarding this condition and how to treat it at a higher rate than the rest of the country. The AI Company then sells this data, which while anonymized, shows this community suffers from this issue.

Here are some potential buyers:

  • A supplement company coming out with a tonic to soothe chronic cough. They place advertisements for this area around the symptoms of interstitial respiratory disease and claim it is a “natural option” to treat the condition. Their sales boom.
  • An insurance provider for the area buys the data. Realizing they will be seeing a spike in claims for some treatments before those claims even enter their system, they amend clinics patients can go to in order to cut costs. Less people are able to access health care.
  • A pharmaceutical company takes out advertisements for their latest drug to treat chronic cough. Patients start requesting this drug at local medical providers after being told it’s “the best” despite there being a cheaper generic. 
  • A company that wants to cut corners with their air filters buys land in this area, knowing their additional pollution may not be noticed as quickly.

And so on, and so forth. While companies can already do some of these things by tracking data of searches via Google, the consolidation of demographic information under medical providers would make the task much easier. Are such privacy violations really worth saving a few minutes? 

I don’t wish to cast an entirely bleak view of artificial intelligence in medicine, despite my list above. There are potential use cases with narrow AI; AI trained to do one specific task. However, I recommended a healthy level of skepticism whenever interfacing with an AI product. “Move Fast and Break Things” might be the motto of Silicon Valley, but we cannot let breaking people become a standard practice in medicine.

Acknowledgements: Thank you to Brie McDonald for help editing this piece.

References

1. Nicola Jones. AI hallucinations can’t be stopped — but these techniques can limit their damage. Nature Web site. https://www.nature.com/articles/d41586-025-00068-5. Accessed 3/17, 2026.

2. Chelli M, Descamps J, Lavoué V, et al. Hallucination rates and reference accuracy of ChatGPT and bard for systematic reviews: Comparative analysis. J Med Internet Res. 2024;26:e53164. https://www.jmir.org/2024/1/e53164https://doi.org/10.2196/53164http://www.ncbi.nlm.nih.gov/pubmed/38776130. doi: 10.2196/53164.

3. Jaimi Dowdell, , Steve Stecklow, Chad Terhune, and, Rachael Levy. As AI enters the operating room, reports arise of botched surgeries and misidentified body parts. https://www.reuters.com/investigations/ai-enters-operating-room-reports-arise-botched-surgeries-misidentified-body-2026-02-09/. Updated 2026. Accessed 3/17, 2026.

4. Ramaswamy A, Tyagi A, Hugo H, et al. ChatGPT health performance in a structured test of triage recommendations. Nat Med. 2026. https://doi.org/10.1038/s41591-026-04297-7. doi: 10.1038/s41591-026-04297-7.

5. Bean AM, Payne RE, Parsons G, et al. Reliability of LLMs as medical assistants for the general public: A randomized preregistered study. Nat Med. 2026;32(2):609–615. https://doi.org/10.1038/s41591-025-04074-y. doi: 10.1038/s41591-025-04074-y.

6. IQ and the wealth of nations, Wikipedia Web site. https://en.wikipedia.org/wiki/IQ_and_the_Wealth_of_Nations. Updated 2026. Accessed 3/17, 2026.

7. Dan Samorodnitsky, Kevin Bird, et al. Journals that published richard lynn’s racist ‘research’ articles should retract them. https://www.statnews.com/2024/06/20/richard-lynn-racist-research-articles-journals-retractions/. Updated 2024. Accessed 3/17, 2026.

8. Julia Angwin, Jeff Larson, Surya Mattu and Lauren Kirchner. Machine bias. ProPublica. 2016. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.

9. Craig S. Smith. What large models cost you – there is no free AI lunch. Forbes Web site. https://www.forbes.com/sites/craigsmith/2023/09/08/what-large-models-cost-you–there-is-no-free-ai-lunch/. Updated 2023. Accessed 3/17, 2026.

10. Dave Smith. OpenAI says it plans to report stunning annual losses through 2028—and then turn wildly profitable just two years later. Fortune Web site. https://fortune.com/2025/11/12/openai-cash-burn-rate-annual-losses-2028-profitable-2030-financial-documents/. Updated 2025.

Refusing (to let go of) AI: An Ignatian Pedagogy for a Real AI Literacy

Editor’s note: We welcome a guest blog post from Maxwell Gray, Digital Scholarship Librarian at Marquette University.

My personal, professional relationship with generative AI (genAI) is complicated.

Like many academic librarians, I identify as a deeply anti-AI librarian, who believes AI, especially genAI, designed and built by technocratic oligarchs outside any democratic process represents a real crisis for workers, the environment and human cognition.

But I don’t believe AI refusal represents a productive alternative to the uncritical adoption of AI in higher education. Rhetorically, I don’t think AI refusal will persuade many audiences in academic libraries and higher education to approach AI critically or ethically. I actually worry AI refusal may accidentally cause some audiences to misunderstand the choice as being literally between either simplistic refusal or uncritical adoption.

As a digital scholarship librarian in Jesuit higher education, my response to genAI has been a pedagogy of engagement with the real, lived experiences of students, faculty and staff vis-à-vis genAI. I take this language of engagement and real, lived experiences from the tradition of Ignatian pedagogy where this language represents a “serious, down-to-earth engagement with the real” in the form of the “concrete, lived experience in all its diversity and particularity.”

This kind of pedagogy may take the form of direct experience and contact with the world through the senses and emotions. How do different genAI tools respond to the same prompts? How does the same genAI tool respond to the same prompts for different users? How do the different “styles” or “voices” of different genAI tools, or of the same genAI tool in response to different prompts, make us feel in our bodies, hearts and minds?

Or this kind of pedagogy may take the form of making connections between different varieties of knowledge, or between knowledge and action. How do different genAI use policies in different workplaces reflect different data privacy frameworks in different social contexts? How do problems of informants, confidentiality and reciprocity in anthropology resonate with problems of authorship, data privacy and intellectual property in relation to genAI? (These are real examples from my experience teaching professional graduate students and anthropology major capstone students this semester.)

Ultimately, this kind of pedagogy should take the form of reflecting on experience and knowledge to decide upon the best, most meaningful course of action in the world. This kind of reflection, often called discernment in the tradition of Ignatius and the Society of Jesus, produces a kind of interiority oriented toward personal transformation and social justice. Why do I choose to use genAI tools in the ways I do? How may I use, or not use, genAI tools differently in response to other desires and callings? (These are abstract examples from my experience leading professional development sessions for faculty and staff over the past two years in collaboration with colleagues at Marquette’s Center for Teaching and Learning.)

To be clear, I’m not saying anti-AI librarians who believe in AI refusal don’t often already practice similar pedagogies of engagement with the real world. I’m not saying genAI is inevitable or there is no alternative. Instead, I’m trying to share language and perspective I think are more productive for stopping the uncritical adoption of AI in academic libraries and higher education.

What may it mean to be anti-AI librarians who don’t believe in AI refusal? What may it mean to be anti-AI librarians who believe in serious engagement, with the realities of AI and the realities of our colleagues and patrons? Jerome Nadal, one of Ignatius’s early companions, observed about the Jesuits (as opposed to monastic orders like the Benedictines) that “the whole world is our home.” What may it mean to be anti-AI librarians for whom the whole world, including AI, is our home in this moment when AI, especially genAI, often represents real injustices and indignities?

In a key text for Jesuit higher education, Dean Brackley, S.J, envisions Jesuit colleges and universities as being called to a mission of proyección social. Brackley writes, “Social projection includes all those means by which the university communicates, or projects, knowledge beyond the campus to help shape the consciousness of the wider society.” When I reread Brackley in this moment, I hear him saying that being at home in the world must not mean becoming comfortable with the realities of injustice, but instead must mean promoting justice in the world.

In my pedagogy in and around the library, I have tried to share knowledge I have learned from information studies, media studies and digital humanities with colleagues and patrons from across the university to help shape and raise consciousness on campus of genAI and its injustices and indignities. Over time I have learned doing this work requires seriously engaging with the real interests of students, faculty and staff, and the most effective ways of connecting with them that address their real fears, attachments and desires around genAI.

To other anti-AI librarians in academic libraries, I propose, “let us say yes to who or what turns up” in our classrooms or workshops and for our students and colleagues, wherever they are on their personal AI literacy journeys. In this way, we may open educational contexts where real engagement and discernment can take place, whether in a one-shot for students or over a series of professional development sessions for faculty and staff.

The former Superior General of the Society of Jesus Adolfo Nicolás writes, “Depth of thought and imagination in the Ignatian tradition involves a profound engagement with the real, a refusal to let go until one goes beneath the surface.” What may it mean to be anti-AI librarians who “refuse to let go” of AI until we go beneath the surface of reality toward more critical futures of AI in academic libraries and higher education? What companions may we find to join us in this work? What converts may we inspire?

I Bought Slop: A Conversation on the Accidental Purchase of AI-Generated Material 

Editor’s note: This guest blog post is by Natasha Finnegan, Cataloging and Metadata Librarian, Nicole Kulp, Serials and Electronic Resources Librarian, Sara Wheatley, Acquisitions Specialist, and Emily Zerrenner, Research and Instructional Services Librarian, all at Salisbury University. Emily is a regular ACRLog contributor, but because this post heavily features perspectives beyond her own, we have chosen to use the ACRLog Guest account.  

This is a discussion between four librarians about Large Language Model- and/or AI-generated books. It was spurred by the influx of “AI-slop” purchased unknowingly by staff. 

How can you tell if a book is AI-generated? What does that even mean? 

Sara:  When receiving books, alarm bells start going off when a book has a specific “look” (usually the print-on-demand look & lack of author name).  Next, I flip through the book, I flip through every book, regardless of AI suspicion; for books that I suspect to be AI-generated, I look to see if there is any level of references, which they tend to lack.  Then I investigate the actual text and look for repetitive writing, bulleted lists, misspellings, etc. And of course, there are always the obvious AI-generated images and charts.  When a book meets these criteria, I seek a second opinion, usually from our cataloger.  Admittedly, sometimes I fail my perception check when it comes to AI books and they come back to me for review after some else suspects their AI status. 

Natasha:  From a cataloging perspective, we have already purchased the book by the time it gets in my hands. The first tip-off is usually that there is no record of the book on OCLC or very small holdings. That forces me to slow down and thoroughly examine the material in hand, rather than just checking the usual metadata to be sure I have the correct edition in hand. 

Most of the time, I do copy-cataloging; when I do original cataloging it is because it is a unique item or it is self-published. The real trick to discerning if it is AI-generated versus not-the-most-polished human-created work. 

For now, here are my biggest clues that something is AI-generated:   

  • Organization of the material  
    • a lot of summaries
    • feels like an internet page
    • bulleted lists (ah, irony)
  • Source of the information – few, if any references 
  • AI slop imagery 

Once I find enough evidence, I take it to our Acquisitions Librarian and the selector of the material, and we research the author. Key questions to answer: how often they are publishing whole books, any reviews, publisher credentials, and their other books’ ratings.  

What I don’t do: I rarely read the whole book, no one has time for that! I doubt anyone has actually reviewed the material inside except perhaps to remove the AI watermarks. Hopefully, my inspection is as far as anyone has to go. I also do not run it through any AI checkers, most of them are faulty. 

Emily:  As a selector and as the title alludes to, I am one of the librarians who bought AI-slop. After Sara and Natasha’s investigation, the book in question came to me. I found the book on a GOBI spotlight list for student wellness in community colleges. It was listed as published under Mango Press, which didn’t immediately alert me since AI-generated slop is often self-published. I thought it would be a good addition to our student health collection. In Google Books, the publisher is listed as “Da Vinci’s Designs.” As of December 2025, there were no reviews for it on Amazon, Google Books, or Goodreads. It is in 11 libraries, according to Worldcat.

Nicole:  For ebooks, the tells are hard to find. As there is no print preview for most titles, we cannot see errors or lack of references. So, we must rely on the very basic metadata our vendor provides. Some red flags may include lack of author information or that no other libraries have the material. Truthfully, we are not given enough information to tell at the purchase level. Once we click on the purchase button, there are no refunds. Something that would greatly improve our process would be a preview of all titles so that we could check for references and AI generated tables and images. Refunds would also be helpful! 

Why are AI-generated books an issue? Why should we care? 

Nicole:  AI-generated books could be, and very likely are, full of incorrect information. One example that comes to mind is a very inaccurate diagram model of the human body. A student worker pointed this out to us last semester. It was in a textbook, albeit not one assigned in her classes. How are students supposed to learn how to help themselves and others medically if we can’t trust textbooks to contain accurate anatomy diagrams?  Inaccurate information can and will cause people to harm themselves and others. Learning requires a set of facts to learn from. When you cheapen the materials with inaccurate information, how will anyone learn anything?

Sara: AI-generated books are an issue for a multitude of reasons.  Chief among them is that we cannot trust the information to be accurate (how can we when it is scraping all of the, often highly inaccurate, internet).  Secondly, we should not undermine real authors and researchers who put so much time and effort into their craft.  It’s ridiculous that a few pushes of some keys and several seconds can create something akin to what would take a human years to do. 

Natasha:  Librarians have the goal of providing resources that fulfill their patrons’ information needs. We have selectors that cull through the enormous offerings of the modern world and hopefully choose things that are helpful and interesting to our users. When I am selecting art books, I want a book that has good pictures and truthful information about the artist or movement. 

How can I, as a selector, trust the information inside is accurate? Because it has been created by someone who is hopefully passionate about the topic, passionate enough to correct mistakes and try to avoid errors. There are no texts that are 100% error-free, but human creators can learn.  

But AI cannot do that. It cannot be passionate about anything, there is no desire to make the material as accurate as possible. There is no thinking. The quality of generative AI is based on training materials, and it puts whatever appears to fit in the output. Once the AI-generated material is over 70% correct, I think most people will consider that to be good enough — but 70% is not good enough if it’s for vital topics like medical, law, or science. 

The fundamental question of whether we can trust the information is dependent on whether that formula happened to be correct. We should care, because AI doesn’t care. 

Emily:  To me, Generative AI is a solution to a problem we didn’t have. Librarians are in the business of trust, and how can we ask our patrons to trust us if the materials we’re buying are potentially inaccurate slop? I’m also really disappointed that GOBI would spotlight a poorly written, citationless AI-generated book on a community college list. Librarians don’t have time to check every single thing they’re buying for their collections – if it’s a community college with maybe one librarian holding down the fort, they certainly don’t. We need to hold vendors accountable for what they offer for purchase, and vendors need to hold publishers accountable. If one of those entities doesn’t, our academic standards are lowered. 

What do you do if you discover that you’ve bought an AI-generated book, despite your best efforts? 

Emily:  Well, what I DID do was despair about it. Tell all my AI-skeptic friends. Felt like the cow in the meme we’ve chosen as the cover photo for this post. Then had Sara return the book and vowed to try to do even more digging on the books I choose to spend our taxpayer money on.  

Nicole:  Since ebooks are non-returnable, I would likely suppress the record from appearing in search results after reviewing it to be sure it is AI and that the information is untrustworthy. As far as damage mitigation, it is too much to go back through all the titles that we own electronically, but I would suppress as individual titles are brought to my attention. For databases, I would consider severing the relationship for lack of control over UI and/or if I encountered too much inaccurate AI content. I am also concerned that OA books are potentially more susceptible to AI publishing mills. For this, I would review the material before activating or deactivating as necessary. 

Natasha:  Review the book: does it serve the purpose that led me to purchase it in the first place? If no, then I will attempt to return it. If they don’t allow a return, eat the cost and throw it in the recycling. Maybe complain to everyone who will listen and get in an argument with my pro-AI partner, depends on the mood. 

Sara:  Attempt to return it to the vendor.  If they refuse to return it, add to the AI Jail for use educating students about AI tells. 

Picture of our AI Jail where we store AI books that are remaining in our possession.  The sign on the side says “AI Jail: These books are not to be trusted.  Proceed with caution!” 

What can we do about our big vendors pushing AI-slop on us? 

Emily:  I mean, part of what we can do is what this post is doing. It’s unacceptable that GOBI spotlights AI-slop, or that Libby uses GenAI to run their book suggestion feature. We as their customers should tell them so. Keep the pressure on. Shame is a powerful tool (just saw that San Francisco Comic Con revoked their allowance of AI-generated art after backlash), and so is our spending power.  

Sara:  I have had to push back at two of our vendors (GOBI & Midwest) with varying degrees of success… (once GOBI took the book back, no questions asked, a second time GOBI took the book back but reminded me that it was technically against their return policy, and Midwest took a book back, but let me know that it was a one-time curtesy). GOBI stated in the fall that they had implemented an AI-generated tag.  We struggled to find this tag and found it buried in options when doing an advanced search. 

Natasha: I don’t necessarily think that big vendors are pushing AI-slop. It’s easiest for them to just sell whatever fits their particular algorithm. No humans involved means better bottom line. They certainly don’t want to add to their costs — they don’t want to add paying a human to do the boring work of checking an AI’s work (or reverse-centaur as Cory Doctorow discusses in his blog). So how to push back? Demand actual tags that work, request that vendors verify the publisher, have them provide the author’s rate of publications. That’s the least they can do, and probably the only thing worth their time. They likely don’t check the work visually when offering it to purchasers. 

Nicole:  We can push for greater transparency around AI incorporation and for more control over our user experience. What it will boil down to is the power of the purse. One library cancelling a database or refusing to buy from a repeat offender won’t do much, but if enough of us say no then it will squeeze out any profits they make from pushing AI. Also, we need to stop putting it on a pedestal. The higher the pedestal, the more incentive for vendors to push it.  

What happens when we can’t tell anymore or there isn’t a push for transparency? 

Emily:  Have a little scream about it. There are feelings wrapped up in this, and that’s important to acknowledge them in order to continue pushing for what we want and what our patrons deserve. I’m also going to go back to the book Imagination: A Manifesto by Ruha Benjamin. As I said in my previous book round up on ACRLog, it dares us to imagine a world without these techno-utopianists pretending they are going to save us. 

Sara:  Stare off into the void and wonder how we got here… But really, keep calling out the authors that utilize AI, refuse to purchase and send back AI books to the distributors.  And when the distributors get mad about returns, remind them that if there was some level of transparency, we wouldn’t have purchased the book, thus necessitating a return (or loss for them on items that have already been physically processed). At this point we have to hit them where it hurts, their wallets.  That’s the only thing they’re going to listen to. If they stop making money on AI-slop, maybe they’ll stop producing it. 

Nicole:  We are going to have to fact-check academic materials at such a level that it will be pretty much impossible to simply purchase a book. We will either need to spend insane time to overcome the “ease” that AI has pushed and/or only purchase books direct from trusted authors and publishers, which also goes against everything I stand for in terms of getting information disseminated.   

Natasha:  We are already there, but for now we just want to catch the slop. That’s all that librarians have ever tried to do, keep the good stuff and toss the bad. Don’t waste your time, don’t become paranoid. If we can’t tell anymore, then it isn’t necessarily slop. 

Why are you interested in this topic?   

Natasha:  I’m a creative, I write and make art and enjoy others’ efforts to make the world their own, to share their voice and emotions. It takes work to really speak in your own voice: learning from the things you like, practicing, failing, experiencing, and experimenting. I understand the urge to get a lift from technology, but when it makes choices for the creative, both the artist and the enjoyer lose the uniqueness of that effort and voice. In information science, being able to track and verify the information stream is critical to others building on our society’s discoveries. My job as an academic librarian requires me to support information literacy and thus, here I am. 

Nicole:  I care very deeply about making accurate and trustworthy information accessible to everyone.  Now that AI has become entrenched in academics, I am genuinely frightened about the future of learning when people are too lazy to look for accurate information, or even worse, are incapable of wading through the slop to determine what is factual.  By training students to rely on AI, we are missing critical steps in information literacy that will have rippling effects. AI is not a calculator or a mechanical pencil as some have compared it. It will upend all we know about thinking and learning by diminishing critical thought and generating confidence in inaccurate information. *Gestures around broadly. 

Sara:  Being anti-AI is the hill I will die on.  There are so many reasons why AI use is bad (I won’t deny that there are good applications of AI, but there is much more bad about it and that bad far outweighs the good).  Not only is it destroying the environment and our ability to think critically (or even at all), it is also taking opportunity away from individuals.  AI is trained on information that a human spent years creating, then takes seconds to spit something out that is often incorrect.  Half the point of doing anything is the journey to get there; any amount of effort or struggle put into anything makes the outcome that much greater.  Also, I’m tired of questioning everything I see every day, my anxiety cannot take this level of paranoia. 

Emily:  I consider myself very AI-savvy; I’ve been AI-critical in a variety of spaces like LOEX Fall Focus and the Generative AI in Libraries conference. A lot of emotions came up when I realized that I had, indeed, bought AI-slop: defensiveness, guilt, anger, and despair, to name a few. I’m supposed to be an expert in this – how did I get tricked into buying a book that said a whole lot of nothing, lacked citations, and had horrible illustrations?  I know that sounds incredibly dramatic, but as someone who has confidently said that GenAI is ruining the information landscape and making my job actively harder, it was a blow. I want to share this experience, though, so other librarians can see it’s getting easier and easier to fall into these traps. 

The illustration from chapter 2, “Eating for Energy and Wellness,” in College Student Health Guide by Jules Carson. It features poorly illustrated food, such as fish, cheese, fruits, and vegetables. The carrot in particular looks rather dubious.

Any final parting words?  

Natasha:  Most humans try to do the right thing, most people aren’t aware of the issues involved with AI, and a lot of people are struggling to make ends meet so it’s not surprising they turn to scammy methodologies of money-making. AI-slop creates mistrust of formerly reliable information-providing institutions. We can’t lose that trust, we need to verify the information coming into our libraries as best we can. Authors, publishers, and vendors need to step up their game. 

Sara:  I just hope that the end of the AI wasteland is coming soon; that enough people realize how harmful it is, and we can all collectively do better. 

Nicole: While I am a huge fan of all the ways technology has improved our lives, I do think we are at a point where the dangers of AI are overpowering benefits. Rather than advancing technology to help increase accessibility, it seems to be a money grab and a profit maximization scheme to the detriment of us all. 

Emily:  Have you bought AI slop? Do you have other thoughts about the topic we didn’t hit on? Feel free to comment below. 

References: 

Benjamin, R. (2025). Imagination: A manifesto. W. W. Norton & Company. 

Doctorow, C. (2025, December 5). Pluralistic: the reverse-centaur’s guide to criticizing AI. Retrieved January 13, 2026. https://pluralistic.net/2025/12/05/pop-that-bubble/ 

McCrary, Q. D. (2026, January). Are we ghosts in the machine? AI, agency, and  the future of libraries. The Journal of Academic Librarianship, 52(1). https://doi.org/10.1016/j.acalib.2025.103181 https://www.sciencedirect.com/science/article/pii/S0099133325001776?dgcid=raven_sd_via_email 

PimaLib_InfoIntegrity. (2025, March 7). Did a robot write this? Tips for identifying ebooks written by AI. Pima County Public Library Blogs. Retrieved January 13, 2026. https://www.library.pima.gov/blogs/post/did-a-robot-write-this-tips-for-identifying-ebooks-written-by-ai/  

SFCC Library. (2025, January 5). Misinformation and media literacy: how to spot AI-generated content. Retrieved January 13, 2026. https://libraryhelp.sfcc.edu/misinformation-literacy/spot-ai-generated-content 

Images: 

“AI Jail” 

Wheatley, S. (2026). AI jail.  

“Cow Ocean” Meme: 

Claudio Travesias Necotata Mousseigne. (2016, February 12). SOLO EN NECOCHEA!!! Facebook. https://www.facebook.com/travesiasnecotataclaudio/posts/pfbid02yRuavj8V959robBfRwqrn3RnmUY8pZAZHyqPcRvudaWMXMqHcZbVzEKUV24YksEvl 

Rose Johnson. (2025, November 26). This is so depressing I went and ate a whole block of cheese in protest. Facebook.com. https://www.facebook.com/groups/400135894490699/posts/1602251974279079/ 

treesforcities. (2025, November 18). Guys I can’t go the rest of my life asking if it’s AI or not. Threads. https://www.threads.com/@treesforcities/post/DRMPjXkgiWa/guys-i-cant-go-the-rest-of-my-life-asking-if-its-ai-or-not 

“Dubious Carrot” 

Carson, J. (2025). College Student Health Guide. Mango Publishing. 

OpenAlex and values-aligned tools

It is within our sphere of influence to learn about and use values-aligned infrastructure and leverage our positions as information professionals to teach and encourage others to use values-aligned infrastructure as well.

Two different kinds of bridges crossing a river. Photo by Katelyn G on Unsplash

Academia needs to untangle from corporate entities our ability to understand and evaluate scholarly activity. Companies like Elsevier and Clarivate have effectively embedded evaluation methods into academia that rely on their proprietary tools (Scopus and SciVal; Web of Science and Journal Citation Reports and InCites, respectively). Conveniently, these products also cast judgement on the “legitimacy” of scholarly publication venues and whether they are valuable as lines in a tenure portfolio and/or in library subscription decisions, which raises questions about conflict of interest. One would assume journals owned by Elsevier are rarely excluded from the Scopus corpus, for example.

The need to sever indexing and evaluation of scholarly activity from corporate control dovetails with the need to regain a degree of control over the entire scholarly publishing enterprise. One path might look like a significant shift toward venues that are owned-and-operated by academic institutions or scholarly societies or academic alliances as opposed to for-profit publishers, but a basic barrier to this shift is the inconsistent and incomplete indexing of such venues in scientific knowledge graphs like Scopus and Web of Science. In their recent article on the subject, Nazarovets et al. (2026) conclude with this suggestion: “…the uneven visibility of UJs [university journals] highlights a structural blind spot in global research assessment: unless more inclusive infrastructures like DOAJ and OpenAlex are more widely recognized and further improved in terms of comprehensiveness in coverage and metadata reliability, large parts of scholarly production will remain invisible even if a move is made away from using WoS [Web of Science] and Scopus.” This is just one possible benefit of investing in open infrastructure like OpenAlex (as recommended by UNESCO). Open infrastructure can be defined as free, foundational tools and resources upon which open science/scholarship can be conducted, shared, and explored.

For those unfamiliar with OpenAlex, it is essentially a large database of scholarly works and related metadata. OpenAlex focuses on comprehensive inclusion of scholarly works as opposed to “curation” of legitimate sources as practiced by proprietary services. It has a web interface and can be queried using an API, but at its core, it is a piece of infrastructure dedicated to the public domain via CC0 and actively maintained by a 501(c)3 nonprofit. Of course, relying on publisher-reported citation data may be a dubious proposition to begin with, but in the meantime, as we work to recognize and reward other types of research impact evidence, let’s at least try to use comprehensive citation data.

Is OpenAlex as good as its competitors? In my experience, it has usually been as good or better for most purposes, like generating a list of an authors’ publications inclusive of preprints, or charting an institution’s open access publishing trends over time. Most folks who have taken a deep dive seem to agree. Culbert et al. (2024) analyzed OpenAlex reference coverage in comparison to Web of Science and Scopus and found it to be “comparable” even before OpenAlex’s recent improvements, which included an addition of over 50 million new works. In a Katina review, Ho (2025) called OpenAlex “…a promising and reliable alternative to traditional subscription-based citation databases for researchers, university administrators, research institutions, and government funding bodies that are interested in research activities and potential collaborations.” Other scholarly discovery tools like Overton are using OpenAlex for foundational data.

Folks will need to try it for themselves to see if the usability holds up to their standards, but for those on the fence, consider how simply using such tools might be a tiny contribution to a different future for research infrastructure. Coordinators involved in the Barcelona Declaration on Open Research Information had this to say about the declaration’s commitment to supporting infrastructures for open research information:

“What we think is important for organizations is to take seriously their responsibility for supporting these infrastructures—which can only continue to exist, and develop further, if they are both used and supported. That includes financial support but could also involve participating in governance and in-kind efforts, e.g., contributing data and helping improve data quality. Regarding financial contributions, here again Sorbonne University is a great example—when they unsubscribed from Web of Science, they redirected part of that budget to supporting OpenAlex. In general, we advocate making investments in open infrastructure an integral part of institutional budgets.”

For many of us in academic libraries, we don’t have significant resources to invest or divest, although voting with our money is still worth considering if we have authorization to do so. However, what is certainly within our sphere of influence is to learn about and use values-aligned infrastructure and leverage our positions as information professionals to teach and encourage others to use values-aligned infrastructure as well. At my institution, we’ve begun with small projects like integrating OpenAlex into our local scholarly profile platform and working to add our institutional repository as a source of OpenAlex data. Whether it be OpenAlex or another values-aligned tool, we can vote with our usage as a small daily act of support and/or resistance. By teaching about these tools, separate small acts become collective power.

“We Couldn’t Generate an Answer for your Question”

Editor’s note: We welcome a guest blog post from Jay Singley, Document Delivery and Circulation Desk Manager at North Carolina School of Science and Mathematics.

In March 2025, Ex Libris unveiled their AI-powered Research Assistant tool for institutions using Summon. Within a week, Summon users reported error messages with specific search terms and topics. The first error reports shared in a listserv for Summon users contained “Tulsa race riot” and “Tulsa race massacre.” Test searches by reference librarians and systems librarians containing these search terms generated no results.

I work in an academic library in a public state university system serving high school students and have since turned on Research Assistant in our Summon Preview Environment (a beta testing environment hidden from users). I have begun testing terms and topics systematically using a methodology akin to Matthew Reidsma’s auditing of algorithms, which seeks to expose the implicit biases of opaque technological systems.

In addition to “Tulsa race riot” and “Tulsa race massacre,” my audit of “controversial” search terms conducted in March-April 2025 turned up error results for the following:

  • Genocide in Palestine
  • Gaza war
  • Rwandan genocide
  • Armenian genocide
  • Genocides across the world
  • History of genocides
  • lynching
  • lynching in the united states
  • lynchings in the united states
  • january 6
  • covid
  • covid data
  • COVID-19

Ex Libris responded to initial reports about “Tulsa race riot” and “Tulsa race massacre” error results with the following:

“This is due to safeguard policies enforced by our AI service provider to support ethical and responsible AI use. It’s not something the Primo or Summon applications control.

If a query does not return a result, please try different phrasing. For example, ‘tulsa black wall street’ will return results that are directly related to the race riots.”

Despite requests for more information in the same listserv, Ex Libris has not provided a comprehensive response to librarians’ concerns about who the third-party provider is, what the ethical and responsible AI use policies are, whether local control can be made available, what known search terms and topics are obstructed, and more. As one librarian aptly noted, academic users want and need access to information that may be blocked by AI use policies meant for the general public.

An essential note is that Summon’s Research Assistant is not limited to an institution’s catalog. Instead, the AI tool searches the entire Central Discovery Index of Ex Libris’ available records regardless of whether the material is actually available to the user or not. (As of June 2025, Ex Libris has provided an option to limit search results to a user’s institutional catalog.) When the AI generates an error message (or perhaps more aptly refuses to run a search or share results), it is not necessarily because the material is unavailable. Rather, the third party “safeguard” policies obstruct the search to begin with.

In running test searches, I also uncovered troubling instances of suppressed results. I recently saw the Oscar-winning documentary No Other Land, co-directed by Palestinian and Israeli filmmakers. I decided to ask Research Assistant about this film. Searches for most of the directors generated the same no results error messages. A search for one of the co-directors generated partial and inaccurate results. Research Assistant made several guesses as to what the film No Other Land was about and why it was important. Could this be an example of biased third-party policies preventing searches about this Palestinian film and filmmakers? Is it also possible that Research Assistant is accurately reflecting the lack of materials related to No Other Land available in Ex Libris’ entire Central Discovery Index and widespread censorship of this film (and more broadly Palestinian people, culture, and experiences) in the United States? Are there truly no academic sources talking about this film and these filmmakers, one of whom was recently attacked outside his own home by Israeli settlers? More research is needed to clarify why “controversial” searches return error messages, inaccurate results, or partial results. The current state of Research Assistant partial results and error notices for “controversial” topics warrants more inquiry.

I know with near certainty that if a student user at my workplace received an error message when searching for “Tulsa race massacre,” they would switch their topic and probably not tell me or their instructor about the error message. To my users, “we couldn’t generate an answer for your question” translates to “your topic is not worthy of pursuing—change it.”

Uncritically adopting AI tools in discovery systems will perpetuate, if not exacerbate, existing biases and suppression of minoritized people. Try this safer topic. Try this approved topic. Try this unobstructed topic.

Libraries, museums, and information sciences are a frontline in resisting the federal administration’s targeting of minoritized people as well as their whitewashing, disinforming, censoring, and defunding tactics. In the age of AI, our access to information is not spared from these attacks. We must uncover answers to the questions Safiya Noble, Joy Buolamwini, and others elicit: Who and what is safeguarded through AI tools in discovery searches? Whose safety and comfort are prioritized? At whose expense?