AI Models and Collection Development in Health Sciences Libraries

 

Morrison, Mary

Portillo, I., & Carson, D. (2025). Making the most of Artificial Intelligence and Large Language Models to support collection development in health sciences libraries. Journal of the Medical Library Association : JMLA, 113(1), 92–93. https://doi.org/10.5195/jmla.2025.2079

Summary:

For the project, health sciences librarians at Chapman University in Irvine, Ca, evaluated the four AI models (ChatGPT 4.0, Google Gemini, Perplexity, and Microsoft Copilot) over the course of six months using two different prompts in an effort to streamline their work and aid efficiency. 

For the first prompt, each generative AI model was asked to produce a current eBook list of titles published over the last two years on topics related to pharmacy, communication sciences and disorders, physical assistant, and physical therapy. For the second prompt, the tools were asked to find current subject gaps in the collection and to create a list of recommended call numbers in specific ranges that needed attention. Each collection list was uploaded into the generative AI models using the Create List function found in Sierra which is an Integrated Library System. 

Afterwards, the results were assessed for “quality, accuracy, the presence of fabricated titles (often referred to as ‘hallucinations’) if references were provided, correct citation details and accurate LC call numbers” (Portillo & Carson, 2025). All of the models gave inconsistent responses. Only two of the AI models gave sources for the titles they created. While Copilot was the most accurate in generating titles, Gemini and ChatGPT had many hallucinations and incorrect information. Some tools gave more hallucinations on different days with the same prompt. Because of this, none of the generative AI models are currently adequate tools for collection development. 

The second prompt got better results with all models giving some type of “helpful analysis and accurate LC call numbers” (p. 93). The subject gaps identified by each model were different, but they all had reasoning as to what areas needed further development and the research generated was found to be useful by the researchers for use in their current collection development work. 

The findings of the research support the idea that LLMs are not ready to serve as “primary information retrieval systems” for library collections and are better used as supplementary tools for “analyzing the subject coverage of their collection, identifying subject gaps, and highlighting areas for health science programs that may not be as well represented in a library collection”

 (p. 93).

Evaluation: Overall, I found this article to be a brief and concise research endeavor for a very focused group (health sciences libraries). I appreciated that they used four different AI models to test the accuracy of their results from the two collection based prompts. It was especially interesting to see how each model fared and how they had similarities and differences. I wasn’t surprised by the findings. I find AI tools especially helpful for many tasks, but in tasks that require higher order thinking or multiple steps that require some degree of nuance, there is still a great deal of room for improvement. I feel that AI will improve rapidly and that soon it will be something that is more useful in increasing librarians’ efficiency and reducing their workload.