Comparative review on Primo Research Assistant, Scopus AI, Web of Science Research Assistant and a explainer for AI search for librarians

May was a busy month for me in terms of output.

[Article] Comparative review of Primo Research Assistant, Scopus AI and Web of Science Research Output

First, I had two pieces of work published in the Katina Magazine that I am quite proud of.

First, a comparative review of Primo Research Assistant, Scopus AI, and Web of Science Research Assistant—written by yours truly—was published.

These are three of the most commonly used academic search tools in the library world, so I was honoured to be invited to write the piece.

Summon Research Assistant was released just as I was finishing the article. From a quick glance, it appears functionally equivalent to Primo Research Assistant.

This was one of the most technically complex pieces I’ve written. The original draft was more than twice as long. Eventually, with the help of editors and copyeditors, we decided to split the article into two parts. The first part focuses solely on how these three systems generate direct answers using Retrieval-Augmented Generation (RAG).

A second part will be forthcoming, comparing other features such as finding seminal papers, topic maps, and top experts.

[Article] A "explainer" of AI search for librarians

Even after splitting the review into two, the writing was still too long and unwieldy because I kept digressing into technical explanations.

Eventually, the editors had the brilliant idea of pulling out all the technical concept detours and publishing them separately as an “explainer” piece titled A Librarian’s Guide to AI in Academic Search Tools. 

To be honest, when I first saw the title, I thought it was a bit overreaching. Understanding “AI”—even within just the field of information retrieval—is no easy task. I’m not even sure I fully understand it myself.

I expanded the piece a little to smooth out the rough edges (remember, it started as essentially footnotes to the comparative review). It covers the following topics:

1. Constructing an Answer with Retrieval Augmented Generation (RAG)

2. Understanding (Vector) Embedding Search

3. Why the Use of Embeddings in Retrieval Reduces Interpretability

4. Embedding Search in Practice

5. Why Embedding Search Leads to Less Reproducible Results

6. Reranking with Embedding Search

7. Hybrid Search and Rerankers

In some ways, writing this piece stressed me out even more. Leaving aside imposter syndrome (I’m self-taught!), I struggled to strike the right balance between conciseness, technical accuracy, and accessibility for librarians with no background in information retrieval.

A “fun” game I played was pasting chunks of text into frontier LLMs like o3 and Gemini 2.5 Pro and asking them to critique my writing. They were merciless—and made it clear how much expertise is required to be both concise and technically accurate! In case you’re worried about hallucinations, I’ve found that top LLMs are generally solid on the fundamentals of information retrieval—likely because much of the content is open access and there's a wealth of teaching material available.

In the end, I came to a conclusion: when writing technical explanations, you can only really choose two out of three of the following:

a) Concise

b) Comprehensive and Accurate

c) Comprehensibility to laypersons

I chose to sacrifice (b) in favour of being concise and comprehensible—even to librarians without a background in AI or information retrieval.

I’m not sure I succeeded. Even now, I’m unhappy with the opening sentence and I think should have changed some of the examples and analogies I used in the piece. I also really itch to revise and add more sections.

Still, even if the piece isn’t 100% technically precise, I believe it points in approximately the right direction for any librarian who wants to understand these emerging tools.

[ADV]  Want to Go Deeper into AI and Information Retrieval?

As I write this, I’ve just wrapped up my 1.5-hour “Master Class” – Understanding the Fundamentals of AI in Academic Search. In total, 127 librarians and researchers from around the world registered—thank you for your support!

But 1.5 hours barely scratches the surface.

That’s why I’ve teamed up with my colleague, Senior Librarian Bella Ratmelia, to offer a more comprehensive course titled AI-Powered Search in Libraries: A Crash Course on Understanding the Fundamentals for Library Professionals.

It will be conducted online as part of FSCI 2025 (FORCE11 Scholarly Communication Institute in partnership with UCLA Library) from July 22–24, 6-9 pm (Pacific Time) in three sessions of three hours each.

Bella and I co-designed the course using sound pedagogical practices and plenty of interactive activities to help you build your intuition for technical concepts like embeddings, LLMs, and RAG. The extra time will also allow us to experiment with AI academic search tools. Unlike my previous talks, we’ll also dedicate an entire session to testing methodology. No coding knowledge is required.

I’m excited to have more time to share and learn together. If you’re interested, please register here.

Scholarships are available for participants from the Global South, but applications must be submitted by June 6, 2025.

[Recording] Playing Devil’s Advocate on AI Search Engines

This month, I was invited to give several talks. Some weren’t recorded, and others were private sessions.

However, my keynote at the Librarian Futures Virtual Summit (hosted by Technology from Sage) was recorded and is available for public viewing.

In this talk, I took a different approach—playing devil’s advocate on the topic of AI search. This is a technique I’ve used before in my career, blogging about then new developmenets like citation based mapping services (e.g. Connected papers), Institutional Repositories, Web Scale Discovery Services (e.g. Summon), mobile related library services and more in the past.

I actually have content for a longer version of this talk, which I’ll likely revisit in a future blog post.

Conclusion 

I’ve been exploring information retrieval and AI for almost three years now. This year, I decided to try sharing what I’ve learned in a more structured and deliberate way. I hope my work helps librarians better navigate and understand this fascinating—if sometimes bewildering—topic.

Text has been copyedited with the help of gpt4o.