Hi Aaron - great post as always. I think that outside of academia there are many librarians working in this middle ground space, who are very visible within their own organisations. This is the bread and butter of the medical library world. We both teach and undertake searches within this space, supporting junior doctors, research offices, guideline creation, and clinical improvement projects. However, sadly we are also in a sector where publishing our work is not encouraged or actively supported. We likely to need to source support for undertaking research into the quality/validity of AI tools within our safety-first area of the library world, much of the research in this space is currently being undertaken by clinical staff without our input. Thanks for the musings :)
I can't help feeling that there are many valuable insights here but that - as you point out - researchers are going to use some of these tools with little knowledge (and without reading your article). If the library's role is discovery in the broad sense, not just limited to carrying out systematic reviews, where its expertise is acknowledged, how can the library demonstrate its relevance? Maybe your workshops, or for those who cannot attend them, some practical guidance on using Undermind, Consensus, and so on.
This post was directed to librarians who may underestimate the value of these tools.
How to demonstrate expertise (or even gain it) and be trusted is a different topic....
Frankly, I think most researchers worth the name, can generally tell if a search engine has high retrieval capability. They may be less aware of what happens under the hood, what weaknesses each tool has, something a librarian with the right expertise can help with....
Interesting post! I'm curious what you would define as a "difficult query" and in what disciplinary area(s) was that query in? I often find Google Scholar does not give me relevant results when searching in certain fields (like deeper inorganic chemistry queries). I think your post would benefit from more specifics.
Whether a question is “difficult” is a fairly deep question, and I have been thinking about it for a while.
I think one way to describe “difficult queries” is to define it by what it is not. So a broad query like “Is there an open access citation advantage” is not a difficulty query because there are many many possible relevant papers.
I find many librarians like to test with broad queries with many possible relevant papers. These tests are very poor stress tests even with conventional search much less modern search.
I also suspect, librarians are used to relatively poor relevancy ranking, so most searches if they can find say 5 out of the top 10 relevant (for a easy query where there are many possible matches) or they have to search through top 50 and/or do a lot of filtering/reformulating to find 1-2 relevant papers (where there are few), they consider it decent.
Of course, even if a query has only one possible relevant match, it can still be easy to match if all you need to do is to type in a common obvious keyword and it is retrieved, so having few possible relevant matches is at most a necessary but not sufficient condition. (Arguably it is not even a necessary condition if your task is to find all or most of the set).
Difficult queries are as you might expect, queries where the terminology is tricky (though this can be subjective), maybe the paper you are looking for is on a new area so everyone using different terms, maybe the paper you are looking for is in a different discipline than you expected using different terms so you miss the obvious keyword (this is subjective). Other factors are whether there are a lot of hard negatives or near misses to confuse the search etc.
If you look at formal Information Retrieval benchmarks they have many interesting ways to create challenging benchmarks.
BRIGHT - This is a reasoning intensive benchmark where you locate relevance by reasoning, not term overlap. It is not easy for lexical or even conventional semantic matching.
e.g. a coding question whose answer is a piece of documentation you can only locate by understanding the logic and syntax of the functions involved, with no shared vocabulary between question and answer.
I don’t think most users try this because it is futile for conventional search. But you will be amazed at how good the modern modern LLMs like ChatGPT or agentic search tools are starting to be at this.
2. BrowseComp Plus - An improved BrowseComp test with a fixed corpus, this is a test which is constructed so there is only ONE possible answer, but is extremely difficult to locate if you didnt already know the answer because it requires multiple constraints to be satisfied simultaneously (most of which is difficult to filter or search for)
e.g Find me a paper that has condition A, B, C, D, E and it is hard to filter even for one condition. This is made even harder in the BrowseComp Plus because the corpus is populated with hard negatives or near misses.
The hard queries I test generally fall into this type.
OBLIQ-Bench - clearest recent benchmarks for latent or implicit relevance, where the target document does not state the query terms directly.
This is the newest and possibly hardest benchmark. These include tip of the tongue queries, queries that are analogue seeking (eg query includes a math equation and query is to retrieve documents that use the same method to solve). Lucky there are descriptive queries that describe a certain “latent property”. e.g. search through tweets to show that subtly show a certain attitude.
Some of these can be adapted to literature review tasks…
If you look at all 3, they tend to work such that if you already have the correct document in hand you can verify it is correct . But going in the other direction is difficult. This is called the “asymmetry of verification” or "retrieval-verification asymmetry"
Also a common weakness as I mentioned in a footnote in this post is they tend to focus on “needle in haystack” problems, eg finding the ONE answer that is correct, not the more common task of trying to find ALL or most papers in your topic.
Hi Aaron, thanks for another great article. I have a question: Did you ever stumble across hallucinated references in undermind? I recently did and don't know if I just was to blue-eyed thinking that this couldn't happen with that tool or that this wasn't a thing until now.
Hallucinated in what sense? It can "misinterpret" a paper but it should never completely fabricate a paper
The way such tools is they use deterministic code/ non LLM methods to verify the reference is something from their index.
It is possible for them to have minor errors in citation eg slightly wrong pub year but that's more to do with the source index being unclean (eg index merging different versions of papers).
But if it's a true fabrication or ghost reference, the only way is if the index it is using Has been polluted somehow with fake papers...
Obviosly undermind took a chapter headline or something out of the book and fabricated the reference?
This is my conversation with undermind (in German but i figure, that should'nt be a problem). I asked, if it hallucinated the reference:
--- answer undermind ----
Nein — das scheint **nicht halluziniert** zu sein.
Ich habe es im Paper-Korpus gefunden als **[Mon08]**:
- *Il ricordo degli eroi: gli attori della memoria e l'Associazione nazionale alpini nel primo dopoguerra*
- **M. Mondini**
- **2008**
Aber: Die Metadaten sind **sehr dünn**:
- kein Abstract
- kein PDF
- Journal/Publikationsort nicht sauber erfasst
- Seitenangabe wirkt etwas roh
Also präziser gesagt: **Der Eintrag existiert im Korpus, aber er ist bibliographisch schwach abgesichert.** Wenn du willst, kann ich als Nächstes prüfen, **wo genau der Text erschienen ist** und ob es ein **Kapitel, Aufsatz oder Buchbeitrag** ist.
--- end undermind answer ----
Here I asked the follow up that I'd like to get the real reference or a source where that one came from:
Das passt sehr gut zu deinem verdächtigen Datensatz *Il ricordo degli eroi: gli attori della memoria e l'Associazione nazionale alpini nel primo dopoguerra*: Meine **Inference** ist, dass das **kein eigenständiger Aufsatz**, sondern **ein Kapitel- oder Abschnittstitel im Buch** ist, vermutlich beginnend auf **S. 105**. ([books.google.com](https://books.google.com/books/about/Alpini.html?id=7kEhAQAAIAAJ&utm_source=openai))
Hmm this looks like the newer Projects feature. I haven't used that as much as the og classic.
Even for the classic, I would say only the original report that is generated initially and the list of references is pretty much invulnerable. AKA if you mouse over a citation, and it shows a overlay with details of the paper, it is definitely something it found from the search index. Anything else, is vulnerable.
Yes, its the project thing. And actually it came from the original search report I launched. Not from the chat. It was one of the search results, it came up with. The overlay is there. With details. The doi links to a 404 on a bookshop webpage. So I guess undermind is reaching out into the wild, while searching?
Ah. okay, this is almost certainly a case where the index Undermind is using is outdated. The fact that there is an overlay means the LLM didn't make it up, but it came directly from the search index. But Undermind's search index (from Semantic Scholar) can have errors and link to outdated information.
To be clear, if the error is due to the index, the LLM is blameless and it isn't technically a LLM hallucination.
When you say, "friction is pedagogically useful and one could even argue better search results removes friction!" obviously, you know that this is backwards. Most people on campus would call us crazy if we told them: "The purpose of library databases is to frustrate and confuse undergraduates, forcing them to ask librarians for help, thereby giving us a chance to teach them important IL concepts." But I think that you are going to write more about this weirdness in another post ...
Indeed. I have actually discussed this with librarians who defend a somewhat watered down version of this and I will steel man that argument in the next post...
Hi Aaron - great post as always. I think that outside of academia there are many librarians working in this middle ground space, who are very visible within their own organisations. This is the bread and butter of the medical library world. We both teach and undertake searches within this space, supporting junior doctors, research offices, guideline creation, and clinical improvement projects. However, sadly we are also in a sector where publishing our work is not encouraged or actively supported. We likely to need to source support for undertaking research into the quality/validity of AI tools within our safety-first area of the library world, much of the research in this space is currently being undertaken by clinical staff without our input. Thanks for the musings :)
Yep. We librarians are researchers too.
I can't help feeling that there are many valuable insights here but that - as you point out - researchers are going to use some of these tools with little knowledge (and without reading your article). If the library's role is discovery in the broad sense, not just limited to carrying out systematic reviews, where its expertise is acknowledged, how can the library demonstrate its relevance? Maybe your workshops, or for those who cannot attend them, some practical guidance on using Undermind, Consensus, and so on.
This post was directed to librarians who may underestimate the value of these tools.
How to demonstrate expertise (or even gain it) and be trusted is a different topic....
Frankly, I think most researchers worth the name, can generally tell if a search engine has high retrieval capability. They may be less aware of what happens under the hood, what weaknesses each tool has, something a librarian with the right expertise can help with....
Interesting post! I'm curious what you would define as a "difficult query" and in what disciplinary area(s) was that query in? I often find Google Scholar does not give me relevant results when searching in certain fields (like deeper inorganic chemistry queries). I think your post would benefit from more specifics.
Whether a question is “difficult” is a fairly deep question, and I have been thinking about it for a while.
I think one way to describe “difficult queries” is to define it by what it is not. So a broad query like “Is there an open access citation advantage” is not a difficulty query because there are many many possible relevant papers.
I find many librarians like to test with broad queries with many possible relevant papers. These tests are very poor stress tests even with conventional search much less modern search.
I also suspect, librarians are used to relatively poor relevancy ranking, so most searches if they can find say 5 out of the top 10 relevant (for a easy query where there are many possible matches) or they have to search through top 50 and/or do a lot of filtering/reformulating to find 1-2 relevant papers (where there are few), they consider it decent.
Of course, even if a query has only one possible relevant match, it can still be easy to match if all you need to do is to type in a common obvious keyword and it is retrieved, so having few possible relevant matches is at most a necessary but not sufficient condition. (Arguably it is not even a necessary condition if your task is to find all or most of the set).
Difficult queries are as you might expect, queries where the terminology is tricky (though this can be subjective), maybe the paper you are looking for is on a new area so everyone using different terms, maybe the paper you are looking for is in a different discipline than you expected using different terms so you miss the obvious keyword (this is subjective). Other factors are whether there are a lot of hard negatives or near misses to confuse the search etc.
If you look at formal Information Retrieval benchmarks they have many interesting ways to create challenging benchmarks.
BRIGHT - This is a reasoning intensive benchmark where you locate relevance by reasoning, not term overlap. It is not easy for lexical or even conventional semantic matching.
e.g. a coding question whose answer is a piece of documentation you can only locate by understanding the logic and syntax of the functions involved, with no shared vocabulary between question and answer.
I don’t think most users try this because it is futile for conventional search. But you will be amazed at how good the modern modern LLMs like ChatGPT or agentic search tools are starting to be at this.
2. BrowseComp Plus - An improved BrowseComp test with a fixed corpus, this is a test which is constructed so there is only ONE possible answer, but is extremely difficult to locate if you didnt already know the answer because it requires multiple constraints to be satisfied simultaneously (most of which is difficult to filter or search for)
e.g Find me a paper that has condition A, B, C, D, E and it is hard to filter even for one condition. This is made even harder in the BrowseComp Plus because the corpus is populated with hard negatives or near misses.
The hard queries I test generally fall into this type.
OBLIQ-Bench - clearest recent benchmarks for latent or implicit relevance, where the target document does not state the query terms directly.
This is the newest and possibly hardest benchmark. These include tip of the tongue queries, queries that are analogue seeking (eg query includes a math equation and query is to retrieve documents that use the same method to solve). Lucky there are descriptive queries that describe a certain “latent property”. e.g. search through tweets to show that subtly show a certain attitude.
Some of these can be adapted to literature review tasks…
If you look at all 3, they tend to work such that if you already have the correct document in hand you can verify it is correct . But going in the other direction is difficult. This is called the “asymmetry of verification” or "retrieval-verification asymmetry"
Also a common weakness as I mentioned in a footnote in this post is they tend to focus on “needle in haystack” problems, eg finding the ONE answer that is correct, not the more common task of trying to find ALL or most papers in your topic.
Hmm, I definitely should devote a post on this…
Hi Aaron, thanks for another great article. I have a question: Did you ever stumble across hallucinated references in undermind? I recently did and don't know if I just was to blue-eyed thinking that this couldn't happen with that tool or that this wasn't a thing until now.
Hallucinated in what sense? It can "misinterpret" a paper but it should never completely fabricate a paper
The way such tools is they use deterministic code/ non LLM methods to verify the reference is something from their index.
It is possible for them to have minor errors in citation eg slightly wrong pub year but that's more to do with the source index being unclean (eg index merging different versions of papers).
But if it's a true fabrication or ghost reference, the only way is if the index it is using Has been polluted somehow with fake papers...
Obviosly undermind took a chapter headline or something out of the book and fabricated the reference?
This is my conversation with undermind (in German but i figure, that should'nt be a problem). I asked, if it hallucinated the reference:
--- answer undermind ----
Nein — das scheint **nicht halluziniert** zu sein.
Ich habe es im Paper-Korpus gefunden als **[Mon08]**:
- *Il ricordo degli eroi: gli attori della memoria e l'Associazione nazionale alpini nel primo dopoguerra*
- **M. Mondini**
- **2008**
Aber: Die Metadaten sind **sehr dünn**:
- kein Abstract
- kein PDF
- Journal/Publikationsort nicht sauber erfasst
- Seitenangabe wirkt etwas roh
Also präziser gesagt: **Der Eintrag existiert im Korpus, aber er ist bibliographisch schwach abgesichert.** Wenn du willst, kann ich als Nächstes prüfen, **wo genau der Text erschienen ist** und ob es ein **Kapitel, Aufsatz oder Buchbeitrag** ist.
--- end undermind answer ----
Here I asked the follow up that I'd like to get the real reference or a source where that one came from:
--- answer undermind ----
Ja — **es kommt im Buch vor**, und zwar sehr wahrscheinlich **im Inhaltsverzeichnis** von *Alpini: parole e immagini di un mito guerriero* von Marco Mondini. Google Books zeigt unter **Contents** den abgeschnittenen Eintrag **“… e l’Associazione nazionale Alpini nel primo … | 105”**. ([books.google.com](https://books.google.com/books/about/Alpini.html?id=7kEhAQAAIAAJ&utm_source=openai))
Das passt sehr gut zu deinem verdächtigen Datensatz *Il ricordo degli eroi: gli attori della memoria e l'Associazione nazionale alpini nel primo dopoguerra*: Meine **Inference** ist, dass das **kein eigenständiger Aufsatz**, sondern **ein Kapitel- oder Abschnittstitel im Buch** ist, vermutlich beginnend auf **S. 105**. ([books.google.com](https://books.google.com/books/about/Alpini.html?id=7kEhAQAAIAAJ&utm_source=openai))
Sicher belegt ist außerdem, dass das Buch selbst 2008 bei Laterza erschienen ist und 255/256 Seiten hat. ([books.google.com](https://books.google.com/books/about/Alpini.html?id=7kEhAQAAIAAJ&utm_source=openai))
--- end answer undermind ----
Hmm this looks like the newer Projects feature. I haven't used that as much as the og classic.
Even for the classic, I would say only the original report that is generated initially and the list of references is pretty much invulnerable. AKA if you mouse over a citation, and it shows a overlay with details of the paper, it is definitely something it found from the search index. Anything else, is vulnerable.
Yes, its the project thing. And actually it came from the original search report I launched. Not from the chat. It was one of the search results, it came up with. The overlay is there. With details. The doi links to a 404 on a bookshop webpage. So I guess undermind is reaching out into the wild, while searching?
Ah. okay, this is almost certainly a case where the index Undermind is using is outdated. The fact that there is an overlay means the LLM didn't make it up, but it came directly from the search index. But Undermind's search index (from Semantic Scholar) can have errors and link to outdated information.
To be clear, if the error is due to the index, the LLM is blameless and it isn't technically a LLM hallucination.
Thank you for the explanation!
When you say, "friction is pedagogically useful and one could even argue better search results removes friction!" obviously, you know that this is backwards. Most people on campus would call us crazy if we told them: "The purpose of library databases is to frustrate and confuse undergraduates, forcing them to ask librarians for help, thereby giving us a chance to teach them important IL concepts." But I think that you are going to write more about this weirdness in another post ...
Indeed. I have actually discussed this with librarians who defend a somewhat watered down version of this and I will steel man that argument in the next post...