Brilliant dissection of the workflow vs reasoning divide. The citation gap test really exposes how these tools are pattern-matchers rather than flexible thinkers, something that matters alot when venturing outside predefined templates. The speed-reliability tradeoff is intresting too becuase for most standarized lit reviews, I'd probably take Undermind's 8-minute workflow over 30-minute open-ended reasoning anyway.
While Asta's PaperFinder fails at doing both tasks, it supports follow-ups--so after you get the paper, you can say "Identify papers that are related to this paper that could have been cited but are not cited by it".
I see Elicit.com just launched their agentic platform with 6 Ai workflows. Haven't tried yet to see if they are similar to Scispace agents that can mix and match or just 6 fixed workflows
Brilliant dissection of the workflow vs reasoning divide. The citation gap test really exposes how these tools are pattern-matchers rather than flexible thinkers, something that matters alot when venturing outside predefined templates. The speed-reliability tradeoff is intresting too becuase for most standarized lit reviews, I'd probably take Undermind's 8-minute workflow over 30-minute open-ended reasoning anyway.
While Asta's PaperFinder fails at doing both tasks, it supports follow-ups--so after you get the paper, you can say "Identify papers that are related to this paper that could have been cited but are not cited by it".
https://asta.allen.ai/share/5cdd8677-e98a-448f-921c-5ad3fb9ac8d2
I actually know and give credit to fact that ai2 paper finder has a known item search mode but it's not enough?
Your example in the follow up shows it does not understand or do the task correctly.
Firstly it gives you recent papers e.g. 2019 papers which could not have been cited.
Secondly, I see no evidence it actually checked the reference list to confirm it wasn't cited. Based on the reasoning trace it wasn't.
I see Elicit.com just launched their agentic platform with 6 Ai workflows. Haven't tried yet to see if they are similar to Scispace agents that can mix and match or just 6 fixed workflows