Systematically improving the visibility of your research on social media -Visual abstracts, Video abstracts , Plain language abstracts and more
In November 2019's Systematically improving the visibility of your research on social media - what would this look like? , I shared two initatives, one by a University Library and another by a Researcher/Publisher to try to experiment to improve the visibility of their research papers posted on Social Media.
Toby Green then with OECD systematically experimented with the Kudos platform to track which of his efforts resulted in more views and downloads of his paper.
As noted in Maximizing dissemination and engaging readers: The other 50% of an author's day: A case study part of the problem of trying to apply traditional web marketing techniques for tracking social media efforts or campaigns for academic papers is that you often did not have access to granular usage statistics. Journal articles published on publisher and aggregator platforms either didn't display any download statistics or if they did would often just show an cumulative figure which isn't sufficient if you want to see what promotional activities were most successful.
While some platforms are starting to provide such statistics for authors and even intergrate them with other platforms, a workaroud if such statistics weren't available to be to share the Kudos Landing page instead and use views of that as a proxy for the success of your efforts.
At the library level, I also wrote about University of Manchester Libraries' OpenAccess+ service. The service did not just autotweet papers from their Twitter account but tried some of the following, including an algorithm that leveraged on almetrics to try to identify twitter users who were likely interested in the topic of the paper they were tweeting (based on past history of tweeting articles in the same journal title).
Other things they did mostly automated via script included
Identifies and links to Open Access verion (via Unpaywall etc)
Identifies and links to available research data (via Scholix)
Summarises papers finding using Machine Learning (via Scholarcy)
Tags potentially interested account (see algorithm above)
One of the more interesting questions not explored in that post was do we have any evidence at all that any of these activities actually "work".
Take tweeting papers. Do we have any evidence tweeting papers helps with impact (however you measure it). Are there proven techniques that are known to work?
It seems to me when posting academic papers on social media for promotional purposes, there are 3 factors at play.
What content you post / how you post the message
When and how often you post the message
Who you can reach - aka the reach/number of followers of your account that is doing the sharing
The second point - when you post the message strikes me as something that isn't particular to academic content and the standard ways of figuring this out should suffice.
The third point is probably the most important but seems a difficult one to alter in the short term, so in this blog post we will focus on the first point (but see an idea I have at the end of the post), what content and how we should post.
As an aside, one idea I have seen floating around in papers and talks is the idea of "Academic SEO", the idea you can optimise your paper's title, your abstract with the magic number of words or keyword so it floats on top of keyword searches.
I must admit to be quite skeptical on whether anyone has really cracked this and the papers I have read that claim to show you how to do this are generally disappointing. But in this case let's assume the paper has been published so these things are set in stone.
What should you tweet?
Clearly a tweet promoting a academic paper would have all the basics, link to the paper, a good image, appropriate hash tags etc. But can you do more beyond posting the title, author and link to the paper?
As mentioned earlier, UoM service tweeted summaries generated by the Scholarcy service but there are other options. One could tweet either a traditional abstract or
Visual Abstract (also known as Graphical abstract)
Video Abstract
Plain Language abstract
to try to attract interest.
Sample Visual Abstract - Template
Producing such additional work isn't trivial (but some tools are starting to emerge to ease this task- see later), so libraries who are looking to help with value add services to improve visibility of their researcher's work might consider moving to support such tasks.
But do we even have evidence this helps? This blog post reviews some of the recent studies that have attempted to study this.
1. Does posting visual abtracts on Twitter help improve impact?
While tools like Scholarcy and Paper Digest (see Nature article on summarizers and Semantic Scholar's TLDR generater ) attempts to summarise findings of a paper, the output is generally in text. Is there evidence that posting visuals would have greater impact? Perhaps enticing more people to click through to read?
This is where the idea of a visual abstract comes in.
Doing a quick exploration around the topic suggests Visual abstracts (sometimes known as graphical abstracts) is something most commonly seen in STEM medical journals, particularly with journals from certain publishers or platforms such as BMJ, LWW and is not unknown to journals in Elsevier Journals etc.
It was first introduced by Dr. Andrew Ibrahim for Annals of Surgery in 2016 and today we see visual abstracts for some articles in journals such as NJEM, Chest Journal , Transplantation , American Journal of Nephrology and more. According to Dr Ibrahim
It's unclear to me who creates these visual abstracts. In some journals, I see submission guidelines for visual abstracts, where they can be submitted by authors along side their manuscripts (and in some cases it is stated such supplementary material is not peer reviewed).
I also wonder if in other cases, such visual abstracts might be created post publication for dissemination on social media by social media accounts of journals e.g. NEJM
An example of a visual abstract
Various journals have produced guidelines for the creation of such visual abstracts , for example here but clearly generating a visual abstract isn't trivial and currently they are rare and are done more for clinical trials and systematic reviews.
But do they work? If they "work" however "work" is defined one could imagine libraries providing a service helping to craft such visual abstracts or at least make it easy for such visual abstracts to be made,
In particular, Scholarcy is now experimenting with machine generated Visual abstracts!
Based off Mike Morrison's better poster format, Scholarly tries to identify the main finding and suitable figure to add to the template. I've played with an early version and while it is still raw , it looks promising as a starter tool, that you can built off it.
Studies on impact of visual abstracts
A Picture Is Worth a Thousand Views: A Triple Crossover Trial of Visual Abstracts to Examine Their Impact on Research Dissemination, conducts a pretty well designed experiment to study the impact of Visual Abstracts.
The key improvement of this paper over earlier papers on this topic is that instead of just comparing tweets with just the citation and tweets with the visual abstracts, for each paper three type of tweets are sent
Citation only - tweets included the study title, the authors, and author institutions
Key figure - tweets included the citation-only information along with a key figure from the article.
Visual abstract - tweets included the citation-only information along with a visual abstract created by the AJN visual abstract team.
The inclusion of "key figure" tweets was to study the impact of having any visual as opposed to just text.
The sample was 40 papers from American Journal of Nephrology (AJN) and tweeted between December 2018 and October 2019 from the AJN Twitter account (@AmJNephrol).
To reduce ordering effects each of the 40 papers were randomised into 1 of 6 groups, some groups would post tweets with the visual abstract first, then Key figure, then citation only, while other groups would do it in a different order.
It was found whether comparing between visual abstracts, key figure or citation only tweets, the group they were in (which determines order of tweets), did not significantly differ in various metrics (average number of impressions, engagements, retweets, and link clicks) by group, showing that ordering of the tweets sent among the 3 types had little impact.
So far so good.
But how did the 3 types compare to each other?
On first glance the results look good
"Visual abstract tweets had more than twice as many views as citation-only tweets (1351, SD 1053 vs 639, SD 343; P<.001) and nearly twice as many views as key figure tweets (1351, SD 1053 vs 732, SD 464; P<.001) (Table 1)."
Twitter also has an engagement score which includes "all the clicks anywhere on the tweet", including replies, retweets, clicks on links, hashtags, avatar, to expand the tweet and more.
While engagement rate is just number of engagements (likes, retweets, replies, follows and likes) divided by impressions (number of times users have seen the tweet on twitter)
In general, both engagement score and engagement rate for Visual abstracts tweets were signifiantly higher than the other two groups.
For example
"Visual abstracts had 5 times the engagement of citation-only tweets (P<.001) and more than 3.5 times the engagement of key figure tweets (P<.001)."
Edit: Similar results can be found in A Crossover Randomized Trial of Visual Abstracts Versus Plain-Text Tweets for Disseminating Orthopedics Research, where 10 manuscripts each from the Journal of Arthroplasty was randomly tweeted with visual abstracts and without. Tweets with visual abstracts had higher Twitter engagement scores after 7 days and 30 days.
All this sounds exciting. But ultimately, you could argue if a tweet did not lead the reader to click in through to the paper it did not do it's job.
But with engagement scores and rates (which do include link clicks among other metrics) much higher for tweets with visual abstracts , you would think this would be a given.
Surprising the answer is no.
"There was no significant difference in the number of link clicks for visual abstract tweets and key figure tweets (P=.35), or for visual abstract tweets and citation-only tweets (P=.35)."
The point here it seems is that while tweets with visual abstracts increases "engagements" (likes, retweets, views of the tweet) it does not help much in terms of link click-throughs
As noted by the authors, this is because the click link rates of visual abstracts is very low 0.6% , though it may be due to the nature of social media. (See also a recent study of bit.ly links from Twitter to published papers which showed also low click rates)
"that although social media can boost article exposure, that exposure is shallow. Social media can convey a message quickly, but it rarely pulls viewers in for deeper consideration. Additionally, many viewers may be more interested in quickly viewing and evaluating the key elements of a paper rather than clicking through to the article link, which is often hidden behind a paywall."
I am no expert in altmetrics but I think this has been the general story of over a decade of altmetrics research that social media reach does not usually translate to traditional scholarly impact, probably because the tweets on papers that catch the fancy of people on Twitter (most of whom are not academics) generally do not correlate with academic concepts of impact and merit. It's a different audience.
Is use of twitter in general correlated or causes higher citations?
The authors try to argue that a low click rate isn't fatal and cites various studies that claim to show tweets are associated with higher citations (which often is what researchers are ultimately aiming for).
Personally in my humble opinion correlational type studies generally can be treated with suspicion and my sense of the literature is on the balance they don't even show correlations between tweets and citations in most studies.
What are left are the few randomized trials trying to test this effect.
Randomized Controlled Trial of Social Media: Effect of Increased Intensity of the Intervention (no effect, was follow up to earlier study which also had no effect on 30 day visits to website)
Effect of Promotion Via Social Media on Access of Articles in an Academic Medical Journal: A Randomized Controlled Trial. (has effect on 30 day visits to website)
To tweet or not to tweet, that is the question: A randomized trial of Twitter effects in medical education (no effect)
These studies generally measure improvement in website vists etc in a 30 day window/altmetrics. I would argue even if an effect was found (usually wasn't), it's unclear how that would translate to citation count differences.
For randomised control trials that tries to compare effect on citation rate as opposed to something like website visits , I found only two.
Twitter promotion predicts citation rates of cardiovascular articles: a preliminary analysis from the ESC Journals Randomized Study (preliminary, has effect)
Does Tweeting Improve Citations? One-Year Results from the TSSMN Prospective Randomized Trial. (has effect)
Side note : While doing a quick look through, it seems to me the cardiologists seem far ahead of the curve trying to figure out social media in academia. Beyond various attempts with Randomised control trials to measure the effect, there are pieces like Social media and citations: what do cardiologists need to know? and A Cardiothoracic Surgeon’s Playbook for Social Media and Digital Scholarship. Though none of it is ground breaking to a Twitter veteran, they might still be helpful for thos new to Twitter.
I haven't looked closely at these papers but most papers seem to show no effect with the exception of the last one. But that paper has been seriously questioned by Phil Davis in a series of posts on the Scholarly Kitchen. Including here, here followed by the author's response and a final reply by Phil Davis
I think it's fair to say the findings of this paper is somewhat under the cloud
All in all, my tenative assessment is that even if tweeting visual abstracts helps with social media engagement (which seems correct based on my intitution ), we don't have much evidence they will lead to higher citations. Of course that might be okay, if it isn't your main purpose.
Visual abstracts vs plain language abstracts vs Video abstracts vs published abstract
Interestingly visual abstracts aren't the only things you can try to produce as alternatives to the traditional abstract. You can also do
Plain language abstracts
Video abstracts.
Plain language abstracts are
While video abstracts are video versions of the normal abstracts
So how do these four types of abstracts - traditional, plain language, visual and abstract stack up against one another?
In - Video abstracts and plain language summaries are more effective than graphical abstracts and published abstracts this was studied.
Two papers were chosen which had all 4 types of abstracts and users were randomly assigned to 1 of 8 groups.
From paper - https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0224697
Their careers "science, science-related, non-science, or undergraduate" was surveyed and in addition they were surveyed on various things including
a 6-question quiz (1 Multiple choice, 5 True/False) to determine if they understood the abstract/paper they saw
Questions on how much they enjoyed the abstract, whether they thought they understood the abstract and whether they would want to see more of such abstracts
Their "preference for getting scientific information via written summaries, graphics/infographics, videos, audio sources like podcasts, and reading the original research paper"
Interestingly
"videos had the highest scores for comprehension, understanding, enjoyment, and desired updates (Median(M)>4 of 6, M>3 of 4, M>3 of 4, M>2 of 4 for videos respectively) "
Like the authors I find this surprising, even without taking into account that in the survey most respondents stated they preferred getting information in the written form.
Plain language summaries did next best and often had equal scores to videos except videos were generally as good if not better except in the particular case of comprehension of science-related participants for one of the papers.
There was in general little significant differences between the groups of users by career.
The papers goes into detail with further analysis (eg noting significant differences in the results between the two papers studied) and I suspect the findings here would change depending on the type of paper used and the amount of background knowledge needed. I personally found one of the papers more difficult to understand than the other which would probably affect the type of abstract I would need
Conclusion and some thoughts
What works for social media for promotion of academic content has a lot more unknowns than knowns. A lot of the existing research also seems heavily focused on medical articles, it would be interesting to see if the results extend to other fields like Business, Social Sciences.
As a long time Twitter user who uses it regularly for professional purposes, my gut feel is that good summaries of papers are greatly appreciated.
When I am in the mood and start spamming Twitter chains summarising interesting papers (on Library Science, Information Science), I generally get good responses with some followers even telling me they value such contributions greatly. As such, I am not surprised adding a visual abstract to a Twitter helps a ton with engagements in terms of retweets and likes.
The trouble is, my sense is most people will be satisified with my interpretion of the paper (which might be biased or wrong!) and maybe only the rare person would be intrigued enough to follow up and read the full paper and my twitter following has generally grown to encompass a pretty niche audience of librarians, publishers, bibliometricians who are inclined towards academics!
This probably explains why in general it is hard to show the impact of tweeting on link click throughs, much less citations!
That said, the pathway to lead someone to get attracted to, read the abstract, download the paper and finally cite it is a complicated story and who knows how many who do not immediately click in to read the paper from the Tweet eventually come across it again and it sparks recognition , leading to downloads, reading and citing.
Also sometimes your purpose is not to be cited, but to be read or even be known which is the case for industry and practioners.
My gut is beyond what you tweet, who you tweet to, or can reach is probably by far the number one most important factor.
And this is something that the library cannot do on the behalf of the researcher. An experienced person who is successful at using Twitter for professional purposes can give tips on what works to build one's network but it cannot be something that is built over night (and the social network used might need to vary depending on where one's community is at).
University of Manchester Libraries' OpenAccess+ service's attempt to bypass this problem is to try to identify twitter users who might likely be interested in the topic (identified by altmetrics to find top users who have tweeted papers in the same journal in the past) and tweeting at them - a intriguing idea.
A refinment of the idea I think is instead of using tweets of articles in the same journal titles as a critera one should identify who has tweeted related papers instead. Say identify 50 related papers and see if anyone tweeted at least 2-3 of them?
So perhaps some sort of similarity match to researcher profiles. After all, many articles in the same journal are on radically different topics, Or perhaps try to find related authors first who have published related papers and try to identify their social media accounts (via wikidata, ORCID?). But i suppose the data on such items is quite sparse currently.
Using techniques used by new emerging tools like Connected papers and other similar tools it might be interesting to see if it provides better targetting.
Think of it as a recommender system that actively looks for users who have already expressed their interest on Twitter!