Elsevier Data Fetcher - official free tool to do more with Elsevier APIs such as Scopus without coding
In recent years, Elsevier has provided relatively good access to their content via APIs, whether it be via the Scopus API for metadata including bibliometrics or even full-text via Sciencedirect API as long as you are an institutional customer, typically at no additional fee.
My suspicion is this feature isn't that well known even among librarians. But even if you were aware of it, you might still find it difficult to leverage APIs because this typically requires some amount of coding. This has changed with Elsevier releasing a free tool called Data Fetcher that provides a GUI so that you can extract data from various Elsevier APIs without coding. This is how it is described.
The API Query tool, affectionately known as the Data Fetcher for its ability to fetch data from Scopus, PlumX, SciVal, and other APIs, was built in-house by an inspired Elsevier employee who wanted to help his clients more easily generate and analyze vast swaths of API data. In addition, the tool supported and continues to support analytical and intelligence use cases with needs beyond what's available in the .com interfaces of Elsevier's Research Products, but that depends on the data available from those products. The menu-driven interface of the tool makes it easy to use and serves both casual users without the time or background to code as well as advanced coders and scripters not wishing to re-invent that wheel.
I first learnt about this tool via a Webinar, which I highly recommend you watch to see how some libraries are using this tool.
As the tool's page says, it enables you to handle use cases beyond what is available using the respectively Elsevier web interfaces.
Besides the potential to go beyond the download limits of the web interfaces of Scopus and Scival (the Scopus API will download in batches until all the data is downloaded subject to quotas of course), there are other use cases.
Extract data together from multiple Elsevier sources using multiple APIs - Scopus + Scival +PlumX.
One major use case with this tool is that you can combine data from multiple Elsevier APIs. In my specific case, you can combine data extracted from Scopus together with Scival (which we have a subscription) as well as from PlumX which offers Altmetrics.
Imagine this common use, you are trying to use Scopus to extraction citation metrics for a certain paper.
On the Scopus details page, you see additional details like the following.
Scopus document details page
Some of this information is useful, particularly additional information of altmetrics from PlumX (particularly if the item is so new there aren't many traditional citations yet).
But when you try to export this information into csv you see that only the following limited fields are available.
Scopus Export document settings
There are all the traditional bibliographic and traditional citation fields you can export but where's the altmetrics from PlumX?
It turns out that you can only export this information using PlumX APIs and Elsevier Data Fetcher allows you to combine the use of Scopus API and PlumX metrics API to extract data from both, yielding you a spreadsheet combining Scopus and PlumX data together.
My first thought was this doesn't help me since my institution does not subscribe to Plumx Analytics. But it turns out that PlumX Metrics API is available for all API users with an active Scopus subscription!
Not all Elsevier APIs are free - Scival API, Embase API are only available if your institution has access to them!
But how does one use Data Fetcher?
Preliminary steps for Using Data Fetcher
As at time of writing (June 2023), you can't just download and use Data Fetcher from the webpage as the zip file is password protected. You need to write in to datafetchersupport@elsevier.com to request the password.
Once you get the password, you can unzip the file and extract all the files on-top your computer. No installation is needed.
Obviously, you will need to have a Scopus API Key, which you can get from the Elsevier Developer Portal if your institution has access. Check with your librarian (not me!) if you have problems here.
Elsevier API Keys
In some instances, you may also require an institution token on top of the API key, if you intend to use the API outside your Campus IP range. Please email Elsevier support if you need this.
Once you have these pieces of information ready, you can now start using Data Fetcher. Run the program and enter the API key and Institutional token if necessary under "Credentials".
The next step is to set up the query. The nice thing is it uses the exact same Scopus search syntax, so you can test it first in Scopus before entering it here.
The example query I am using involves querying for all documents with the affiliation of my institution and PubYear from 2020 to 2022.
Scopus Query in Data Fetcher
Note : You can also use ScienceDirect API, by turning on the use_Sd mode to yes, this will override the default Scopus query.
There are tons of features and setting you can touch in Data Fetcher, but there's a high chance you will want to touch Output->fields.
This is where the magic happens,
Adding additional fields to the output in Data Fetcher
Take your time to play around with the fields, in my example above, I added PlumX metrics. This is free for all Scopus customers I believe.
Some of the fields I added to output
Above shows the interface after I added some of the output fields. You can see besides PlumX Metrics, I added SciVal output fields. The latter is available only if your institution has access to Scival.
There are various other options you can change, but for my purposes this is the meat of it. The extraction led to 2301 documents, and it was done in 20 minutes.
Below shows the output file in the output folder. I also screen captured some of the fields of interest, but it's basically the same output you get from Scopus enhanced with additional fields from PlumX and Scival.
Sample output - I only screen captured the columns of interest (click to expand)
Other functionality of Data Fetcher
What I showed you was just the tip of the iceberg on what Data Fetcher can do.
Querying by list of IDS
Another use case involves uploading a file of IDs, in the example below DOIs, so the search is based on those IDs. While you can do this in the Scopus web interface, this will break once you have too many IDs.
Using a file of IDs to query in Data Fetcher
You are not limited to just DOIs. But you can list Scopus-ID, EID etc (see below)
Some IDs you can use in Data Fetcher
Creating Co-networks and Topic Models
While querying you can also ask Data Fetcher to create network files and run Topic Models (plugins that are included by default), if you turn the settings to Yes.
Co-network and Topic Models turned on
If they are turned on, you will find appropriate files available in the output folder
Sample output file from Data Fetcher - Conetwork plugin turned on
VOSviewer co-author network is based on author strings, using Scopus data fetcher , this "API enabled" network can be more disambiguated using IDs(6) pic.twitter.com/fU3Gx51Nbt
— Aaron Tay (@aarontay) May 13, 2023
The outputs you can get from Data Fetcher are really flexible.
See for example the following section from the manual.
In particular, you can even do analysis by authorship positions from authors.txt and do a analysis by authorship position.
One use case, analysis by authorship positions! I can imagine this has several real use cases here (4) pic.twitter.com/pcuCLD8ScM
— Aaron Tay (@aarontay) May 13, 2023
Conclusion
Even with this long blog post, I feel I have barely scratched the surface of what this tool can do. Have fun playing!
