The SCOOP: The Metric System – Measuring Impact in Scholarly Communication

A Brief History of Metrics: From the Journal Impact Factor to Altmetrics

Ask an author why he or she has published their work in an academic journal, and one of the common responses you will hear is that they want to establish themselves as an expert in their field. Their reputation as an expert, and by extension the value attributed to their published work, is usually tied to where that article was published, or more critically what rank that journal holds among other journals in the field. Journal ranking calculated according to a journal’s impact factor has historically been the predominant metric for measuring the impact and quality of scholarship. However, reliance upon this flawed means of measuring the importance of one’s contribution to the field may have contributed to some of the problems with the current system of scholarly publishing.

With the growth of alternative means of disseminating scholarly works (e.g., open access publishing and pre-print servers) and online availability of related scholarly objects (e.g., presentation slides and data sets), alternative means of measuring impact were needed. In response, the field of altmetrics was born. Altmetrics measure scholarly impact in the online environment in which scholarly dissemination now largely lives. Altmetrics counts citations as well as hits, mentions, downloads, and tweets from a variety of sources across publishing, academia, and social media. Parallel to the rise of altmetrics and measuring the impact of individual pieces of scholarship came the growing need for tools to help authors to improve identification and linking of academic works to themselves as their creators. Below are some of these tools and resources to better measure the impact of scholarly works and improve linking of scholarly works with their creators:

Tools for Measuring Impact and Assuring Author Identity

Metrics Toolkit – http://www.metrics-toolkit.org/resources/

The Metrics Toolkit provides information about many available research metrics across disciplines, including how each metric is calculated, where you can find it, and how each should (and should not) be applied. You’ll also find examples of how to use metrics in grant applications, CVs, and promotion dossiers. Through the Toolkit, you can browse the metrics you want to learn more about or you can choose metrics that will be best for your use case by filtering by discipline, research output, and impact type categories.

ORCID – https://orcid.org/

If you have ever published an article, participated in grant-funded research, written a blog, posted a dataset online, or created any other myriad of scholarly objects, you will want to create a free profile on ORCID and obtain your unique 16-digit identifier. ORCID (Open Researcher and Contribution ID) aims to solve the name ambiguity problem in scholarly communication by creating a registry of persistent unique identifiers for individual researchers and an open and transparent linking mechanism between ORCID, other ID schemes, and research objects such as publications, grants, and patents. Increasingly, grant funders and publishers are requiring grantees and authors to have an ORCID. (For an example profile, please see my ORCID page at https://orcid.org/0000-0003-2314-5264)

ImpactStory Profiles – https://profiles.impactstory.org

Founded by the researchers who coined the term “altmetrics,” ImpactStory is an open-source website that helps researchers explore and share the online impact of their research. Users create an ImpactStory Profile with either a Twitter account or an ORCID number, and in return, ImpactStory gathers data from across the web to tell a story about an individual’s scholarly footprint.

Publish or Perish – https://harzing.com/resources/publish-or-perish

A freely downloadable software application, Publish or Perish retrieves and analyzes academic citations based on search parameters entered by the user. It uses a variety of data sources, including Google Scholar and Microsoft Academic Search, to obtain raw citations, then analyzes the data and presents metrics such as citation counts, the H-index and its variants, and the g-index. The on-screen display of results can be saved to a variety of output formats for further reference and analysis.

Recommended Further Reading:

• The ACRL Scholarly Communications Toolkit includes a section on “Measuring Impact,” which includes links to articles on the history of bibliometrics as well as to additional resources and tools for measuring article-level and journal-level metrics as well as altmetrics. https://acrl.libguides.com/scholcomm/toolkit/impact
• In 2013, SPARC released a primer on article-level metrics. This guide still stands as a useful tool for understanding this important metric for measuring impact at the article level. https://sparcopen.org/wp-content/uploads/2016/01/SPARC-ALM-Primer.pdf
• In a special Bulletin of the Association for Information Science and Technology learn more about the What, Why and Where of Altmetrics. http://www.asis.org/Bulletin/Apr-13/Bulletin_AprMay13_Final.pdf
• A brief but informative article on the “Grand Challenges in Measuring and Characterizing Scholarly Impact.” https://www.frontiersin.org/articles/10.3389/frma.2016.00004/full

The SCOOP, Scholarly COmmunication and Open Publishing, is a monthly column published to inform ATLA members of recent developments, new resources, or interesting stories from the realm of scholarly communication and open access publishing.