Name: Amazon.com ratings and reviews
Can apply to: Books
Metric definition: Amazon.com ratings are submitted by end users and allotted on a one- to five-point scale. Amazon.com reviews are text commentary submitted by readers. Both apply to individual books.
Metric calculation: Ratings and reviews are retrieved based upon a book’s ISBN. Overall book ratings are calculated as a type of weighted average according to a “machine learning algorithm”, about which few details are known. User verification seems to play a part in the weighting for both reviews and ratings, meaning in theory these data are more difficult to game. Other users can upvote Amazon.com reviews as “helpful”, which impacts the visibility of said review and therefore the influence that the review has upon prospective readers.
Data sources: User-submitted Amazon.com ratings and reviews.
Appropriate use cases: Review content, especially from notable reviewers (e.g., a positive review from a Nobel laureate), can be used to demonstrate the popular reception of a book. “Number of reviews” has been recommended as a means to understand the wider popularity of a book. According to Thelwall & Kousha (2016), “Amazon book reviews presumably reflect to some extent the teaching impact (Kousha and Thelwall, 2008) or cultural benefit of books (White, Boell, Yu et al. 2009), rather than their impact on future research.”
Limitations: It is possible for those who do not own an item to submit both Amazon.com reviews and ratings (cf. the phenomenon of the Three Wolf Moon t-shirt).
Inappropriate use cases: Ratings should not be interpreted as measurements of quality or influence, nor a lack thereof. Users might leave poor reviews for a variety of reasons, including delayed shipping or a philosophical disagreement with the central thesis of a work.
Available metric sources: Amazon.com, PlumX
Transparency: Data on Amazon.com is fully auditable, in that it is possible to read all reviews and ratings for an item. However, it is possible for reviewers to rate and review a title using a pseudonym, so these data are not completely protected from falsification.
Timeframe: Reviews have been available since at least 2010. Ratings have been tested periodically since approximately 2014. Both are available for books of any age.