Review Components and Editorial Policies

Peer Review

The Charleston Advisor publishes critical reviews of online resources for libraries. To maintain a high level of accuracy and integrity, all formal reviews are peer reviewed by experienced librarians on our Editorial Board. Our reviewers come from all areas of librarianship and from all types of libraries, ensuring that products are reviewed by people who actually use them. A review may be accepted by the Managing Editor contingent upon changes that are suggested by reviewers. All reviews must receive at least two positive recommendations to publish. This allows us to maintain the high-quality reviews our readers have come to expect. Reports from the field, opinion editorials (op eds), and other non-review articles are not peer reviewed and are published at the discretion of the Managing Editor and Publisher.

The Charleston Advisor Rating Guidelines

As a critical evaluation tool for Web-based electronic resources, The Charleston Advisor uses a rating system that scores each product on the basis of four elements: Content, User Interface/Searchability, Pricing, and Purchase/Contract Options. A Composite Score averaging these elements provides an at-a-glance rating that is prominently displayed near the top of each review. The rating system is based on a five-star model (with one star being the lowest and five being the highest). As each of the four elements is rated, it is assigned a ranking with granularity no finer than one-half star (e.g., 3.5 stars). These are equally weighted and averaged for an overall Composite Score. Brief evaluative comments follow each ranking, especially in cases in which a product or service receives less than the highest value in a category.

The Charleston Advisor is meant to be a critical evaluation tool, not just a descriptive review of a product. Our reviewers take great care in the ranking section of the review because it is one of the most visible yet controversial components of a review. The following brief guidelines help explain the ranking process.

Content

Special attention is paid to the intended audience for which the product is targeted: Does the product meet the intended users’ needs? The content is compared with competitive products on the market, and any major omissions or special strengths are factored into the rating.

User Interface/Searchability

The user interface and search engine are evaluated in terms of meeting the intended purposes: Is the product intuitive and easy to use? Are advanced searching features available if the product warrants it? Are graphics and other screen design features in keeping with the intent of the product and its audience? Is the search engine reliable, and does it provide consistent results? Are there special features, installation requirements, plug-ins, or other special software requirements? If so, is the product easy to use or more of a nuisance?

Pricing

The value of the product in relation to its cost must be assessed. A high price alone does not necessarily mean a low ranking, but the product is evaluated in terms of content, user interface, and value-added features. However, vendors that resell duplicative content in different “packages,” with enough difference to force libraries into acquiring these different packages, may be marked down. Vendors that are flexible (or inflexible) in their pricing options are noted. Products that are open access or freely available do not receive a rating in this category.

Purchase/Contract Options

The contract provisions that accompany a service are viewed in terms of accepted national guidelines (e.g., those adopted by major organizations such as the International Coalition of Library Consortia, Association of Research Libraries, and the American Library Association). Factors that may be considered include definition of acceptable users, archiving provisions (when appropriate), lease/ownership of data, interlibrary loan provisions, redistribution of information provisions, or other peculiar or interesting issues. Products that are open access or freely available do not receive a rating in this category.

Composite Score

The overall score is an average of the above four elements with equal weighting for each. The Composite Score is rounded up to the next highest fraction of a star when necessary (e.g., 3 ¾ stars).

The Charleston Advisor Review Components

Title of Product or Resource: the official product name
Reviewed by: name, affiliation, and email address of the reviewer
Original Date of Review: date of the initial review of the product by The Charleston Advisor
Date Last Updated: if the product has been reviewed before, this is the date of the latest version
Composite Score: the arithmetic composite average of all elements in the score box
Abstract: an executive summary of the key points of the review
Pricing Options: pricing options available to subscribers
Product Description: a narrative description of the product and its content
User Interface: a detailed examination of the user interface, navigation, and searching within the product
Critical Evaluation: examination of the strengths and weaknesses of the product
Similar Products: a brief description of other products or services that compete with the product that is being reviewed or otherwise may be of interest to readers
Contract Provisions: detailed description of the options and features of the standard contracts offered by the vendor
Authentication: description of the authentication and security options available to subscribers
References: citations to other reviews of the product and related materials
Score Box: the Composite Score for each resource is based on four elements—Content, User Interface/Searchability, Pricing, and Purchase/Contract Options
Contact Information Box: corporate address, phone, email, and URL
Metadata Box: Elements assigned to each review for better searching online. Included are Free Text Keywords, Primary Categories, Type of Product Being Reviewed, Target Audience, and Access (purchasing model)