1.

The Literature Search

A new research project almost always begins with a literature search—or at least it should. The goal of the literature search is to evaluate the state of our communal knowledge on a topic before embarking on a quest of adding to that knowledge. Since science is either about confirming or refuting existing knowledge or developing new knowledge, a thorough understanding of the current state of communal knowledge is essential. Additionally, this literature search will form a foundation for all of the five goals of citations. Note that a literature search is not about finding relevant papers, it is about reading relevant papers.

Unfortunately, literature searches are rarely done as well as they should be. Here are a few hints to improve literature searches:

Starting a literature search always leads to a difficult question: How do you know when to stop? There will always be important papers that you never find. This is the nature of modern science. Knowing when to quit (or pause) the literature search and begin the new work is a matter of judgment and experience.

2.

Verify, Verify, Verify

One of the most pervasive problems with citations is that they are frequently incomplete or inaccurate. It is the job of the authors to verify the accuracy of the references. Editors, copy editors, and reviewers are not responsible for reference accuracy and are not expected to check references for accuracy. And while copy editors try to flag incomplete or improperly formatted references, it is the authors that ultimately must fix the errors found. Why not do the work up front to ensure that the references are complete, accurate, and properly formatted? It will only save time and effort in the end, and indicate to the editors and reviewers that you care enough to pay attention to these important details.

Alas, far too few authors take this advice seriously. Several studies have found that between 34 and 67% of references in a variety of medical and biomedical journals contained errors.³* These errors can be broken down into major and minor errors. A major error means that the article could not be found given the information in the citation. One study found that major errors occur in 7% of the citations from one class of medical journals.⁴ Minor errors include punctuation or spelling mistakes, mistakes in the article titles, mistakes in the name and initials of the author(s), and citation style mistakes. These errors serve as irritants to the reader—they can still find the article, but have to put more effort into it to do so.

It is probably obvious that the main cause of errors in citations is simple sloppiness on the part of the author. There is another problem, however, that may also be at work: copying citations from other papers. In other words, some authors commit a cardinal sin of citations and add a reference without ever having read that paper. Copying citations from other papers without actually looking up and reading that paper can result in a propagation of errors that are never corrected⁵ (kind of like the telephone game that my daughters like to play). A slightly less egregious form is the abstract citation: citing a paper after reading only the abstract. Both types of unread cites should be avoided: cite only papers you have read.

3.

Other Problems with Citations

There are other reasons why a specific reference does not fulfill the goals set out above and thus does not benefit the reader.

Spurious citations: citations that are not needed but are included anyway. Sometimes these citations are added at the last minute, after the paper is written, to give the impression that a literature search and proper citation work have been done. Often they include redundant citations, where the extra citations do not add any value beyond the first one. An example given by Brian Thompson⁶ was “related work on the technique has been carried out by numerous researchers.^1-101” The problem is obvious: an interested reader must wade through far too much literature to get the needed background. Sometimes spurious cites are meant to give an impression of erudition by citing an obscure, historical reference (if the referenced work is in a foreign language, all the better).⁷ In all such cases, simply asking the question “If the reader looks up this reference, will it be time well spent?” will be enough to decide if that reference is spurious.

Biased citations: references added (or omitted) for reasons other than meeting the five goals of citations. Biases include overciting of friends’ or colleagues’ work, omitting cites to the work of rivals, and gratuitous citations in an attempt to curry favor with a boss or potential referee.

Self-cites: citations to one’s own work. There is nothing wrong with self-citations, per se. After all, the work represented in a single paper is often just the latest result of a larger ongoing project. As such, citations to one’s earlier work are often perfectly appropriate. Self-cites are a problem when they are either spurious or biased.⁸ Knowing as we do the tendencies of many scientists toward self-promotion, one fears that self-cites may be designed to boost the recognition of the author rather than increase the value of the paper to the reader.

Excluding contrary evidence: a form of biased citations where citations to prior work whose conclusions or data contradict the current work are omitted. Since one of the five goals of citations is to explicitly contrast the new work with prior work containing conflicting data or conclusions, avoiding such conflict (for whatever reason) does not serve the interest of science.

In the end, authors must find a balance between too many and too few citations. The literature base even on very narrow topics is often vast, and it can be difficult to pick a small subset to cite.

4.

Conclusions

To do a good job of providing citations in a scientific publication, one must keep in mind the multiple goals of proper citing. But like other aspects of good science writing, a simple theme has emerged: make the paper reader-centric, not author-centric. While it is common to choose citations that make the paper more valuable to the author (by demarcating what is novel, for example), good citations make the paper more valuable to the reader. Unfortunately, doing a good job of citing requires more work from the authors. But careful citing is worth the effort if your goal is a quality scientific publication.