last updated:
06/28/2005

Editor Note: This document was prepared by Steve Duck as the then editor of the Journal of Social and Personal Relationships (JSPR) to be distributed to student reviewers. It was also distributed to participants at the International Network on Personal Relationships 10th New Scholars Pre-Conference, which took place at the University of Oklahoma in May 1998. It may therefore refer to editorial policies that were in place at the time, but no longer are. It contains, however, a lot of useful advice and information about the review process, that may be especially helpful to new reviewers and authors. The current editor of any journal will usually provide you with the information that is considered essential about the journal’s policy. You may also find guidelines, especially about the journal’s mission, scope, and type or articles that it publishes, in the “Instructions to Authors” page in any recent issue or on the journal’s website. Also feel free to ask the editor about anything that is making you doubtful. My own changes or additions to the original text are marked by ellipsis… points or by [square brackets].

João Moreira,
IARR New Professional Representative
February, 2005

Guidelines for New Reviews for Journal of Social and Personal Relationships

Prepared by
Steve Duck for members of the International Network on Personal Relationships.
Copyright 1993

What happens during reviewing?

When authors submit a paper for review, it is sent out typically to two [but sometimes to three or even four] referees who are asked to make a professional, expert judgment about its suitability for publication. Note that this judgment requires a different standard than whether you just like the paper or enjoyed reading it, though both of those things may be relevant. Acceptance for publication is the highest form of standard to be applied to work that you may read and it requires that the paper's driving question is interesting and theoretically informative, that the work is well designed and well executed, that the paper is a good, clear and informative report of the work that was done, and that the paper makes a contribution to the literature on the relative topic. As a referee you are implicitly asked to stand in the place of other experts in the field and make judgments on these points, but of course, you are not alone and do not bear the full responsibility for any decision that is ultimately made on the paper.

Reviewers are typically given a date by which they should have reported on the manuscript. It is entirely appropriate to return a MS [standard abbreviation for “manuscript”] if you cannot deal with it by the time requested. We all have other things to do.

However, it is appropriate to make that call immediately and not to hang onto the paper for several weeks and then return it. By the time those several weeks have elapsed, it will be too late for the editor to find an alternative reviewer. If you do decide to review the paper then you should do so in a timely fashion and be sure to return the report by the time requested.

Once the reviews are in place or the deadline is reached, whichever happens first, the editor will then make a decision about the MS based on a personal reading of the MS and also on the advice from the expert referees. The editor will compose a decision letter and return it to the authors along with copies of the referees' reports. The editor will also return to the referees a copy of the decision letter and any other reviews that were received, so you will at that point have a chance to compare your review with that of the other referee(s).

Don't forget that reviewing is a very significant professional service that takes up quite a bit of time and you should be sure to list on your vita the fact that you have done reviewing for a particular journal in a particular year.

How does reviewing work?

As you may know, when a journal reviews a paper for possible publication it is usual for this to be done anonymously and blind: that is, the reviewers will not normally be told who wrote the paper (at least not before the reviewers have completed the review) and the author will not normally be told who the reviewers were at any point, though people often try to guess -- in both directions.

Some reviewers send in a report [that is meant to be sent to the manuscript authors, see below] and no letter, some send in a letter [intended to be read by the editor only] and no review, most send in both. BE SURE TO NOTE THE MS CODE AT THE TOP OF YOUR REVIEW AND ON THE LETTER IF YOU SEND ONE. For various practical reasons it is less helpful if you send in a letter on its own because a letter heading identifies your institution -- and so someone will have to go to extra trouble to cut that off the copies in order to retain anonymity - a small point, but a point of professional etiquette. It is also less helpful to be sent a report and no letter merely because reports can sometimes get separated from their envelopes and if there is no identifying attachment then it may not be clear whose report is which (for example if two "unlettered" reports on the same MS come in on the same day in envelopes that get thrown away when the report is copied, it may not ever be possible to work out whose report on the MS is which). [Of course, nowadays most of the process is held over e-mail, so many of these issues are outdated. In any case, the distinction between a text to be forwarded to the authors and another intended for the editor’s eyes only is, and certainly will remain, important.]

Another advantage to the letter-plus-report is that you can say things in one that are not said in the other. The report that you submit [possibly as an e-mail attachment] will usually be sent on unedited to the authors and yet you may want to make some remark that you would not like the authors to see. Such remarks can be put in the cover letter (which could be quite brief) [or in the main e-mail message, or in a separate attachment, possibly using a form the editor e-mailed to you together with the manuscript]. For example while your report may conclude that the work is not suitable for publication, your letter may go further and make clear to the editor that you very strongly urge a revision rather than just think a revision might make a slight difference or might not. Also if the work strikes you as publishable on its merits but on a boring topic, you might want to reserve such a comment for the editor. Sometimes a report leads to a soft rejection but the letter makes it clear that you really were just being polite and the editor should take a hard line. In any case you are able in a letter to add to your remarks in a way that allows the editor to make more useful evaluations of your report. In most cases, however, the report speaks for itself and the cover letter need only say "Hi! here it is; hope it's useful". Another small point is that letters have dates on and can help the editor keep track of things.

In all of the remarks that you make about a paper that you review, remember the purposes for which your advice was sought: 1) to advise the editor about suitability for publication; 2) to advise the authors about the work that they report in the paper. Sometimes these goals conflict, since the paper really may not be publishable as it stands and the work on which it is based may not be adequate to be salvaged in a revised paper, but you can think of a good design change that might help them to do better work and then write a new report. It is fine to mention it even if you think the paper itself has to go down the tube entirely. The report will be sent to the authors unedited and so should be constructive and supportive as far as possible (that does not mean it should not be critical, but it should be professionally courteous). Sometimes people research topics that you think are stupid and it is appropriate to comment in the report that you found the topic unexciting, but you should be able to say why and give a rationale rather than just state the judgment. The letter to the editor might make that sort of comment more explicit and give the editor a sense of why readers might not find the topic worthwhile. Remember, however, that people do research very often because they think it is important and so slighting comments about their work should be kept out of the report. A well argued case against a paper is often helpful to authors, since it indicates things that they can defend against or responses they need to prepare. A simply vituperative evaluation does not help anyone.

What should your report include and how should it be presented?

It is helpful for you to begin by summarizing the theme of the paper as you see it. What do you believe the authors were trying to do? This could be two sentences. You could then comment on the theoretical value of such a theme. If done well, would study of such a theme advance our theoretical understanding of relationships and why? Then comment broadly on whether the authors achieved their objectives or failed to convince. Then give an evaluation of the paper as a paper submitted for publication - and there is your first paragraph complete.

If you think that a paper had great data that were poorly interpreted then here is where you say it. If you think that the idea is great but the study design stinks then say it here. If you think the study should be reanalyzed using a different procedure, or reconstructed hypotheses or given different interpretations then make such points in the early stages.

Subsequent paragraphs should back up the evaluation that you made in the first paragraph. Some people put the major points first and go on to the less important ones. Some take the paper section by section. Some argue about the thesis of the paper and the extent to which the results back up the claims made in the introduction and discussion. There is not really one perfect way to do this and on different occasions you may choose to do it in different ways. However in all your writings about the paper you should remember that a well reasoned critique is more important than an evaluative remark that is not substantiated. You do not really need to give a full commentary on grammar and punctuation except in cases where they detract from the paper's value or are more noticeable than should be or where they create ambiguity. (Incidentally some reviewers return a marked up copy of the MS to be sent to authors, but that is rare and not necessary).

Things you can comment on:

Appropriateness of introductory survey of literature. Were important papers omitted? Was correct interpretation of existing studies given? Are there papers that would help the argument if the authors cited them? Is the argument well built? Are the hypotheses well-derived and interesting? If the authors have asked a dumb question then it hardly matters if they have done a well designed study and done appropriate analyses. If on the other hand they have not made out a persuasive case for a good study then you should try to help them by importing your expertise and knowledge.

Appropriateness of the design to test hypotheses.

Appropriateness of data analyses. Were all the data analyses correct and appropriate? Were too many analyses done? Were any omitted that could have aided interpretation?

Correctness of interpretation of results.

Appropriateness and cogency of discussion of results, first in relation to the study at a local level (discussing any details of the study that did not work properly, for example, or the fact that 15% of respondents were suspicious, or that the sample was asked to fill out questionnaires privately but no checks were made on this, and so on) and at the general level of relationship to the existing literature and other studies that were discussed in the introduction. Has the case been overstated or misrepresented? Were inconsistent results properly discussed and accounted for? Does the study mean what the authors want us to believe it means? Are plausible alternative explanations for results left unruled-out?

Length of the MS. Most MSS do not need to be as long as they are while others need expansion. If you thought that the paper was too long for the value of the data then say so. [Some journals], for example, publish [special “brief articles” sections]… and a paper may not be acceptable as a [standard] article but could be acceptable as a brief article if appropriately revised.

What decisions can you recommend?

[These alternatives may vary from one journal to another, and the editor will probably let you know when asking for a review. As of 1993, the options available for JSPR were:]

a) Accept as is. Out of all the 8000 odd reports that I have seen, there has been only one MS where two referees made this recommendation about it. But it is a possible recommendation.

b) Accept after correction of the following minor errors....[list them]

c) Accept after minor revision when the authors have had a chance to react to the following relatively minor points of criticism or suggestion....[list them]

d) Revise and resubmit to for decision by the editor alone, no extra review. This is usually suggested when the MS is in pretty good shape but there is one relatively major problem that seems to be handle-able yet you cannot be absolutely sure it will be uncontentiously handled. In the previous recommendations the revisions are usually trivial (spelling, headings, inclusion of a missing reference). In this one there is usually some idea that needs to be worked out better, but one that does not really undercut the value of the report even as it stands.

e) Revise and resubmit for another round of reviews. This is by far the most common recommendation apart from reject. In this case the MS has considerable interest and you could see a really excellent version of it being an exciting paper that people would want to read ..... but there are some major weaknesses in the present form of the MS. For instance the stats may be misinterpreted and although you suspect that a better analysis would come out in support of the authors' claims, you'd like to find out for sure. Or you might suggest an entirely new line of argument or some new analysis that would strengthen the paper if it comes out right. Or the paper might have good data but be pretty badly written or overstated or might have missed out great chunks of literature or might conflict with something else you know about that you'd like to see included and resolved. Be aware that not all such R&R [revise and resubmit] MSs go out to the same referees next time around so your points should be ones that are representative of points others in the field might make. Don't just grind a personal ax! As a referee you are supposed to be representing scholarship in the field as a whole and the readership as a whole. Referees very often agree substantially about the strengths and weaknesses of papers and that confirms this pious hope about refereeing. Sometimes they disagree also. That's life. However, authors are not supposed (or not supposed by me anyway) to be writing for the referees personally but for the field as a whole -- so new referees are often sought for revised versions of papers. Thus your comments should not be too picky or personally yours.

f) Revise and resubmit as a brief article. All the points in (e) above apply but in addition the paper is really preliminary or focused on a very local issue or on development of a scale procedure and should not be given a full page-allocation. It is interesting, provocative, and well-conducted, but preliminary, work that will stimulate other scholars to develop and use it. (NB some papers are [currently] submitted [to JSPR] as Brief Articles in the first place -- they will typically be under 15 MSS pages -- and should be reviewed for what they are.)

g) None of the above --- i.e., REJECT. This is always a hard recommendation to reach but you need to bear a few things in mind. Not all publishable papers are publishable in a journal of JSPR's stature. Not all rejected papers are rubbish and so you are not deriding a paper by rejecting it; some just need a lot more work and more than can be accomplished by a substantial revision (e.g., the study needs to be strengthened by the gathering of new data in a new design; the paper is premature for publication and proper amounts of data have not been gathered or analyzed; JSPR is not the best outlet for a paper with these themes; the work is OK but not of JSPR standard). Distinguish between a rejection of the work itself (which may be terribly badly done) and the paper describing the work (which may be awfully written though the work itself is good and makes sense). To reject you need to be sure that the paper cannot be salvaged by a rewrite; that even if the study were perfectly described and conducted it would not add enough to our theoretical knowledge to merit publication. Also keep in mind, on the other hand, that JSPR has [as of 1993] only 640 pages per year and there is an encouraging amount of really good work out there that needs our space.

Ethical issues

The review process is a privileged one and the MSs that you are sent should be treated as such. For example, you should not talk to other people about it in a way that might identify the author, if you think that you know who wrote it. Especially at conferences, you might be sitting next to the person who wrote it! If the reviewing of the MS gives you some good ideas of your own to go and test, then that is fine, however, and is one of the advantages of reviewing. You may use what you learn, but of course it is not ethical to plagiarize phrases or concepts or techniques from such a process.

If you are a graduate student or recent Ph.D., then it is acceptable for you to show your advisor (or a senior mentor-colleague) your review before you send it back to JSPR just in case you are worried that you may have "done it wrong" or in case you would like advice on a particular point. It would not be right to pass the MS round the department coffee room and talk freely about it, but your professional relationship with a senior advisor or colleague may be useful in helping you to learn the reviewing process and it will make you a better professional in the end also.

Finally, if you have any doubts whatever or any questions about the process, do not feel shy about asking the Editor or Associate Editors. You can learn a lot by asking what might seem like a dumb question. Don't be afraid to learn by asking. We are asking you to do us a favor and we should at least be willing to answer your questions in return.