These resources cover topics such as rater reliability, the journal review process, and tips for reviewing journal articles and books.
Boring, E.G. (1951). The book review. American Journal of Psychology, 64, 281-283.
Cicchetti, D.V., Conn, H.O., & Eron, L.D. (1991). The reliability of peer review for manuscript and grant submissions: A cross-disciplinary investigation. Behavioral and Brain Sciences, 14, 119-135. or http://www.researchgate.net/publication/231753896_The_reliability_of_peer_review_for_manuscript_and_grant_submissions_A_cross-disciplinary_investigation
Epstein, S. (1995). What can be done to improve the journal review process. American Psychologist, 50, 883-885.
Perlman, D., & Dean, E. (1987). The wisdom of Solomon: Avoiding bias in the publication review process. In D.N. Jackson & J.P. Rushton (Ed.), Scientific excellence: Origins and assessment (pp. 204-221), Beverly Hills, CA: Sage.
Peters, D.P., & Ceci, S.J. (1982). Peer review practices of psychological journals: The fate of published articles, submitted again. The Behavioral and Brain Sciences, 5, 187-195. or http://www.researchgate.net/publication/231878497_Peer-review_practices_of_psychological_journals_The_fate_of_published_articles_submitted_again
Shebilske, L. (1997). How to review a journal article. ISSPR Bulletin, 13(2), 19-20.
Sternberg, R. J. (Ed.). (2005). Reviewing scientific works in psychology. Washington, DC: American Psychological Association.
Wellman, H. M., & Cialdini, R. (1980, Fall). Guidelines for reviewing research in developmental psychology. Newsletter of the Division of Developmental Psychology, American Psychological Association, 50-64.