How to peer review a paper: a step-by-step guide with a complete example

Editors have measured review quality for decades with the Review Quality Instrument, which scores whether a review comments on the importance of the question, the originality of the paper, the strengths and weaknesses of the method, the presentation, and the interpretation of results, and whether the comments are constructive and substantiated.[27] Three findings stand out. Credentials barely predict quality: at the BMJ, reviewer characteristics explained only 8% of the variation in review quality, and the one factor associated with better reviews was training in epidemiology or statistics.[23] At an emergency-medicine journal, academic rank did not help, and neither did formal training in critical appraisal; the only predictors were working in a university hospital and being within ten years of finishing training.[24] Earlier, at a general medical journal, 43% of reviews were rated good, and reviewers under 40 who were blinded to the authors' identities were far more likely to produce one.[26]

Training helps only slightly: in a randomized trial, a self-taught package raised review quality by a small amount that was not editorially significant and did not last, although trained reviewers found more inserted errors.[25] Signing reviews raised quality, courtesy and time spent in another randomized trial.[29] The lesson: work systematically, check the methods against a checklist, and substantiate every comment.

COPE's guideline puts it plainly: “agree to review only if you have the necessary expertise to assess the manuscript and can be unbiased in your assessment,” and “Ensure you declare all potential competing, or conflicting, interests.” Do not agree if you work at the same institution as the authors or were a recent mentor, mentee, collaborator or joint grant holder, and never “agree to review a manuscript just to gain sight of it with no intention of submitting a review.”[22] Respond either way: editors send 2.4 invitations for every completed review, and three-quarters say finding reviewers is the hardest part of their job.[1] If you cannot review, suggest someone who can.

Two published guides for reviewers converge on the same first move. August and Brouwer's three-step method begins with “reading the entire manuscript to get a sense of the paper as a whole,” before any evaluation.[31] Wiley's reviewer guide describes the first read-through as a quick pass to form an initial impression and to identify major flaws that could affect the manuscript's credibility, before a detailed second read.[34] On that first read, answer four questions and write nothing else down:

1. What is the research question, and is it stated? An unclear question is the first thing editors and reviewers cite.
2. Can this design answer it? Poor study design is the reason editors rank first, and it usually cannot be fixed in revision.[3]
3. Is it new or useful? Lack of new or useful knowledge accounted for 44 to 76% of rejections at AJR, in every country.[4]
4. Do the conclusions match the results? Overinterpretation ranks second among reviewers' reasons for rejection.[2]

Use the checklist for the study design, not your memory. August and Brouwer recommend consulting the EQUATOR Network's reporting guidelines during evaluation, with STROBE for observational studies.[31] Journals that endorse CONSORT publish more complete trial reports, and the omissions the checklists target are the ones that make published research unusable: poor description of interventions made 40 to 89% of trial reports non-replicable, and a third of adequately registered trials changed their primary outcome between registration and publication.[5][6][7]

• Randomized trials: registration and its date, randomization method, allocation concealment, blinding, pre-specified outcomes, sample-size rationale, participant flow, harms. Checklist: CONSORT 2025.
• Observational studies: eligibility, exposure and outcome definitions, confounders and how they were handled, bias, missing data, sensitivity analyses. Checklist: STROBE.
• Systematic reviews: full search strategy, selection and extraction process, risk-of-bias method, synthesis method, protocol registration. Checklist: PRISMA 2020.
• Case reports: consent, de-identification, timeline, diagnostic reasoning. Checklist: CARE.

Mohty and Melo's editorial adds the section-by-section discipline: confirm that “the data are presented consistently and accurately” and “identify any gaps or inconsistencies in data reporting that could undermine the study's findings.”[32]

Most journals do not have one: only 23% of 107 leading biomedical journals use specialist statistical review for all articles, and adding one improves the final manuscript measurably.[10][11] So the ordinary reviewer is usually the only statistical check, and the errors are common: half of psychology papers using significance tests contained a p-value inconsistent with its test statistic, and standards in medical statistics are described in the review literature as “generally low.”[12][13] Check, in order:

1. Was the primary analysis pre-specified, and is the analysed population the randomized or enrolled population?
2. Do the tests match the data type and design, and were assumptions addressed (clustering, repeated measures, proportional hazards)?
3. Are effect estimates reported with confidence intervals, not p-values alone?
4. How many comparisons were made, and was multiplicity handled for secondary outcomes and subgroups?
5. How was missing data handled, and do the numbers in the tables, text and abstract agree?

Reviewers are measurably softer on positive results. Given two versions of a fabricated trial identical except for the direction of the primary result, orthopaedic reviewers recommended the positive version 97.3% of the time versus 80.0% for the null version, found fewer errors in it, and gave its identical methods a higher score.[14] The corrective is deliberate: read the results as if they were null and ask whether the methods would still satisfy you. Then read the discussion for the specific failures Mohty and Melo name, “overinterpretation” of incomplete results, and for causal language that the design cannot support.[32]

A quarter of quotations in medical journal articles do not accurately reflect the cited source, language models fabricate references at rates that were measured at 55% and 18% for two model versions, and the Retraction Watch Database holds more than sixty thousand retraction records that a reference list can silently cite.[15][16][17] You cannot check every reference by hand in five hours, so check the ones the argument rests on: the citations behind the novelty claim, the comparator, and any statement that the literature “consistently shows” something. Confirm the paper exists, that it says what is claimed, and that it has not been retracted.

Every published guide describes the same anatomy, and the editors' quality instrument scores it: a summary that proves you understood the study, major issues that are located and substantiated, minor issues, and a recommendation that matches the comments.[27][30][34] COPE's wording: “Be objective and constructive in your review, providing feedback that will help the authors to improve their manuscript,” and “be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.” Most feedback belongs in the report the authors will see; confidential comments to the editor “should not be a place for denigration or false accusation.”[22] PLOS's instruction to its reviewers is the same: any remark that might help strengthen the paper should go to the authors.[33]

Reviewers advise. At the Journal of General Internal Medicine, reviewers agreed on rejection only slightly better than chance, yet editors rejected 88% of manuscripts when reviewers were unanimous for rejection and 20% when they were unanimous against it.[18] At AJR, 10.2% of major papers were accepted, 54.1% rejected and 35.7% rejected with an invitation to resubmit, and more of the variation in scores came from the reviewer than from the manuscript.[28] Sedaghat and colleagues' guide defines the recommendation categories editors expect, from accept through conditional acceptance to reject.[30] Your recommendation carries weight in proportion to how well the comments support it.

COPE: “Respect the confidentiality of the peer review process and refrain from using information obtained during the peer review process for your own or another's advantage,” and “Do not involve anyone else in the review of a manuscript (including early career researchers you are mentoring), without first obtaining permission from the journal.” If you suspect an ethics problem, “contact the editor directly and do not attempt to investigate on your own.”[22]

Uploading a confidential manuscript to an external AI tool can breach that confidentiality. NIH prohibits its peer reviewers from using generative AI to analyze or formulate critiques, and JAMA Network has published guidance for authors, peer reviewers and editors on the use of AI, language models and chatbots; check your journal's policy before you paste anything anywhere.[20][35] Between 6.5% and 16.9% of review text at four AI conferences showed signs of substantial LLM modification, which is exactly the pattern journals are now screening for.[19] PeerReviewAI is built for authors reviewing their own manuscript before submission; it is not offered as a tool for writing journal reviews.