Reviewing other people's work is a sacred duty. You have to keep in mind that you are requested to evaluate the hard work of a researcher who may have spent days and nights doing the research and documenting it. Also, the researcher is a human being who will feel insulted if the criticism is harsh, ill-phrased, not justified, or not constructive. Sometimes, authors can guess who is the reviewer, and will start feeling bad about you. Who wants this to happen? Be as polite as you can, give clear indications and constructive comments, and do not forget to bring up one or two aspects of the work (if you can), especially if you are recommending the paper for rejection.

As soon as you are solicited for a review:

1. Quickly glance through the paper and check whether you have a conflict of interest or lack the knowledge the review.


2. When this is the case, write immediately to the chair/editor who requested the review explaining why you need to decline.


3. It is a good idea, when you cannot do the review yourself, to propose one or more names of researchers who would be appropriate for the task.


4. When you accept to do the review, (try to) return the review on time.

I find it useful to follow the pattern below when writing a review:

1. Write a 2 sentence (2 or 3 lines) summary of the paper and its contributions.


2. Acknowledge the significance of the research question raised by the authors (or the lack thereof).


3. Acknowledge the innovative aspects of the contributions (or the lack thereof).


4. List your positive comments in support of the acceptance.


5. List your negative comments in support of the rejection.


6. Make a clear statement on whether you recommend that the paper be accepted as is, accepted after modification, conditionally accepted, or rejected. Often, the possible choices are given in the review template. In case you want to express an opinion that is not among the offered choices, contact the conference chair or the journal editor.


7. Now, you can finally write detailed comments to the authors, highlighting incorrect statements, mathematical errors, unclear notations, typos, etc.


8. Often you are given the opportunity to send confidential comments to the organizers/editors. Use this opportunity to bring up any conflict of interest you had forgotten to mention and share concerns that could be discussed among the reviewers or used by the organizers for a final decision.

Here is some expert advice on how to write reviews:

1. How to write a review, by Boi V. Faltings.


2. How to review, by Toby Walsh.