Project: Peer Review

Assigned: Monday, Apr 17, 2017

Due: Friday, Apr 21, 2017 at noon

Collaboration: You should write your initial review independently, but discuss your review with the rest of your group. You can (and should) make revisions to your review after your discussion if your position changes.

Submitting your work: Send a one paragraph summary of your group discussion and all three reviews to me and the group whose proposal you reviewed.

Overview

Peer review plays an important role in the academic world, and it gives us a useful opportunity to help improve the final projects for this course. While there are many reasons academics use peer review, one important one is to gather outside perspective. It is easy to overlook the shortcomings of your own work when you are fully immersed in it, and sometimes an outside observer can quickly spot flaws that you would not notice on your own. In most cases, peer review happens after work has been completed, but we will conduct our reviews before you spend too much time implementing your proposed work. This should give you feedback early enough in the project that you will be able to adjust your work to address reviewer comments, which will hopefully result in a better end product.

Step 1: Independent Review

Every member of your project group will review the same proposal. You should prepare your review independently before you discuss the proposal with your group members. Your initial review should include the following components:

Summary

Give a short summary of the visualization in the proposal. What questions could it help you answer? How will the visualization use interaction? What are the sources of data and how will it be used? Try to avoid giving feedback in this section. The goal here is to show the authors of the proposal what was clear and what could use additional detail. This section should be at least 3–4 sentences.

Strengths

Identify three strong components of the proposed visualization. You could choose to focus on the choice of question the visualization will help to answer, the design of the proposed visualization, or any other aspect of the proposal. For each component, describe what it is and why you think it will make for a successful visualization.

Weaknesses

Identify three weaknesses in the proposal. Are there potential problems with the questions the visualization will help to answer? Is the design easy to understand? Does the interaction seem useful or gratuitous? Again, describe each weakness and give an explanation for why you included it. You should save your suggestions for the next section.

General Comments

This is generally the longest part of a review. You can ask questions about other aspects of the visualization that you did not identify as strenghts or weaknesses, offer suggestions on how to address weaknesses, or just give ideas on possible improvements to the visualization. Think carefully about the scope of the project and the effort required to take your advice; this is not the place to propose an entirely different visualization.

Step 2: Reviewer Discussion

Once you have completed your review independently you can move on to the next step in the peer review process. Every member of your group should have reviewed the same proposal. Meet with them to discuss your identified strengths, weaknesses, and suggestions. You should read each others’ reviews and look for common themes. Did you all agree that the animation described in the proposal seemed gratuitous? Does the design of the visualization make intuitive sense? Likewise, watch for characteristics that you may disagree about; did you identify something as a weakness when another reviewer thought it was a strenght? You do not have to agree, but it is good to understand why your opinions differ. If your opinion changed as a result of the discussion, please update your review.

Your discussion, including time to read your group members’ reviews, should take about half an hour. One member of your group must write a one paragraph summary of your discussion to include with your reviews. Once you have finished your discussion, email all three reviews and your one paragraph summary to me and the group whose proposal you reviewed. Your reviews and summary are due by noon on Friday, Apr 21, 2017.