Due: February 19 at 7:00pm
The goal of this project is to introduce you to the design and execution of user studies for evaluating data visualizations. You will design an experiment inspired by the classic work of Cleveland and McGill on graphical perception, with the goal of measuring how accurately people interpret values from different types of charts.
In this project, you will: 1Design an experimental protocol, build an interactive experiment interface using D3, collect data from participants, and analyze and visualize the results. While part of the goal is to replicate established findings, the project will also allow you to extend that prior work by evaluating new visualization designs that were not included in the original studies.
The final deliverable will be a publicly accessible webpage that documents your methods, presents your results, and discusses their implications. You will also create a presentation to the class about your project to discuss your experiment and findings.
Before designing your experiment, you must carefully read the following papers:
In both studies, participants were shown one visualization at a time with two data points highlighted. Participants were asked to estimate the ratio between the two highlighted values. For example, a prompt might ask: "What percentage is the smaller bar relative to the larger bar?"< Participants responded with a numerical percentage, like 75%
The accuracy of these judgments provides a quantitative measure of how well different visual encodings support perceptual estimation.
Decide which types of visualizations you want to evaluate. Cleveland and McGill originally tested five bar-based and angle-based designs (e.g., aligned bars, stacked bars, pie chart angles). Heer and Bostock replicated these designs and added size-based encodings (e.g., area of circles, rectangles).
Examples of new encodings you might consider include (note, these are just suggestions)
Organize your experiment into blocks, where each block corresponds to a visualization type. The order of blocks should be randomized or counterbalanced across participants to reduce learning effects.Within each block:
Using D3, build an interactive interface that presents one stimulus at a time. Each stimulus should:
The interface must include a slider for entering a percentage estimate for each stimulus visualization, along with a "next" button to advance to the next trial
For each trial, record the following information:
These responses should be stored automatically by the interface (e.g., in a JavaScript array). At the end of the experiment, the interface should indicate completion and allow participants to download their data as one CSV file.
Recruit participants from the class and beyond (e.g., friends, family, or other students). The simplest approach is to host the experiment on a webpage and share the link. Participants can email you their downloaded CSV files, or, optionally, you may implement a backend (e.g., using a server and database) to automatically collect responses.
Be sure to provide clear instructions. Keep the total experiment duration reasonable (e.g., 10–15 minutes). You will need to test the experiment
For each visualization type, compute accuracy using the Cleveland and McGill error metric, which measures the deviation between the perceived and true ratios.
For each visualization design:
These statistics will allow you to compare visualization designs while explicitly representing uncertainty. For visualization types for which there is data from the Cleveland and McGill paper (or the Heer and Bostock replication), include the data from these experiments. That is, display earlier results in your visualization (e.g., using different color) so that those can be compared to the result you gathered.
Your final report should be presented as a webpage and include the following sections:
You will turn in your work as a publicly accessible webpage containing the report. You should also include a link to the experiment interface so I can try it. Email me URL of your report webpage.
You will also present your project and results to the class. We will have project presentations on February 23. Your presentation will be 20-minutes (+ 5 minutes of Q/A)