Contrast and Scatterplots: Taking Research from Pre-registration to Open-access Publication.
Author: Gabriel Strain
.png)
Our recently published paper, “The Effects of Contrast on Correlation Perception in Scatterplots” investigated a novel technique for manipulating the contrast of points within scatterplots, with the aim of improving people’s accuracy in correlation perception. This consisted of a pair of experiments conducted online using crowdsourced participants. Our stimuli were generated using the R language, and we built mathematical models that provide evidence for our manipulation being able to partially correct for people’s biases when estimating correlation from scatterplots.
Applying open research practices
- Study pre-registration: both experiments described in the paper had their hypotheses and analysis plans pre-registered with the Open Science Framework. Links to these pre-registrations can be found at the end of the case study.
- Open Data, Materials, and Code: both experiments are hosted at Pavlovia.org, and links to them are contained within the paper itself. These experiments and the stimuli included in them are permanently and freely available to be re-used and adapted. The anonymised full data set and all code associated with the project is permanently and freely available in an online repository (GitHub).
- Dockerisation: An online repository contains instructions detailing how to fully recreate the computational environment used during analysis, allowing for fully reproducible analyses, is permanently and freely available. This also means that the paper itself is executable and fully reproducible.
- Open Access: The paper is published with gold open-access, making it freely and publicly available in perpetuity.
Overcoming challenges
Writing a fully executable and reproducible paper was initially a challenge. All figures and analyses are dynamically generated from R code, making formatting and document preparation significantly more time-consuming at first. Once this was done, however, it meant that figures in the paper could be changed without the need for manual recreation. This also meant we could write the code needed to analyse and visualise the data before data collection was complete.
Benefits of using these open research practices
Our emphasis on reproducibility and replicability means our research findings are inherently more robust. There can be no doubt that we did exactly what we planned to do, and all code and analysis can be independently checked and verified by anyone. Making all parts of the paper freely and publicly available, including publishing as gold open-access and facilitating straightforward replication, removes barriers to science that have traditionally been in place.
Top tip
Make sure to factor in the time and training required to make your work fully reproducible, as it can be time-consuming and technically challenging.
Related links
GitHub repository
See project code and materials on GitHub.
Read more
OSF repository
View the project materials on OSF.
Read more