The research cycle and questionable research practices
Group activity
Discuss the following practices: are they questionable? Why?
A researcher tests several different statistical models on the same dataset but only reports the one that gives the strongest effect.
A researcher preregisters their study design, runs the study as planned, and publishes the results even though they found no effect.
After running an experiment, a researcher changes the stated hypothesis to match the observed results and rewrites the paper introduction.
A research team decides to increase their sample size mid-study because they realise the initial sample lacks the statistical power to detect medium effects. They clearly document this change in their published article.
An author declines to share their raw data when requested by other researchers, without offering any clear reason (e.g. confidentiality, ethical restrictions).
Group activity
Play the game three times to publish three papers: the goal is to publish all three!
You need to balance Integrity and Career points and beat the reviewers lottery.
A few studies looking at the effectiveness of peer-review (Smith 2006; White 2003; Jefferson et al. 2002, 2007; Bruce et al. 2016; Garcia-Costa et al. 2022; Stephen 2022).
Little empirical evidence that peer review ensures quality.
Changes to manuscript contents are limited.
Statistical content increases after review.
Training reviewers does not improve manuscript.
Most suggestions are ignored when re-submitting to a different journal.