Post #1604863

2025-10-14 06:55 UTC

@nicebread So, this means that non-preregistered studies should be evaluated more harshly because there is less transparency? I can imagine some social scientists (political science, sociology) would disagree if they cannot preregister their observational research. This maybe different for experiment-heavy psychology. In the end, there will always be a concern that some standard advantages one kind of research and disadvantages another.

Replies (3)

@tomstafford@mastodon.online 2025-10-14 07:11

@ingorohlfing @nicebread surely the contrast is meant to be preregistered experiments vs experiments lacking a preregistration? We have to trust reviewers not to blindly apply a heuristic across all methodologies, right??

Open ##1604864
@auzdavenice@mastodon.social 2025-10-14 09:21

@ingorohlfing observational work is incredibly easy to pre-register Pre-registration is literally posting the Methods section ahead of time alongside an implicit (hopefully explicit) promise of, "I'll let y'all know how this changes as the work unfolds" That's it There's no pre-reg popo that go around beating people if they deviate from some grand Platonic ideal You can do it for literally any kind of science

Open ##1604866
@nicebread@scicomm.xyz 2025-10-16 05:13

@ingorohlfing I see three situations here: (a) Researchers who are transparent and preregister get a more critical review (because deviations are found etc.) (b) Prereg / non-prereg get the same level of criticism (c) Non-prereg gets a more critical review, because of undisclosed flexibility and higher risk of bias. I read the current review guidelines primarily as „we want to avoid (a)“, and I fully agree to that. Beyond, I lean towards (c) *if* a preregistration would have been feasible. A prereg mitigates risk of bias; if that tool of mitigation is not present/not possible, I expect other measures of ensuring trust.

Open ##1604873