Post #2979467
2023-05-12 19:13 UTC
Spin notwithstanding, the technical details within preprint itself make it abundantly clear that the method doesn't work.
In a "juiced" test set with as many fake papers as real ones, the indicators that they use have a sensitivity of 86% and a false alarm rate of 44%.
Yes, they flag 44% of the known real papers as fake.
That's not a detector, it's a coinflip.
This should be a profound embarrassment to everyone involved with the preprint and Science story alike.
https://www.medrxiv.org/content/10.1101/2023.05.06.23289563v1.full.pdf
Replies (1)
-
@ct_bergstrom@fediscience.org 2023-05-12 19:32
To test their indicator, the authors conjecture that a valid indicator should meet three criteria based on a questionnaire sent to authors: (i) Authors of fake publications are reluctant to provide critical information as revealed by their response – or non-response – to the questionnaire by the editor, (ii) the number of fake publications increases steadily over time, and (iii) journals with a low to medium impact factor are most affected. There's a huge problem here.