Elektrine lite

← Feed

@satrevik@fediscience.org

Post #1475734

2025-03-25 12:13 UTC

Imagine you preregistered study with tests A and B for a directed hypothesis H, but did not specify one-tailed or two-tailed testing. Test A is in the predicted direction, but p = between .05 and .09. Test B is in the non-predicted direction, but p < .05. How do you report results? #PreRegistration #OpenScience #NHST #SignificanceTesting #pValue

Replies (4)

  • @satrevik@fediscience.org 2025-03-25 19:09

    Oops, I lost the results so far when I updated the response options to make them clearer. But the first responses were: one for the first option, one for the third option, and two "other" (perhaps just curious to see results, which you can do by clicking the eye icon).

    Open ##2292058

  • @satrevik@fediscience.org 2025-03-27 11:53

    My own two cents (that may diverge from common practice): Option 1 is a valid and probably preferable approach. With a preregistered directed hypothesis, this is a more severe test of the hypothesis (but would have been better if the one-tailed testing was preregistered). Option 2 may be valid, if the preregistration doesn't make it sufficiently clear that the hypothesis was directional. Option 3 I would say is not a valid approach. Two-tailed testing should be used to test whether there is ANY association between variables (positive or negative). A significant effect in one of the tested direction should thus not be used to argue against a directional hypothesis (which you are not testing).

    Open ##2292059

  • @auzdavenice@mastodon.social 2025-03-25 19:37

    @satrevik@fediscience.org Report both; lend more weight in discussion/interpretation to B But this likely isn't a clear-cut finding if A and B diverge Edit: I might have responded to the old wording, but am not sure!

    Open ##2292060

  • @olibrendel@scicomm.xyz 2025-03-26 05:49

    @satrevik@fediscience.org Is this a realistic question ? With two tests A and B I suppose you mean two different statistical approaches ? I could see that one is not significant but the other is, but when they go in different directions, I would suppose that you have a problem with your basic statistical assumption. Either the mean of population X is higher than that of population Y, it cannot be smaller at the same time. But perhaps I don't understand your question, then please give an example.

    Open ##2292063