Reporting Guidelines

• Reporting Guidelines
• Effect Existence and significance
  • Interpretation Rules of Thumb
  • Template Sentence

These guidelines can be referred to by citing the package:

• Makowski, D., Ben-Shachar M. S. & Lüdecke, D. (2019). Understand and Describe Bayesian Models and Posterior Distributions using bayestestR. Available from https://github.com/easystats/bayestestR. DOI: 10.5281/zenodo.2556486.

Reporting Guidelines

Based on the previous comparison of point-estimates and indices of effect existence, we can draw the following recommendations.

To minimally describe the posterior distribution of a parameter, we suggest reporting 1) the median as an index of centrality, 2) the 89% CI (using HDI rather than quantiles) as an index of centrality and, in the context of null-hypothesis testing, the Probability of Direction (pd) for effect existence and, especially in the context of confirmatory analyses, the ROPE percentage (full, i.e., based on the full posterior distribution) with an explicitly specified range for effect significance.

Effect Existence and significance

The pd and the ROPË are two indices that give different and independent information: The pd is a marker of existence, consistency and direction of a parameter (whether a parameter has a consistent effect in one or another direction), whereas the percentage in ROPE is a index of significance (in its primary meaning); informing us whether a parameter is related or not to a non-negligible change (in terms of magnitude) in the outcome.

Interpretation Rules of Thumb

The following thresholds are presented as landmarks only, and any use of such “labels” should be explicitly justified. Please consider with caution.

• Probability of Direction (pd): In most cases, it seems that the pd corresponds to the frequentist one-sided p value through the formula p value = (1-pd/100) and to the two-sided p value (the most commonly reported) through the formula p value = 2*(1-pd/100). Thus, a pd of 95%, 97.5% 99.5% and 99.95% corresponds approximately to a two-sided p value of respectively .1, .05, .01 and .001. Thus, for convience, we recommend using the following reference values:

  • pd <= 95% ~ p > .1: uncertain
  • pd > 95% ~ p < .1: possibly existing
  • pd > 97%: likely existing
  • pd > 99%: probably existing
  • pd > 99.9%: certainly existing

• ROPE (full): Extra caution is required as its interpretation highly depends on other parameters such as sample size and ROPE range.

  • > 99% in ROPE: negligible (we can accept the null hypothesis)
  • > 97.5% in ROPE: probably negligible
  • <= 97.5% & >= 2.5% in ROPE: undecided significance
  • < 2.5% in ROPE: probably significant
  • < 1% in ROPE: significant (we can reject the null hypothesis)

Note: If you have any advice, opinion or such, we encourage you to let us know by opening an discussion thread or making a pull request.

Template Sentence

Based on these suggestions, a template sentence for minimal reporting of a parameter based on its posterior distribution could be:

• “the effect of X has a probability of pd of being negative (Median = median, 89% CI [***…***, ***…***]) and can be considered as significant (ROPE% in ROPE).”