Different Output between Stata and ggeffects

Daniel Lüdecke

2018-10-11

Why is the output from Stata different from the output from ggeffect?

Stata’s equivalent to the marginal effects produced by ggeffects is the margins-command. However, the results are not always identical. For models from non-gaussian families, point estimates for the marginal effects are identical, but the confidence intervals differ.

Here is an explanation, why there is a difference. First, we fit a logistic regression model.

set.seed(5)

data <- data.frame(
  outcome = rbinom(100, 1, 0.5),
  var1 = rbinom(100, 1, 0.1),
  var2 = rnorm(100, 10, 7)
)

m <- glm(
  outcome ~ var1 * var2, 
  data = data, 
  family = binomial(link = "logit")
)

Example with graphical output

The Stata plot

This is the code in Stata to produce a marginal effects plot.

The resulting image looks like this.

The ggeffects plot

When we use ggeffects, the plot slighlty differs.

As we can see, the confidence intervals in the Stata plot are outside the plausible range of [0, 1], which means that the predicted uncertainty from the Stata output has a probability higher than 1 and lower than 0, while ggpredict() computes confidence intervals within the possible range.

Example with numerical output

Here is another example with numeric output. First, we fit a model with survey-design.

library(survey)
set.seed(123)

df1 <- data.frame(
  id = seq(1, 100, by = 1),
  gender = as.factor(rbinom(n = 100, size = 1, prob = 0.50)),
  working = as.factor(rbinom(n = 100, size = 1, prob = 0.40)),
  income  = sample(0:100000, 100),
  pweight = sample(50:500, 100, replace = TRUE)
)
  
des <- svydesign(
  id = ~id,
  weights = ~pweight,
  data = df1
)

m <- svyglm(
  gender ~ working * income,
  family = quasibinomial(),
  data = df1,
  design = des
)

The same exact analysis in R (using ggpredict()) and Stata slightly differs. We get the same point estimate, but different confidence intervals again.

The Stata output

margins, at(working=(0 1) income=(100 1000 10000))

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         _at |
          1  |   .4906395   .1375135     3.57   0.001     .2177829    .7634961
          2  |   .4893027   .1356353     3.61   0.000     .2201729    .7584325
          3  |   .4759466   .1173992     4.05   0.000     .2430012     .708892
          4  |    .579421   .1687542     3.43   0.001      .244576    .9142659
          5  |   .5773197    .166619     3.46   0.001     .2467115    .9079279
          6  |   .5561654   .1454702     3.82   0.000      .267521    .8448097
------------------------------------------------------------------------------

The ggeffects output

Conclusion

I guess Stata is getting the confidence intervals wrong here. It looks like predictions and standard errors returned in Stata are on the (transformed) response scale. The confidence intervals are then computed by estimate +/- 1.98 * standard error, which may lead to confidence intervals that are out of reasonable bounds (e.g. above 1 or below 0 for predicted probabilities).

The transformed estimate (on the response scale) is always between 0 and 1, and the same is true for the transformed standard errors. However, adding or substracting approx. 2 * transformed SE to the transformed estimate does no longer ensure that the confidence intervals are within the correct range.

The more precise way to do the calculation is to calculate estimates, standard errors and confidence intervals on the (untransformed) scale of the linear predictor and then back-transform.

See examples:

Less appropriate approach

More precise approach