Appendix 1: Using cobalt with Other Preprocessing Packages

f.build()

f.build() is a small tool that can be helpful in quickly specifying formula inputs to functions. An example is provided below:

data("lalonde", package = "cobalt")
covs <- subset(lalonde, select = -c(treat, re78))
f.build("treat", covs)

## treat ~ age + educ + race + married + nodegree + re74 + re75
## <environment: 0x7fd99aa54c00>

The function creates a formula object from two inputs: the first argument is the quoted name of the variable to be the left hand side (response) variable in the formula; the second argument is a vector of right hand side (predictor) variable names or a data frame, the variable names of which are to be the predictor variables. The utility of f.build() is that the user does not have to manually type out the name of every covariate when entering a formula into a function. It can be used simply in place of a formula, as in the following examples, which make use of the objects defined above:

# Generating propensity scores using logistic regression
p.score <- glm(f.build("treat", covs), data = lalonde, family = "binomial")$fitted.values

# Using matchit() from the MatchIt package
library("MatchIt")
m.out <- matchit(f.build("treat", covs), data = lalonde, method = "nearest")

f.build() can also be used in the Matching, optmatch, ebalance, and formula interfaces in bal.tab().

Note: in an earlier version of cobalt, the first argument of f.build() was unquoted; this has been changed to reap the benefits of standard evaluation, including the ability to loop through response variables.

splitfactor() and unsplitfactor()

Some functions (outside of cobalt) are not friendly to factor or character variables, and require numeric variables to operate correctly. For example, some regression-style functions, such as ebalance() in ebal, can only take in non-singular numeric matrices. Other functions will process factor variables, but will return output in terms of dummy coded version of the factors. For example, lm() will create dummy variables out of a factor and drop the reference category to create regression coefficients.

To prepare data sets for use in functions that do not allow factors or to mimic the output of functions that split factor variables, users can use splitfactor(), which takes in a data set and the names of variables to split, and outputs a new data set with newly created dummy variables. Below is an example splitting the race variable in the Lalonde data set into dummies, eliminating the reference category ("black"):

head(lalonde)

lalonde.split <- splitfactor(lalonde, "race")
head(lalonde.split)

It is possible to undo the action of splitfactor() with unsplitfactor(), which takes in a data set with dummy variables formed from splitfactor() or otherwise and recreates the original factor variable. If the reference category was dropped, its value needs to be supplied.

lalonde.unsplit <- unsplitfactor(lalonde.split, "race", 
                                 dropped.level = "black")
head(lalonde.unsplit)

Notice the original data set and the unsplit data set look identical.

get.w()

get.w() allows users to extract weights from the output of a call to a preprocessing function in one of the supported packages. Because each package stores weights in different ways, it can be helpful to have a single function that applies equally to all outputs. twang has a function called get.weights() that performs the same functions with slightly finer control for the output of a call to ps(). See the section “Comparing balancing methods” in the main vignette for examples of the use of get.w().

Using bal.tab() with twang

Generalized boosted modeling (GBM), as implemented in twang, can be an effective way to generate propensity scores and weights for use in propensity score weighting. bal.tab() functions similarly to the functions bal.table() and summary() when used with GBM in twang. Below is a simple example of its use:

library("twang")
data("lalonde", package = "cobalt") ##If not yet loaded
covs0 <- subset(lalonde, select = -c(treat, re78, nodegree, married))

ps.out <- ps(f.build("treat", covs0), data = lalonde, 
             stop.method = c("es.mean", "es.max"), 
             estimand = "ATT", n.trees = 1000, verbose = FALSE)
bal.tab(ps.out, stop.method = "es.mean")

## Call
##  ps(formula = f.build("treat", covs0), data = lalonde, n.trees = 1000, 
##     verbose = FALSE, estimand = "ATT", stop.method = c("es.mean", 
##         "es.max"))
## 
## Balance Measures
##                 Type Diff.Adj
## prop.score  Distance   0.4781
## age          Contin.   0.0143
## educ         Contin.  -0.0921
## race_black    Binary   0.0436
## race_hispan   Binary  -0.0049
## race_white    Binary  -0.0387
## re74         Contin.  -0.0001
## re75         Contin.   0.0403
## 
## Effective sample sizes
##            Control Treated
## Unadjusted 429.000     185
## Adjusted    52.778     185

The output looks a bit different from twang’s bal.table() output. First is the original call to ps(). Next is the balance table containing mean differences for the covariates included in the input to ps(). Last is a table displaying sample size information, similar to what would be generated using twang’s summary() function. The “effective” sample size is displayed when weighting is used; it is calculated as is done in twang. See the twang documentation, ?bal.tab, or “Details on Calculations” in the main vignette for details on this calculation.

When using bal.tab() with twang, the user must specify the ps object, the output of a call to ps(), as the first argument. The second argument, stop.method, is the name of the stop method(s) for which balance is to be assessed, since a ps object may contain more than one if so specified. bal.tab() can display the balance for more than one stop method at a time by specifying a vector of stop method names. If this argument is left empty or if the argument to stop.method does not correspond to any of the stop methods in the ps object, bal.tab() will default to displaying balance for all stop methods available. Abbreviations are allowed for the stop method, which is not case sensitive.

The other arguments to bal.tab() when using it with twang have the same form and function as those given when using it without a conditioning package, except for s.d.denom. If the estimand of the stop method used is the ATT, s.d.denom will default to "treated" if not specified, and if the estimand is the ATE, s.d.denom will default to "pooled", mimicking the behavior of twang. The user can specify their own argument to s.d.denom, but using the defaults is advised.

If sampling weights are used in the call to ps(), they will be automatically incorporated into the bal.tab() calculations for both the adjusted and unadjusted samples, just as twang does.

mnps objects resulting from fitting models in twang with multinomial treatments are also compatible with cobalt. See the section “Using cobalt with multinomial treatments” in the main vignette. iptw objects resulting from fitting models in twang with longitudinal treatments are also compatible with cobalt. See the Appendix 3 vignette. ps.cont objects resulting from using ps.cont() in WeightIt, which implements GBM for continuous treatments, are also compatible. See the section “Using cobalt with continuous treatments” in the main vignette.

Using bal.tab() with Matching

The Matching package is used for propensity score matching, and was also the first package to implement genetic matching. MatchIt calls Matching to use genetic matching and can accomplish many of the matching methods Matching can, but Matching is still a widely used package with its own strengths. bal.tab() functions similarly to Matching’s MatchBalance() command, which yields a thorough presentation of balance, and makes Matching the only package of those integrated with cobalt to display variance ratios by default. Below is a simple example of the use of bal.tab() with Matching:

library("Matching")
data("lalonde", package = "cobalt") #If not yet loaded
covs0 <- subset(lalonde, select = -c(treat, re78, nodegree, married))

fit <- glm(f.build("treat", covs0), data = lalonde, family = "binomial")
p.score <- fit$fitted.values
match.out <- Match(Tr = lalonde$treat, X = p.score, estimand = "ATT")

bal.tab(match.out, formula = f.build("treat", covs0), data = lalonde)

## Balance Measures
##                Type Diff.Adj
## age         Contin.   0.2144
## educ        Contin.  -0.1025
## race_black   Binary   0.0108
## race_hispan  Binary  -0.0070
## race_white   Binary  -0.0038
## re74        Contin.   0.0500
## re75        Contin.   0.0886
## 
## Sample sizes
##                      Control Treated
## All                      429     185
## Matched                  185     185
## Matched (Unweighted)     159     185
## Unmatched                270       0

The output looks quite different from Matching’s MatchBalance() output. Rather than being stacked vertically, balance statistics are arranged horizontally in a table format, allowing for quick balance checking. Below the balance table is a summary of the sample size before and after matching, similar to what Matching’s summary() command would display. The sample size can include a “weighted” and “unweighted” count; the “weighted” count is the sum of the matching weights, while the “unweighted” is the count of units with nonzero matching weights.

The input to bal.tab() is similar to that given to MatchBalance(): the Match object resulting from the call to Match(), a formula relating treatment to the covariates for which balance is to be assessed, and the original data set. This is not the only way to call bal.tab(): instead of a formula and a data set, one can also input a data frame of covariates and a vector of treatment status indicators, just as when using bal.tab() without a conditioning package. For example, the code below will yield the same results as the call to bal.tab() above:

bal.tab(match.out, treat = lalonde$treat, covs = covs0)

The other arguments to bal.tab() when using it with Matching have the same form and function as those given when using it without a conditioning package, except for s.d.denom. If the estimand of the original call to Match() is the ATT, s.d.denom will default to "treated" if not specified; if the estimand is the ATE, s.d.denom will default to "pooled"; if the estimand is the ATC, s.d.denom will default to "control". The user can specify their own argument to s.d.denom, but using the defaults is advisable. In addition, the use of the addl argument is unnecessary because the covariates are entered manually as arguments, so all covariates for which balance is to be assessed can be entered through the formula or covs argument. If the covariates are stored in two separate data frames, it may be useful to include one in formula or covs and the other in addl.

Using bal.tab() with optmatch

The optmatch package is useful for performing optimal pairwise or full matching. Most functions in optmatch are subsumed in MatchIt, but optmatch sees use from those who want finer control of the matching process than MatchIt allows. The output of calls to functions in optmatch is an optmatch object, which contains matching stratum membership for each unit in the given data set. Units that are matched with each other are assigned the same matching stratum. The user guide for optmatch recommends using the RItools package for balance assessment, but below is an example of how to use bal.tab() for the same purpose. Note that some results will differ between cobalt and RItools because of differences in how balance is calculated in each.

#Optimal full matching on the propensity score
library("optmatch")
data("lalonde", package = "cobalt") #If not yet loaded
covs0 <- subset(lalonde, select = -c(treat, re78, nodegree, married))

fit <- glm(f.build("treat", covs0), data = lalonde, family = "binomial")
lalonde$p.score <- fit$fitted.values #get the propensity score
fm <- fullmatch(treat ~ p.score, data = lalonde)

bal.tab(fm, formula = f.build("treat", covs0), data = lalonde)

## Call
##  fullmatch(x = treat ~ p.score, data = lalonde)
## 
## Balance Measures
##                Type Diff.Adj
## age         Contin.   0.1204
## educ        Contin.  -0.0635
## race_black   Binary   0.0054
## race_hispan  Binary  -0.0059
## race_white   Binary   0.0005
## re74        Contin.   0.0172
## re75        Contin.   0.0854
## 
## Sample sizes
##         Control Treated
## All         429     185
## Matched     429     185

Most details for the use of bal.tab() with optmatch are similar to those when using bal.tab() with Matching. Users can enter either a formula and a data set or a vector of treatment status and a set of covariates.

Using bal.tab() with CBPS

The CBPS (Covariate Balancing Propensity Score) package is a great tool for generating covariate balancing propensity scores, a class of propensity scores that are quite effective at balancing covariates among groups. CBPS includes functions for estimating propensity scores for binary, multinomial, and continuous treatments. bal.tab() functions similarly to CBPS’s balance() command. Below is a simple example of its use with a binary treatment:

library("CBPS")
data("lalonde", package = "cobalt") #If not yet loaded
covs0 <- subset(lalonde, select = -c(treat, re78, nodegree, married))

#Generating covariate balancing propensity score weights for ATT
cbps.out <- CBPS(f.build("treat", covs0), data = lalonde)

## [1] "Finding ATT with T=1 as the treatment.  Set ATT=2 to find ATT with T=0 as the treatment"

bal.tab(cbps.out)

## Call
##  CBPS(formula = f.build("treat", covs0), data = lalonde)
## 
## Balance Measures
##                 Type Diff.Adj
## prop.score  Distance  -0.0157
## age          Contin.  -0.0085
## educ         Contin.   0.0050
## race_black    Binary   0.0003
## race_hispan   Binary  -0.0001
## race_white    Binary  -0.0002
## re74         Contin.   0.0003
## re75         Contin.   0.0021
## 
## Effective sample sizes
##            Control Treated
## Unadjusted 429.000     185
## Adjusted   110.799     185

First is the original call to CBPS(). Next is the balance table containing mean differences for the covariates included in the input to CBPS(). Last is a table displaying sample size information. The “effective” sample size is displayed when weighting (rather than matching or subclassification) is used; it is calculated as is done in twang. See the twang documentation, ?bal.tab, or “Details on Calculations” in the main vignette for details on this calculation.

The other arguments to bal.tab() when using it with CBPS have the same form and function as those given when using it without a conditioning package, except for s.d.denom. If the estimand of the original call to CBPS() is the ATT, s.d.denom will default to "treated" if not specified, and if the estimand is the ATE, s.d.denom will default to "pooled". The user can specify their own argument to s.d.denom, but using the defaults is advisable.

When using CBPS and bal.tab() with continuous treatments, the same guidelines apply as when using bal.tab() with continuous treatments without a conditioning package. See the section “Using cobalt with continuous treatments” in the main vignette for more details.

CBPSContinuous objects resulting from fitting models in CBPS with continuous treatments are also compatible with cobalt. See the section “Using cobalt with continuous treatments” in the main vignette. CBPS objects resulting from fitting models in CBPS with multinomial treatments are also compatible with cobalt. See the section “Using cobalt with multinomial treatments” in the main vignette. CBMSM objects resulting from fitting models in CBPS with longitudinal treatments are also compatible with cobalt. See the Appendix 3 vignette.

Using bal.tab() with ebal

The ebal package implements entropy balancing, a method of weighting for the ATT that yields perfect balance on all desired moments of the covariate distributions between groups. Rather than estimate a propensity score, entropy balancing generates weights directly that satisfy a user-defined moment condition, specifying which moments are to be balanced. ebal does not have its own balance assessment function; thus, cobalt is the only way to assess balance without programming, which the ebal documentation instructs. Below is a simple example of using bal.tab() with ebal:

library("ebal")
data("lalonde", package = "cobalt") #If not yet loaded
covs0 <- subset(lalonde, select = -c(treat, re78, race))

#Generating entropy balancing weights
e.out <- ebalance(lalonde$treat, covs0)

## Converged within tolerance

bal.tab(e.out, treat = lalonde$treat, covs = covs0)

## Balance Measures
##             Type Diff.Adj
## age      Contin.       -0
## educ     Contin.        0
## married   Binary        0
## nodegree  Binary        0
## re74     Contin.       -0
## re75     Contin.       -0
## 
## Effective sample sizes
##            Control Treated
## Unadjusted 429.000     185
## Adjusted   247.644     185

First is the balance table containing mean differences for covariates included in the original call to ebalance. In general, these will all be very close to 0. Next is a table displaying effective sample size information. The “effective” sample size is calculated as is done in twang. See the twang documentation, ?bal.tab, or “Details on Calculations” in the main vignette for details on this calculation. A common issue when using entropy balancing is small effective sample size, which can yield low precision in effect estimation when using weighted regression, so it is important that users pay attention to this measure. That said, in simulations by Zhao and Percival (2015), entropy balancing reliably had smaller empirical standard errors than did covariate balancing propensity scores.

The input is similar to that for using bal.tab() with Matching or optmatch. In addition to the ebalance object, one must specify either both a formula and a data set or both a treatment vector and a data frame of covariates.

Using bal.tab() with designmatch

The designmatch package implements various matching methods that use optimization to find matches that satisfy certain balance constraints. bal.tab() functions similarly to designmatch’s meantab() command but provides additional flexibility and convenience. Below is a simple example of using bal.tab() with designmatch:

library("designmatch")
data("lalonde", package = "cobalt") #If not yet loaded
covs0 <- subset(lalonde, select = -c(treat, re78, race))

#Matching for balance on covariates
dmout <- bmatch(lalonde$treat,
                dist_mat = NULL,
                subset_weight = NULL,
                mom = list(covs = covs0,
                           tols = absstddif(covs0, lalonde$treat, .005)),
                n_controls = 1,
                total_groups = 185)

##   Building the matching problem... 
##   GLPK optimizer is open... 
##   Finding the optimal matches... 
##   Optimal matches found

bal.tab(dmout, treat = lalonde$treat, covs = covs0)

## Balance Measures
##             Type Diff.Adj
## age      Contin.   0.0038
## educ     Contin.   0.0054
## married   Binary   0.0000
## nodegree  Binary   0.0054
## re74     Contin.  -0.0120
## re75     Contin.  -0.0076
## 
## Sample sizes
##           Control Treated
## All           429     185
## Matched       185     185
## Unmatched     244       0

The input is similar to that for using bal.tab() with Matching or optmatch. In addition to the designmatch() output object, one must specify either both a formula and a data set or both a treatment vector and a data frame of covariates. The output is similar to that of optmatch.

Using bal.tab() with other packages

It is possible to use bal.tab with objects that don’t come from these packages using the default method. If an object that doesn’t correspond to the output from one of the specifically supported packages is passed as the first argument to bal.tab, bal.tab will do its best to process that object as if it did come from a supported package. It will search through the components of the object for items with names like "treat", "covs", "data", "weights", etc., that have the correct object types. Any additional arguments can be specified by the user.

The goal of the default method is to allow package authors to rely on cobalt as a substitute for any balancing function they might otherwise write. By ensuring compatibility with the default method, package authors can have their users simply supply the output of a compatible function into cobalt functions without having to write a specific method in cobalt. A package author would need to make sure the output of their package contained enough information with correctly named components; if so, cobalt functions can be used as conveniently with the output as it is with specifically supported packages.

Below, we demonstrate this capability with the output of optweight, which performs a version of propensity score weighting using optimization. No bal.tab method has been written with optweight output in mind; rather, optweight was written to have output compatible with the default method of bal.tab.

library("optweight")
data("lalonde", package = "cobalt")

#Estimate the weights using optimization
ow.out <- optweight(treat ~ age + educ + married + race + re74 + re75,
                    data = lalonde, estimand = "ATE", tols = .01)

#Note the contents of the output object:
names(ow.out)

##  [1] "weights"   "treat"     "covs"      "s.weights" "estimand" 
##  [6] "focal"     "call"      "tols"      "duals"     "info"

#Use bal.tab() directly on the output
bal.tab(ow.out)

## Call
##  optweight(formula = treat ~ age + educ + married + race + re74 + 
##     re75, data = lalonde, tols = 0.01, estimand = "ATE")
## 
## Balance Measures
##                Type Diff.Adj
## age         Contin.   0.0000
## educ        Contin.   0.0100
## married      Binary  -0.0100
## race_black   Binary   0.0100
## race_hispan  Binary   0.0000
## race_white   Binary  -0.0100
## re74        Contin.  -0.0100
## re75        Contin.   0.0085
## 
## Effective sample sizes
##            Control Treated
## Unadjusted 429.000 185.000
## Adjusted   349.421  52.041

The output is treated as output from a specifically supported package. See ?bal.tab.default for more details and another example.