1 Introduction

This package provides a collection of functions to simulate data, estimate item parameters and calculation of likelihood ratio test in multistage designs. In multistage tests, different groups of items (blocks) are presented to test persons depending on their response behavior to previous item groups. Multistage testing is thus a simple form of adaptive testing. If data are collected on the basis of such a multistage design and the items are estimated using the Conditional Maximum Likelihood (CML) method, Glas (1988) has shown, that the item parameters are biased. While Eggen & Verhelst (2011) propose to avoid the bias using the marginal maximum likelihood (mml) method, Zwitser & Maris (2015) showed in their work, that taking the applied multistage design in consideration and including it in the estimation of the item parameters, the estimation of item parameters is not biased using the CML method. Their proposed solution is implemented in our package. In this package, the approach of Zwitser & Maris (2015) is implemented. It was ensured that the necessary specification of the multistage design is as simple as possible and will be described in detail below. For the multistage design, the elementary symmetric function has to be calculated several times, so that this is as efficient as possible, the relevant functions have been written in RCPP (Eddelbuettel & Balamuta, 2017). In addition to estimating the item parameters for multistage designs, it is also possible to estimate the item parameters for conventional designs. In order to further increase the efficiency, the algorithm of Verhelst, Glas, & Van der Sluis (1984) was used for the elementary symmetry function.

The package is well tested, but errors can still be exist. If you find a bug, please report it to us (with an working example) preferred via github: issues

2 Package

Below are a few sample applications of our package, which should provide information for the start:

2.1 Rasch model without multistage design

For the estimation of item parameters of a simple 1-PL model, the function tmt_rm must be called. Here only the data set has to be passed (as matrix or data.frame) to estimate item parameters. Additional arguments allow to turn off the estimates of the standard errors (for example, for time reasons) or to switch from “nlminb” to “optim” (as in the example below). Per default the items are normalized to sum = 0 as recommended by Glas (2016).

2.2 Rasch model with simple multistage design

If the item parameters are to be estimated for a multistage design, for which the package was genuinely built, then the design must first be created. For the multistage design, it is necessary that each block is defined, as exemplified below with B1, B2, etc. The name of the block is arbitrary. The name of the block follows with “=~” the vector with the items of the respective block. If all blocks have been defined, the start block(s) must be defined. The starting block(s) is/are described with a name and the sign “==” (double equal sign) followed by the particular block. Finally, all branches must be defined. Each branch has a name followed by the “:=” character in the starting block and all other blocks in the branch. Each block must be followed in parentheses by the minimum (first number) and maximum (second number) of solved items in the respective block.

Keeping the syntax is very important, otherwise the design can not be translated.

component syntax example
block =~ B1 =~ c(i1, i2, i3, i4, i5)
starting block == Start == B2
branch := b1 := Start(minSolved,maxSolved) + B1(minSolved, maxSolved)

To estimate the item parameters the function tmt_rm must be called with the additional information of the multistage design. Additional arguments allow to turn off the estimates of the standard errors (for example, for time reasons) or to switch from “nlminb” to “optim” (as in the example below).

  # Example for multistage-design
  mstdesign <- "
    B1 =~ c(i1, i2, i3, i4, i5)
    B2 =~ c(i6, i7, i8, i9, i10)
    B3 =~ c(i11, i12, i13, i14, i15)

    # define starting Block
    Start == B2

    # define branches
    b1 := Start(0,2) + B1(0,5)
    b2 := Start(3,5) + B3(0,5)
  "
    # generate item parameters with corresponding names to the multistage design
  items <- seq(-1,1, length.out = 15)
  names(items) <- paste0("i",1:length(items))
  
    # generate random data under given multistage design
  dat <- tmt_sim(mstdesign = mstdesign, 
                      items = items, 
                      persons = 500, 
                      mean = 0, 
                      sd = 1)
    # estimate the item parameters under the given multistage-design
  dat.rm <- tmt_rm(dat = dat, 
                  mstdesign = mstdesign, 
                  optimization = "optim")
  
    # print summary of item parameters
  summary(dat.rm)
#> 
#> Call:
#>   tmt_rm(dat = dat, mstdesign = mstdesign, optimization = "optim")
#> 
#> 
#> Results of Rasch model (mst) estimation: 
#> 
#> Difficulty parameters: 
#>              est.b_i1   est.b_i2   est.b_i3   est.b_i4   est.b_i5   est.b_i6
#> Estimate   -0.7211684 -0.8292145 -0.7750037 -0.5428493 -0.4548954 -0.3668290
#> Std. Error  0.1467064  0.1469657  0.1468081  0.1467566  0.1469998  0.1029584
#>               est.b_i7   est.b_i8  est.b_i9 est.b_i10 est.b_i11 est.b_i12
#> Estimate   -0.08228339 -0.2054576 0.1535489 0.2491042 0.5489391 0.7174808
#> Std. Error  0.10244709  0.1025473 0.1027568 0.1030808 0.1499833 0.1504586
#>            est.b_i13 est.b_i14 est.b_i15
#> Estimate   0.5849513 0.7748798 0.9487972
#> Std. Error 0.1500235 0.1507871 0.1523017
#> 
#> CLL: -2029.587 
#> Number of iterations: 71 
#> Number of parameters: 15

2.3 Likelihood-ratio Test

The likelihood ratio test of Andersen (Andersen, 1973) is also implemented. The estimated item parameters either from an simple 1-PL model or from a 1-PL model with multistage design could be passed to the function tmt_lrtest. This function is also a generic function (like tmt_rm), which calls the specific function for data with/without multistage design. For the application on very large data sets it is possible to parallelize the tmt_lrtest function. For this purpose only the amount of cores has to be passed as additional argument (it is recommended to use three cores, if possible).

References

Andersen, E. B. (1973). A goodness of fit test for the Rasch model. Psychometrika, 38(1), 123–140. doi:10.1007/BF02291180

Eddelbuettel, D., & Balamuta, J. J. (2017). Extending extitR with extitC++: A Brief Introduction to extitRcpp. PeerJ Preprints, 5, e3188v1. doi:10.7287/peerj.preprints.3188v1

Eggen, T., & Verhelst, N. D. (2011). Item Calibration in Incomplete Testing Designs. Psicologica: International Journal of Methodology and Experimental Psychology, 32(1), 107–132.

Glas, C. (1988). The Rasch model and multistage testing. Journal of Educational Statistics, 13(1), 45–52. doi:10.3102/10769986013001045

Glas, C. A. W. (2016). Maximum-likelihood estimation. In W. van der Linden (Ed.), Handbook of item response theory: Volume two: Statistical tools (pp. 197–236). New York, NY: CRC Pres. doi:10.1007/978-1-4612-4230-7_5

Verhelst, N., Glas, C., & Van der Sluis, A. (1984). Estimation problems in the rasch model: The basic symmetric functions. Computational Statistics Quarterly, 1(3), 245–262.

Zwitser, R. J., & Maris, G. (2015). Conditional statistical inference with multistage testing designs. Psychometrika, 80(1), 65–84. doi:10.1007/s11336-013-9369-6