To cite package markovchain in publications use:

Tae Seung Kang

Giorgio Alfredo Spedicato and Tae Seung Kang. markovchain: discrete time Markov chains made easy

Corresponding BibTeX entry:

  @Manual{,
    title = {markovchain: discrete time Markov chains made easy},
    author = {Giorgio Alfredo Spedicato},
    month = {08},
    year = {2016},
    note = {R package version 0.6},
  }