greybox v0.5.8 (Release data: 2020-02-08)
==============
Changes:
* assoc() function now has a method parameter, which allows to force the function to use specific measure of association.
* assoc() now can produces just a vector or a non-square matrix, depending on the value of y.
* New function pcor() - partial correlations.
* Both assoc() and pcor() now give warnings in cases with categorical variables and pearson / spearman / kendall correlations. They will also convert the factors into the numeric values in order to be able to proceed with calculations.
* And a minor optimisation in assoc() - it now fully relies on cor.test(), ignoring cor() function.
* Use NumDeriv for the calculation of Hessian for plogis and pnorm.
* Use non-MLE variance in the calculation of vcov in case of mixture distribution models.
* summary() now also calculates the R^2 and R^2-adj, but does not print them out.
* Minor improvements in speed of mcor() and determination().
* alm() now does not reestimate parameters if distribution=c("dnorm","dlnorm").
* Updated vignette of alm() with more reasonable explanation of the occurrence variable.
Bugfixes:
* alm(): AR model with provided parameters would not work correctly.
* alm(): fixed the entropy calculation for the dinvgauss.
* plot.greybox() did not work in case of the infinite bounds.
* A bugfix for lmCombine() + alm(), which caused alm() to drop some of the variables, failing the lmCombine(). Now if fast=TRUE, alm() will not do that.
* stepwise() did not calculate the number of parameters correctly in case of distribution="dnorm" and factor variables.
* Fix for alm() when data is not provided, but is in the environment.
greybox v0.5.7 (Release data: 2019-12-10)
==============
Changes:
* plot.greybox() now also produces LOWESS lines on the scatterplots. There is a parameters that regulates this.
* A bit of tuning of plot.greybox() graphs.
* Inverse Gaussian distribution for alm() is now available and works okay. Note that this is a non-conventional model. Check vignette for alm() function.
* Legend for tableplot() is now by default FALSE.
Bugfixes:
* alm() would be stuck in some cases, when determination returns NaNs.
* Fix of a bug in alm() for the new version of R, where class(matrix) is now c("matrix","array"), and not just "matrix".
greybox v0.5.6 (Release data: 2019-10-29)
==============
Changes:
* plot.greybox() now produces 4 plots instead of one: Fitted over time, Standardised residuals vs Fitted, Absolute Residuals vs Fitted and Q-Q plot with the specified distribution. Should be useful for model diagnostics.
* In fact, plot.greybox() can now also produce Residuals Squared vs Fitted, ACF and PACF. Choose your value!
* An update in the alm() vignette, explaining the Box-Cox Normal distribution.
* Updates in lmCombine and lmDynamic - now we produce the correct covariance matrix of the parameters from the functions.
* rstudent() is now also available, together with the plots. See plot.greybox() for more detail.
* Tuned the vcov calculation for alm(). Hopefully, it will result in more accurate covariance matrix of parameters.
* alm() now estimates df for the distribution="dt" if it is not provided.
Bugfixes:
* Fixed issues with lmCombine and lmDynamic, when no variables are selected.
* A bugfix in the interval construction for the Student's distribution.
* Don't round the importance in lmCombine!
greybox v0.5.5 (Release data: 2019-09-19)
==============
Bugfixes:
* Removed c.ts(), as it caused recursive behaviour.
greybox v0.5.4 (Release data: 2019-09-16)
==============
Changes:
* The function measures() now allows selecting between Naive and Arithmetic Mean benchmarks for relative measures.
* A small optimisation in alm(), removing the unnecessary data from the memory.
* New method - c.ts() - enhancing the work of ts() class of stats package. Combines several ts objects in one, stacking them one after another.
* lmDynamic() now reports the weights for the models.
Bugfixes:
* Fix for spread() and names of variables with spaces and special characters.
* Similar fix for stepwise().
* Fixes for the names with spaces and special characters in lmCombine() and lmDynamic().
* Fixed a bug for alm(), which sometimes produced "y" instead of the proper name of the response variable.
* A bugfix in ro() with co=FALSE - it produced the forecasts for the wrong samples. Thanks to Zhan Peng for reporting the bug! :)
greybox v0.5.3 (Release data: 2019-07-31)
==============
Changes:
* The summary is now more consistent with the ones from smooth package v2.5.2.
* Corrected typos in the outputs.
* Renamed RelMAE into rMAE and RelRMSE into rRMSE for consistency purposes. Similar renaming of RelAME into rAME and RelMIS into rMIS.
Bugfixes:
* lmCombine and lmDynamic did not work if there was only one variable to combine.
* A fix for lmCombine and lmDynamic in case of "dalaplace".
* A fix for the selected variable in occurrenceModel with data.table object.
* predict.greybox() did not work correctly in case of no newdata provided.
* determination, association and mcor did not work with "data.table" and "tbl" classes well. Now they just change the classes of these objects to data.frame as a temporary solution.
greybox v0.5.2 (Release data: 2019-06-15)
==============
Changes:
* Added references to the vignettes in the documentation.
* Functions for Box-Cox Normal distribution (based on the original 1964 paper): dbcnorm(), pbcnorm(), rbcnorm() and qbcnorm().
* alm() now supports Box-Cox Normal distribution (thus box-cox transform of the data).
* Tuned the initial parameters in alm() for recursive models.
* Tuning in the code for the recursive model. It should work approximately 2 times faster now...
* Cosmetic changes in rmc(). In case of "mcb", the methods in the same group as the selected one, have the same darker colour of the mean.
* Rolling back the "quiet" parameter to "silent".
* Switch off entropy of NegBin and Poisson in alm() for now... Otherwise it becomes too inflated and unreasonably difficult.
Bugfixes:
* Fixed a bug with the estimation of models, where some of the variables did not have any variability (i.e. in cases of occurrence model).
* alm() with distribution="plogis" sometimes would not work well with Arima() function.
* A bugfix in plot(forecast), where a list was used in some of the cases for the holdout.
* A bugfix for dropping dummies in case of a multicollinearity in reused regression.
greybox v0.5.1 (Release data: 2019-04-27)
==============
Changes:
* The parameter "silent" is now renamed into "quiet" in ro(), xregExpander(), xregMultiplier() and xregTransformer() functions. It only specifies the output in console now.
* Similarly, "bruteForce" is now "bruteforce" in lmCombine(), lmDynamic() and determ(),
* "B" is now "parameters" in alm(),
* "checks" is now "fast" (note, the opposite meaning),
* "style" is now "outplot" in rmc().
* All the functions now return "fitted" instead of "fitted.values". This is not a big deal, the fitted() method will work as always.
* forecast() function now accepts h - the forecast horizon. If h!=nrow(newdata), then the function will either cut off values, or produce forecasts for the explanatory variables.
Bugfixes:
* alm() would not do checks properly when only one explanatory variable was used.
* lmCombine() and lmDynamic() would not work because they would refer to non-existent "ourModel". Thanks leungi for the bug report!
* lmCombine() and lmDynamic() used to warn about the computational time after doing things, not before that.
* predict.alm() would not work well if only one-step-ahead prediction was needed.
greybox v0.5.0 (Release data: 2019-04-20)
==============
Changes:
* New function - polyprod(), returning the product of two polynomials.
* alm() now has parameters ar and i, which define the order of respective elements of ARIMA model.
* alm() now checks for stationarity of AR.
* alm() now produces the fitted for the zero observations.
* alm() now accepts parameters for the nloptr.
* In case of occurrence model, the expected entropy is added to the likelihood.
* Tuning of the initials for the recursive model.
* alm() with distribution=c("pnorm","plogis") and ARI(p,d) now produces adequate estimates of probability and forecasts.
* Error measures imported from smooth. Accuracy() function is renamed into measures().
greybox v0.4.2 (Release data: 2019-03-10)
==============
Changes:
* nParam, nobs, sigma and AICc, BICc methods for the "varest" class of the functions for "vars" package. Should allow selection using the corrected AIC and BIC.
* graphmaker() now does not need for forecast to start at the right place. If start(actuals)=start(forecast), it will place forecast at the end.
* graphmaker() now also allows not to reset par, so that you can add arbitrary elements to the graph.
* rmc() now returns the exponentiated values of means and intervals in case of distribution="dlnorm".
* nParam() method has been renamed into nparam().
* A new method - actuals(), which returns actuals from the model (similar to "getResponse" of forecast package).
* nobs() and actuals() now also have a hidden parameter all, which determines, whether to return all the values or only demand sizes in case of occurrence model. The default behaviour for nobs is FALSE, and for actuals is TRUE. This is driven by the functions relying on these methods.
* xregExpander() now allows specifying how to fill in the gaps for the lagged variables.
Bugfixes:
* Fixed a bug in stepwise(), due to which the wrong fitted values were generated.
* Annoying bug with occurrence!="none" models and factors.
* lmCombine and lmDynamic did not work correctly with NAs.
* alm produced errors for simple regressions and occurrence.
* stepwise would produce errors, when there was no variability in the data.
greybox v0.4.1 (Release data: 2019-01-27)
==============
Changes:
* The extended vignette on marketing analytics tools (tableplot, spread, cramer, mcor, assoc and determ).
* determination() can now be smart and use stepwise(). This might be especially useful for cases of fat regressions diagnostics.
* stepwise() now also works with factors. And faster than the previous version, when dat is numeric.
* use parameter in mcor(), cramer() and assoc(). By default NAs are removed.
* lmDynamic and lmCombine now work with factors and with all the alm distributions.
* lmDynamic and lmCombine now also have the parameter paralle, which defines whether to make calculations in parallel or not.
* alm() now removes NaNs if they are present in the data.
Bugfixes:
* Fixes in mcor and assoc, which did not work correctly in some cases of factors provided as x or y.
* xregExpander() did not work appropriately when extrapolate=FALSE.
* Fixed an annoying bug in predict.greybox(), when the newdata did not contain the response variable.
* stepwise() sometimes produced data with wrong colnames. Now it doesn't.
greybox v0.4.0 (Release data: 2019-01-04)
==============
Changes:
* Added Burnham & Anderson in the library.bib in the vignettes.
* summary() now prints the name of the response variable.
* lmCombine now returns logLik adequate to the selected distribution.
* Sample size, estimated parameters and degrees of freedom are now also returned in the summary.
* Use Choleski decomposition in vcov.alm function instead of solve.
* xregMultiplier() function, allowing producing cross-products of variables.
* Two new cool functions: tableplot() - produces plots for the two categorical variables, showing graphically, where the most frequent values happen; spread() - plots a matrix of scatterplots / boxplots / tableplots, depending on the type of the provided variable.
* Added a clarification about the most efficient use of RMC (together with RelMAE / RelMSE).
* alm() now works with factors.
* spread() now allows doing log transforms of numerical data.
* New function: cramer() - that calculates Cramer's V and the according statistics. Good for measuring the association between the categorical variables.
* New function: mcor() - multiple correlation between the numerical and catgorical variables.
* New function: association() aka assoc() - returns the matrix of measures of association (the values depend on the types of variables under consideration).
* xregExpander() now has 'extrapolate' parameter which allows deciding whether the missing values need to be extrapolated or not.
* graphmaker() now does not plot forecast if it is NA. In addition, the legend is now slightly more flexible.
* summary() now prints the df for dchisq and size for the dnbinom.
* tableplot() now also accepts dataframes, plotting the first two columns.
* rmc() now should work much faster in cases of distribution=c("dnorm","dlnorm").
* plot.rmc() now only resets par(), when style="lines". In the "mcb" style, it won't change par, so that the user can add any elements they want.
* alm() now estimates sigma parameter for dfnorm directly using likelihood.
Bugfixes:
* lmCombine() and lmDynamic() did not work well when the data.frame was provided as data.
* pointLik() did not work with lmCombine() because scale parameter was not available. Similarly, it did not work with stepwise() in case of normal distribution.
* predict.alm() was misbehaving in case of non-null occurrence.
* determination() now works with factors.
* Additional explanations for RMC.
* Bugfix in alm() for cases of occurrence and the provided factors.
* vcov.alm would not work in cases of occurrence model having different set of variables than the sizes one.
* nParam() did not take the number of parameters in the occurrence part into account.
* plot.predict.alm() now works fine in case of newdata=NULL.
* Fixes in alm() for dalaplace, dnbinom and dchisq distributions and the usage of factors.
* predict.alm() would not work for the models with intercept only.
* alm() in cases of distribution="plogis" or "pnorm" some times could not produce errors correctly (due to exp(huge number)). Now it does.
greybox v0.3.3 (Release data: 2018-11-27)
==============
Changes:
* Student t distribution in alm.
* Beta distribution in alm. When will I stop? I guess, I'll do that when I stop procrastinating...
* New functions for three parameter log normal distribution.
* New function for the non-linear transformation of the provided variables - xregTransformer. Use with care!
* Renamed parameter "b" into "scale" in laplace, alaplace and s functions.
* lmCombine now returns a matrix with the selected variables and the respective information criteria.
Bugfixes:
* Corrected a typo of "plogos" in alm.
* is-functions for greybox now rely on "inherits" function.
* Some bugfixes in alm() with dchisq. But there's a lot of confusion there, including stuff in predict.alm.
* Stepwise was not calculating the number of degrees of freedom correctly in case of distribution="dnorm".
* Stepwise did not call for alm in case of distribution!="dnorm".
* Bugfix in rs, rlaplace and other r-functions, where thethe duplicates of the provided parameters were removed. This caused problems in cases of huge samples, when identical random numbers could have appeared.
greybox v0.3.2 (Release data: 2018-10-25)
==============
Changes:
* Updates in the vignette of alm.
* Although the square link is tricky in case of Chi Squared distribution, it is the correct thing to do. The alm() function now checks if the generated mu_t is positive, and if not, it returns a big number, forcing the solver to stick with the positive solution.
* If inverting Hessian fails, return very big values (meaning high uncertainty).
* predict.alm() now saves the original level of probability.
* New initialisation for plogis, pnorm, dpois and dnbinom in alm().
* dpois, plogis, pnorm and dnbinom now use maxeval=500. All the others have 100. This should improve the estimates of parameters in difficult cases.
* We now return only that data, that was used in the model construction in alm().
* lmCombine and lmDynamic now should work with the distributions of alm(). The only two that are not 100% correct are dchisq and dfnorm - the fitted values of those are incorrect.
* removed getResponse.alm. Now getResponse.greybox does what is necessary.
* Residuals of dnbinom and dpois are now calculated as y - fitted.
* predict with distriubion="dnbinom" in cases, when scale is not available, is now calculated based on the definition of scale via variance and mean.
* pointLik.ets() is now calculated differently, so that sum(pointLik) is close to the logLik produced by ets() function. The problem with logLik of ets() is that it is not calculated correctly, chopping off some parts of normal distribution. Total disaster!
* New set of distribution functions - for Asymmetric Laplace Distribution (ALD).
* alm() now estimates models with Asymmetric Laplace Distribution with predefined alpha parameter. This is equivalent to the quantile regression with alpha quantile, but is done from the likelihood point of view. It also allows estimating alpha in sample.
* The correct prediction and confidence intervals for the alm() with ALD.
* predict function now also works, when newdata is not provided (although why would you want to do that?).
Bugfixes:
* predict.alm() sometimes produced NAs in the lower bound.
* When having varying probability, plot.predict sometimes struggled to use the correct value.
* plot.predict.greybox() now passes values from ellipsis to graphmaker.
* The intervals for dnorm are now corrected for the cases of occurrence model.
* plot.predict works differently when there are Inf values in the bounds.
* predict() did not work correctly for simple linear regression.
* alm() returned a vector in data for cases of the model with intercept only.
* predict.alm() with distribution=c(dlaplace, ds, dfnorm) did not work in some cases of fixed level of probability.
* predict.alm() now writes lower and upper values in the existing elements of the list instead of creating the new ones.
* predict.alm() did not produce prediction intervals for "dnorm".
* plot.greybox() now checks whether there is a need to transform the data to the binary variable or not.
greybox v0.3.1 (Release data: 2018-09-07)
==============
Changes:
* Corrected some typos in README.md and added description of several functions.
* predict() and forecast() functions now produce confidence and prediction intervals for the provided holdout sample data. forecast() is just a wrapper around predict().
* Normal and log-normal distributions are now available in alm().
* rmc() now uses alm().
* stepwise(), lmCombine() and lmDynamic() can now also be constructed with distributions from alm(). They use lm() in case of "dnorm" and alm() otherwise.
* alm() now does not return vcov if you didn't ask for it (should increase speed of computation for large datasets).
* alm() can be constructed with the provided vector of parameters (needed for vcov method).
* We now use well-known analytical solutions for the cases of distribution="dnorm" of alm() and other functions.
* Code of lmCombine and lmDynamic is slightly simplified.
* We now use Choleski decomposition for the calculation of the inverse of matrices in alm.
* distribution="dlogis" is now available for alm().
* alm() now also supports logit and probit models, which are called using distribution="plogis" and distribution="pnorm" respectively (reference to the names of respective CDFs in R).
* alm() now has occurrence parameter, which allows dealing with zeroes in the data. In this case, a mixture distribution can be used.
* alm() with dlnorm now also returns analytical covariance matrix instead of hessian based one.
* stepwise(), lmCombine() and lmDynamic() now rely on .lm.fit() function, when distribution="dnorm", so the speed of calculation should be substantially higher.
* New functions for class checks: is.greybox(), is.alm(), is.greyboxC(), is.greyboxD(), is.rmc() and is.rollingOrigin().
* stepwise() now calculates only the necessary correlations. This allows further inceasing the speed of computation.
* alm() uses its own mean function, so this should also increas its speed.
* Correct prediction intervals for the model with the occurrence part and a new parameter in prediction function - side - which allows producing one-sided PIs.
* stepwise() should now work better with big data.
* Futher optimisation of stepwise in order to decrease the used memory.
* alm() and all the other functions now return "data" instead of "model" and don't produce terms and qr. This should save some space.
* vcov.alm() now uses call in order to reestimate the model.
* rmc() now returns groups of methods. This can be used for analytical purposes.
* alm() now uses a more refined parameters for vcov calculation for "dchisq" and returns a slightly different call with vcov.
* pointLik.alm() method for alm class.
* alm() now extracts meaningful residuals depending on the distribution used. e.g. dnorm -> y - mu, dlnorm -> log(y) - mu
* stepwise() now allows defining occurrence model. So now you can do something like: stepwise(ourData, distribution="dlnorm", occurrence=stepwise(ourData, distribution="plogis"))
* predict function now returns probabilities for the lower and upper intervals. So if you had side="upper", then the lower will be "0", and the upper will be the specified level.
* dpois and dnbinom distributions in alm. alm() allows producing prediction intervals for both of them. But covariance matrix of parameters for dnbinom might be hard to calculate...
* The dispersion parameter of dnbinom in alm() is now estimated separately, which now solves a lot of problems.
* Renamed parameter A into B for alm(). Very serious thing!
* distribution="dchisq" in alm() now estimates the non-central Chi Squared distribution. The returned scale corresponds to the estimated number of degrees of freedom, while mu is the exponent of the expectation.
* rmc() now colours the lines depending on the number of groups. If there's only one, then there's one group and the differences are not significant.
* Started a new vignette for the alm() function.
* graphmaker() is now moved from smooth to greybox.
Bugfixes:
* Fixed a bug with the style="line" in rmc(), where the grouping would be wrong in cases, when one method significantly differs from the others.
* logLik previously was not calculated correctly for the mixture models.
* Bugfix in hessian calculation, when Choleski decomposition works...
* Bugfix in pointLik for the models with occurrence.
* predict() function failed with newdata with one observation.
* Initials of both Poisson and NegBin in case of non-zero data are now taken with logs. This leads to more robust starting points.
greybox v0.3.0 (Release data: 2018-08-05)
==============
Changes:
* New cool function - lmDynamic() - that constructs a dynamic linear regression based on point ICs.
* New set of functions for Folded normal distribution.
* New function - alm - Advanced Linear Model.
* Folded normal distribution for rmc() with value="a".
* Proper model for chi-squared distribution in alm and rmc.
* Renamed distributions in the alm function.
Bugfixes:
* determination() function did not work in cases of 2 variables.
* vcov() and confint() were misbehaving when nVars==1.
greybox v0.2.3 (Release data: 2018-08-02)
==============
Changes:
* determination() now automatically drops variables with no variability.
* New function - nemenyi() - imported from TStools with minor bugfixes and corrections.
* It appears that Nikos is against the move of nemenyi() function from TStools to greybox. This was a misunderstanding between the two of us. So no nemenyi() function here, nothing to see here, move along!
* New function for multiple comparison of methods based on regression analysis - rmc(). This is a parametric analogue of nemenyi test. The function works with errors, their absolute and squared values and relies on lm / glm.
* New methods imported from smooth: errorType, pointLik and pAIC.
Bugfixes:
* plots of ro() were misaligned in case of co=FALSE
* ro() now also returns the correct actual values (previously they could be cut off when ci=FALSE).
greybox v0.2.2 (Release data: 2018-05-25)
==============
Changes:
* New description of the package and badges in README.md
* New function - determination() - returns R-squares for the provided data. This can be useful when you need to analyse the multicollinearity effect.
* nParam method for logLik class.
* BICc - new method for the classes, implementing, guess what?
* Updated description of the package in the help file.
* ro() now returns a class and has print and plot methods associated with it.
* ro() is much more flexible now, returning whatever you want in an adequate format.
* New methods for the greybox functions: confint, vcov.
* Renamed "combiner" into "lmCombine", because it makes more sense. We will use "combine" name for a more general function that would combine forecasts from arbitrary provided models (e.g. smooth, forecast and lm classes).
Bugfixes:
* sigma() method returned the wrong standard error in cases of combined models.
greybox v0.2.1 (Release data: 2018-05-01)
==============
Changes:
* New description of the package and badges in README.md
greybox v0.2.1 (Release data: 2018-05-01)
==============
Changes:
* print.summary now specifies digits. Summary does not round up anything. This corresponds to the normal behaviour of these methods.
* Implemented Laplace distribution, which is useful when models are estimated using MAE.
* Sped up qs() and qlaplace() functions using the inverse cumulative functions.
* New function - ro() - Rolling origin.
Bugfixes:
* qs() returned weird values when several 0 and 1 were specified as probabilities.
greybox v0.2.0 (Release data: 2018-03-10)
==============
Changes:
* combiner now uses a more clever mechanism in case of bruteForce==FALSE.
* combiner now also checks if the provided data has ncol>nrow and sets bruteForce if it has.
* Use Kendall Tau as default in cor() for stepwise.
* Don't use Kendall Tau as default everywhere - only for fat regressions.
* New summary and print methods for models from stepwise. No statistical tests printed, only confidence intervals and ICs.
* AICc for smooth functions in case of iSS models should take only the demand sizes into account, not all the parameters.
greybox v0.1.1 (Release data: 2018-03-05)
==============
Changes:
* We now do not depend on smooth. We suggest it. It's smooth that should depend on greybox!
* New function imported from smooth - AICc.
* New functions for the S distribution (the maximisation of likelihood of which corresponds to the minimum of HAM): ds, ps, qs, rs.
* stepwise now returns the object of two classes: greybox and lm.
* combiner now returns three classes: greybox, lm and greyboxC.
* nParam is moved to greybox from smooth.
Bugfixes:
* If smooth is not installed, plot forecasts using simpler function.
* The forecasts are now produced for the combined models in cases of fat regressions.
greybox v0.1.0 (Release data: 2018-03-03)
==============
* Initial release. stepwise() and xregExpander() are imported here from smooth package.
* combiner() function that combines lm() models. This thing is in the development right now.
* combiner() has a meaningful summary() now. Working to make it more accesible to lm functions.
* summary() for combiner now returns the list of values.
* stepwise() should now perform slightly better.
* combiner() can now be smart and use stepwise for the models pool creation.
* combined lm model can now be used together with predict() and forecast() functions.
* plot() and forecast() methods for the combined functions.