Academic
Publications
A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection

A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection,Ron Kohavi

A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection   (Citations: 1233)
BibTex | RIS | RefWorks Download
We review accuracy estimation methods and compare the two most common methods cross- validation and bootstrap Recent experimen­ tal results on artificial data and theoretical re cults m restricted settings have shown that for selecting a good classifier from a set of classi­ fiers (model selection), ten-fold cross-validation may be better than the more expensive ka\p one-out cross-validation We report on a large- scale experiment—over half a million runs of C4 5 and aNaive-Bayes algorithm—loestimale the effects of different parameters on these al gonthms on real-world datascts For cross- validation we vary the number of folds and whether the folds arc stratified or not, for boot­ strap, we vary the number of bootstrap sam­ ples Our results indicate that for real-word datasets similar to ours, The best method lo use for model selection is ten fold stratified cross validation even if computation power allows using more folds
Cumulative Annual
Sort by: