How do I get the training accuracies for each fold in k-fold cross validation in R?

Issue

I would like to evaluate whether the logistic regression model I created is overfit. I’d like to compare the accuracies of each training fold to the test fold, but I don’t know how to view these in R. This is the k-fold cross validation code:

library(caret)
levels(habitatdata$outcome) <- c("absent", "present") #rename factor levels 
set.seed(12)
cvIndex <- createFolds(factor(habitatdata$outcome), 5, returnTrain = T) #create stratified folds
ctrlspecs <- trainControl(index = cvIndex, 
  method = "cv", 
  number = 5, 
  savePredictions = "all", 
  classProbs = TRUE) #specify training methods
set.seed(123)
model1 <- train(outcome~ ist + hwt, 
  data=habitatdata,
  method = "glm",
  family = binomial, trControl = ctrlspecs) #specify model

How do I view the training accuracies of each fold?

Solution

Look at model1$resample – it should give you a table with Accuracy (and Kappa) for each fold.

Answered By – rw2

Answer Checked By – Pedro (AngularFixing Volunteer)

Leave a Reply

Your email address will not be published.