Isotonic Regression calibration for SVM

Issue: fail to calibrate SVM scores using isotonic regression.

Approach:
I used the code written on this blog (https://www.analyticsvidhya.com/blog/2016/07/platt-scaling-isotonic-regression-minimize-logloss-error/)
to calibrate the SVM outputs from a dataset called Fall detection (Fall Detection Data from China | Kaggle). I first split the data randomly into training and test set. The test set has 1000 samples. I use e1071:::svm package to fit the training data and get decision scores for training the calibration training. Then, I train the entire SVM and predict on the training set and get the decision values as testing decision values for calibration testing. But I got the following error:

Error in cut.default(x0, breaks = x, labels = FALSE, include.lowest = TRUE) :
‘breaks’ are not unique

Also, I tried using some calibration packages that provide Isotonic regression: CORElearn, Iso, but they return NA values as probability calibrated outputs…

Anyone can provide some insights on a standard way of calibrating SVM scores based on isotonic regression?

Here is my code:

# Load package
library(regtools)

library(regtools)fall <- read.csv("data/falldetection.csv")
fall$ACTIVITY <- as.factor(fall$ACTIVITY)
fall.svmout <- qeSVM(fall,'ACTIVITY')
# Get the indicies for testing
tstIdxs <- fall.svmout$holdIdxs
# Get all other indicies other than testing indicies
trnIdxs <- setdiff(1:nrow(fall),tstIdxs)

ycol <- fall.svmout$ycol # get the label column 
trnX <- fall[trnIdxs,-ycol] # get the training X
trnY <- fall[trnIdxs,ycol] # get the training Y
tstX <- fall[tstIdxs,-ycol] # get the testing X
tstY <- fall[tstIdxs,ycol] # get the testing Y
trn <- fall[trnIdxs,] # get the training set
tst <- fall[tstIdxs,] # get the testing set


count <- 1
trnMat<-matrix(NA, nrow(trn), length(levels(trnY)))
tstMat<-matrix(NA, nrow(tst), length(levels(trnY)))
frml <- fall.svmout$formula
for(i in levels(trnY)){
  trn$ACTIVITY <- as.factor(ifelse(trnY==i, 1, 0))
  model <- e1071::svm(frml,data=trn,kernel = "radial",decision.values=TRUE)
  trnMat[,count] <- model$decision.values
  pred <- predict(model, tst, decision.values = TRUE)
  tstMat[,count] <- as.vector(attr(pred, "decision.values"))
  count <- count + 1
}

Then, you can apply the calibration method to each trnMat[,i] with ifelse(trnY==i, 1,0) as labels for training an isotonic regression model and predict on tstMat[,i].

© Copyright 2013-2020 Analytics Vidhya