Comparing Actual vs predicted values

machine_learning
#1

Team,

i am predicting top 3 skills for a candidate in python and Comparing Actual vs predicted values.

I am stuck with comparing actual vs predicted values top three classes when we compare

and print the results actual vs predicted results, the predicted values are sorted by

top probabilities and mapped to classes where as in actual we have only class labels

and when we compare the values are deviating. please suggest solutions to fix

insights much appreciated

Regards,

Tony

#2

Hi @train.bi can you be a little more descriptive with an example … is it a classification problem or You are preditcing top 3 skills and want to check the predicted top 3 with actual top 3 ??

#3

yes. its a classification problem based on skills count we need to get the top three skills of the person. i have extracted probabilities and applied sort to get top three probabilities and map to classes. now i am stuck with comparing actual vs predicted values top three classes when we compare and print the results actual vs predicted results, the predicted values are sorted by top probabilities and mapped to classes where as in actual we have only class labels

#4

@train.bi
Hi it purely depends like how you would like to approach as in if at all the top 3 class being predicted is same as actual its perfect and if only 2 of the top 3 are matching the actual how much bad it is for you particular goal , if and only if u want accuracy to be 100% i.e top 3= actual top 3 we can directly calculate error as cases where prediction went wrong upon total cases …Otherwise if u want to give some particular weight-age to different scenarios.

Also look at the below link it has some interesting evaluation metrics:
http://sdsawtelle.github.io/blog/output/mean-average-precision-MAP-for-recommender-systems.html