When do you push a model into production

model
regularization
accuracy
optimization
preprocessing

#1

Hello,

Suppose I am trying to predict a contiuous target feature/ variable/field say Housing price which is based on a set of “n” independent parameters.

Having carried out all the data preprocessing steps including checking relationships among independent variable and multi-collinearity. We ran an algorithm say Linear Regression and got say 78% accuracy of the model. Then we followed it up with Ridge / lasso for regularization and perhaps we imporved our accuracy to 84%.

Now my question is where do we Stop?.

When can we say that our model is optimized and we can do no better than this to imporve its accuracy?

When can we be sure or confident enough of pushing our model into production?

As always thank you for your help and support :slight_smile:


#2

Hi @mohitlearns,

First of all, when you say you got 78% accuracy, are you talking about the training accuracy or the validation accuracy?

My suggestion would be to make a validation set from your existing training set and then validate your model using the validation set. You can stop the training once you see that there is no improvement in the validation accuracy.

There is no clear answer to that. Ideally, we should train until we get 100% accuracy but it is not practically possible. It also depends on the type of data we are dealing with. Sometimes we can say that 90% accuracy is good enough and in certain cases, even 95% is not enough. So, it all depends on the type of data and we should always keep trying to improve accuracy.

When the model’s performance on both the training as well as the validation set is good, we can say that the model is ready to be pushed for production.