In case of missing values for continuous variables, we perform following steps to handle it.
Ignore these observations
Replace with general average
Replace with similar type of averages
Build model to predict missing values
Can you suggest me the methods to handle missing values if data is binary (1/0 or M/F) or categorical variables.
There is various ways to handle missing values of categorical ways.
Ignore observations of missing values if we are dealing with large data sets and less number of records has missing values
Ignore variable, if it is not significant
Develop model to predict missing values
Treat missing data as just another category
The same steps apply for a categorical variable as well.
Replace by most frequent value
Replace using an algorithm like KNN using the neighbours.
Predict the observation using a multiclass predictor.
Hope this helps.
You can also look at this article:
Tutorial on data exploration that comprises missing value imputation, outliers, feature engineering, variable creation in data science and machine learning
Generalised Low rank models can generate missing values by themselves. You can have a look at -
thanks to share the Stanford paper great chapter 5
can you explain how to replace by most frequent value?
the second option you mentioned.
appreciate your reply.
It simply means replacing the missing values using the mode of the column. You can calculate the mode using
@haneeshb ,for replacing by most frequent value
you can do like ,
df[“example”] = df.example.fillna(df.example.mode,inplace = True) ,
where mode represents the most frequent value out if n values
I am new to machine learning , help me out with my model.
I have a dataset that has two categorical columns one column with 100 unique entries and the second one with 136 unique entries. The size of the dataset is 20k observations. Onehotencoding or dummies result in the exploding of the dataset. How should I preprocess these columns to predict my linear regression model?
Are there any working examples with KNN to treat missing values of categorical data , both for nominal and ordinal types?