Hi,
I am working on a data science project in python and while data exploration I have found a feature with skewed distribution. I want to apply log transformation to reduce the skewness of the feature but it is giving an error value because feature has zero values. Should I drop observation with zero values or there are methods to deal with this situation?
Regards,
Steve
There are various methods to deal with it, I am listing some of these:
Hope this helps!
Regards,
Imran
Hi Steve,
Do log(x+1) transformation. 
This is the best way to avoid error created by log transformation and is widely used among data scientists.
Hope this helps.
Regards,
Aayush
Just look for the smallest non zero entry in your data, let this be e.g. x, then add x/2 to this smallest values and compute the log .
Hope this helps!
Regards,
Rohit
Use Box-Cox transformation for data having zero values.This works fine with zeros (although not with negative values). However, often the square root is not a strong enough transformation to deal with the high levels of skewness (we generally do sqrt transformation for right skewed distribution) seen in real data. If you are using are than can use function boxcox.fit() in package named geoR.
Hope this helps.
Regards,
Dharm
Thank you sir. sir i have little confusion in it. we will need to use log(x+1) transformation for the whole data set or only the values that are in negative or zero.?i shall be thankful
Whole data set. You need to apply same transformation to each i/p to be consistent.
I am not a statistician or expert but I have doubts about adding constant to feature with many zeros. Because when we add a constant values (like +1 or more) to all zeros along with other values, non-effective values (zeros) will be effective.
Does not this effect the prediction negatively?
I prefer to apply log(X - (min(X) - 1)) to eliminate log0 and negativity problems.
