Ttykuu

I'm currently working on a classification problem and I've a numerical column which is left skewed. i've read many posts where people are recommending to take log transformation or boxcox transformation to fix the left skewness.

So I was wondering what would happen If I left the skewness as it is and continue with my model building? Are there any advantages of fixing skewness for classification problem (knn, logistic regression)?

There are issues that will depend on specific features of your data and analytic approach, but in general skewed data (in either direction) will degrade some of your model's ability to describe more "typical" cases in order to deal with much rarer cases which happen to take extreme values.

Since "typical" cases are more common than extreme ones in a skewed data set, you are losing some precision with the cases you'll see most often in order to accommodate cases that you'll see only rarely. Determining a coefficient for a thousand observations which are all between [0,10] is likely to be more precise than for 990 observations between [0,10] and 10 observations between [1,000, 1,000,000]. This can lead to your model being less useful overall.

"Fixing" skewness can provide a variety of benefits, including making analysis which depends on the data being approximately Normally distributed possible/more informative. It can also produce results which are reported on a sensible scale (this is very situation-dependent), and prevent extreme values (relative to other predictors) from over- or underestimating the influence of the skewed predictor on the predicted classification.

You can test this somewhat (in a non-definitive way, to be sure) by training models with varying subsets of your data: everything you've got, just as it is, your data without that skewed variable, your data with that variable but excluding values outside of the "typical" range (though you'll have to be careful in defining that), your data with the skewed variable distribution transformed or re-scaled, etc.

As for fixing it, transformations and re-scaling often make sense. But I cannot emphasize enough:

Fiddling with variables and their distributions should follow from properties of those variables, not your convenience in modelling.

Log-transforming skewed variables is a prime example of this:

• If you really think that a variable operates on a geometric scale,
  and you want your model to operate on an arithmetic scale, then log
  transformation can make a lot of sense.


• If you think that variable operates on an arithmetic scale, but you
  find its distribution inconvenient and think a log transformation
  would produce a more convenient distribution, it may make sense to
  transform. It will change how the model is used and interpreted,
  usually making it more dense and harder to interpret clearly, but
  that may or may not be worthwhile. For example, if you take the log of a numeric outcome and the log of a numeric predictor, the result has to be interpreted as an elasticity between them, which can be awkward to work with and is often not what is desired.


• If you think that a log transformation would be desirable for a
  variable, but it has a lot of observations with a value of 0, then
  log transformation isn't really an option for you, whether it would
  be convenient or not. (Adding a "small value" to the 0 observations
  causes lots of problems-- take the logs of 1-10, and then 0.0 to
  1.0).