9

After reading through a few references, I have come to know that for machine learning in general, it is necessary to normalize features so that no features are arbitrarily large ($centering$) and all features are on the same scale ($scaling$).

However, I'm having a bit of difficulty in visualizing the impact of this for a decision tree. Does data normalization impact the decision tree structure? If yes, how?

Raaj
  • 711
  • If a column you are using is a combination of two others, then it might be a good idea to normalise those beforehand (or rescale in some manner so that each column contributes in the way you want to the final value). – Guillermo Mosse Sep 16 '19 at 08:23

1 Answers1

12

Normalization should have no impact on the performance of a decision tree. It is generally useful, when you are solving a system of equations, least squares, etc, where you can have serious issues due to rounding errors. In decision tree, you are just comparing stuff and branching down the tree, so normalization would not help.

vdesai
  • 658
  • Is there a way I can prove this formally? Any reference to this would help.

    I'm thinking data would be centered and scaled but structure would remain the same.

    However, I need to prove this. Intuitively, I'm thinking I should show there's no information gain change.

     – Raaj Sep 11 '14 at 17:28
  • Go to section on advantages of decision tree : http://en.wikipedia.org/wiki/Decision_tree_learning – vdesai Sep 11 '14 at 17:32