Misleading Info on Entropy Formula for Decision Tree

Course Q&A Machine Learning Specialization Advanced Learning Algorithms
1

Hi,

I am working on Decision Tree practice lab section 4.1. I found that below statement is misleading because taking 0 as input of logarithm with any basis is treated as infinity, instead it’s written as log2(0)=0.

  • For implementation purposes, 0log2(0)=0. That is, if p_1 = 0 or p_1 = 1, set the entropy to 0

I thought I didn’t need an if-else statement to deal with p_1=0 or p_1=1 until I noticed that I relied to wrong statement.

Feel free to correct me if I’m wrong.

Thanks,
awidianto

2

Hello @awidianto,

  1. mathematically, \lim_{p \to 0} p\log(p) = 0
  2. computationally, we cannot calculate 0log(0) because log(0) is inf, so usually we add a very small number like 10e-7 to inside the log so that we are instead calculating 0 * log(0+10e-7) which will also gives you a 0.
  3. if we don’t add that small constant, we can use the if-else statement.

Cheers,
Raymond