
Advanced Statistics Advanced Probability and Statistics Math Forum 
 LinkBack  Thread Tools  Display Modes 
August 22nd, 2011, 01:51 AM  #1 
Newbie Joined: Aug 2011 Posts: 2 Thanks: 0  Calculating information gain (decision trees)
Hi guys! I have a problem with the calculation of my information gain of my decision tree. During the lectures, we saw a rather easy example. However, I'm not able to come to the correct solution and it's frustrating me! The decision tree deals with whether or not to play outside, according to the weather conditions, and the example goes as follows: Attribute Outlook (sunny):  yes  yes  yes  no  no Put another way: info[2,3] = entropy(2/5, 3/5) = 2/5log(2/5)3/5log(3/5) = 0.971 bits However, when I type 2/5log(2/5)3/5log(3/5) on my calculator, I never get 0.971 bits... I know I'm doing something wrong but I can't seem to remember the rules according to logarithms. Can someone tell me how to come to the correct answer? Thanks! 
August 22nd, 2011, 12:52 PM  #2 
Global Moderator Joined: Nov 2006 From: UTC 5 Posts: 16,046 Thanks: 938 Math Focus: Number theory, computational mathematics, combinatorics, FOM, symbolic logic, TCS, algorithms  Re: Calculating information gain (decision trees)
You're probably using the wrong logarithm base. Base 2 gives bits; base e gives nats; etc. You can convert a logarithm of any base to base 2 by dividing the result by log (in that base) of 2.

August 22nd, 2011, 01:49 PM  #3 
Newbie Joined: Aug 2011 Posts: 2 Thanks: 0  Re: Calculating information gain (decision trees)
Ok thanks! I typed the following on my calculator and now it works: (2/5)*(log(2/5)/log(2))(3/5)*(log(3/5)/log(2)) = 0.971 Again thanks! 
August 22nd, 2011, 04:44 PM  #4 
Global Moderator Joined: Nov 2006 From: UTC 5 Posts: 16,046 Thanks: 938 Math Focus: Number theory, computational mathematics, combinatorics, FOM, symbolic logic, TCS, algorithms  Re: Calculating information gain (decision trees)
No problem. I had answered the question earlier, actually, but it was lost in the server move. 

Tags 
calculating, decision, gain, information, trees 
Thread Tools  
Display Modes  

Similar Threads  
Thread  Thread Starter  Forum  Replies  Last Post 
What is the most efficient way to gain mathematical insight?  k4ff3  Academic Guidance  7  April 12th, 2013 06:20 AM 
Cumulative net gain and Stochastic Process... I'm lost!  Juliayaho  Advanced Statistics  4  November 7th, 2012 08:34 AM 
gain/loss ratios?  circadianpulse  Economics  2  December 18th, 2010 06:20 AM 
The expected gain, given a Binomial Distribution?  Zephyr  Advanced Statistics  1  October 2nd, 2007 05:44 PM 
What is the most efficient way to gain mathematical insight?  k4ff3  New Users  2  December 31st, 1969 04:00 PM 