
Advanced Statistics Advanced Probability and Statistics Math Forum 
 LinkBack  Thread Tools  Display Modes 
October 5th, 2014, 03:49 AM  #1 
Newbie Joined: Oct 2014 From: Warwick, UK Posts: 1 Thanks: 0  entropy of combined events
Hi there, suppose I toss a fair coin twice. Suppose that I can perfectly observe every outcome, so in the end I observe each of the 4 events {HH},{TH},{HT},{TT}. The informational content measured by entropy would be 1/2log(1/2)*4=2. But now suppose that I know that I would be able to observe only 2 events: E1:={H,H} E2:={TH,HT,TT} How do I redefine the measure of entropy to account for such cases? Many thanks, D 
October 5th, 2014, 04:27 AM  #2 
Math Team Joined: Nov 2010 From: Greece, Thessaloniki Posts: 1,989 Thanks: 132 Math Focus: pre pre pre pre pre pre pre pre pre pre pre pre calculus 
$\mathbb{P}\left(\{H,H\}\right) = \frac{1}{4}$ and $\mathbb{P}\left(\{TH,HT,TT\}\right) = \frac{1}{2^6}$, $H = \frac{1}{4}\log_{2}\left(\frac{1}{4}\right) \frac{1}{2^6}\log_{2}\left(\frac{1}{2^6}\right) = \frac{1}{2}+\frac{6}{64}=\frac{38}{64}=\frac{19}{3 2}$. For the first one, all the events observed can occur with the same probability $\frac{1}{4}$, so $H =4\frac{1}{4}\log_{2}\left(\frac{1}{4}\right)=2$, which means that we get full information 2bits (two sides of the coin). Last edited by ZardoZ; October 5th, 2014 at 04:48 AM. 

Tags 
combined, entropy, events 
Thread Tools  
Display Modes  

Similar Threads  
Thread  Thread Starter  Forum  Replies  Last Post 
question combined mode radiation and convection ?  rsoy  Physics  2  November 14th, 2013 11:54 AM 
combined probabilities  zengjinlian  Algebra  0  August 11th, 2013 01:23 PM 
How to compute entropy?  acepsut  Economics  0  June 28th, 2013 04:28 AM 
Entropy H is continuous in p(i)  iVenky  Computer Science  1  March 27th, 2013 05:16 AM 
calculating the entropy  safyras  Algebra  0  November 6th, 2011 01:50 AM 