My Math Forum entropy of combined events

 October 5th, 2014, 03:49 AM #1 Newbie   Joined: Oct 2014 From: Warwick, UK Posts: 1 Thanks: 0 entropy of combined events Hi there, suppose I toss a fair coin twice. Suppose that I can perfectly observe every outcome, so in the end I observe each of the 4 events {HH},{TH},{HT},{TT}. The informational content measured by entropy would be -1/2log(1/2)*4=2. But now suppose that I know that I would be able to observe only 2 events: E1:={H,H} E2:={TH,HT,TT} How do I redefine the measure of entropy to account for such cases? Many thanks, D
 October 5th, 2014, 04:27 AM #2 Math Team   Joined: Nov 2010 From: Greece, Thessaloniki Posts: 1,989 Thanks: 131 Math Focus: pre pre pre pre pre pre pre pre pre pre pre pre calculus $\mathbb{P}\left(\{H,H\}\right) = \frac{1}{4}$ and $\mathbb{P}\left(\{TH,HT,TT\}\right) = \frac{1}{2^6}$, $H = -\frac{1}{4}\log_{2}\left(\frac{1}{4}\right) -\frac{1}{2^6}\log_{2}\left(\frac{1}{2^6}\right) = \frac{1}{2}+\frac{6}{64}=\frac{38}{64}=\frac{19}{3 2}$. For the first one, all the events observed can occur with the same probability $\frac{1}{4}$, so $H =-4\frac{1}{4}\log_{2}\left(\frac{1}{4}\right)=2$, which means that we get full information 2bits (two sides of the coin). Last edited by ZardoZ; October 5th, 2014 at 04:48 AM.

 Tags combined, entropy, events

 Thread Tools Display Modes Linear Mode

 Similar Threads Thread Thread Starter Forum Replies Last Post r-soy Physics 2 November 14th, 2013 11:54 AM zengjinlian Algebra 0 August 11th, 2013 01:23 PM acepsut Economics 0 June 28th, 2013 04:28 AM iVenky Computer Science 1 March 27th, 2013 05:16 AM safyras Algebra 0 November 6th, 2011 01:50 AM

 Contact - Home - Forums - Top