My Math Forum  

Go Back   My Math Forum > College Math Forum > Advanced Statistics

Advanced Statistics Advanced Probability and Statistics Math Forum

LinkBack Thread Tools Display Modes
October 5th, 2014, 03:49 AM   #1
Joined: Oct 2014
From: Warwick, UK

Posts: 1
Thanks: 0

entropy of combined events

Hi there, suppose I toss a fair coin twice. Suppose that I can perfectly observe every outcome, so in the end I observe each of the 4 events {HH},{TH},{HT},{TT}. The informational content measured by entropy would be -1/2log(1/2)*4=2.

But now suppose that I know that I would be able to observe only 2 events:

How do I redefine the measure of entropy to account for such cases?

Many thanks,
rocketman150 is offline  
October 5th, 2014, 04:27 AM   #2
Math Team
Joined: Nov 2010
From: Greece, Thessaloniki

Posts: 1,989
Thanks: 133

Math Focus: pre pre pre pre pre pre pre pre pre pre pre pre calculus
$\mathbb{P}\left(\{H,H\}\right) = \frac{1}{4}$ and $\mathbb{P}\left(\{TH,HT,TT\}\right) = \frac{1}{2^6}$,

$H = -\frac{1}{4}\log_{2}\left(\frac{1}{4}\right) -\frac{1}{2^6}\log_{2}\left(\frac{1}{2^6}\right) = \frac{1}{2}+\frac{6}{64}=\frac{38}{64}=\frac{19}{3 2}$.

For the first one,
all the events observed can occur with the same probability $\frac{1}{4}$, so

$H =-4\frac{1}{4}\log_{2}\left(\frac{1}{4}\right)=2$, which means that we get full information 2bits (two sides of the coin).

Last edited by ZardoZ; October 5th, 2014 at 04:48 AM.
ZardoZ is offline  

  My Math Forum > College Math Forum > Advanced Statistics

combined, entropy, events

Thread Tools
Display Modes

Similar Threads
Thread Thread Starter Forum Replies Last Post
question combined mode radiation and convection ? r-soy Physics 2 November 14th, 2013 10:54 AM
combined probabilities zengjinlian Algebra 0 August 11th, 2013 01:23 PM
How to compute entropy? acepsut Economics 0 June 28th, 2013 04:28 AM
Entropy H is continuous in p(i) iVenky Computer Science 1 March 27th, 2013 05:16 AM
calculating the entropy safyras Algebra 0 November 6th, 2011 01:50 AM

Copyright © 2018 My Math Forum. All rights reserved.