My Math Forum  

Go Back   My Math Forum > College Math Forum > Advanced Statistics

Advanced Statistics Advanced Probability and Statistics Math Forum

LinkBack Thread Tools Display Modes
January 11th, 2017, 07:52 AM   #1
Joined: Dec 2016
From: US

Posts: 6
Thanks: 0

Maximal entropy distribution

What is the maximal entropy distribution for a positive random variable for which we know the first and second moment?

$\displaystyle X>0$

$\displaystyle E[X]=a$

$\displaystyle E[X^2]=4a^2$
mike22 is offline  
January 11th, 2017, 09:15 AM   #2
Senior Member
Joined: Apr 2014
From: Glasgow

Posts: 2,099
Thanks: 703

Math Focus: Physics, mathematical modelling, numerical and computational solutions
I have no idea, but this site might help:

Discrete version
Suppose $\displaystyle S = \{x_1,x_2,...\}$ is a (finite or infinite) discrete subset of the reals and we choose to specify n functions $\displaystyle f_1,...,f_n$ and n numbers $\displaystyle a_1,...,a_n$. We consider the class C of all discrete random variables X which are supported on S and which satisfy the n conditions

$\displaystyle {\displaystyle \operatorname {E} (f_{j}(X))=a_{j}\quad {\mbox{ for }}j=1,\ldots ,n} $

If there exists a member of C which assigns positive probability to all members of S and if there exists a maximum entropy distribution for C, then this distribution has the following shape:

$\displaystyle {\displaystyle \operatorname {Pr} (X=x_{k})=c\exp \left(\sum _{j=1}^{n}\lambda _{j}f_{j}(x_{k})\right)\quad {\mbox{ for }}k=1,2,\ldots }$

where the constants c and λj have to be determined so that the sum of the probabilities is 1 and the above conditions for the expected values are satisfied. Conversely, if constants c and λj like this can be found, then the above distribution is indeed the maximum entropy distribution for our class C.
Benit13 is offline  

  My Math Forum > College Math Forum > Advanced Statistics

distribution, entropy, maximal

Thread Tools
Display Modes

Similar Threads
Thread Thread Starter Forum Replies Last Post
conversion of entropy SophiaRivera007 Physics 2 January 13th, 2017 03:01 AM
entropy of combined events rocketman150 Advanced Statistics 1 October 5th, 2014 04:27 AM
How to compute entropy? acepsut Economics 0 June 28th, 2013 04:28 AM
Entropy H is continuous in p(i) iVenky Computer Science 1 March 27th, 2013 05:16 AM
calculating the entropy safyras Algebra 0 November 6th, 2011 01:50 AM

Copyright © 2018 My Math Forum. All rights reserved.