
Advanced Statistics Advanced Probability and Statistics Math Forum 
 LinkBack  Thread Tools  Display Modes 
January 11th, 2017, 08:52 AM  #1 
Newbie Joined: Dec 2016 From: US Posts: 6 Thanks: 0  Maximal entropy distribution
What is the maximal entropy distribution for a positive random variable for which we know the first and second moment? $\displaystyle X>0$ $\displaystyle E[X]=a$ $\displaystyle E[X^2]=4a^2$ 
January 11th, 2017, 10:15 AM  #2 
Senior Member Joined: Apr 2014 From: Glasgow Posts: 2,087 Thanks: 700 Math Focus: Physics, mathematical modelling, numerical and computational solutions 
I have no idea, but this site might help: https://en.wikipedia.org/wiki/Maximu...y_distribution Discrete version Suppose $\displaystyle S = \{x_1,x_2,...\}$ is a (finite or infinite) discrete subset of the reals and we choose to specify n functions $\displaystyle f_1,...,f_n$ and n numbers $\displaystyle a_1,...,a_n$. We consider the class C of all discrete random variables X which are supported on S and which satisfy the n conditions $\displaystyle {\displaystyle \operatorname {E} (f_{j}(X))=a_{j}\quad {\mbox{ for }}j=1,\ldots ,n} $ If there exists a member of C which assigns positive probability to all members of S and if there exists a maximum entropy distribution for C, then this distribution has the following shape: $\displaystyle {\displaystyle \operatorname {Pr} (X=x_{k})=c\exp \left(\sum _{j=1}^{n}\lambda _{j}f_{j}(x_{k})\right)\quad {\mbox{ for }}k=1,2,\ldots }$ where the constants c and λj have to be determined so that the sum of the probabilities is 1 and the above conditions for the expected values are satisfied. Conversely, if constants c and λj like this can be found, then the above distribution is indeed the maximum entropy distribution for our class C. 

Tags 
distribution, entropy, maximal 
Thread Tools  
Display Modes  

Similar Threads  
Thread  Thread Starter  Forum  Replies  Last Post 
conversion of entropy  SophiaRivera007  Physics  2  January 13th, 2017 04:01 AM 
entropy of combined events  rocketman150  Advanced Statistics  1  October 5th, 2014 05:27 AM 
How to compute entropy?  acepsut  Economics  0  June 28th, 2013 05:28 AM 
Entropy H is continuous in p(i)  iVenky  Computer Science  1  March 27th, 2013 06:16 AM 
calculating the entropy  safyras  Algebra  0  November 6th, 2011 01:50 AM 