Maximum-likelihood examples: Normal density

This notebook gives examples of maximum-likelihood estimation for the Normal density, and assumes that the definitions in the notebook "statistical_modeling.nb" have already been evaluated.

Global user definitions

Colors representing two classes

Data colors

data1Color = Red;
data2Color = Blue;

style1  =  {data1Color, 
          PointSize[ .02]};
style2  =  {data2Color, PointSize[ .02]};

Classification region colors

class1Color  =  RGBColor[1, 1,  .5];
class2Color  =  RGBColor[ .5, 1, 1];

classColors  =  {class1Color,  class2Color};

Default plot range

pr  =  {{-1, 1}, {-1, 1}};

No frame ticks

nF = FrameTicks -> None;
FS = Sequence[yS,  nF];

Example A: Uniform distributions

Generate data from two uniform classes

Number of points in each class

n1  =  100;
n2  =  100;

Generate data

data1  =  
        Table[{ru2[{pr[[1, 1]], 0}], 
            ru2[pr[[2]]]}, {n1}];
data2  =  
        Table[{ru2[{0, pr[[1, 2]]}], 
            ru2[pr[[2]]]}, {n2}];

Visualize data

gd1  =  
        PlotData2D[data1, pr, style1, nS];
gd2  =  
        PlotData2D[data2, pr, style2, nS];
gd  =  Show[gd1, gd2, FS];

Gaussian probability density functions

Compute Gaussian pdfs

pdf1  =  gaussian[data1, {x, y}];
pdf2  =  gaussian[data2, {x, y}];

View Gaussian pdfs

ShowTwoGaussianPDFs[pdf1, pdf2, pr, x, y, yS, imSize];

Classification of input space

ShowTwoClassificationRegions[pdf1, pdf2, pr, x, y, classColors, gd, 
        Axes -> False, Frame -> True, FS];

Example B: different uniform distributions

Generate data from two non-rectangular uniform classes

Number of points in each class

n1  =  100;
n2  =  100;

Specialized data generation

SpecialDataGeneration[ 
          flag_] := 
Module[{x, 
            y}, 
{x, y}  =  
            r2[1]; 
While[-flag*x  >  
              flag * y,  {x, y}  =  r2[1]]; {x, y}];

Generate data

data1  =  Table[SpecialDataGeneration[1], {n1}];
data2  =  
        Table[SpecialDataGeneration[-1],  {n2}];

Visualize data

gd1  =  
        PlotData2D[data1, pr, style1, nS];
gd2  =  
        PlotData2D[data2, pr, style2, nS];
gd  =  Show[gd1, gd2,  FS];

Gaussian probability density functions

Compute Gaussian pdfs

pdf1  =  gaussian[data1, {x, y}];
pdf2  =  gaussian[data2, {x, y}];

View Gaussian pdfs

ShowTwoGaussianPDFs[pdf1, pdf2, pr, x, y, yS, imSize];

Classification of input space

ShowTwoClassificationRegions[pdf1, pdf2, pr, x, y, classColors, gd, 
        Axes -> False, Frame -> True, FS];

Example C: Gaussian distributions

Generate data from two Gaussian classes

Definition of Gaussians

μ1  =  { .3,  .3};
Σ1  =   .07 {{1. , - .5}, {- .5,  .5}};

μ2  =  {- .5, - .5};
Σ2  =   .04 {{1. , - .5}, {- .5,  .5}};

Number of points in each class

n1  =  100;
n2  =  100;

Generate data

data1  =  
        Table[rgBounded[{μ1,  Σ1}, pr], {n1}];
data2  =  
        Table[rgBounded[{μ2,  Σ2}, 
            pr], {n2}];

Visualize data

gd1  =  
        PlotData2D[data1, pr, style1, nS];
gd2  =  
        PlotData2D[data2, pr, style2, nS];
gd  =  Show[gd1, gd2,  FS];

Gaussian probability density functions

Compute Gaussian pdfs

pdf1  =  gaussian[data1, {x, y}];
pdf2  =  gaussian[data2, {x, y}];

View Gaussian pdfs

ShowTwoGaussianPDFs[pdf1, pdf2, pr, x, y, yS, imSize];

Classification of input space

ShowTwoClassificationRegions[pdf1, pdf2, pr, x, y, classColors, gd, 
        Axes -> False, Frame -> True, FS];

Example D: different Gaussian distributions

Generate data from two Gaussian classes

Definition of Gaussians

μ1  =  { .3,  .3};
Σ1  =   .07 {{1. , - .5}, {- .5,  .5}};

μ2  =  {- .5, - .5};
Σ2  =   .06 {{1. ,  .5}, { .5,  .5}};

Number of points in each class

n1  =  100;
n2  =  100;

Generate data

data1  =  
        Table[rgBounded[{μ1,  Σ1}, pr], {n1}];
data2  =  
        Table[rgBounded[{μ2,  Σ2}, 
            pr], {n2}];

Visualize data

gd1  =  
        PlotData2D[data1, pr, style1, nS];
gd2  =  
        PlotData2D[data2, pr, style2, nS];
gd  =  Show[gd1, gd2,  FS];

Gaussian probability density functions

Compute Gaussian pdfs

pdf1  =  gaussian[data1, {x, y}];
pdf2  =  gaussian[data2, {x, y}];

View Gaussian pdfs

ShowTwoGaussianPDFs[pdf1, pdf2, pr, x, y, yS, imSize];

Classification of input space

ShowTwoClassificationRegions[pdf1, pdf2, pr, x, y, classColors, gd, 
        Axes -> False, Frame -> True, FS];