Fisher: reduced-dimensionality classification, example #2

Generate data from two Gaussian classes

Definition of Gaussians

μ1 = {.1, .1} ; RowBox[{RowBox[{Σ1, , =, , RowBox[{.07, RowBox[{{, RowBox[{RowBox[{{, RowBox[{1., ,, -.4}], }}], ,, {-.4, .2}}], }}]}]}], ;}]

μ2 = {-.1, -.1} ; RowBox[{RowBox[{Σ2, , =, , RowBox[{.07, RowBox[{{, RowBox[{RowBox[{{, RowBox[{1., ,, -.4}], }}], ,, {-.4, .2}}], }}]}]}], ;}]

Number of points in each class

n1 = 100 ; n2 = 100 ;

Generate data

data1 = Table[rg[μ1, Σ1], {n1}] ; data2 = Table[rg[μ2, Σ2], {n2}] ;

Visualize data

gd1 = PlotData2D[data1, pr, style1, nS] ; gd2 = PlotData2D[data2, pr, style2, nS] ; gdata = Sh ... gd2, FrameTrue, FrameTicksNone, yS, PlotRangeAll, ImageSize400] ;

[Graphics:../HTMLFiles/index_92.gif]

Fisher linear discriminant

w = Fisher[data1, data2] ;

Visualize Fisher projections

VisualizeProjections[w, data1, data2, style1, style2, {.5, .5}, ImageSize400] ;

[Graphics:../HTMLFiles/index_95.gif]

One-dimensional Fisher-based classification

Project data onto Fisher discriminant line

d1pca = ProjectFisher[w, data1//N] ; d2pca = ProjectFisher[w, data2//N] ;

Compute corresponding 1d Gaussian pdfs

p1 = ComputePDF1D[d1pca // N, Red] ; p2 = ComputePDF1D[d2pca // N, Red] ;

Display resulting pdfs

pl1 = ShowTwoGaussianPDFs1D[p1, p2, {-.5, .5}, x, AxesNone, ImageSize400] ; Show[pl1, yS] ;

[Graphics:../HTMLFiles/index_99.gif]

Error over training data

Decision boundary

db = ComputeDecisionBoundary1D[p1, p2, x, {-.1, .1}] ;

Classification error based on 1d Gaussian pdfs

ErrorOverTrainingSet[w, data1, data2]

1.

Compare to 2d classificationGaussian

Compute Gaussian pdfs

pdf1 = gaussian[data1, {x, y}] ; pdf2 = gaussian[data2, {x, y}] ;

Classification of input space

pr = AbsoluteOptions[gdata, PlotRange]//Last // Last ;

ShowTwoClassificationRegions[pdf1, pdf2, pr, x, y, classColors, gdata, AxesFalse, FrameTrue, FrameTicksNone, ImageSize400, yS] ;

[Graphics:../HTMLFiles/index_106.gif]

Created by Mathematica  (October 20, 2003)