Fisher: reduced-dimensionality classification, example #1

Generate data from two Gaussian classes

Definition of Gaussians

μ1 = {1, 1} ; RowBox[{RowBox[{Σ1, , =, , RowBox[{.07, RowBox[{{, RowBox[{RowBox[{{, RowBox[{1., ,, -.4}], }}], ,, {-.4, .2}}], }}]}]}], ;}]

RowBox[{RowBox[{μ2, , =, , RowBox[{{, RowBox[{0, ,, 1.3}], }}]}], ;}] RowBox[{RowBox[{& ... owBox[{.07, RowBox[{{, RowBox[{RowBox[{{, RowBox[{1., ,, -.4}], }}], ,, {-.4, .2}}], }}]}]}], ;}]

Number of points in each class

n1 = 100 ; n2 = 100 ;

Generate data

data1 = Table[rg[μ1, Σ1], {n1}] ; data2 = Table[rg[μ2, Σ2], {n2}] ;

Visualize data

gd1 = PlotData2D[data1, pr, style1, nS] ; gd2 = PlotData2D[data2, pr, style2, nS] ; gdata = Sh ... gd2, FrameTrue, FrameTicksNone, yS, PlotRangeAll, ImageSize400] ;

[Graphics:../HTMLFiles/index_72.gif]

Fisher linear discriminant

w = Fisher[data1, data2] ;

Visualize Fisher projections

VisualizeProjections[w, data1, data2, style1, style2, {0, 2}, ImageSize400] ;

[Graphics:../HTMLFiles/index_75.gif]

One-dimensional Fisher-based classification

Project data onto Fisher discriminant line

d1pca = ProjectFisher[w, data1//N] ; d2pca = ProjectFisher[w, data2//N] ;

Compute corresponding 1d Gaussian pdfs

p1 = ComputePDF1D[d1pca // N, Red] ; p2 = ComputePDF1D[d2pca // N, Red] ;

Display resulting pdfs

pl1 = ShowTwoGaussianPDFs1D[p1, p2, {0, 2}, x, AxesNone, ImageSize400] ; Show[pl1, yS] ;

[Graphics:../HTMLFiles/index_79.gif]

Error over training data

Decision boundary

RowBox[{RowBox[{db, =, RowBox[{ComputeDecisionBoundary1D, [, RowBox[{p1, ,, p2, ,, x, ,, RowBox[{{, RowBox[{1, ,, 1.5}], }}]}], ]}]}], ;}]

Classification error based on 1d Gaussian pdfs

ErrorOverTrainingSet[w, data1, data2]

3.5

Compare to 2d classificationGaussian

Compute Gaussian pdfs

pdf1 = gaussian[data1, {x, y}] ; pdf2 = gaussian[data2, {x, y}] ;

Classification of input space

pr = AbsoluteOptions[gdata, PlotRange]//Last // Last ;

ShowTwoClassificationRegions[pdf1, pdf2, pr, x, y, classColors, gdata, AxesFalse, FrameTrue, FrameTicksNone, ImageSize400, yS] ;

[Graphics:../HTMLFiles/index_86.gif]

Created by Mathematica  (October 20, 2003)