PCA: reduced-dimensionality classification, example #2

Generate data from two Gaussian classes

Definition of Gaussians

μ1 = {.1, .1} ; RowBox[{RowBox[{Σ1, , =, , RowBox[{.07, RowBox[{{, RowBox[{RowBox[{{, RowBox[{1., ,, -.4}], }}], ,, {-.4, .2}}], }}]}]}], ;}]

μ2 = {-.1, -.1} ; RowBox[{RowBox[{Σ2, , =, , RowBox[{.07, RowBox[{{, RowBox[{RowBox[{{, RowBox[{1., ,, -.4}], }}], ,, {-.4, .2}}], }}]}]}], ;}]

Number of points in each class

n1 = 100 ; n2 = 100 ;

Generate data

data1 = Table[rg[μ1, Σ1], {n1}] ; data2 = Table[rg[μ2, Σ2], {n2}] ;

Visualize data

gd1 = PlotData2D[data1, pr, style1, nS] ; gd2 = PlotData2D[data2, pr, style2, nS] ; gdata = Sh ... gd2, FrameTrue, FrameTicksNone, yS, PlotRangeAll, ImageSize400] ;

[Graphics:../HTMLFiles/index_51.gif]

Principal Component Analysis

pca = PCA[data1 ~ Join ~ data2 // N] ;

gpca = PCAPlot[pca, 2, {Black, Thickness[.01]}, nS] ; Show[gdata, gpca, yS, PlotRangeAll, ImageSize400] ;

[Graphics:../HTMLFiles/index_54.gif]

Visualize PCA projection

RowBox[{RowBox[{VisualizeProjections, [, RowBox[{pca[[2, 1]], ,, data1, ,, data2, ,, style1, ,, style2, ,, RowBox[{{, RowBox[{1.2, ,, 1.2}], }}], ,, ImageSize400}], ]}], ;}]

[Graphics:../HTMLFiles/index_56.gif]

One-dimensional PCA-based classification

Project data onto first PC

d1pca = PCAConvert[data1//N, pca, 1] ; d2pca = PCAConvert[data2//N, pca, 1] ;

Compute corresponding 1d Gaussian pdfs

p1 = ComputePDF1D[d1pca // N, Red] ; p2 = ComputePDF1D[d2pca // N, Red] ;

Display resulting pdfs

pl1 = ShowTwoGaussianPDFs1D[p1, p2, {-2, 2}, x, AxesNone, ImageSize400] ; Show[pl1, yS] ;

[Graphics:../HTMLFiles/index_60.gif]

Error over training data

ErrorOverTrainingSet[pca[[2, 1]], data1, data2]

40.5

Compare to 2d classificationGaussian

Compute Gaussian pdfs

pdf1 = gaussian[data1, {x, y}] ; pdf2 = gaussian[data2, {x, y}] ;

Classification of input space

pr = AbsoluteOptions[gdata, PlotRange]//Last // Last ;

ShowTwoClassificationRegions[pdf1, pdf2, pr, x, y, classColors, gdata, AxesFalse, FrameTrue, FrameTicksNone, ImageSize400, yS] ;

[Graphics:../HTMLFiles/index_66.gif]

Created by Mathematica  (October 20, 2003)