Basic image processing: Part III

Note: This notebook assumes that all definitions in ipcv_definitions.nb have been evaluated first. It also assumes that the directory in the Set image directory points to an image directory with the following images: binary_tst.ppm, ball1.ppm, ball2.ppm, ball_data.ppm, hall1.ppm, hall2.ppm, outdoor1.ppm and outdoor2.ppm.

Set image directory

If[StringQ[$ImagePath], $path = $ImagePath,  $path = "."]; 
    SetDirectory[$path];

Connected-component labeling in binary images: simple example of blob finding

Define 8x8 binary test image

tst = {{1, 1, 0, 1, 1, 1, 0, 1}, {1, 1, 0, 1, 0, 1, 0, 1}, {1, 1, 1, 
            1, 0, 0, 0, 1}, {0, 0, 0, 0, 0, 0, 0, 1}, {1, 1, 1, 1, 0, 1, 0, 
            1}, {0, 0, 0, 1, 0, 1, 0, 1}, {1, 1, 0, 1, 0, 0, 0, 1}, {1, 1, 0, 
            1, 0, 1, 1, 1}};

ShowImage[BlobImage[Reverse[tst]] , 20];

After first pass of blob-finding algorithm

Same color denotes partially connected region.

tstb1 = BlobPass1[tst];
ShowImage[BlobImage[tstb1] // Reverse, 
        20];
BlobCount[tstb1]

After second pass of blob-finding algorithm

Same color denotes fully connected region.

tstb = BlobFind[tst];
ShowImage[BlobImage[tstb]  //  Reverse, 
        20];
BlobCount[tstb]

Operating on binary images: morphological operators

Load binary test image

img = 
        ReadImage["binary_tst.ppm"]  /.  
          1 -> 255;
scale  =  2;

Display image

ShowImage[img, scale];

Median filtering

tmp = MedianFilter[img, 2];
ShowImage[tmp, scale];

Morphological operators

Dilation operator

Structural element definition

s  =  Table[1, {3}, {3}];
s // MatrixForm

First dilation

g1 = 
        ShowImage[img, 2,  255, 0, 
          DisplayFunction -> Identity];
g2 = 
        ShowImage[tmp = Dilation[img, s], 2, 255, 0, 
          DisplayFunction -> Identity];
Show[GraphicsArray[{g1, g2}], 
        ImageSize -> Last[Dimensions[img]]*scale*2];

Second dilation

g2 = 
        ShowImage[tmp2 = Dilation[tmp, s], 2, 255, 0, 
          DisplayFunction -> Identity];
Show[GraphicsArray[{g1, g2}], 
        ImageSize -> Last[Dimensions[img]]*scale*2];

Erosion operator

Structural element definition

s  =  Table[1, {3}, {3}];
s // MatrixForm

First erosion

g1 = 
        ShowImage[img, 2,  255, 0, 
          DisplayFunction -> Identity];
g2 = 
        ShowImage[tmp = Erosion[img, s], 2, 255, 0, 
          DisplayFunction -> Identity];
Show[GraphicsArray[{g1, g2}], 
        ImageSize -> Last[Dimensions[img]]*scale*2];

Second erosion

g2 = 
        ShowImage[tmp2 = Erosion[tmp, s], 2, 255, 0, 
          DisplayFunction -> Identity];
Show[GraphicsArray[{g1, g2}], 
        ImageSize -> Last[Dimensions[img]]*scale*2];

Closing operator (dilation followed by erosion)

Structural element definition

s  =  Table[1, {3}, {3}];
s // MatrixForm

Closing

g1 = 
        ShowImage[img, 2,  255, 0, 
          DisplayFunction -> Identity];
g2 = 
        ShowImage[tmp = Closing[img, s], 2, 255, 0, 
          DisplayFunction -> Identity];
Show[GraphicsArray[{g1, g2}], 
        ImageSize -> Last[Dimensions[img]]*scale*2];

Opening operator (erosion followed by dilation)

Structural element definition

s  =  Table[1, {3}, {3}];
s // MatrixForm

Opening

g1 = 
        ShowImage[img, 2,  255, 0, 
          DisplayFunction -> Identity];
g2 = 
        ShowImage[tmp = Opening[img, s], 2, 255, 0, 
          DisplayFunction -> Identity];
Show[GraphicsArray[{g1, g2}], 
        ImageSize -> Last[Dimensions[img]]*scale*2];

Larger structural element definition

s  =  Table[1, {5}, {5}];
s // MatrixForm

Opening

g1 = 
        ShowImage[img, 2,  255, 0, 
          DisplayFunction -> Identity];
g2 = 
        ShowImage[tmp = Opening[img, s], 2, 255, 0, 
          DisplayFunction -> Identity];
Show[GraphicsArray[{g1, g2}], 
        ImageSize -> Last[Dimensions[img]]*scale*2];

Connected-component labeling after two morphological operators

Connected components in original image

tmp = BlobFind[img];
ShowImage[BlobImage[tmp], scale];
BlobCount[tmp]

Connected components following Closing operator

s = Table[1, {3}, {3}];
m1 = Closing[img, s];

tmp = BlobFind[m1];
ShowImage[BlobImage[tmp], scale];
BlobCount[tmp]

Connected components following (1) Closing and then (2) Opening operator

m2 = Opening[m1, s];

tmp = BlobFind[m2];
ShowImage[BlobImage[tmp], scale];
BlobCount[tmp]

Case study #1: find the ball

Load two color images of a ball in a room

img1 = ReadImage["ball1.ppm"];
img2 = ReadImage["ball2.ppm"];
scale  =  1;

Display images

g1 = 
        ShowImage[img1, scale, 255, 0, 
          DisplayFunction -> Identity];
g2 = 
        ShowImage[img2, scale, 255, 0, 
          DisplayFunction -> Identity];
Show[GraphicsArray[{g1, g2}], 
        ImageSize -> Dimensions[img1][[2]]*
            scale*2];

Load pre-processed image of ball for Gaussian color model

ball = ReadImage["ball_data.ppm"];

Show image (for building color model)

ShowImage[ball, 1];

Build Gaussian color model

Eliminate background pixels from data

rgb = 
        DeleteCases[Partition[Flatten[ball], 3], 
          ball[[1, 1]]];

Compute the mean μ and covariance matrix Σ over RGB pixel values

{μ  =  Mean[rgb]  //  N, Σ = 
          CovarianceMatrix[rgb]  //  N};
MatrixForm  /@  {μ,  Σ}

Gaussian color-model based ball detection: image #1

Select image

img = img1;

Display image

ShowImage[img, scale];

Gaussian distance image

gd = GaussianDistanceImage[img, μ, Σ];
ShowImage[Max[gd] - gd, scale, 1.5 Max[gd]];

Perform Gaussian-distance thresholding on original image

threshold = 22. ;

gt = 
        GaussianThresholdImage[img, 
          threshold, {μ, Σ}];
ShowImage[gt, scale];

Blob finding on noisy threshold image

tmp = BlobFind[gt];
ShowImage[BlobImage[tmp], scale];
BlobCount[tmp]

Clean up binary threshold image (w/Opening operator)

gt2 = Opening[gt, Table[1, {5}, {5}]];
ShowImage[gt2, scale];

Blob finding on clenaed-up threshold image

tmp = BlobFind[gt2];
ShowImage[BlobImage[tmp], scale];
BlobCount[tmp]

Superimpose original image with cleaned-up threshold image

ShowImage[BitAnd[gt2, img], scale];

Gaussian color-model based ball detection: image #2

Select image

img = img2;

Display image

ShowImage[img, scale];

Gaussian distance image

gd = GaussianDistanceImage[img, μ, Σ];
ShowImage[Max[gd] - gd, scale, 1.5 Max[gd]];

Perform Gaussian-distance thresholding on original image

threshold = 22. ;

gt = 
        GaussianThresholdImage[img, 
          threshold, {μ, Σ}];
ShowImage[gt, scale];

Blob finding on noisy threshold image

tmp = BlobFind[gt];
ShowImage[BlobImage[tmp], scale];
BlobCount[tmp]

Clean up binary threshold image (w/Opening operator)

gt2 = Opening[gt, Table[1, {5}, {5}]];
ShowImage[gt2, scale];

Blob finding on clenaed-up threshold image

tmp = BlobFind[gt2];
ShowImage[BlobImage[tmp], scale];
BlobCount[tmp]

Superimpose original image with cleaned-up threshold image

ShowImage[BitAnd[gt2, img], scale];

Case study #2: find the graduate student

Load a foreground and background image of a hallway

img1 = ReadImage["hall1.ppm"];
img2 = ReadImage["hall2.ppm"];
scale  =  1;

Display images

g1 = 
        ShowImage[img1, scale, 200, 0, 
          DisplayFunction -> Identity];
g2 = 
        ShowImage[img2, scale, 200, 0, 
          DisplayFunction -> Identity];
Show[GraphicsArray[{g1, g2}], 
        ImageSize -> Dimensions[img1][[2]]*
            scale*2];

Example processing steps #1

Smooth both images (Gaussian filtering)

gf = GaussianFilter[11, 1.25]  //  N;
s1 = Mask[img1, gf];
s2 = Mask[img2, gf];
g1 = 
        ShowImage[s1, scale, 200, 0, 
          DisplayFunction -> Identity];
g2 = 
        ShowImage[s2, scale, 200, 0, 
          DisplayFunction -> Identity];
Show[GraphicsArray[{g1, g2}], 
        ImageSize -> Dimensions[img1][[2]]*
            scale*2];

Subtract background from foreground image

ShowImage[diff = Abs[s2 - s1], scale, 100];

Threshold difference image

Threshold image

threshold = 40. ;

th = ThresholdImage[diff, threshold];
ShowImage[th, scale];

Blob finding on unprocessed threshold image

tmp = BlobFind[th];
ShowImage[BlobImage[tmp], scale];
BlobCount[tmp]

Manipulate binary threshold image

Closing operator

th2 = Closing[th, Table[1, {5}, {5}]];
ShowImage[th2, scale];

Dilation operator

th3 = Dilation[th2, Table[1, {3}, {3}]];
ShowImage[th3, scale];

Blob finding on processed threshold image

blb = BlobFind[th3];
ShowImage[BlobImage[blb], scale];
BlobCount[blb]

Keep only largest detected blob

bl2 = KeepLargestBlob[blb];
ShowImage[BlobImage[bl2], scale];

Superimpose original image with cleaned-up threshold image

v = bl2  // Flatten //  Frequencies  //  Last  //  
          Last;
bl2 = bl2  /.  v -> 255;
ShowImage[BitAnd[bl2, img2], scale, 100];

Example processing steps #2

Subtract background from foreground image

ShowImage[diff = Abs[img2 - img1], scale, 100];

Threshold difference image

Threshold image

threshold = 40. ;

th = ThresholdImage[diff, threshold];
ShowImage[th, scale];

Blob finding on unprocessed threshold image

tmp = BlobFind[th];
ShowImage[BlobImage[tmp], scale];
BlobCount[tmp]

Manipulate binary threshold image

Median filter

th2 = MedianFilter[th, 3]; ShowImage[th2, scale];

Closing operator

th3 = Closing[th2, Table[1, {7}, {7}]];
ShowImage[th3, scale];

Blob finding on processed threshold image

blb = BlobFind[th3];
ShowImage[BlobImage[blb], scale];
BlobCount[blb]

Keep only largest detected blob

bl2 = KeepLargestBlob[blb];
ShowImage[BlobImage[bl2], scale];

Superimpose original image with cleaned-up threshold image

v = bl2  // Flatten //  Frequencies  //  Last  //  
          Last;
bl2 = bl2  /.  v -> 255;
ShowImage[BitAnd[bl2, img2], scale, 100];

Case study #3: find the people...

Load a foreground and background image of a hallway

img1 = ReadImage["outdoor1.ppm"];
img2 = ReadImage["outdoor2.ppm"];
scale  =  1;

Display images

g1 = 
        ShowImage[img1, scale, 200, 0, 
          DisplayFunction -> Identity];
g2 = 
        ShowImage[img2, scale, 200, 0, 
          DisplayFunction -> Identity];
Show[GraphicsArray[{g1, g2}], 
        ImageSize -> Dimensions[img1][[2]]*
            scale*2];

Example processing steps #1

Smooth both images (Gaussian filtering)

gf = GaussianFilter[11, 1.25]  //  N;
s1 = Mask[img1, gf];
s2 = Mask[img2, gf];
g1 = 
        ShowImage[s1, scale, 200, 0, 
          DisplayFunction -> Identity];
g2 = 
        ShowImage[s2, scale, 200, 0, 
          DisplayFunction -> Identity];
Show[GraphicsArray[{g1, g2}], 
        ImageSize -> Dimensions[img1][[2]]*
            scale*2];

Subtract background from foreground image

ShowImage[diff = Abs[s2 - s1], scale, 100];

Threshold difference image

Threshold image

threshold = 20. ;

th = ThresholdImage[diff, threshold];
ShowImage[th, scale];

Blob finding on unprocessed threshold image

tmp = BlobFind[th];
ShowImage[BlobImage[tmp], scale];
BlobCount[tmp]

Manipulate binary threshold image

Closing operator

th2 = Closing[th, Table[1, {5}, {5}]];
ShowImage[th2, scale];

Blob finding on processed threshold image

blb = BlobFind[th2];
ShowImage[BlobImage[blb], scale];
BlobCount[blb]

Superimpose original image with cleaned-up threshold image

ShowImage[BitAnd[th2, img2], scale, 200];

Example processing steps #2

Subtract background from foreground image

ShowImage[diff = Abs[img2 - img1], scale, 100];

Threshold difference image

Threshold image

threshold = 20. ;

th = ThresholdImage[diff, threshold];
ShowImage[th, scale];

Blob finding on unprocessed threshold image

tmp = BlobFind[th];
ShowImage[BlobImage[tmp], scale];
BlobCount[tmp]

Manipulate binary threshold image

Median filter

th2 = MedianFilter[th, 3]; ShowImage[th2, scale];

Closing operator

th3 = Closing[th2, Table[1, {5}, {5}]];
ShowImage[th3, scale];

Blob finding on processed threshold image

blb = BlobFind[th3];
ShowImage[BlobImage[blb], scale];
BlobCount[blb]

Superimpose original image with cleaned-up threshold image

ShowImage[BitAnd[th3, img2], scale, 100];