| |
| Monday, October 13, 2008 |
| Activity 20: Neural Networks |
Pattern recognition simply is making the computer recognize patterns we recognize visually. This begs the question: why not model pattern recognition the way our brain works? And this is what neural networks is about. Neural networks are composed of neurons arranged in a layer that is fully connected with the preceding layer. The layers are called the input, hidden and output layers respectively. The input layer is where the input features are fed and directed to the hidden layer which is again directed to the output layer. Neurons pass information using an "all-or-nothing" approach which is dependent on the strength of its connection to adjacent neurons.The tansig function is one of the most often used activation functions is pattern recognition problems.
For neural networks I used MATLAB instead of Scilab since MATLAB has an inbuilt Neural Networks toolbox while Scilab needs ANN. I used the parameters from Activity 18 for the 4 classes of images which are: Area, Height over Width, Average R, and Average G. Below is the code I used:
training = [5345 1.000000 0.758862 0.129754
6128 0.958763 0.756430 0.136960
5541 1.011494 0.814644 0.108821
1239 1.076923 0.602028 0.283912
1224 1.078947 0.629195 0.280192
1206 1.108108 0.611142 0.289349
439 0.444444 0.253336 0.508687
368 0.304348 0.248109 0.521104
372 0.444444 0.255933 0.518223
2080 1.000000 0.520683 0.374020
2064 1.040000 0.529217 0.382716
1977 1.083333 0.582884 0.345742 ]';
test = [ 5421 1.000000 0.795391 0.112691
5162 0.932584 0.826950 0.100648
5281 1.098765 0.788015 0.122597
1264 1.051282 0.633988 0.271862
1179 1.105263 0.618832 0.278147
1191 1.105263 0.615691 0.280110
306 0.409091 0.243423 0.517028
297 0.304348 0.245041 0.518146
304 0.239130 0.252019 0.514459
2000 0.980000 0.499452 0.372944
1956 1.020408 0.525111 0.379350
1906 1.062500 0.522208 0.392979 ]';
training_out = [0 0; 0 0; 0 0; 0 0; 0 1; 0 1; 0 1; 0 1; 1 1; 1 1; 1 1; 1 1]';
net=newff(minmax(training),[25,2],{'tansig','tansig'},'traingd');
net.trainParam.show = 50;
net.trainParam.lr = 0.01;
net.trainParam.lr_inc = 1.05;
net.trainParam.epochs = 1000;
net.trainParam.goal = 1e-2;
[net,tr]=train(net,training,training_out);
a = sim(net,training);
a = 1*(a>=0.5);
b = sim(net,test);
b = 1*(b>=0.5);
diff = abs(training_out - b);
diff = diff(1,:) + diff(2, :);
diff = 1*(diff>=1);
Again, 100% classification was observed.
Acknowledgments
Thanks to Jeric for helping me with the code. I understood this activity! i give myself 10/10 neutrinos!
 |
posted by poy @ 2:43 AM  |
|
|
|
|
| Activity 19: Linear Discriminant Analysis |
Moving on to our discussion about pattern recognition we study Linear Discriminant Analysis or LDA. Here we create a discriminant function from predictor variables. Here I tried classifying 5 centavo and 25 centavo coins.
This time since both are (should be) circular, I remove the parameter of Height over Width leaving three parameters for classifications: Area, R (NCC) and G (NCC). The code I used is shown below:
x = [1239 0.602028 0.283912;
1224 0.629195 0.280192;
1206 0.611142 0.289349;
2080 0.520683 0.374020;
2064 0.529127 0.382716;
1977 0.582884 0.345742];
xt = [1264 0.633988 0.271862;
1179 0.618832 0.278147;
1191 0.615691 0.280110;
2000 0.499452 0.372944;
1956 0.499452 0.379350;
1906 0.522208 0.392979];//Test class
y = [1;
1;
1;
2;
2;
2;];
x1 = [1239 0.602028 0.283912;
1224 0.629195 0.280192;
1206 0.611142 0.289349];
x2 = [2080 0.520683 0.374020;
2064 0.529127 0.382716;
1977 0.582884 0.345742];
n1 = size(x1, 1);
n2 = size(x2, 1);
m1 = mean(x1, 'r');
m2 = mean(x2, 'r');
m = mean(x, 'r');
x1o = x1 - mtlb_repmat(m, [n1, 1]);
x2o = x2 - mtlb_repmat(m, [n2, 1]);
c1 = (x1o'*x1o)/n1;
c2 = (x2o'*x2o)/n2;
C = (n1/(n1+n2))*c1 + (n2/(n1+n2))*c2;
p = [n1/(n1+n2); n2/(n1+n2)];
CI = inv(C);
for i = 1:size(xt, 1)
xk = xt(i, :);
f1(i) = m1*CI*xk' - 0.5*m1*CI*m1' + log(p(1));
f2(i) = m2*CI*xk' - 0.5*m2*CI*m2' + log(p(2));
end
class = f1 - f2;
class(class >= 0) = 1;
class(class < 0) = 2;
Again, 100% classification was shown.
Acknowledgments
Thanks to Ed for the code! I totally rocked this activity! I give myself 10/10 neutrinos!
 |
| posted by poy @ 12:41 AM |
|
|
|
| Sunday, October 12, 2008 |
| Activity 18: Minimum Distance Classification |
Moving on to more awesome applications of image processing, we have pattern recognition. Face recognition, retinal scans, object classification - the stuff of SciFi movie legends are actually used by scientists for their value in scientific analysis. One of the most common form of pattern recognition is to recognize different objects from data gathered on a training set.
In this activity we recognized Coke bottle caps, 25 centavo coins, Supplemantary medicine and 5 centavo coins.
The test and training images were white balanced first using the reference white algorithm. After which a code shown below was performed to classify each object and see whether the computer would be able to classify the objects properly. The characteristics I used to make these qualifications are as follows: pixel area, ratio of height and width, average red component (NCC), and average green component (NCC).
For the feature set I calculated the pixel area by summing up the binarized image. The ratio of height and width is computed as the maximum vertical minus the minimum vertical divided by its horizontal counterpart. Since the cropped images I used have the same dimensions, I just got the average red/green over the whole image. The feature vector of a class is represented by the mean of feature vectors of all training images used. The method used to classify the test images is the Euclidean distance. The training feature set which have the shortest distance over the test features is the class of that test image.
Since the objects I gathered are distinct from each other, I obtained correct classification for all 12 test objects. The code I used is threefold. The first is the code I used for the Training Set, the second for the Test and the third for the Classification.
First:
chdir('G:\poy\poy backup\physics\186\paper 18');
parameters = []; //1 = area, 2 = h/w, 3 = r, 4 = g
se = ones(3,3);
for n = 1:3
for i = 1:3
I = imread("images" + string(i) + ".JPG");
//Binarizing
I2 = I(:, :, 1) + I(:, :, 2) + I(:, :, 3);
I2 = I2/3;
I2(I2 > 0.40) = 2;
I2(I2 <= 0.40) = 1;
I2(I2 == 2) = 0;
//Closing operation
I2 = erode(dilate(I2, se), se);
//Area
area = sum(I2);
parameters(n, i, 1) = area;
//height/width
[v, h] = find(I2 == 1);
parameters(n, i, 2) = (max(v) - min(v))/(max(h) - min(h));
//average red and green (NCC)
ave = I(:, :, 1) + I(:, :, 2) + I(:, :, 3);
r = I(:, :, 1)./ave;
g = I(:, :, 2)./ave;
r = r.*I2;
g = g.*I2;
parameters(n, i, 3) = sum(r)/area;
parameters(n, i, 4) = sum(g)/area;
end
end
fprintfMat("param1.txt", parameters(:, :, 1));
fprintfMat("param2.txt", parameters(:, :, 2));
fprintfMat("param3.txt", parameters(:, :, 3));
fprintfMat("param4.txt", parameters(:, :, 4));
Second:
chdir('G:\poy\poy backup\physics\186\paper 18');
parameters = []; //1 = area, 2 = h/w, 3 = r, 4 = g
se = ones(3,3);
for n = 13:24
for i = 13:24
I = imread("images" + string(i) + ".JPG");
//Binarizing
I2 = I(:, :, 1) + I(:, :, 2) + I(:, :, 3);
I2 = I2/3;
I2(I2 > 0.40) = 2;
I2(I2 <= 0.40) = 1;
I2(I2 == 2) = 0;
//Closing operation
I2 = erode(dilate(I2, se), se);
//Area
area = sum(I2);
parameters(n, i, 1) = area;
//height/width
[v, h] = find(I2 == 1);
parameters(n, i, 2) = (max(v) - min(v))/(max(h) - min(h));
//average red and green (NCC)
ave = I(:, :, 1) + I(:, :, 2) + I(:, :, 3);
r = I(:, :, 1)./ave;
g = I(:, :, 2)./ave;
parameters(n, i, 3) = sum(r)/area;
parameters(n, i, 4) = sum(g)/area;
end
end
fprintfMat("param1-2.txt", parameters(:, :, 1));
fprintfMat("param2-2.txt", parameters(:, :, 2));
fprintfMat("param3-2.txt", parameters(:, :, 3));
fprintfMat("param4-2.txt", parameters(:, :, 4));
Third:
//Classifying the test images
parameters_train(:, :, 1) = fscanfMat("param1.txt");//area
parameters_train(:, :, 2) = fscanfMat("param2.txt");//h/w
parameters_train(:, :, 3) = fscanfMat("param3.txt");//mean(r)
parameters_train(:, :, 4) = fscanfMat("param4.txt");//mean(g)
for i = 1:4
for j = 1:4
m(i, j) = mean(parameters_train(i, :, j));
end
end
clear parameters_train;
normfactor = max(m(:, 1));
m(:, 1) = m(:, 1)/normfactor;
test(:, :, 1) = fscanfMat("param1-2.txt");
test(:, :, 2) = fscanfMat("param2-2.txt");
test(:, :, 3) = fscanfMat("param3-2.txt");
test(:, :, 4) = fscanfMat("param4-2.txt");
test(:, :, 1) = test(:, :, 1)/normfactor;
for i = 1:4
for j = 1:4
for k = 1:4
t(k) = test(i, j, k);
end
for l = 1:4
d = t' - m(l, :);
dist(i, j, l) = sqrt(d*d');//distance of image(i, j) on class l
end
end
end
correct = 0;
for i = 1:4
for j = 1:4
shortest = find(dist(:,j,i) == min(dist(:,j,i)));
if shortest == i
correct = correct + 1;
end
end
end
Acknowledgments:
Thanks Ed for the code. I give myself 9 neutrinos for this activity, things are still a bit fuzzy for me.
 |
| posted by poy @ 9:08 PM |
|
|
|
|
|
|
