Draw divisory MLP line together with chart in MATL

Question

I need to plot the divisory line together with the graph below:

The code I used to train the MLP neural network is here:

circles = [1 1; 2 1; 2 2; 2 3; 2 4; 3 2; 3 3; 4 1; 4 2; 4 3];
crosses = [1 2; 1 3; 1 4; 2 5; 3 4; 3 5; 4 4; 5 1; 5 2; 5 3];

net = feedforwardnet(3);
net = train(net, circles, crosses);

plot(circles(:, 1), circles(:, 2), 'ro');
hold on
plot(crosses(:, 1), crosses(:, 2), 'b+');
hold off;

But I'd like to show the line separating the groups in the chart too. How do I proceed? Thanks in advance.

Answer 1

First off, you're not training your neural network properly. You'd have to use both circles and crosses as input samples into your neural network and the output will have to be a two neuron output where [1 0] as the output would denote that the circles class is what the classification should be and [0 1] is what the crosses classification would be.

In addition, each column is an input sample while each row is a feature. Therefore, you have to transpose both of these and make a larger input matrix. You'll also need to make your output labels in accordance with what we just talked about:

X = [circles.' crosses.'];
Y = [[ones(1, size(circles,1)); zeros(1, size(circles,1))] ...
     [zeros(1, size(crosses,1)); ones(1, size(crosses,1))]];

Now train your network:

net = feedforwardnet(3);
net = train(net, X, Y);

Now, if you want to figure out which class each point belongs to, you simply take the largest neuron output and whichever one gave you the largest, that's the class it belongs to.

---

Now, to answer your actual question, there's no direct way to show the "lines" of separation with Neural Networks if you use the MATLAB toolbox. However, you can show regions of separation and maybe throw in some transparency so that you can overlay this on top of the figure.

To do this, define a 2D grid of coordinates that span your two classes but with a finer grain... say... 0.01. Run this through the neural network, see what the maximum output neuron is, then mark this accordingly on your figure.

Something like this comes to mind:

%// Generate test data
[ptX,ptY] = meshgrid(1:0.01:5, 1:0.01:5);
Xtest = [ptX(:).'; ptY(:).'];

%// See what the output labels are
out = sim(net, Xtest);
[~,classes] = max(out,[],1);

%// Now plot the regions
figure;
hold on;
%// Plot the first class region
plot(Xtest(1, classes == 1), Xtest(2, classes == 1), 'y.');
%// Add transparency
alpha(0.1);

%// Plot the second class region
plot(Xtest(1, classes == 2), Xtest(2, classes == 2), 'g.');
%// Add transparency
alpha(0.1);

%// Now add the points
plot(circles(:, 1), circles(:, 2), 'ro');
plot(crosses(:, 1), crosses(:, 2), 'b+');

The first two lines of code generate a bunch of test (x,y) points and ensures that they're in a 2 row input matrix as that is what the network inputs require. I use meshgrid for generating these points. Next, we use sim to simulate or put in inputs into the neural network. Once we do this, we will have two output neuron neural network responses per input point where we take a look at which output neuron gave us the largest response. If the first output gave us the largest response, we consider the input as belonging to the first class. If not, then it's the second class. This is facilitated by using max and looking at each column independently - one column per input sample and seeing which location gave us the maximum.

Once we do this, we create a new figure and plot the points that belonged to class 1, which is the circles, in yellow and the second class, which is the crosses, in green. I throw in some transparency to make sure we can see the regions with the points. After, I plot the points as normal using your code.

---

With the above code, I get this figure:

---

As you can see, your model has some classification inaccuracies. Specifically, there are three crosses that would be misclassified as circles. You'll have to play around with number of neurons in the hidden layer as well as perhaps using a different activation function but this certainly is enough to get you started.

Good luck!