Using AI for categorization

AI systems for categorisation

The freeware [[FuzzyCOPE?]] application is used to learn how fuzzy rules can be extracted from analysis of the weights of a neural network. This is a mouth full, but will be explained step-by-step.

See wikipedia for information on neural networks, however, we're creating a Fuzzy neural network (FuNN) here. The difference with a normal neural network is that we will not treat the network as a black box, but use it to extract the learned results in the form of rules. The result will list a number of rules like: "if property11 is small and property 2 is large, then the sample belongs to category 3".

Note that FuzzyCOPE/3 runs on Linux using the WINE package.

Fuzzy sets

The first you'll need to know is what is meant by fuzzy, which is made too complicated all the time. It basically is a variation on set theory. In this theory, an element is either in a set or it isn't. A fuzzy variable can partly be in a set, we indicate this with a number between 0 and 1 to indicate this. A membership function is used to calculate this number.

Rules extraction

On to the neural network stuff. A neural network is a great technique to classify information, except for the fact that afterwards you know that it has learned, but you have no idea what it learned.

That's where the "extracting rules" piece from the introduction comes into play. We will attempt to train a neural network and after the training, we'll try to extract the fuzzy logic that the network learned. In practice, we'll try to derive the membership functions from the weights of the network.

The data

What we're trying to categorize here are flowers, and in particular three species of irises: setosa, versicolor and virginica. For 150 samples (50 of each type of flower), four variables have been determined and measured:

As said, we know beforehand that there are three species hidden in the data file. We are going to categorize the datafile in the three species in the next steps. So afterwards, we'll need to be able to say things like: "well, the crown leaf length of this particular iris is 7.5 cm, so it's probably the setosa iris."

To make it easy to make such determinations, we must divide the values of the variables into intervals. Let's assume someone used a tool to do exactly this.

Preparing the data

We have a set of data, iris1.dat. You can open this file in a text editor. The header contains some interesting information. You'll see the first four columns of data containing lengths of the iris leaves. These will have to be normalized, i.e. reworked in a value between 0 and 1. This can be done with the Data Analysis Tool of FuzzyCOPE/3. With the Shuffle button, the data is placed out of order. The data can be saved as a training file.

Setup of the FuNN

We'll need to determine a few things by hand before we can set up the FuNN. In this section, we'll just discover them one by one before setting up the FuzzyCOPE/3 tool.

In our data file, each line has four characteristics of an iris flower (see above). These four characteristics will be used to separate the datafile into three particular types of irises.

In neural network speak, we have four input variables and three output variables. The output variables can have only two values: one iris is either member of a particular type or not. This means that the number of actions in our neural network must be 2.

Now for the internals of our FuNN. We'll need to look at the data ourselves and set up some intervals for the input variables. As said before, we assume this has already been done. For each interval, our fuzzy neural network must determine a membership function. That means specifying the number of membership functions in our FuNN.

The number of rules that will be derived must be stated as well. This will take some trial-and-error. It's best to start with a number equal to the number of membership functions.

Extraction of rules

After the data is prepared,