Collaborative Neural Meshes - Ending Spam: Bayesian Content Filtering and the Art of Statistical Language Classification [Electronic resources] نسخه متنی

Collaborative Neural Meshes

The term neural network is rather generic. The Wikipedia (http://www.wikipedia.org) definition of a neural network is “a mathematical model for information processing based on a connectionist approach to computation.” The original inspiration for the technique was from examination of bioelectrical networks in the brain formed by neurons and their synapses. In a neural network model, simple nodes (or neurons, or units) are connected together to form a network of nodes—hence the term “neural network.”

The type of network discussed here is closer to a neural mesh. It is similar to a classification network in that it deals with discovering collaborative results, but the neural mesh approach is implemented with a different architecture.

Contrary to a classification group, which requires only as many nodes as are necessary to make a positive identification, in a neural mesh all nodes within a certain proximity to the input “neuron” are queried. Individual nodes are interconnected with other nodes to form a mesh of synaptic circuitry. The interconnections between nodes are best built using specific logic to identify nodes that are more contextually similar to each other. Once the results from all relevant nodes have been returned, the results from the most reliable nodes are then used to make a final decision.

Reliable nodes are determined based on how accurate they have been in the past. Reliability is generally calculated by keeping track of the number of correct and incorrect classifications each node has made against the mail of each of the other users. This series of neural decisions is stored in the system as temporary data. Each decision contains a list of all the users (or nodes) involved in the decision and what the decision was. If the user determines that the message was erroneously classified, the retraining process will trigger a lookup of the neural decision. Decisions that were incorrect will then

be subtracted from the affected nodes’ “correct” totals and added to their “incorrect” totals, which could result in the “rewiring” of certain interconnections. At execution time, these interconnections will identify reliable nodes as closest. These will also be nodes that are more contextually similar to the user. Figure 14-1 shows an example of nodes interconnected in this way. Nodes can be interconnected in a variety of different practical implementations. In the neural mesh paradigm, the interconnected roots are shifted based on reliability and will provide a structure in which the most reliable nodes are the most closely interconnected.

Figure 14-1: Neural mesh interconnections

A practical implementation of this approach can use varying “strengths” and distances. The reliability (and results) are then combined to form a probability based on the results.

The advantage of using a neural network rather than a classification group is that the filter is capable of “learning” which users have datasets closer to their own mail behavior, therefore providing better results. Apart from being very cool to watch, this also is what makes collaborative algorithms such as these successful, since there is such a diverse set of users on most systems.

Another application for a neural mesh is to create dynamic dataset groups to better scale system resources. For example, if our neural mesh includes 100 users, and it finds that 25 of those users appear to have an extremely high reliability rate with one another, those 25 users could potentially be clustered together using a single dataset with only minimal train-on-error data in each user’s own database, which could be merged at run time. Merged groups are sometimes also created by systems administrators to provide a global set of data for all users, allowing individual users to store necessary training data and merge it with the global information at run time. This approach allows users to quickly ramp up their filtering by using a global group without the need to wait for their own corpus to become mature before their training data is applied to the decision. The same approach can be used to merge groups of contextually similar users together, using only minimal training data to “tweak” the parent group.

Neural Declustering

An inverted approach to neural networking, called neural declustering, can be used to break up a series of global groups into smaller, more contextually similar groups. Let’s say you started out with three servers and 20,000 users on each server. Each of the three servers could be split in half, to form six nodes of 10,000 users each, and linked into a neural network group. The neural network would then be able to determine which of the six nodes were closest to one another. The nodes could then be split up based on their contextual similarities into 12 different parent datasets. These datasets would use the neural network to find the ones that they were the most contextually similar to, merge them again, and split up into 24 different datasets, and so on. Figure 14-2 illustrates the process of declustering three large systems into several smaller ones based on similarities.

Figure 14-2: Neural declustering