 |
||
|
||
|
||
|
||
|
||
|
||
| AI Sourcebook, Part I: Neural Networks | |
|
I often receive questions related to the implementation of various neural network architectures. There are many books and articles explaining the theory behind the particular models, but most of them are difficult to understand and even more difficult to implement. Turning the theory into practical design can be a challenging task, even for the most experienced programmer. Of course, you could use various commercial controls: it may be the way to go on large-scale projects, but it takes the control out of your hands and leave you with less knowledge on the inner workings of NNs. Several Web sites contain documentation, ready-to-(re)use and well documented source code and even complete example applications for a variety of real-world problems. Some of them contain true programming gems, but are difficult to find using standard search engines. So, without further ado, here is a list of sites featuring NN-related source code.
Applets for Neural Networks and Artificial Life
Links to related sites with source for competitive learning methods, backpropagation,
neural nets for constraint satisfaction and optimization, simulated annealing,
Bayesian networks, etc.
Backprop C++ code
These programs lets you create networks of arbitrary size, and uses a sigmoid
function to compute the output of each unit, weights and biases are adjusted
with the delta rule as described by the PDP researchers . The net has three
layers which are fully interconnected, and uses on-line learning. It was successfully
used for character and number recognition.
Binary Hopfield Neural Network Applet
A Hopfield network is an associative memory that works by calculating how much
a node is in agreement with the nodes that it is connected to. This Java applet,
written by Matt Hill, demonstrates the inner workings of such network.
BPN.class
This XOR Java applet is a little example that implements a 3-input XOR gate
with a 3 layered neuronal network.
BSOM
Bayesian Self-Organizing Map Simulation in Java by Akio Utsugi.
Connectionist Models of Cognition
Created by Simon Dennis and Devin McAuley, Connectionist Models of Cognition
is a Web-based textbook designed to introduce the key concepts in the area of
neural networks. It targets both faculty and students interested in the application
of parallel distributed processing models to cognitive phenomena either in a
teaching or research capacity. The textbook provides hands-on modeling experience
with a variety of standard architectures and the opportunity to begin developing
your own models. A background in connectionist theory is not required. The Java-based
BrainWave connectionist simulator is embedded within the materials, allowing
you to interact with the figures as you work through the exercises. Designed
for the introductory cognitive science course on connectionist models at the
University of Queensland, BrainWave employs a highly graphical, direct manipulation
interface - much like a drawing program - allowing students to focus on the
models and not the interface.
DemoGNG
This Java applet developed by Hartmut S. Loos implements several methods related
to competitive learning. Here is a list
with all available models and distributions.
Freeware and Shareware Tools for implementing Neural Networks and Connectionism
One of the best resources for NN-related topics on the Web, maintained by Pacific
Northwest National Laboratory.
jaNet
jaNet package is a Java neural network toolkit. With jaNet you can design, test,
train and optimize and save a neural network an external file. You can then
include such network in your private application using the jaNet.backprop package.
Java Demos
The Computation and Neural Networks Laboratory at at Wayne State University
has developed Java demonstrations of neural net concepts. Includes Support Vector
Machine, Self-Organizing Maps, Clustering via Simple Competitive Learning, Perceptron,
Backprop, links to other similar sites, etc.
L.A.N.E.
The intent and design of L.A.N.E. is to provide a foundation with which any
neural net can be created with an elaborate set of tools that will facilitate
the realization of any architecture. Users may structure there applications/uses
as a set of primitive operations that can be composed using scripts to best
suit the user's needs. Neural networks can be quickly generated and displayed
graphically. Besides C/C++, it supports Tcl/Tk.
Neural Networks at
your Fingertips
Thanks to Karsten Kutza, this excellent page features source for Adaline network,
Backpropagation network, Hopfield model, Bidirectional Associative Memory, Boltzmann
machine, Counterpropagation network, Self-Organizing Map and Adaptive Resonance
Theory model. You can download
the complete package in a comprehensive .zip file. This file will extract to
a directory structure, containing the C source, MS Visual C++ 4.0 makefile,
Win32 executable, and the generated output for each of the above mentioned programs.
Neural Nets C++ Library
Rodrigo de Salvo Braz wrote some basic classes to make the job of writing some
neural network-like models easier. This library defines a set of classes that
are essential to neural networks, like pattern, batch and neural net itself.
It does not define, though, classes for internal components like units, for
example, because the implementations of many models do not need that level and
it would make the applications be heavier and more complicated than necessary.
However, there are some problems with the library: the backpropagation example
does not work in all cases, so we are expecting the necessary changes...
Neural Networks, Connectionist Systems, and Neural Systems
This section at CMU Artificial Intelligence Repository features more than 60
various NN packages and simulators. You'll find source code for networks written
in Scheme, Lisp, Pascal and C/C++.
Neural Network in Visual Basic
Small, easy to understand project that implements backpropagation algorithm
in VB.
Neural Networks for Face Recognition
This web page provides an implementation (in C++) of the backpropagation algorithm
described in Chapter 4 of the textbook Machine Learning. It also includes the dataset discussed in Section 4.7 of the book,
containing over 600 face images.
Perceptron
This Java applet is an example of a basic perceptron which can recognize whether
a pattern belongs to one group or another.
Source code
for Neural Networks - Site1,
Site2
These Russian sites contain source code for various programs implementing neural
network models, along with the "good old fashioned AI" (inference
engines, expert systems, NLU) and Genetic Algorithm examples in Pascal, Delphi,
VB and C.
Neural Net (Python, Java)
The purpose of this project is to create a neural net that can be trained on
reading the numbers 1 through 4. The numbers are placed in a 5 x 4 grid of 0s
and 1s, and also "grayscale' values of 0.1 through 0.9.
The Pattern Recognition Basis of AI
This excellent site offers programs with a command line (non-GUI) interface.
The source is suitable for Unix and DOS, there are 16-bit DOS binaries (uses
hardware floating point) for all the programs and a Windows 3.1 binary for backprop.
The newest Unix version requires the free Tcl/Tk package. GUI based versions
of backprop for Unix and Windows plus non-GUI versions of backprop, k-nearest-neighbor,
LVQ1, DSM, simple clustering, ART I, feed-forward counter propagation, interactive
activation network, a simple BAM, Hopfield and Boltzman networks and the linear
pattern classifier.
XOR
Backpropagation example from Generation 5: XOR Net in C++ with several excellent accompanying essays.