ANN 学习

最近公司做一个商品预测项目，看了多篇Paper均使用ANN或者ENN，因此借此机会学习一下神经网络

首先，这里介绍了众多的神经网络工具
http://www.oschina.net/project/tag/401/Neural-Network
在其中，我分别测试了Neuroph 和Jonn 以及 FANN ,最终FANN 比较好用，Neuroph 运行时太废CPU，JONN年代久远，与我使用的JRE不一致，并且没法找到合适的JRE 。

下面是FANN 的使用方法

Installation

Unzip the fann21.zip file anywhere you want. fannExplorer and fannKernel do not use or change registry settings or menus. Install and enable Flash Player and JavaScript
in the web browser you want to use. Flash Player version 7 is required, version 9 is recommended. To uninstall simply delete the fann directory and subdirectories where you installed it.

Starting up

Double-click fannKernel.exe in the fann\fannSoap\fannKernel\Release subdirectory or start it from a console/commandline. Your firewall may display a warning at this
point, you need to allow fannKernel to communicate locally with fannExplorer.



Then start your favourite web browser and type in or copy/paste the web browser link shown with your local machine name instead of SonyMobile which is the machine this
is written on. If you have JavaScript and Flash 7 or later enabled then you will be greeted with the fannExplorer main window. You can save this as a shortcut/favourite/bookmark in your browser. You can run multiple fannExplorer clients simultaneously, the
fannKernel is multi-threaded, so you only need to run one fannKernel when you train several neural networks at the same time. Note that using multiple threads is only faster if your processor has multiple CPU cores.

Loading a neural network

In the fannExplorer menu choose the menu items indicated with numbers - in any order. For example to load the xor sample: Load (1) xor.net with the Load Neural Network,
load (2) xor.train with Load Training Data, load (3) xor.test with Load Test Data, finally (4) select Show All.



In the Controller you can click on the header lines to access different functionalities: Topology gives basic information about the neural network. Algorithm allows
you to choose the exact parameters used by the neural network. Training contains functions related to training. Testing lets you evaluate the error and output of the neural network.

Training a neural network

To try out the xor sample click on the training pane header. First click on Randomize or Initialize. You can see the relative weights change in the Weight Graph window.
Then click on Train. You can see the Mean Square Error Plot (MSE) being updated with a measure of how well the neural network approximates the desired output. It should rapidly approach 0.001 for this sample. If not then click Randomize and Train again.

Testing a neural network

Click on the testing pane header then click on Execute. In the Output Plot you can now see the values the neural network was trained to reproduce as green dots and the
current output that the neural network produces as red dots.

Animating the training

Switch back to the Training pane. Click on Randomize to reset the neural network then click on Update output plot while animating. Now use the Animate button instead
of the Train button. While this slows down the training process dramatically, it can be very instructive to view the weights, MSE and output.

You can try this again with a the BesselJ sample. First load besselj.net, besselj.train and besselj.test as before. Since the BesselJ function is more difficult for a neural network to approximate, adjust
the network parameters: On the Algorithm pane set the training algorithm to Incremental. Back on the Training pane set Maximum number of epochs to 6000, set Stop when mse falls below to 0, set Epochs between mse reports to 100, and set Epochs between display
updates to 500. Click on Randomize and Animate.

Preparing your own data

First you have to convert or create training and test data files in the FANN file format. It is a simple format that you can create with a text editor or a script. The
first line contains three integers: Number of data pairs, number of inputs and number of outputs. Next follows the data pairs: One line with the input data values and one line with the output data values. All numbers are separated by blank space characters.
See the FANN C fann_read_train_from_file and the example files if you need more details. You can not create new data files in fannExplorer, only edit simple ones.
To get good initial results normalize your input/output data in a -1 to 1 interval for use with the symmetric activation functions or 0 to 1 for the rest. Also use a 0 to 1 interval for the cascade training
with the default settings. You can do this with the large number of scaling functions in FANN 2.1 or simply use a script or spreadsheet. Note that the floating point format depends on your C compiler if you use exponential notation (see printf/scanf and the
formats in the FANNPRINTF/FANNSCANF macros for your compiler).

Working with files

Neural network files are placed in the net subdirectory below the fannKernel's startup directory. The files have extensions named according to their content. Fann neural
network definitions have extension .net, training data have .train and test data have .test.
If you follow the same naming convention and place copies of your own fann files in the net subdirectory, then they become available in the fannExplorer's Load dialogs, so you can train your own neural networks.

Creating your own neural network

Load your data files with fannExplorer then either create a New Neural Network from the File menu or try enabling Cascade Training on the Training pane to get a network
created for you. If you want to use Cascade Training then you need to read the FANN C documentation and pay attention to the tooltips.
To get started try loading the xor.train and xor.test files in fannExplorer - there is no need to load a neural network file for this sample - one will be created with cascade training. Edit the data files
by pressing the Edit buttons on the Train and Test panes replacing the -1 values with 0. Press Apply when you are done. You can save the data files with new names from the File menu - you could also have used a text editor to do this. On the Training pane
turn Cascade training mode on. Change 'Maximum number of neurons' to 5 and 'Neurons between mean square error tests' to 1. Then Reset the network and press Train. Use the Execute button on the Testing pane to try the little marvel or save it from the File
menu.
The time series predicting sunspot sample is in a form suitable for cascade training if you want to try a more demanding example. The time series prediction works by presenting as input the values for time
t-1, t-2, t-4 and t-8 and as output the value for t. The data was generated and normalized in a 0.2 to 0.8 range with a script using almost 300 years of observed sunspot activity. The test data contains the data for recent years that were excluded from the
training data set. You will need a little patience and have cascade training generate about 15-20 hidden neurons for this sample to work. After you have generated a network you can turn off cascade training and see if you can improve the results with normal
training and by adjusting the parameters on the Algorithm pane.

Performance

If you have a neural network with many neurons or connections then close the Weight Graph Window. It will not be informative and updating it may take so long that you
will experience timeouts. If you have a large test data set then do not use the Output Plot with animation. The grid component that you can access from the Weight Graph, MSE Plot and Edit Test/Training Data is only suitable for small data sets. Use a text
editor or script for larger data sets. The reason behind the fannExplorer/fannKernel design was to make it very simple to run one kernel on a machine in a classroom and have students access it via a browser, this design implies the outlined performance limitations.

Scripting

Scripting samples with C#, VB and SML.Net are included in the fannDotNet directory. You can also find samples on using Mathematica via .Net if you are using Windows
or via web services otherwise. If you want to integrate and distribute a neural network that you have created with an application written in a .Net language, then you can obtain a precompiled Windows DLL from the FANN website and use Platform/Invoke to access
it (see MSDN for details). If your neural network has been trained then you only need to declare functions for loading and running it in your C# or VB code.

Where to go from here

While you are using fannExplorer you can access its online help page which has more detailed information. If you are not familiar with neural networks then you will
need a good text book to get satisfactory results. You can also find information on neural networks and backpropagation on Wikipedia. For more details see the FANN C or C++ documentation.
Is it time for a break? Then see a preview of
what is happening in the freegoldbar programming corner. Or have a go at the DeathStacks game.

Security notice

Use a firewall when you are connected to the internet and do not run fannKernel on a webserver. fannKernel is intended to be used in a friendly environment such as on
a local machine or in an intranet behind a firewall. fannKernel uses SOAP over tcp/ip by default on port 2718 (first four digits of the exponential constant) to communicate with fannExplorer clients. fannKernel does not implement authentication or encryption.
If you want absolute security then you can unplug your network cable while using fannKernel. Otherwise use a good firewall to disallow incoming connections.

License

Your neural networks are your own property. The FANN C Library and C++ wrapper are distributed under LGPL. fannKernel and fannExplorer are copyright 2007/2550 by freegoldbar
at yahoo dot com. All rights reserved. For private and educational use only. Not for sale, commercial or military use. No warranties given or implied. No suitability for any purpose.