Neural Network Design Flow

Figure 2.6: The Neural Network Design Flow


Typically, the Neural Network design flow contains 5 major steps as shown in Figure 2.6. The first step involves in data-set preparation. In this step, based on requirements of an application, raw data is collected, processed and transformed to form a data-set that contains a collection of input-target pairs. The data-set is then divided into 3 small sets, including training set, validation set, and test set. The training set is used to train a Neural Network, the validation set is used to prevent over-fitting issue during the training process, and the test set is used to evaluate the Neural Network stability to test if it could cope with a completely new data-set.


Once the data-set is available, the Neural Network structure will be configured accordingly. This step involves defining how many input nodes, hidden nodes, and output nodes of the Neural Network. The activation functions for hidden layers and output layer are also being defined in this step. Both pre-processing and post-processing methods are configured to transform data-set inputs into the correct range that the Neural Network can use and reverse its outputs to the range that match data-set targets. This step also allows users to choose the correct training algorithm as well as the cost function to be used in a training procedure.


After the Neural Network is being configured, a selected training algorithm that uses the training set will teach Neural Network to learn data-set input-target mapping. This training algorithm is based on the optimization process that minimizes the errors by adjusting the Neural Network weights. During that training process, the validation set is used to prevent over-fitting issue.


The trained Neural Network is then evaluated in step 4 to test how it responds to complete new data-set. Evaluation techniques such as confusion matrix, ROC curves, regression curves will be used to decide if the Neural Network is required to be retrained due to poor generalization.


If the trained Neural Network copes well with a new data-set and its performance meets desired criteria in terms of robustness and accuracy, it will be deployed to a given application in step 5.