epoch and calculating mean square error for training set Neural Network

artificial-intelligence, neural-network

Solution

y question is about Neural Network Training. I already searched about this but, there is no good explanation about it.

There are dozens of good explanations on the web, and in the literature, one such example may be the book by Haykin: Neural Networks and Learning machines

So for the first one, how to calculate mean square error? (I know this is silly, but I really don't get it)

In the most simple terms, mean squared error is defined as

sum_i 1/n (desired_output(i) - model_output(i))^2

So you simply calculate the mean of the squares of the errors (differences between your output, and the desired one).

Now when should we calculate the mean square error? does it when we already take all pairs? or does we calculate it for each pair?

Both methods are used, one is called batch learning, and one is online learning. So all next questions have the answer "both are correct, depending whether you are using batch or online learning". Which one to choose? Obviously - it depends, but for a sake of simplicity I would suggest starting with batch learning (so you compute the error over all training samples and then update).

Problem

Original source