Using large input values with Auto Encoders

autoencoder, matlab

Solution

The process of convert your inputs to the range [0,1] is called normalization, however, as you noticed, the sigmf function is not adequate for this task. This link maybe is useful to you.

Suposse that your inputs are given by a matrix of N rows and M columns, where each row represent an input pattern and each column is a feature. If your first column is:

vec =

   -0.1941
   -2.1384
   -0.8396
    1.3546
   -1.0722

Then you can convert it to the range [0,1] using:

%# get max and min
maxVec = max(vec);
minVec = min(vec);

%# normalize to -1...1
vecNormalized = ((vec-minVec)./(maxVec-minVec))

vecNormalized =

    0.5566
         0
    0.3718
    1.0000
    0.3052

As @Dan indicates in the comments, another option is to standarize the data. The goal of this process is to scale the inputs to have mean 0 and a variance of 1. In this case, you need to substract the mean value of the column and divide by the standard deviation:

meanVec = mean(vec);
stdVec = std(vec);

vecStandarized = (vec-meanVec)./ stdVec

vecStandarized =

    0.2981
   -1.2121
   -0.2032
    1.5011
   -0.3839

Problem

I have created an Auto Encoder Neural Network in MATLAB. I have quite large inputs at the first layer which I have to reconstruct through the network's output layer. I cannot use the large inputs as it is,so I convert it to between [0, 1] using `sigmf` function of MATLAB. It gives me a values of 1.000000 for all the large values. I have tried using setting the format but it does not help. Is there a workaround to using large values with my auto encoder?

Original source

Related problems