Keras summation Layer acting weird, summing over training set

Question

I am having trouble understanding the basic way Keras works. I am experimenting with a single summation layer, implemented as a Lambda layer using tensorflow as a backend: ``` from keras import backend as K test_model = Sequential() test_model.add( Lambda( lambda x: K.sum(x, axis=0), input_shape=(2,3)) ) x = np.reshape(np.arange(12), (2,2,3)) test_model.predict(x) ``` This returns: ``` array([[ 6., 8., 10.], [ 12., 14., 16.]], dtype=float32) ``` Which is very weird, as it sums over the first index, which to my understanding corresponds to the index of the training data. Also, if I change the axis to `axis=1` then the sum is taken over the second coordinate, which is what I would expect to get for `axis=0`. What is going on? Why does it seem like the `axis` chosen effects how the data is passed to the lambda layer?

Accepted Answer

The `input_shape` is the shape of one sample of the batch. It doesn't matter if you have 200 or 10000 samples in a batch, all the samples should be (2,3).

But the batch itself is what is passed along from one layer to another. A batch contains "n" samples, each sample with the `input_shape`:

- Batch shape then is: (n, 2, 3) -- n samples, each sample with input_shape = (2,3)

You don't define "n" when `input_shape` is required, because "n" will be defined when you use `fit` or another training command, with the `batch_size`. (In your example, n = 2)

This is the original array:

```
[[[ 0  1  2]
  [ 3  4  5]]

[[ 6  7  8]
  [ 9 10 11]]]

Sample 1 = [ 0  1  2], [ 3  4  5]
Sample 2 = [ 6  7  8], [ 9 10 11]
```

Summing on index 0 (the batch size dimension) will sum sample 1 with sample 2:

```
[ 6 8 10], [12 14 16]
```

Summing on index 1 will sum the first dimension of one sample's input shape:

```
[ 3, 5, 7 ], [15, 17, 19]
```

Keras summation Layer acting weird, summing over training set

Solution

Problem