Change row values to zero if less than row standard deviation

Question

I want to change all values of a row to zero if they are less than the standard deviation of that row. ``` set.seed(007) X <- data.frame(matrix(sample(c(5:50), 100, replace=TRUE), ncol=10)) X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 1 37 10 43 45 11 17 39 13 13 44 2 10 24 32 16 7 50 41 47 9 39 3 23 49 46 35 16 30 22 10 11 46 4 41 46 19 28 47 39 27 40 49 13 5 29 23 49 10 50 17 42 43 7 31 6 31 26 11 36 35 43 45 29 33 9 7 21 12 5 21 29 12 31 30 7 30 8 32 24 8 43 9 17 35 44 41 8 9 20 44 39 8 40 17 27 45 14 37 10 50 8 5 48 27 15 15 12 30 15 ``` The lines below appear to do the job, but is terribly slow on my actual use-case and I'm a bit unsure what sapply is returning.... ``` Y <- t(sapply(1:nrow(X), function(i) sapply(1:ncol(X), function(j) ifelse(X[i,][[j]] < sd(X[i,]), 0, X[i,][[j]])))) [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [1,] 37 0 43 45 0 17 39 0 0 44 [2,] 0 24 32 0 0 50 41 47 0 39 [3,] 23 49 46 35 16 30 22 0 0 46 [4,] 41 46 19 28 47 39 27 40 49 13 [5,] 29 23 49 0 50 17 42 43 0 31…

Accepted Answer

I suspect this is slower that the apply solution, but since there is no need to add the data.frame step and the fact that apply.data.frame is notoriously slow, I may still "win" or "keep even" at least until the other contestants tumble to the fact that I use a matrix object.

```
set.seed(007)
X <- matrix(sample(c(5:50), 100, replace=TRUE), ncol=10)
X[ sweep(X, 1, apply(X,1,sd) ) < 0 ] <- 0
```

Note that Richardo and I both got the same different starting point than the OP although I think he needed to transpose if he wants a row operation:

```
> X
   X1 X2 X3 X4 X5 X6 X7 X8 X9 X10
1  50  0 34 36 41 31  0 18 45  20
2  23 15 18 17 22 38 28 32 45   0
3   0 40 50  0 39 40 40 43 16  46
4   0  0 46  0 25 33 36 33 39   0
5  16 25 50 22 46 38 30  0 22  38
6  41  0  0 43 19 22 35 31  0  31
7  20 30 33 27  0 12 26 25  0  29
8  49  0 27 41 42  0 27 25 40  21
9   0 50 49 43 46 22 20 33 21  42
10 26 19 21 26 49 17 24 47 24  13
```

Added note: I was playing around with the rowMeans function to see if I could come up with a vectorized alternative to apply(X,1,sd) version of sd():

```
sqrt(rowSums((X[1:10, ]-rowMeans(X))^2)/9)
```

So:

```
 sdbyrow <- function(mat) sqrt(rowSums((mat-rowMeans(mat))^2)/(ncol(mat)-1) )
 all.equal(apply(X,1,sd), sdbyrow(X) )
#[1] TRUE
```

Change row values to zero if less than row standard deviation

Solution

Problem

Related problems