Row sum for large term-document matrix / simple_triplet_matrix ?? {tm package}

Question

So I have a very large term-document matrix: ``` > class(ph.DTM) [1] "TermDocumentMatrix" "simple_triplet_matrix" > ph.DTM A term-document matrix (109996 terms, 262811 documents) Non-/sparse entries: 3705693/28904453063 Sparsity : 100% Maximal term length: 191 Weighting : term frequency (tf) ``` How do I get the rowSum (frequency) of each term? I tried: ``` > apply(ph.DTM, 1, sum) Error in vector(typeof(x$v), nr * nc) : vector size cannot be NA In addition: Warning message: In nr * nc : NAs produced by integer overflow ``` Obviously, I know about `removeSparseTerms`: ``` ph.DTM2 <- removeSparseTerms(ph.DTM, 0.99999) ``` Which cuts down the size a bit: ``` > ph.DTM2 A term-document matrix (28842 terms, 262811 documents) Non-/sparse entries: 3612620/7576382242 Sparsity : 100% Maximal term length: 24 Weighting : term frequency (tf) ``` But I still cannot apply any matrix-related functions to it: ``` > as.matrix(ph.DTM2) Error in vector(typeof(x$v), nr * nc) : vector size cannot be NA In…

Accepted Answer

OK, after some more Google'ing, I came across the `slam` package, which enables:

```
ph.DTM3 <- rollup(ph.DTM, 2, na.rm=TRUE, FUN = sum)
```

Which works.

Row sum for large term-document matrix / simple_triplet_matrix ?? {tm package}

Solution

Problem