data.table results differ between vector scan and binary search for missing data
data.table, r
Solution
I guess J is more than just a binary search; it's a "join." For each key combination it is given, it has to return something. To turn it off:
DT[J('R','H'),nomatch=0]
Problem
This is from the examples in the data.table introduction. See http://cran.r-project.org/web/packages/data.table/vignettes/datatable-intro.pdf The examples go on that a binary search is faster than a vector scan and produces exactly the same result (see page 5). So here is my example: ``` library(data.table) grpsize = ceiling(10000/26^2) DF <- data.frame(x=rep(LETTERS,each=26*grpsize), y=rep(letters,each=grpsize),v=runif(grpsize*26^2), stringsAsFactors=FALSE) DT = data.table(DF) setkey(DT,x,y) DT[x=='R' & y=='h'] DT[J("R","h")] ``` As expected this returns exactly the same result. One scans every row, the other is a binary search. However, when there are rows that are not existent the results differ. See the following code: ``` DT[x=='R' & y=='H'] DT[J("R","H")] ``` I get the following results ``` # > DT[x=='R' & y=='H', ] # Empty data.table (0 rows) of 3 cols: x,y,v # > DT[J("R","H")] # x y v # 1: R H NA ``` a.) Why is this the case? b.) Is there a way to change the behaviour of the binary search to not return results of non existing rows?