用容差匹配两个非常非常大的向量（速度快！但节省工作空间）

Question

matches: abs(((referencelist - sample[i])/sample[i])*10^6)) < 0.5

referencelist <- read.table(header=TRUE, text="value  name
154.00312  A
154.07685  B
154.21452  C
154.49545  D
156.77310  E
156.83991  F
159.02992  G
159.65553  H
159.93843  I")

sample <- c(154.00315, 159.02991, 154.07688, 156.77312)

    name value      reference
1    A   154.00315  154.00312
2    G   159.02991  159.02992
3    B   154.07688  154.07685
4    E   156.77312  156.77310

我能做的是使用例如外部函数

myDist <- outer(referencelist, sample, FUN=function(x, y) abs(((x - y)/y)*10^6))
matches <- which(myDist < 0.5, arr.ind=TRUE)
data.frame(name = referencelist$name[matches[, 1]], value=sample[matches[, 2]])

或者我可以使用for()循环。

但我的特殊问题是，参考向量大约有1*10^12个条目，而我的样本向量大约有1*10^7个条目。因此，通过使用outer()，我可以轻松地破坏所有的工作空间限制，并且通过使用for()或链接的for()循环，这将需要几天/几周才能完成。

最美好的愿望

Answer 1

您的匹配条件

abs(((referencelist - sample[i])/sample[i])*10^6)) < 0.5

可以重写为

sample[i] * (1 - eps) < referencelist < sample[i] * (1 + eps)

EPS=0.5E-6。

library(data.table)
options(digits = 10)
eps <- 0.5E-6 # tol * 1E6
setDT(referencelist)[.(value = sample, 
                       lower = sample * (1 - eps), 
                       upper = sample * (1 + eps)), 
                     on = .(ref > lower, ref < upper), .(name, value, reference = x.ref)]

   name     value reference
1:    A 154.00315 154.00312
2:    G 159.02991 159.02992
3:    B 154.07688 154.07685
4:    E 156.77312 156.77310

setDT(referencelist2)[.(value = sample, 
                       lower = sample * (1 - eps), 
                       upper = sample * (1 + eps)), 
                     on = .(ref > lower, ref < upper), .(name, value, reference = x.ref)]

   name     value reference
1:    A 154.00315 154.00312
2:    F 154.00315 154.00320
3:    G 159.02991 159.02992
4:    B 154.07688 154.07685
5:    E 156.77312 156.77310