performance matrix r matrix-multiplication

双向频率表，然后是矩阵乘法 - 高运行时间

发布于 2025-01-23 04:23:07 字数 786 浏览 2 评论 0原文

我是R的新手，并尝试将固定矩阵之间的产品计算到2向频率表之间，以适用于dataframe或矩阵中的任何列组合，然后将其除以序列长度（aka的行数为15），问题在于，运行时间在1K序列（1K列）上执行时会急剧增加。目的是用尽可能多的序列使用它（超过10分钟，10K可能超过1小时），

mat1 <- matrix(sample(LETTERS),ncol = 100,nrow = 15)
mat2 <- matrix(sample(abs(rnorm(26,0,3))),ncol=26,nrow=26)
rownames(mat2) <- LETTERS
colnames(mat2) <- LETTERS
diag(mat2) <- 0

test_vec <- c()
for (i in seq(ncol(mat1)-1)){  
  for(j in seq(i+1,ncol(mat1))){
    
    s2 <- table(mat1[,i],mat1[,j]) # create 2-way frequency table
    mat2_1 <- mat2
    mat2_1 <- mat2_1[rownames(mat2_1) %in% rownames(s2), 
                               colnames(mat2_1) %in% colnames(s2)]
    calc <- ((1/nrow(mat1))*sum(mat2_1*s2))
    test_vec <- append(test_vec,calc)

  }}

感谢您的帮助。

原文

I'm new to R, and trying to calculate the product between a fixed matrix to a 2-way frequency table for any combinations of columns in a dataframe or matrix and divide it by the sequence length (aka number of rows which is 15), the problem is that the running time increases dramatically when performing it on 1K sequences (1K columns). the goal is to use it with as much as possible sequences (more than 10 minutes, for 10K could be more than 1hr)

mat1 <- matrix(sample(LETTERS),ncol = 100,nrow = 15)
mat2 <- matrix(sample(abs(rnorm(26,0,3))),ncol=26,nrow=26)
rownames(mat2) <- LETTERS
colnames(mat2) <- LETTERS
diag(mat2) <- 0

test_vec <- c()
for (i in seq(ncol(mat1)-1)){  
  for(j in seq(i+1,ncol(mat1))){
    
    s2 <- table(mat1[,i],mat1[,j]) # create 2-way frequency table
    mat2_1 <- mat2
    mat2_1 <- mat2_1[rownames(mat2_1) %in% rownames(s2), 
                               colnames(mat2_1) %in% colnames(s2)]
    calc <- ((1/nrow(mat1))*sum(mat2_1*s2))
    test_vec <- append(test_vec,calc)

  }}

Thanks for the help.

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

薔薇婲 2025-01-30 04:23:07

这是一种将MAT1转换为数据的方法。表，并将所有列转换为因素，并使用表（...，DEXBUDE = null）

library(data.table)
m=as.data.table(mat1)[,lapply(.SD, factor, levels=LETTERS)]
g = combn(colnames(m),2, simplify = F)
result = sapply(g, function(x) sum(table(m[[x[1]]], m[[x[2]]], exclude=NULL)*mat2)/nrow(m))

检查公平：

sum(result-test_vec>1e-10)
[1] 0

这里有4950组合（100个组合（100 ） *99/2），但是随着Nrow（MAT1）的增加，组合数将很快增加（如您指出的那样）。在这种情况下，您可能会发现并行版本效果很好。

library(doParallel)
library(data.table)
registerDoParallel()

m=as.data.table(mat1)[,lapply(.SD, factor, levels=LETTERS)]
g = combn(colnames(m),2, simplify = F)
result = foreach(i=1:length(g), .combine=c) %dopar%
  sum(table(m[[g[[i]][1]]], m[[g[[i]][2]]], exclude=NULL)*mat2)
result = result/nrow(m)

Here is an approach that converts mat1 to a data.table, and converts all the columns to factors, and uses table(..., exclude=NULL)

library(data.table)
m=as.data.table(mat1)[,lapply(.SD, factor, levels=LETTERS)]
g = combn(colnames(m),2, simplify = F)
result = sapply(g, function(x) sum(table(m[[x[1]]], m[[x[2]]], exclude=NULL)*mat2)/nrow(m))

Check equality:

sum(result-test_vec>1e-10)
[1] 0

Here there are 4950 combinations (100*99/2), but the number of combinations will increase quickly as nrow(mat1) increases (as you point out). You might find in that case that a parallelized version works well.

library(doParallel)
library(data.table)
registerDoParallel()

m=as.data.table(mat1)[,lapply(.SD, factor, levels=LETTERS)]
g = combn(colnames(m),2, simplify = F)
result = foreach(i=1:length(g), .combine=c) %dopar%
  sum(table(m[[g[[i]][1]]], m[[g[[i]][2]]], exclude=NULL)*mat2)
result = result/nrow(m)

回复收藏 0 原文

~没有更多了~