矩阵计算错误

发布于 2024-10-16 03:59:27 字数 268 浏览 1 评论 0原文

我正在使用 R 工具来计算 SVD (svd(m)),它适用于小矩阵,但当我传递它时 20Kx20X 矩阵。处理后,它给出了以下错误

Error in svd(m) : infinite or missing values in 'x'

,我检查过,没有行或列的值全为 0,并且行和中没有重复项 柱子。所有列都有值。

我无法在这里通过 20Kx20K 矩阵:(

I am using R tool to calculate SVD (svd(m)) and it works on small matrix but as I pass it 20Kx20X matrix. After processing, it gives the following error

Error in svd(m) : infinite or missing values in 'x'

I checked and there is no row or column with all 0 values and no duplicate in row and
column. All columns have values.

I cannot past 20Kx20K matrix here :(

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

筱果果 2024-10-23 03:59:27

我猜你的问题与内存大小无关,尽管我无法在我的 4GB 内存机器上处理 20Kx20K 矩阵。

这种猜测的原因是 svd() 内的第一行代码如下:

if (any(!is.finite(x))) 
    stop("infinite or missing values in 'x'")

换句话说,svd() 函数首先测试数据中是否存在任何无限值。

这发生在任何进一步处理之前。因此,如果您有内存问题,这些问题甚至在调用 svd() 之前就会很明显。

我建议您检查无限值:

x <- c(0, Inf, NA, NULL)
which(!is.finite(x))

[1] 2 3

这表明第二个和第三个值被认为是非有限的。换句话说,数据中的任何 NA 值都会导致错误。

I am guessing that your problem is not related to memory size, although I am not able to process a 20Kx20K matrix on my 4GB memory machine.

The reason for this guess is that the first line of code inside svd() is the following:

if (any(!is.finite(x))) 
    stop("infinite or missing values in 'x'")

In other words, the svd() function test first whether there are any infinite values in your data.

This happens before any further processing. So, if you had memory problems, these would be apparent even before your call to svd().

I suggest you check for infinite values:

x <- c(0, Inf, NA, NULL)
which(!is.finite(x))

[1] 2 3

This indicates that the second and third values are considered to be not finite. In other words, any NA values in your data will cause your error.

小嗲 2024-10-23 03:59:27

可能 svd 计算本身也使用大量内存。如果我们与 MATLAB 进行比较,我们会发现 svd 计算分配的内存与矩阵本身使用的内存一样多,因此,如果您已经使用了 3GB 内存,则 svd 计算可能会再分配 3GB,从而获得 6GB 内存。

Possibly the svd calculation itself also uses a lot of memory. If we compare to MATLAB, we see that the svd calculation allocates just as much memory as the matrix itself uses, so if you already ise 3GB of memory, the svd calculation possibly allocates another 3GB, which gives 6GB of memory.

我喜欢麦丽素 2024-10-23 03:59:27

如果您存储 8 字节的双精度数,则 20Kx20K 意味着 8*20,000*20,000/1024/1024 ~ 3GB RAM 可以将整个数据保存在内存中。

我不知道你有多少可用内存,但我会研究 R 可以做什么来根据需要将矩阵序列化到磁盘。

矩阵是稀疏矩阵还是带状矩阵?您可以采取一些措施来减少所需的内存量吗?

矩阵的零空间有多大?条件数(最大与最小特征值之比)是多少?较大的条件数可能表明求解困难。矩阵不必具有零行或零列即可接近奇异。

更新:

根据您的评论,我想说 RAM 是您问题中最少的。听起来好像可以将整个矩阵保存在内存中——如果你能解决所有问题的话。您可以对整个矩阵进行寻址。您正在 64 位操作系统上运行 - 您的 R 版本也是 64 位吗?

不幸的是,SVD 的副产品之一是获取零空间的大小。

您可以使用雅可比迭代获得矩阵的最小特征值。 Lanczos 可能是获得最大特征值的不错选择。要获得所有这些将需要大量工作;您可能只需要评估最小和最大的五个。

每当我在使用某些软件时遇到错误时,我都会立即将其粘贴到谷歌搜索中。至少令人欣慰的是,我不是第一个遇到特定问题的人:

http://www.google.com/search?sourceid=chrome&ie=UTF-8&q=Error +in+svd(m)+:+无限+或+缺失+值+in+'x'

If you're storing doubles that are 8 bytes, 20Kx20K means 8*20,000*20,000/1024/1024 ~ 3GB of RAM to hold the whole thing in memory.

I don't know how much RAM you've got available, but I'd look into what R can do to serialize the matrix out to disk as needed.

Is the matrix sparse or banded? Can you do something to decrease the amount of memory you need?

How large is the null space for you matrix? What's the condition number (ratio of largest-to-smallest eigenvalue)? A large condition number can be an indication of difficulties in solving. A matrix need not have a zero row or column to be nearly singular.

UPDATE:

Based on your comment, I'd say that RAM is the least of your problems. Sounds like it's possible to hold the entire matrix in memory - if you can address it all. You can address the entire matrix. You're running on a 64-bit OS - is your version of R 64-bit as well?

Unfortunately, one of the byproducts of SVD is to get the size of the null space.

You can get the minimum eigenvalue for your matrix using Jacobi iteration. Lanczos might be a good choice for getting the maximum eigenvalue. It'd be a lot of work to get all of them; you might just want the five smallest and largest to assess.

Anytime I experience an error with some software I immediately paste it into a Google search. At least it's comforting to know that I'm not the first to experience a particular problem:

http://www.google.com/search?sourceid=chrome&ie=UTF-8&q=Error+in+svd(m)+:+infinite+or+missing+values+in+'x'

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文