log(1-exp(x)) 的数值精度

发布于 2024-11-26 14:00:45 字数 1666 浏览 0 评论 0原文

我正在用非常大的数字做一些数学运算（我正在使用Python，但这个问题不是Python特定的）。对于一个值，我有一个公式，可以得出 f(t) = Pr(X < t)。我想使用这个公式来获得 Pr(X >= t) = 1 - f(t)。由于 f(t) 返回的值非常接近零，因此我一直在使用对数变换并存储 log( f(t) ) 而不是 f( t）。我的 log( f(t) ) 约为 -1e5 左右。

对于乘法来说，这非常有效。 log(f(t) * g) = log(f(t)) + log(g)。

但是，仅使用 log( f(t) ) 计算 log( 1 - f(t) ) 是非常困难的；当然，我可以暂时对我存储的值求幂并计算 log( 1 - exp( log( f(t) ) )，但这将返回 log( 1 - 0.0 ) = 0.0 因为 log( f(t) ) 非常接近零。

您可能会问，“你为什么关心？如果它接近零，那么 1 减去它就非常接近了。 1.“嗯，你说得很好。你真是个聪明人。

问题是我想用它来对值进行排名，所以我真的很关心其中是否是 log(0.999) 另一个是 log(0.9999)。您可能还会问，“那么，为什么不直接对 log( f(t) ) 进行排名，然后反转呢？获得排名的顺序for log( 1 - f(t) )。”再次，我忍不住指出你的问题有多么好。与你交谈真的很愉快。

但问题是：我不知道不仅仅是想按 1 - f(t) 排名；我实际上想根据 Pr(X >= t) * g(t) = (1 - f( t)) g(t) 服用后。日志，我得到 log( 1 - f(t) ) + log( g(t) );单独基于 f(t) 的排名不会给出正确的答案。

过去，我编写了一个小 Python 函数来从 log(a) 和 log(b) 计算 log(a + b)

def log_add(logA,logB):
    if logA == log(0):
        return logB
    if logA<logB:
        return log_add(logB,logA)
    return log( 1 + math.exp(logB-logA) ) + logA

：首先对它们进行标准化，使它们靠近，然后在它们靠近时求幂。

不幸的是，我无法使用相同的技巧来进行减法，因为没有标准化因子可以带来 log(1) 和 log( f(t) ) 靠得很近，因为它们相距很远。

有谁知道如何解决这个问题？这似乎是一个经典的问题；我真的希望/希望/祈祷有一个聪明的函数可以在位级别上运行，可以从 log(x) 得到 log(1-x) 。另外，如果您知道它是如何工作的，我真的非常想知道。

干杯! 奥利弗

原文

I'm doing some math with really big numbers (I'm using Python, but this question isn't Python specific). For one value, I have a formula that gives me f(t) = Pr(X < t). I want to use this formula to get Pr(X >= t) = 1 - f(t). Because f(t) returns values very close to zero, I've been using a log transform and storing log( f(t) ) instead of f(t). My log( f(t) ) is on the order of -1e5 or so.

For multiplying, this works quite well. log( f(t) * g ) = log( f(t) ) + log(g).

But, it's very difficult to compute log( 1 - f(t) ) using only log( f(t) ); I could, of course, temporarily exponentiate the value I store and compute log( 1 - exp( log( f(t) ) ), but that will return log( 1 - 0.0 ) = 0.0 because log( f(t) ) is so close to zero.

You might ask, "Why do you care? If it's close to zero, then 1 minus it is very close to 1." Well, that's a good point you've made. You're a smart cookie.

The trouble is I want to use this to rank values, so I really care if one is log(0.999) and the other is log(0.9999). You might also ask, "Well, why don't you just rank the log( f(t) ) and then reverse the order to get the rankings for log( 1 - f(t) )." Again, I cannot help but point out how great your questions are. It really has been a pleasure talking to you.

But here's the problem: I don't just want to rank by 1 - f(t); I actually want to rank based on Pr(X >= t) * g(t) = (1 - f(t)) g(t). After taking logs, I get log( 1 - f(t) ) + log( g(t) ); ranking based on f(t) alone won't give the correct answer.

In the past I wrote a little Python function to compute log(a + b) from log(a) and log(b):

def log_add(logA,logB):
    if logA == log(0):
        return logB
    if logA<logB:
        return log_add(logB,logA)
    return log( 1 + math.exp(logB-logA) ) + logA

It helps by first normalizing them so that they're close together and then exponentiating when they're close together.

Unfortunately, I wasn't able to get the same trick to work for my subtraction because there is no normalization factor that will bring log(1) and log( f(t) ) close together because they're so far apart.

Does anyone know how to solve this? It seems like such a classic kind of problem; I'd really expect/hope/pray that there is a clever function that operates on the bit level that can give me log(1-x) from log(x). Also, if you know how it works, I'd really, really love to know.

Cheers!
Oliver

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

雄赳赳气昂昂 2024-12-03 14:00:45

如果 log(f(t)) 确实是 -1e5（或类似的数量级），则 0.0 是 log(1-f(t)) 的最佳浮点表示形式代码>.事实上，f(t) = exp(-1e5) 因此，根据 dmuir 提到的泰勒级数，log(1-f(t)) = -exp(-1e5) code> （这实际上不是一个精确的等式，但它是一个非常好的近似值）。现在，-exp(-1e5) = -3.56e-43430，但 0 到 -4e-324 之间没有浮点数，因此最佳浮点表示形式是 0.0。

因此，对于标准浮点数来说，你想要做的事情是不可能的。

这有关系吗？你说要根据Pr(X >= t) * g(t) = (1 - f(t)) g(t)排名，相当于按排名>log( 1 - f(t) ) + log( g(t) )。我们在上面发现 log(1-f(t)) = -3.56e-43430，因此只有当 log(g(t) 的值不同时，该项才会产生影响)) 的差异不超过这个微小的数字，并且如果您的计算足够准确，可以通过这些微小的数字进行区分（如果您使用标准浮点数，那么您的计算将永远不够准确）。换句话说，如果 log(f(t)) 确实是 -1e5 或类似值，那么您可以仅按 g(t) 排名。

然而，log(f(t)) 的数量级可能是 -1e5，但有时它会采用接近零的值，例如 -10 或 -1。在这种情况下，您不能只是忽略它，而且必须确实按 log(1-f(t)) + log(g(t)) 进行排名。您应该使用 math.log1p 函数：按 log1p(-f(t)) + log(g(t)) 排名。原因是，如果 f(t) 接近于零，则 log(1-f(t)) 不准确，但 log1p(-f(t)) 是准确的。如果 f(t) 非常接近于零，例如当 log(f(t)) = -1e5 时，则 log1p(-f(t)) = 0.0因为这是使用标准浮点数所能做到的最好的结果。

我使用“标准浮点数”这个表达是有原因的。可以使用更精确的浮点数，如果您确实想捕获像 -3.56e-43430 这样的小数字，那么您应该这样做。一种可能性是 Python 中的 mpmath （不幸的是，它似乎不支持 log1p 函数）。请注意，这比标准浮点数慢得多，正如我所说，我认为您不需要它。但是，如果您想更好地了解这些问题，那么值得一试。

If log(f(t)) is indeed -1e5 (or of similar order of magnitude) then 0.0 is the best floating point representation of log(1-f(t)). Indeed, f(t) = exp(-1e5) so, by the Taylor series that dmuir mentioned, log(1-f(t)) = -exp(-1e5) (this is actually not an exact equality, but it is an extremely good approximation). Now, -exp(-1e5) = -3.56e-43430, but there are no floating point numbers between 0 and -4e-324, so the best floating point representation is 0.0.

So, what you want to do is impossible with standard floating point numbers.

Does this matter? You say that want to rank based on Pr(X >= t) * g(t) = (1 - f(t)) g(t), which is equivalent to ranking by log( 1 - f(t) ) + log( g(t) ). We found above that log(1-f(t)) = -3.56e-43430, so this term is only going to make a difference if the different values of log(g(t)) differ by no more than this tiny number and if your computation is accurate enough that it can distinguish by these tiny numbers (if you use standard floating point numbers, then your computation will never be accurate enough). In other words, if log(f(t)) is indeed -1e5 or similar, than you can just rank by g(t).

However, it may be that log(f(t)) is of the order of -1e5, but that it sometimes takes values closer to zero like -10 or -1. In that case, you cannot just ignore it and you have to indeed rank by log(1-f(t)) + log(g(t)). You should write this using the math.log1p function: rank by log1p(-f(t)) + log(g(t)). The reason is that if f(t) is close to zero, then log(1-f(t)) is inaccurate but log1p(-f(t)) is accurate. If f(t) is extremely close to zero, such as when log(f(t)) = -1e5, then log1p(-f(t)) = 0.0 because that is the best it can do using standard floating point numbers.

I am using the expression "standard floating point numbers" for a reason. It is possible to use floating point numbers with more precision, and if you really want to capture tiny numbers like -3.56e-43430 that is what you should do. One possibility is in Python is mpmath (unfortunately, it does not seem to support the log1p function). Be warned that this is much slower than standard floating point numbers, and as I said I don't think you'll need it. However, it's worth a go if you want to understand these issues better.

回复收藏 0 原文

~没有更多了~