预测与实际置信区间之间的比率

发布于 01-21 10:46 字数 1501 浏览 3 评论 0原文

我有一个向量的集合，每个矢量包含6个参数，范围为$ 3 \ sigma $，即，每个参数的平均值$ \ pm 3 \ sigma $（我称其为3sig的3sig）。我的目标是预测每个参数向量的载体，其中包含2498个元素。为此，我创建了一个Randomforest回归剂，并完成了工作。下一步是通过将预测结果与基准值（实际值）进行比较来评估模型性能。

我要做的第一件事是：将数据滤除到1sig，2sig和3sig的置信区间中，并通过分析预测值和实际值之间的比率来查看预测变量如何在这些间隔上执行（特别是我将分析，我将分析该比率的最大值和最小值）。

为了获得预测“ pred”和实际值“ y_test”之间的比率，我使用了

Ratio_All =pred.div(np.exp(y_test).reset_index(drop=True).set_axis([i for i in range(0,28)],axis=1))

REST_INDEX部分是使行的数字在0开始，以便在将两个数据范围划分时不获得NAN值。

有了这个比率，我可以通过appyling .max（）和.min（）过滤最大值和最小值：

plt.figure(figsize=(8, 6), dpi=100)
plt.xscale('log');plt.yscale('linear');plt.xlim(2,2500)
plt.xlabel(r'$\ell$')
plt.ylabel(r'$ \hat{C_\ell}^\mathrm{TT}/C_\ell^\mathrm{TT}$')
plt.axhline(y=1, color='k', linestyle='--')
plt.plot(l, Ratio_All.max(), '-b')
plt.plot(l, Ratio_All.min(), '-r')

作为输出，我得到了数字：

此图对应于3sig置信区间的比率。现在，对于2sig和1sig中的数据中的数据做同样的操作，我应该得到两个相似的曲线，但在3sig的曲线内，因为1sig中的数据是2sig中数据的一个子集，而2sig中的数据是数据的一个子集在3sig（对吗？）。但是...那不是我得到的...实际上，如果我这样做，我会得到以下数字：

“在此处输入映像”

我的问题是：我的方法是错误的，在哪里？我该如何修复？

我的假设是：预测因子和专家在3SIG中预测数据，但与1sig和2sig中的数据相比，情况更糟。解决方案是为每个间隔做一个预测指标，但它不能解释为什么当红色曲线相等时，红色曲线比黑色曲线更糟（一旦它是相同的置信区间）

原文

I have a collection of vectors, each vector contains 6 parameters in the range of $3\sigma$, i.e, each parameter has an average value $\pm 3\sigma$ (i will call it 3sig for short). My Goal is to predict a vector containing 2498 elements for each vector of parameters. To do so I created a RandomForest regressor and it did the job. The next step is to evaluate the model performance by comparing the predicted results with fiducial values (real values).

The first thing I have to do is: filter out the data into confidence intervals of 1sig, 2sig and 3sig and see how the predictor performs on each of these intervals by analyzing the ratio between predicted values and actual values (in particular I shall analyze the maximum value and the minimum value of this ratio).

To get the ratio between predictions "pred" and actual values "y_test", I used

Ratio_All =pred.div(np.exp(y_test).reset_index(drop=True).set_axis([i for i in range(0,28)],axis=1))

The rest_index part is to make the rows' numbers start at 0 in order to not obtain NaN values when dividing the two dataframes.

With this ratio, I can filter the maximum and minimum values by appyling .max() and .min():

plt.figure(figsize=(8, 6), dpi=100)
plt.xscale('log');plt.yscale('linear');plt.xlim(2,2500)
plt.xlabel(r'$\ell
As an output, I get the figure:

This figure corresponds to the maximum and minimum of the ratio in the confidence interval of 3sig. Now, doing the same for data in 2sig and data in 1sig i should get two similar curves but INSIDE the curves for 3sig, because the data in 1sig is a subset of the data in 2sig and the data in 2sig is a subset of the data in 3sig (right???).

But... that's not what I get... actually if I do this, i get the following figure:

(It's the same as before, but for l<30 it's in log scale) Where the black line corresponds to the maximum and minimum of the ratio in 3sig, the green line is the data in 1sig and the yellow line is the data in 2sig. The red line is a "test data" in 3sig that i generated to test again, and it performs A LOT worse than the original training set...
My question is: is my approach wrong and where? how can I fix it?
My hypothesis is: the predictor is and expert in predicting data in 3sig, but is worse when compared to data in 1sig and 2sig. The solution would be to make a predictor to each interval, but it does not explain why the red curve is worse than the black one when they should be equal (once it's the same confidence interval)
)
plt.ylabel(r'$ \hat{C_\ell}^\mathrm{TT}/C_\ell^\mathrm{TT}
As an output, I get the figure:

This figure corresponds to the maximum and minimum of the ratio in the confidence interval of 3sig. Now, doing the same for data in 2sig and data in 1sig i should get two similar curves but INSIDE the curves for 3sig, because the data in 1sig is a subset of the data in 2sig and the data in 2sig is a subset of the data in 3sig (right???).

But... that's not what I get... actually if I do this, i get the following figure:

(It's the same as before, but for l<30 it's in log scale) Where the black line corresponds to the maximum and minimum of the ratio in 3sig, the green line is the data in 1sig and the yellow line is the data in 2sig. The red line is a "test data" in 3sig that i generated to test again, and it performs A LOT worse than the original training set...
My question is: is my approach wrong and where? how can I fix it?
My hypothesis is: the predictor is and expert in predicting data in 3sig, but is worse when compared to data in 1sig and 2sig. The solution would be to make a predictor to each interval, but it does not explain why the red curve is worse than the black one when they should be equal (once it's the same confidence interval)
)
plt.axhline(y=1, color='k', linestyle='--')
plt.plot(l, Ratio_All.max(), '-b')
plt.plot(l, Ratio_All.min(), '-r')

As an output, I get the figure:

This figure corresponds to the maximum and minimum of the ratio in the confidence interval of 3sig. Now, doing the same for data in 2sig and data in 1sig i should get two similar curves but INSIDE the curves for 3sig, because the data in 1sig is a subset of the data in 2sig and the data in 2sig is a subset of the data in 3sig (right???).
But... that's not what I get... actually if I do this, i get the following figure:

(It's the same as before, but for l<30 it's in log scale) Where the black line corresponds to the maximum and minimum of the ratio in 3sig, the green line is the data in 1sig and the yellow line is the data in 2sig. The red line is a "test data" in 3sig that i generated to test again, and it performs A LOT worse than the original training set...

My question is: is my approach wrong and where? how can I fix it?

My hypothesis is: the predictor is and expert in predicting data in 3sig, but is worse when compared to data in 1sig and 2sig. The solution would be to make a predictor to each interval, but it does not explain why the red curve is worse than the black one when they should be equal (once it's the same confidence interval)

分享到QQ

分享到微博