Scikit-Learn如何使用管道以及如何进行逻辑＆＃x2B;山脊回归

Question

两个问题：

我正在尝试运行一个预测流失的模型。我的许多功能都有多重共线性问题。为了解决这个问题，我试图用Ridge来惩罚系数。

更具体地说，我试图运行逻辑回归，但在模型上应用山脊惩罚（不确定是否有意义）...

问题：

1. 选择脊回归分类器是否足够吗？还是我需要选择逻辑回归分类器并用一些参数以ridge惩罚（即logisticRegress（apply_penation = ridge） ridge）

2. 我正在尝试确定特征重要性，并且通过某些研究，IT好像我需要使用它：

https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.selectkbest.html

但是，如果我的模型如何访问此功能，我感到困惑已围绕sklearn.pipeline.make_pipeline函数构建。

我只是想找出哪些自变量在预测我的标签方面最重要。

下面的代码供参考

#prep data
X_prep = df_dummy.drop(columns='CHURN_FLAG')

#set predictor and target variables
X = X_prep #all features except churn_flag
y = df_dummy["CHURN_FLAG"]

#create train /test sets
Xtrain, Xtest, ytrain, ytest = train_test_split(X, y, test_size=0.20)


'''
standard scaler - incoming data needs to standardized before any other transformation is performed on it.
SelectKBest() -> This method comes from the feature_selection module of Scikit-learn. It selects the best features based on a specified scoring function (in this case, f_regression)
 The number of features is specified by the value of parameter k. Even within the selected features, we want to vary the final set of features that are fed to the model, and find what performs best. We can do that with the GridSearchCV method
 Ridge() -> This is an estimator that performs the actual regression -- performed to reduce the effect of multicollinearity.
 GridSearchCV _> Other than searching over all the permutations of the selected parameters, GridSearchCV performs cross-validation on training data.
'''
#Setting up a pipeline
pipe= make_pipeline(StandardScaler(),SelectKBest(f_regression),Ridge())

#A quick way to get a list of parameters that a pipeline can accept
#pipe.get_params().keys()

#putting together a parameter grid to search over using grid search
params={
    'selectkbest__k':[1,2,3,4,5,6],
    'ridge__fit_intercept':[True,False],
    'ridge__alpha':[0.01,0.1,1,10],
    'ridge__solver':[ 'svd', 'cholesky', 'lsqr', 'sparse_cg', 'sag',
'saga']
}
#setting up the grid search
gs=GridSearchCV(pipe,params,n_jobs=-1,cv=5)
#fitting gs to training data
gs.fit(Xtrain, ytrain)

#building a dataframe from cross-validation data
df_cv_scores=pd.DataFrame(gs.cv_results_).sort_values(by='rank_test_score')

#selecting specific columns to create a view
df_cv_scores[['params','split0_test_score', 'split1_test_score', 'split2_test_score',
       'split3_test_score', 'split4_test_score', 'mean_test_score',
       'std_test_score', 'rank_test_score']].head()

#checking the selected permutation of parameters
gs.best_params_

'''
Finally, we can predict target values for test set by passing it’s feature matrix to gs. 
The predicted values can be compared with actual target values to visualize and communicate performance of the model.
'''
#checking how well the model does on the holdout-set
gs.score(Xtest,ytest)

#plotting predicted churn/active vs actual churn/active
y_preds=gs.predict(Xtest)
plt.scatter(ytest,y_preds)

Answer 1

选择山脊回归分类器是否足够吗？或我需要选择逻辑回归分类器并用一些参数以ridge惩罚（即LogisticRegress（apply_penality = ridge）

）

因此脊回归和物流之间的问题归结于您是否试图进行分类或回归。如果您想在某个连续的基础上预测搅拌的数量，如果您想预测某人的流失，或者他们可能会搅拌使用逻辑回归

。重新出现。

我正在尝试确定特征重要性，并且通过一些研究，似乎我需要使用此功能。

通常，您可以通过name_steps属性访问管道的元素。因此，在您的情况下，如果您想访问selectkbest可以做：

pipe.named_steps["SelectKBest"].get_feature_names()

这将为您提供功能名称，现在您仍然需要这些值。在这里，您必须访问模型学习系数。对于Ridge和Logistic回归，应该是这样的：

pipe.named_steps["logisticregression"].coef_

如果您想要一个更详细的教程在这里