使用 pandas 库删除停用词

Question

从列中删除停用词并将同一列中的剩余单词移动到新列。这只能通过使用 pandas 库来完成。停用词存储在字典中。应对整列的每一行执行此操作。

Answer 1

我会将停用词列表存储在列表中而不是字典中：

import pandas as pd

#in the following dataframe
df = pd.DataFrame(['some sentance with a few stopwords the', 'another sentance with other stopwords the a or'], columns = ['col1'])


#say you have the following stopwords
stopword_list = ['the', 'a', 'or']

#creating a copy of the initial column, splitting word of each row in a list
df['col2'] = df['col1'].str.split()

#removing stopwords
df['col2'] = df['col2'].apply(lambda x: [i for i in x if i not in stopword_list])

#joining the list, to get a sentance 
df['col2'] = df['col2'].apply(lambda x: ' '.join(x))