将单词添加到 nltk 非索引字表

发布于 2024-10-29 04:38:54 字数 281 浏览 3 评论 0原文

我有一些代码可以从我的数据集中删除停用词，因为停用词列表似乎没有删除我也想要的大部分单词，我希望将单词添加到此停用词列表中，以便它将删除他们对于这个案子。我用来删除停用词的代码是：

word_list2 = [w.strip() for w in word_list if w.strip() not in nltk.corpus.stopwords.words('english')]

我不确定添加单词的正确语法，并且似乎无法在任何地方找到正确的语法。任何帮助表示赞赏。谢谢。

原文

I have some code that removes stop words from my data set, as the stop list doesn't seem to remove a majority of the words I would like it too, I'm looking to add words to this stop list so that it will remove them for this case.
The code i'm using to remove stop words is:

word_list2 = [w.strip() for w in word_list if w.strip() not in nltk.corpus.stopwords.words('english')]

I'm unsure of the correct syntax for adding words and can't seem to find the correct one anywhere. Any help is appreciated. Thanks.

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

祁梦 2024-11-05 04:38:54

您可以简单地使用追加方法向其中添加单词：

stopwords = nltk.corpus.stopwords.words('english')
stopwords.append('newWord')

或扩展以附加单词列表，正如查理在评论中所建议的那样。

stopwords = nltk.corpus.stopwords.words('english')
newStopWords = ['stopWord1','stopWord2']
stopwords.extend(newStopWords)

You can simply use the append method to add words to it:

stopwords = nltk.corpus.stopwords.words('english')
stopwords.append('newWord')

or extend to append a list of words, as suggested by Charlie on the comments.

stopwords = nltk.corpus.stopwords.words('english')
newStopWords = ['stopWord1','stopWord2']
stopwords.extend(newStopWords)

回复收藏 0 原文

半步萧音过轻尘 2024-11-05 04:38:54

import nltk
stopwords = nltk.corpus.stopwords.words('english')
new_words=('re','name', 'user', 'ct')
for i in new_words:
    stopwords.append(i)
print(stopwords)

import nltk
stopwords = nltk.corpus.stopwords.words('english')
new_words=('re','name', 'user', 'ct')
for i in new_words:
    stopwords.append(i)
print(stopwords)

回复收藏 0 原文

梦情居士 2024-11-05 04:38:54

我在 Ubuntu 机器上的做法是，我在 root 中按 ctrl + F 切换“停用词”。它给了我一个文件夹。我走进里面，里面有不同的文件。我打开“英语”，只有128个单词。添加了我的话。保存并完成。

回复收藏 0 原文

一曲爱恨情仇 2024-11-05 04:38:54

英语停用词是 nltk/corpus/stopwords/english.txt 中的一个文件（我猜它会在这里...我这台机器上没有 nltk..最好的办法是在 nltk 存储库中搜索“english.txt”）

您只需在此文件中添加新的停用词即可。

如果您的停用词列表增加到数百个，也可以尝试查看 bloom 过滤器

回复收藏 0 原文

诠释孤独 2024-11-05 04:38:54

我总是在任何需要它的模块的顶部执行 stopset = set(nltk.corpus.stopwords.words('english')) 。然后可以轻松地向集合中添加更多单词，并且成员资格检查速度更快。

回复收藏 0 原文

无风消散 2024-11-05 04:38:54

也在寻找解决方案。经过一番尝试和错误后，我必须将单词添加到非索引字表中。希望这有帮助。

def removeStopWords(str):
#select english stopwords
cachedStopWords = set(stopwords.words("english"))
#add custom words
cachedStopWords.update(('and','I','A','And','So','arnt','This','When','It','many','Many','so','cant','Yes','yes','No','no','These','these'))
#remove stop words
new_str = ' '.join([word for word in str.split() if word not in cachedStopWords]) 
return new_str

Was also looking for solution on this. After some trail and error I got to add words to the stoplist. Hope this helps.

def removeStopWords(str):
#select english stopwords
cachedStopWords = set(stopwords.words("english"))
#add custom words
cachedStopWords.update(('and','I','A','And','So','arnt','This','When','It','many','Many','so','cant','Yes','yes','No','no','These','these'))
#remove stop words
new_str = ' '.join([word for word in str.split() if word not in cachedStopWords]) 
return new_str

回复收藏 0 原文

薄凉少年不暖心 2024-11-05 04:38:54

 import nltk
 nltk.download('stopwords')
 from nltk.corpus import stopwords
 #add new words to the list
 new_stopwords = ["new", "custom", "words", "add","to","list"]
 stopwrd = nltk.corpus.stopwords.words('english')
 stopwrd.extend(new_stopwords)

 import nltk
 nltk.download('stopwords')
 from nltk.corpus import stopwords
 #add new words to the list
 new_stopwords = ["new", "custom", "words", "add","to","list"]
 stopwrd = nltk.corpus.stopwords.words('english')
 stopwrd.extend(new_stopwords)

回复收藏 0 原文

调妓 2024-11-05 04:38:54

我使用此代码将新的停用词添加到 python 中的 nltk 停用词列表中

from nltk.corpus import stopwords
#...#
stop_words = set(stopwords.words("english"))

#add words that aren't in the NLTK stopwords list
new_stopwords = ['apple','mango','banana']
new_stopwords_list = stop_words.union(new_stopwords)

print(new_stopwords_list)

I use this code for adding new stop words to nltk stop word list in python

from nltk.corpus import stopwords
#...#
stop_words = set(stopwords.words("english"))

#add words that aren't in the NLTK stopwords list
new_stopwords = ['apple','mango','banana']
new_stopwords_list = stop_words.union(new_stopwords)

print(new_stopwords_list)

回复收藏 0 原文