如何用bert nlpaug中的原始令牌替换[unk]令牌

发布于 2025-01-26 13:22:04 字数 842 浏览 3 评论 0 原文

我正在尝试使用nlpaug交换一些单词,但是由于它,它遇到了问题,用[UNK]代币永久取代令牌。我在此处使用这些文档:

我的代码一个示例就是这样:

aug = naw.ContextualWordEmbsAug(action='substitute', top_k=10,
                            aug_min=2, aug_max=4, stopwords=stops, batch_size=25)

aug.augment('You deposit the minimum amount (which is Rs 25/-) and buy tickets (Tickets are your entry fee) and start playing the game with others.')

输出

'[UNK] deposit the entire amount ( which is rs 10 / - ) and buy tickets ( tickets are your entry cost ) and start playing the game with ease.'

我在句子中失去了您,并希望保留此句子。这发生在一个句子的宇宙中,因此一个简单的案例修复将不是这里的答案。任何帮助将不胜感激。

I am trying to use nlpaug to swap some words out but am having issue with it replacing tokens permanently with the [UNK] token. I am using the docs here: https://nlpaug.readthedocs.io/en/latest/augmenter/word/context_word_embs.html

My code an example is as such:

aug = naw.ContextualWordEmbsAug(action='substitute', top_k=10,
                            aug_min=2, aug_max=4, stopwords=stops, batch_size=25)

aug.augment('You deposit the minimum amount (which is Rs 25/-) and buy tickets (Tickets are your entry fee) and start playing the game with others.')

Output

'[UNK] deposit the entire amount ( which is rs 10 / - ) and buy tickets ( tickets are your entry cost ) and start playing the game with ease.'

I have lost you in the sentence and would like to keep this. This happens across a universe of sentences so a simple one case fix will not be the answer here. Any help would be greatly appreciated.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文