还有其他方法可以减少推理时间吗？
我正在使用 Huggingface 的预训练模型对我的 Twitter 数据集（包含 150 万条推文）进行情绪分析。我通过使用加载模型
model = AutoModelForSequenceClassification.from_pretrained("savasy/bert-base-turkish-sentiment-cased")
tokenizer = AutoTokenizer.from_pretrained("savasy/bert-base-turkish-sentiment-cased")
sa= pipeline("sentiment-analysis", tokenizer=tokenizer, model=model, device=0)
，所以我使用我的 GPU (rtx 2060)。我做了一些测试来估计在 150 万条推文中进行推理所需的时间，大约需要 5 个小时。我通过观察给定尺寸所花费的时间来估计这一点。而且看起来它是线性增加的。
size: 10 , 0.2910006046295166 seconds
size: 100 , 1.1989195346832275 seconds
size: 1000 , 12.097795724868774 seconds
size: 10000 , 123.63561415672302 seconds
有没有其他方法可以减少推理时间，或者我应该只运行代码 5 小时？
编辑：
我还观察到 GPU 利用率最大为 %30
我使用的模型 https: //huggingface.co/savasy/bert-base-turkish-sentiment-cased
I am using a pre-trained model from huggingface to do sentiment analysis on my twitter dataset which has 1.5 million tweets. I load the model by using
model = AutoModelForSequenceClassification.from_pretrained("savasy/bert-base-turkish-sentiment-cased")
tokenizer = AutoTokenizer.from_pretrained("savasy/bert-base-turkish-sentiment-cased")
sa= pipeline("sentiment-analysis", tokenizer=tokenizer, model=model, device=0)
so i am using my gpu (rtx 2060). I did some tests to estimate the time it will took to do inference in 1.5 million tweets and it approximately takes 5 hours. I estimated this by observing the time it took with given sizes. And it looks like it increases linearly.
size: 10 , 0.2910006046295166 seconds
size: 100 , 1.1989195346832275 seconds
size: 1000 , 12.097795724868774 seconds
size: 10000 , 123.63561415672302 seconds
Is there any other way to decrease inference time or should i just run code for 5 hours ?
Edit:
I also observed that the gpu utilization is at max %30
The model i use https://huggingface.co/savasy/bert-base-turkish-sentiment-cased
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
如果您尝试使用的数据集确实很大，如果您想减少加载后的加载时间，您可能会减少数据准备和训练阶段之间的时间，但问题是您刚刚加载了数据集到项目中，可能会尝试使用缓存中的数据集，这可以减少推理时间，
If the dataset you're trying to use is definitely large, if you want to decrease the loading time after loading you may decrease the timings between data preparation and training phase, but the problem is you just loaded the dataset to the project, may be trying using the dataset in cache memory which can decrease the inference time bit lower,