attributeError：模块＆＃x27; torch.distributed＆＃x27;没有属性＆＃x27; is_initialized＆＃x27;在pytorch == 1.11.x中
每当我们为训练量的对象创建对象时，
from transformers import Trainer, TrainingArguments
batch_size = 64
logging_steps = len(emotions_encoded["train"]) // batch_size
model_name = f"{model_ckpt}-finetuned-emotion"
training_args = TrainingArguments(output_dir=model_name,
num_train_epochs=2,
learning_rate=2e-5,
per_device_train_batch_size=batch_size,
per_device_eval_batch_size=batch_size,
weight_decay=0.01,
evaluation_strategy="epoch",
disable_tqdm=False,
logging_steps=logging_steps,
push_to_hub=True,
log_level="error")
我都会得到
属性：模块“ torch.distributed”没有属性'is_initialized'
whenever we are creating the object for TrainingArguments
from transformers import Trainer, TrainingArguments
batch_size = 64
logging_steps = len(emotions_encoded["train"]) // batch_size
model_name = f"{model_ckpt}-finetuned-emotion"
training_args = TrainingArguments(output_dir=model_name,
num_train_epochs=2,
learning_rate=2e-5,
per_device_train_batch_size=batch_size,
per_device_eval_batch_size=batch_size,
weight_decay=0.01,
evaluation_strategy="epoch",
disable_tqdm=False,
logging_steps=logging_steps,
push_to_hub=True,
log_level="error")
I am getting
AttributeError: module 'torch.distributed' has no attribute 'is_initialized'
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
为了解决此问题
实际上，Mac不支持分布式培训设施
，因此此问题正在
解决此问题，
请访问您的Transformers软件包，在我的情况下，您可以
在我的情况下安装它，它是
替换line-1024
如果torch.distributed.is_initialized（）和self.local_rank == -1：
和
如果是true和self.local_rank == -1：
重新启动内核
In order to solve this problem
Actually Window and Mac doesn't support distributed training facility
so this issue is occuring
To solve this problem
go to your transformers package where you install it
in my case it is
replace line-1024
if torch.distributed.is_initialized() and self.local_rank == -1:
with
if True and self.local_rank == -1:
Restart your kernal