如何自动将Pytorch模型转换为拥抱面模型?
假设我创建了一个pytorch模型:
class Model(nn.Module):
def __init__(self):
super(Model, self).__init__()
self.model_1_encoder = RobertaModel.from_pretrained('roberta-base')
self.model_2_encoder = RobertaModel.from_pretrained('roberta-base')
self.dropout = nn.Dropout(0.5)
self.linear_model_1_out = nn.Linear(768, 512)
self.linear_model_2_out = nn.Linear(768, 512)
self.linear3 = nn.Linear(512, 512)
self.linear4 = nn.Linear(512, 1)
def forward(self, x):
question_input_ids, attn_mask, xyz_input_ids, xyz_attention_mask = torch.tensor(x['input_ids']), torch.tensor(x['attention_mask']), torch.tensor(x['xyz_ids']), torch.tensor(x['xyz_mask'])
model_1_outputs = self.model_1_encoder(question_input_ids, attention_mask=attn_mask)
model_2_outputs = self.model_2_encoder(xyz_input_ids, attention_mask = xyz_attention_mask)
model_1_outputs = self.dropout(model_1_outputs[0])
model_2_outputs = self.dropout(model_2_outputs[0])
model_2_outputs = torch.sum(model_2_outputs, dim=0)
model_2_outputs = torch.unsqueeze(model_2_outputs, 0)
model_2_outputs = model_2_outputs.repeat(model_1_outputs.shape[0], 1, 1)
model_1_outputs = self.linear_model_1_out(model_1_outputs)
model_2_outputs = self.linear_model_2_out(model_2_outputs)
model_1_and_xyz_sum = model_2_outputs + model_1_outputs
outputs = self.dropout(model_1_and_xyz_sum)
outputs = outputs[:,0,:].view(-1,512)
outputs = torch.relu(self.linear3(outputs))
outputs = self.dropout(outputs)
outputs = torch.relu(self.linear4(outputs))
return outputs
model = Model()
from this 因此,我知道我需要为Pytorch模型创建一个配置文件,该文件将由模型加载。他们引用了如何向变形金刚添加模型?除其他各种输入和步骤外,我的自定义模型中的每一层。我还找到了 this this 他们的论坛上的问题,引用 this 转换Pytorch模型。
如何自动将我的pytorch模型转换为Hugginface模型,以便我可以使用他们的Trainer
类(除了DataCollator
等其他功能之外)?
潜在的最佳伪代码:
my_pytorch_model = Model()
my_huggingface_model = Custom_huggingface_Model.load_model(my_pytorch_model)
Say I created a Pytorch model:
class Model(nn.Module):
def __init__(self):
super(Model, self).__init__()
self.model_1_encoder = RobertaModel.from_pretrained('roberta-base')
self.model_2_encoder = RobertaModel.from_pretrained('roberta-base')
self.dropout = nn.Dropout(0.5)
self.linear_model_1_out = nn.Linear(768, 512)
self.linear_model_2_out = nn.Linear(768, 512)
self.linear3 = nn.Linear(512, 512)
self.linear4 = nn.Linear(512, 1)
def forward(self, x):
question_input_ids, attn_mask, xyz_input_ids, xyz_attention_mask = torch.tensor(x['input_ids']), torch.tensor(x['attention_mask']), torch.tensor(x['xyz_ids']), torch.tensor(x['xyz_mask'])
model_1_outputs = self.model_1_encoder(question_input_ids, attention_mask=attn_mask)
model_2_outputs = self.model_2_encoder(xyz_input_ids, attention_mask = xyz_attention_mask)
model_1_outputs = self.dropout(model_1_outputs[0])
model_2_outputs = self.dropout(model_2_outputs[0])
model_2_outputs = torch.sum(model_2_outputs, dim=0)
model_2_outputs = torch.unsqueeze(model_2_outputs, 0)
model_2_outputs = model_2_outputs.repeat(model_1_outputs.shape[0], 1, 1)
model_1_outputs = self.linear_model_1_out(model_1_outputs)
model_2_outputs = self.linear_model_2_out(model_2_outputs)
model_1_and_xyz_sum = model_2_outputs + model_1_outputs
outputs = self.dropout(model_1_and_xyz_sum)
outputs = outputs[:,0,:].view(-1,512)
outputs = torch.relu(self.linear3(outputs))
outputs = self.dropout(outputs)
outputs = torch.relu(self.linear4(outputs))
return outputs
model = Model()
From this SO question I understand that I need to create a config file for the Pytorch model that will be loaded by the model. They referenced How to add a model to Transformers?, which if I understands correctly requires manual writing of each layer in my custom model in addition to various other inputs and steps. I also found this question on their forum, which referenced this, which had even more steps to complete in order to convert the Pytorch model.
How can I automatically convert my Pytorch model to a Hugginface model so that I can use their Trainer
class (in addition to other functionalities like the DataCollator
, etc.)?
Potential optimal pseudocode:
my_pytorch_model = Model()
my_huggingface_model = Custom_huggingface_Model.load_model(my_pytorch_model)
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论