如何自动将Pytorch模型转换为拥抱面模型?

发布于 2025-02-11 02:42:03 字数 2808 浏览 1 评论 0原文

假设我创建了一个pytorch模型:

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        
        self.model_1_encoder = RobertaModel.from_pretrained('roberta-base')
        self.model_2_encoder = RobertaModel.from_pretrained('roberta-base')

        self.dropout = nn.Dropout(0.5)
        self.linear_model_1_out = nn.Linear(768, 512)
        self.linear_model_2_out = nn.Linear(768, 512)
    
        self.linear3 = nn.Linear(512, 512)
        self.linear4 = nn.Linear(512, 1)
        
    def forward(self, x):
        question_input_ids, attn_mask, xyz_input_ids, xyz_attention_mask = torch.tensor(x['input_ids']), torch.tensor(x['attention_mask']), torch.tensor(x['xyz_ids']), torch.tensor(x['xyz_mask'])
        model_1_outputs = self.model_1_encoder(question_input_ids, attention_mask=attn_mask)
        model_2_outputs = self.model_2_encoder(xyz_input_ids, attention_mask = xyz_attention_mask)
        model_1_outputs = self.dropout(model_1_outputs[0])
        model_2_outputs = self.dropout(model_2_outputs[0])

        model_2_outputs = torch.sum(model_2_outputs, dim=0) 
        model_2_outputs = torch.unsqueeze(model_2_outputs, 0) 
        model_2_outputs = model_2_outputs.repeat(model_1_outputs.shape[0], 1, 1) 

        model_1_outputs = self.linear_model_1_out(model_1_outputs)
        model_2_outputs = self.linear_model_2_out(model_2_outputs)

        model_1_and_xyz_sum = model_2_outputs + model_1_outputs 
        outputs = self.dropout(model_1_and_xyz_sum)
        outputs = outputs[:,0,:].view(-1,512) 
        outputs = torch.relu(self.linear3(outputs))
        outputs = self.dropout(outputs)
        outputs = torch.relu(self.linear4(outputs))
        return outputs

model = Model()

from this 因此,我知道我需要为Pytorch模型创建一个配置文件,该文件将由模型加载。他们引用了如何向变形金刚添加模型?除其他各种输入和步骤外,我的自定义模型中的每一层。我还找到了 this this 他们的论坛上的问题,引用 this 转换Pytorch模型。

如何自动将我的pytorch模型转换为Hugginface模型,以便我可以使用他们的Trainer类(除了DataCollat​​or等其他功能之外)?

潜在的最佳伪代码:

my_pytorch_model = Model()
my_huggingface_model = Custom_huggingface_Model.load_model(my_pytorch_model)

Say I created a Pytorch model:

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        
        self.model_1_encoder = RobertaModel.from_pretrained('roberta-base')
        self.model_2_encoder = RobertaModel.from_pretrained('roberta-base')

        self.dropout = nn.Dropout(0.5)
        self.linear_model_1_out = nn.Linear(768, 512)
        self.linear_model_2_out = nn.Linear(768, 512)
    
        self.linear3 = nn.Linear(512, 512)
        self.linear4 = nn.Linear(512, 1)
        
    def forward(self, x):
        question_input_ids, attn_mask, xyz_input_ids, xyz_attention_mask = torch.tensor(x['input_ids']), torch.tensor(x['attention_mask']), torch.tensor(x['xyz_ids']), torch.tensor(x['xyz_mask'])
        model_1_outputs = self.model_1_encoder(question_input_ids, attention_mask=attn_mask)
        model_2_outputs = self.model_2_encoder(xyz_input_ids, attention_mask = xyz_attention_mask)
        model_1_outputs = self.dropout(model_1_outputs[0])
        model_2_outputs = self.dropout(model_2_outputs[0])

        model_2_outputs = torch.sum(model_2_outputs, dim=0) 
        model_2_outputs = torch.unsqueeze(model_2_outputs, 0) 
        model_2_outputs = model_2_outputs.repeat(model_1_outputs.shape[0], 1, 1) 

        model_1_outputs = self.linear_model_1_out(model_1_outputs)
        model_2_outputs = self.linear_model_2_out(model_2_outputs)

        model_1_and_xyz_sum = model_2_outputs + model_1_outputs 
        outputs = self.dropout(model_1_and_xyz_sum)
        outputs = outputs[:,0,:].view(-1,512) 
        outputs = torch.relu(self.linear3(outputs))
        outputs = self.dropout(outputs)
        outputs = torch.relu(self.linear4(outputs))
        return outputs

model = Model()

From this SO question I understand that I need to create a config file for the Pytorch model that will be loaded by the model. They referenced How to add a model to Transformers?, which if I understands correctly requires manual writing of each layer in my custom model in addition to various other inputs and steps. I also found this question on their forum, which referenced this, which had even more steps to complete in order to convert the Pytorch model.

How can I automatically convert my Pytorch model to a Hugginface model so that I can use their Trainer class (in addition to other functionalities like the DataCollator, etc.)?

Potential optimal pseudocode:

my_pytorch_model = Model()
my_huggingface_model = Custom_huggingface_Model.load_model(my_pytorch_model)

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文