仅添加其他参数后,nn.init.normal_结果更改
我创建一个模型并使用nn.init.normal_初始化参数,但是,我添加另一个参数后的权重变化(没有其他操作,包括iNATION)。 该代码在这里,不注重另一种情况。
def __init__(self):
self.tc_h = torch.nn.Sequential(
torch.nn.Linear(512, 1024),
torch.nn.ReLU(),
torch.nn.Linear(1024, 512)
)
self.tg_h = torch.nn.Sequential(
torch.nn.Linear(512, 1024),
torch.nn.ReLU(),
torch.nn.Linear(1024, 512)
)
self.vc_h = torch.nn.Sequential(
torch.nn.Linear(512, 1024),
torch.nn.ReLU(),
torch.nn.Linear(1024, 512)
)
self.vg_h = torch.nn.Sequential(
torch.nn.Linear(512, 1024),
torch.nn.ReLU(),
torch.nn.Linear(1024, 512)
)
#self.tc_Q = torch.nn.Linear(512, 512)
#self.vc_Q = torch.nn.Linear(512, 512)
#self.tg_Q = torch.nn.Linear(512, 512)
#self.vg_Q = torch.nn.Linear(512, 512)
for layer in [
self.tc_h, self.tg_h, self.vc_h, self.vg_h,
#self.tc_Q, self.tg_Q, self.vc_Q, self.vg_Q,
]:
for name, param in layer.named_parameters():
if name.endswith('weight'):
nn.init.normal_(param, std=1024 ** -0.5)
I create a model and initialize the parameter with nn.init.normal_, however, the weight change after I add another Parameter(no other operation including initalization) .
The code is here, uncomment to get another case.
def __init__(self):
self.tc_h = torch.nn.Sequential(
torch.nn.Linear(512, 1024),
torch.nn.ReLU(),
torch.nn.Linear(1024, 512)
)
self.tg_h = torch.nn.Sequential(
torch.nn.Linear(512, 1024),
torch.nn.ReLU(),
torch.nn.Linear(1024, 512)
)
self.vc_h = torch.nn.Sequential(
torch.nn.Linear(512, 1024),
torch.nn.ReLU(),
torch.nn.Linear(1024, 512)
)
self.vg_h = torch.nn.Sequential(
torch.nn.Linear(512, 1024),
torch.nn.ReLU(),
torch.nn.Linear(1024, 512)
)
#self.tc_Q = torch.nn.Linear(512, 512)
#self.vc_Q = torch.nn.Linear(512, 512)
#self.tg_Q = torch.nn.Linear(512, 512)
#self.vg_Q = torch.nn.Linear(512, 512)
for layer in [
self.tc_h, self.tg_h, self.vc_h, self.vg_h,
#self.tc_Q, self.tg_Q, self.vc_Q, self.vg_Q,
]:
for name, param in layer.named_parameters():
if name.endswith('weight'):
nn.init.normal_(param, std=1024 ** -0.5)
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论