如何找到深度学习模型的大小？

Question

我正在处理同一模型的不同量化实现，主要区别是权重，偏见和激活的精度。因此，我想知道如何找到MBS中32位浮点的模型大小与INT8中的差异之间的区别。我的模型以.pth格式保存。

Answer 1

您可以计算参数和缓冲区的数量。
然后尝试将它们与元素大小相乘，您将具有所有参数的大小。

model = models.resnet18()
param_size = 0
buffer_size = 0
for param in model.parameters():
    param_size += param.nelement() * param.element_size()

for buffer in model.buffers():
    buffer_size += buffer.nelement() * buffer.element_size()

size_all_mb = (param_size + buffer_size) / 1024**2
print('Size: {:.3f} MB'.format(size_all_mb))

它将打印：

大小：361.209 MB

Answer 2

“要计算模型大小（以字节为单位），请将参数数量乘以所选精度的大小（以字节为单位）。例如，如果我们使用 BLOOM-176B 模型的 bfloat16 版本，则有 176*10**9 x 2 字节 = 352GB！”

这个关于 HF 的博客值得一读：https://huggingface.co/blog/hf-位和字节集成

Answer 3

遵循@Prajot的代码，我们可以得到一个精度乘数和多个参数乘数，从而得出一条经验法则。

规则

stats for 8 bit models:
---
7 billion param = 7 GB
13 billion param = 13 GB
175 billion param = 175 GB

extending the rule of thumb
---
16 bits? 2x
24 bits? 3x
32 bits? 4x

的经验计算

TLDR -达到精度乘数

import numpy as np
import matplotlib.pyplot as plt

def cal_size(precision, num_params_in_billion=10**9):
    bits = precision/8
    size_in_gb = (num_params_in_billion/1024**3) * bits
    return round(size_in_gb,2)

# Assuming num_params as 1 (you can change it accordingly)
precisions = list(range(1, 25))
sizes = [cal_size(precision) for precision in precisions]

plt.plot(precisions, sizes, marker='o')
plt.title('Model Size vs. Precision')
plt.xlabel('Precision (bits)')
plt.ylabel('Size (GB)')
plt.grid(True)
plt.xticks(range(1, 25))
plt.show()

# Calculate the slope using linear regression
precisions_arr = np.array(precisions)
sizes_arr = np.array(sizes)
slope, intercept = np.polyfit(precisions_arr, sizes_arr, 1)

print("Slope of the line:", slope, "intercept:", intercept)

结果：精度乘数 = 直线斜率 = 0.1164

参数乘数

if precision = 24 bits, param multiplier = 24*0.1164 = 2.79 ~ 3
if precision = 16 bits, param multiplier = 16*0.1164 = 1.86 ~ 2
if precision = 8 bits, param multiplier = 8*0.1164 = 0.93 ~ 1

模型尺寸计算

model size =  Params in billion x param multiplier
7B-24bits = 7 x 3 = 21 GB
13B-24bits = 13 x 3 = 39 GB
7B-16bits = 7 x 2 = 14 GB
7B-8bits = 7 x 1 = 7 GB

Answer 4

已经编写了一个小型代码来计算模型的大小，具体取决于您的模型的参数次数＆amp; dtype您的模型
当前支持fp32，fp16，bfloat16＆amp; int8

def cal_size(num_params,dtype):
    if dtype == "float32":
        return (num_params/1024**2) * 4
    elif dtype == "float16" or dtype == "bfloat16":
        return (num_params/1024**2) * 2
    elif dtype == "int8":
        return (num_params/1024**2) * 1
    else:
        return -1

if __name__ == "__main__":
    import torchvision.models as models
    model = models.mobilenet_v2()
    #mobilenetv2 of width multiplier 1 has 3.4M params 
    total_params = sum(p.numel() for p in model.parameters())
    model_size = cal_size(total_params,"float32")
    if model_size != -1:
        print("Size of model is :{:.2f} MB".format(model_size))
    else:
        print("Incorrect dtype")

Answer 5

另一种方法是仅计算将权重下载到您的计算机的文件夹的大小（如果使用诸如 Hugging Face 之类的东西）。

import subprocess

def get_folder_size(model_name):
    start_path = '/home/ubuntu/.cache/huggingface/hub'
    folder_name = 'models--' + model_name.replace('/', '--')
    folder_path = os.path.join(start_path, folder_name)
    size = subprocess.check_output(['du', '-sb', folder_path]).split([0].decode('utf-8')
    size_in_bytes = int(size)
    size_in_gb = round(size_in_bytes / (1024 ** 3), 3)  # Convert bytes to GB and round to 3 decimal places
    return size_in_gb

get_folder_size('thenlper/gte-base')