Python Pickle 加载每次迭代花费的时间更长

发布于 2024-11-17 12:41:34 字数 317 浏览 4 评论 0原文

我有一个大小为 4mb 的 gzip 腌制文件。每次我打开文件并加载它时，解封所需的时间都会更长。文件的内容永远不会改变并且大小保持不变。

测试时间：
第一次运行：27.5 秒
第二次运行：44.1s
第三次运行：52.7s

我正在使用的代码如下：

f = gzip.open( filepath, "rb" )
pickleFile = cPickle.Unpickler( f )
paData = pickleFile.load()
f.close()

知道为什么每次运行时加载时间更长吗？

原文

I have a gzip'd pickled file that is 4mb in size. Each time I open the file and load it the time it takes to unpickle takes longer. The contents of the file never change and the size stays the same.

Test Times:
1st Run: 27.5s
2nd Run: 44.1s
3rd Run: 52.7s

The code I'm using is as follows:

f = gzip.open( filepath, "rb" )
pickleFile = cPickle.Unpickler( f )
paData = pickleFile.load()
f.close()

Any idea why it is taking longer to load each time I run?

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

酷遇一生 2024-11-24 12:41:34

我创建了两个测试 pickle 文件 3MB 和 22MB，并将其压缩。

我不明白你在做什么，因为我刚刚做了一个测试，这段代码：

def get_time():
    return int(time.time())

while(count < 10):    
    filepath='/home/shared/word_out.gz'
    start = get_time()
    f = gzip.open( filepath, "rb" )
    pickleFile = cPickle.Unpickler( f )
    paData = pickleFile.load()
    f.close()
    end = get_time()

    print 'Time ellapsed' , end - start
    count+=1

总是打印出与我运行它相同的内容。

3 MB 文件：

Time ellapsed 1
Time ellapsed 1
Time ellapsed 0
Time ellapsed 1
Time ellapsed 1
Time ellapsed 0
Time ellapsed 1
Time ellapsed 1
Time ellapsed 0
Time ellapsed 1

22MB 文件：

Time ellapsed 5
Time ellapsed 4
Time ellapsed 2
Time ellapsed 2
Time ellapsed 3
Time ellapsed 2
Time ellapsed 3
Time ellapsed 2
Time ellapsed 2
Time ellapsed 2

I created two test pickle file 3MBs and 22MBs, and zipped it.

I don't understand what you are doing because I just did a test and this code:

def get_time():
    return int(time.time())

while(count < 10):    
    filepath='/home/shared/word_out.gz'
    start = get_time()
    f = gzip.open( filepath, "rb" )
    pickleFile = cPickle.Unpickler( f )
    paData = pickleFile.load()
    f.close()
    end = get_time()

    print 'Time ellapsed' , end - start
    count+=1

Always prints out the same as I run it.

3 MB File:

Time ellapsed 1
Time ellapsed 1
Time ellapsed 0
Time ellapsed 1
Time ellapsed 1
Time ellapsed 0
Time ellapsed 1
Time ellapsed 1
Time ellapsed 0
Time ellapsed 1

22MB File:

Time ellapsed 5
Time ellapsed 4
Time ellapsed 2
Time ellapsed 2
Time ellapsed 3
Time ellapsed 2
Time ellapsed 3
Time ellapsed 2
Time ellapsed 2
Time ellapsed 2

回复收藏 0 原文

~没有更多了~