写入大文件时 FileOutputStream.close 非常慢

发布于 2024-12-11 08:14:51 字数 919 浏览 0 评论 0原文

我有一个使用以下代码通过 TCP 套接字接收文件的方法:

FileOutputStream fileStream = new FileOutputStream(filename.getName());
while (totalRead < size) {
    if (size - totalRead > CHUNKSIZE) {
        read = getInputStream().read(buffer, 0, CHUNKSIZE);
    } else {
        read = getInputStream().read(buffer, 0, size - totalRead);
    }
    totalRead += read;
    fileStream.write(buffer, 0, read);
    fileStream.flush();

    if (System.currentTimeMillis() > nextPrint) {
        nextPrint += 1000;
        int speed = (int) (totalRead / (System.currentTimeMillis() - startTime));
        double procent = ((double)totalRead / size) * 100;
        gui.setStatus("Reciving: " + filename + " at " + speed + " kb/s, " + procent + "% complete");
    }
}
gui.setStatus("Reciving: " + filename + " complete.");
fileStream.close();

FileOutputStream.close 在接收大文件时花费了很长时间,为什么呢?正如你所看到的,我在每个收到的块上刷新流。

I have a method which receives a file over a TCP socket using this code:

FileOutputStream fileStream = new FileOutputStream(filename.getName());
while (totalRead < size) {
    if (size - totalRead > CHUNKSIZE) {
        read = getInputStream().read(buffer, 0, CHUNKSIZE);
    } else {
        read = getInputStream().read(buffer, 0, size - totalRead);
    }
    totalRead += read;
    fileStream.write(buffer, 0, read);
    fileStream.flush();

    if (System.currentTimeMillis() > nextPrint) {
        nextPrint += 1000;
        int speed = (int) (totalRead / (System.currentTimeMillis() - startTime));
        double procent = ((double)totalRead / size) * 100;
        gui.setStatus("Reciving: " + filename + " at " + speed + " kb/s, " + procent + "% complete");
    }
}
gui.setStatus("Reciving: " + filename + " complete.");
fileStream.close();

FileOutputStream.close is taking really long time when receiving large files, why is that? As you see I'm flushing the stream at every received chunk..

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

碍人泪离人颜 2024-12-18 08:14:51

根据操作系统的不同,flush() 只是强制将数据写入操作系统。对于 FileOutputStream,write() 将所有数据传递到操作系统,因此 flash() 不执行任何操作。其中 close() 可以确保文件实际写入磁盘(或不取决于操作系统)。可以在写入数据时查看磁盘是否还处于繁忙状态。

500 MB 的文件需要 30 秒,意味着您正在以 17 MB/s 的速度写入。这听起来像是一个非常慢的磁盘或文件位于网络共享/驱动器上。


您可以尝试打印此

File file = File.createTempFile("deleteme", "dat"); // put your file here.
FileOutputStream fos = new FileOutputStream(file);
long start = System.nanoTime();
byte[] bytes = new byte[32 * 1024];
for (long l = 0; l < 500 * 1000 * 1000; l += bytes.length)
    fos.write(bytes);
long mid = System.nanoTime();
System.out.printf("Took %.3f seconds to write %,d bytes%n", (mid - start) / 1e9, file.length());
fos.close();
long end = System.nanoTime();
System.out.printf("Took %.3f seconds to close%n", (end - mid) / 1e9);

打印

Took 0.116 seconds to write 500,006,912 bytes
Took 0.002 seconds to close

您可以从速度看出,在该系统上即使在关闭时也不会写入数据。即驱动器不是那么快。

Depending on the OS, flush() does nothing more thaqn force the data to be written to the OS. In the case of FileOutputStream, write() passes all data to the OS, so flush() does nothing. Where as close() can ensure the file is actually written to disk (or not depending on the OS). You can look at whether the disk is still busy when writing the data.

A 500 MB files taking 30 seconds means you are writing 17 MB/s. This sounds like a very slow disk or that the file in on network share/drive.


You can try this

File file = File.createTempFile("deleteme", "dat"); // put your file here.
FileOutputStream fos = new FileOutputStream(file);
long start = System.nanoTime();
byte[] bytes = new byte[32 * 1024];
for (long l = 0; l < 500 * 1000 * 1000; l += bytes.length)
    fos.write(bytes);
long mid = System.nanoTime();
System.out.printf("Took %.3f seconds to write %,d bytes%n", (mid - start) / 1e9, file.length());
fos.close();
long end = System.nanoTime();
System.out.printf("Took %.3f seconds to close%n", (end - mid) / 1e9);

prints

Took 0.116 seconds to write 500,006,912 bytes
Took 0.002 seconds to close

You can see from the speed that on this system its not writing the data even on a close. i.e. the drive is not that fast.

青衫儰鉨ミ守葔 2024-12-18 08:14:51

我在使用文件流时看到了同样的情况。我发现,如果您以读写方式打开文件,它会缓存所有内容,并且在您关闭或处置之前不会写入。芙蓉没有写。但是,如果您的写入扩展了文件的大小,它将自动刷新。

打开为每次写入时自动刷新写入。

I saw the same with using filestream. What I found there was if you open the file as readwrite, it cached everything and didn't write until you close or dispose. Flush didn't write. However if your writes extend the size of the file it would autoflush.

Opening as just write autoflushed on each write.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文