DataInputStream 的 readFully 查询
我使用 dataInputStream 的 readFully 消息来读取固定长度的字节数组,如下所示:
byte[] record = new byte[4660004];
in.readFully(record);
这里的问题是,有时读取这么多字节需要 5 秒以上,相当于 20000 条记录。我正在套接字上接收这些数据。客户端以 4660004 字节的字节数组形式发送数据。有没有办法更快地接收这些数据,因为目前大约需要 5 分钟才能接收 100 万条此类记录。
编辑:: 完整的数据流:
首先我创建流:
static DataInputStream dIn = null;
dIn = new DataInputStream(connection.getInputStream());
msgType = dIn.readByte();
int msgIntLen = dIn.readInt();
processBatch(msgIntType, msgIntLen, dIn, connector);
.
.
private static void processBatch(int msgIntType, int msgIntLen, DataInputStream in,
Connector connector) throws IOException {
int recordIntLen = in.readInt();
byte[] record = new byte[msgIntLen - 4];
in.readFully(record);
}
如果 wudf 有帮助,我应该在哪里包含缓冲?
I am using dataInputStream's readFully message to read a fixed length byte array as:
byte[] record = new byte[4660004];
in.readFully(record);
The problem here is that sometimes it takes more than 5 seconds to read these many bytes, which is equal to 20000 records. And I am receiving this data on socket. Client is sending data as byte array of 4660004 bytes. Is there a way to received this data faster as right now it takes about 5 minutes to 1 million such records.
EDIT:: complete data flow :
first I create the stream :
static DataInputStream dIn = null;
dIn = new DataInputStream(connection.getInputStream());
msgType = dIn.readByte();
int msgIntLen = dIn.readInt();
processBatch(msgIntType, msgIntLen, dIn, connector);
.
.
private static void processBatch(int msgIntType, int msgIntLen, DataInputStream in,
Connector connector) throws IOException {
int recordIntLen = in.readInt();
byte[] record = new byte[msgIntLen - 4];
in.readFully(record);
}
where should I include the Buffering if that wudf help ?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
评论开始滚动,所以转向答案。
使用 BufferedOutputStream 在客户端缓冲输出。确保在写入数据后调用 dlOut.flush() ,以便未发送的字节不会保留在缓冲的输出流中。
使用 BufferedInputStream 在客户端缓冲输入。
因为您只是发送字节数组,所以您可能不需要 DataInputStream/DataOuputStream,除非您将它们用于其他目的。您可以只使用 BufferedInputStream/BufferedOutputStream。
Comments are beginning to scroll, so moving to an answer.
Buffer your output on the client side by using a BufferedOutputStream. Make sure to call
dlOut.flush()
after writing the data, so that unsent bytes don't remain in the buffered output stream.Buffer your input on the client side by using a BufferedInputStream.
Because you are just sending byte arrays, you probably don't need the DataInputStream/DataOuputStream, unless you are using them for an additional purpose. You could just be using BufferedInputStream/BufferedOutputStream.