当前位置： 文江博客话题详情

python套接字在逐行调试时工作正常，但在完整运行时无法工作

发布于 2025-01-20 15:44:30 字数 8489 浏览 1 评论 0原文

我正在研究一个项目，该项目涉及使用文件的每个块的CRC校验和计算文件，例如，此处的每个块40960字节。我的问题是，当我按行调试代码时，一切正常，但是当我完全运行代码时，我会在接收器端获得不同的CRC校验和。我猜这个问题来自缓冲区。

问题来自发件人的一边。

任何帮助都将不胜感激，请让我知道如何完成这项任务。

谢谢。

发件人侧：

import socket
import tqdm
import os
import crcmod

SEPARATOR = "<SEPARATOR>"
BUFFER_SIZE = 40960 # bytes

host = "10.218.105.192"
port = 5001

crc32 = crcmod.mkCrcFun(0x104c11db7, 0, False, 0xFFFFFFFF)

s = socket.socket()
print(f"[+] Connecting to {host}:{port}")
s.connect((host, port))
print("[+] Connected.")

#filename = input('Enter file name --> ')
filename="testfile"
if not os.path.exists(filename):
    print("file-doesn't-exist")
    s.close()
filesize = os.path.getsize(filename)
crc_of_file = open(filename, 'rb')
crc_d = crc_of_file.read()
crc = crc32(crc_d)
# send the filename and filesize
#s.send(f"{filename}{SEPARATOR}{filesize}{SEPARATOR}{crc}".encode())
# start sending the file
chunk_crc=[]
progress = tqdm.tqdm(range(filesize), f"Sending {filename}", unit="B", unit_scale=True, unit_divisor=1024)
with open(filename, "rb") as f:
    while True:
        # read the bytes from the file
        bytes_read = f.read(BUFFER_SIZE)
        if not bytes_read:
            # file transmitting is done
            break
        bytes_read += crc32(bytes_read).to_bytes(4, 'big')# we use sendall to assure transimission in
        # busy networks
        #s.send(f"{this_chunk_crc}\n".encode())
        s.sendall(bytes_read)
        # update the progress bar
        progress.update(len(bytes_read))
# close the socket
s.close()

接收器侧：

import socket
import tqdm
import os
import crcmod
# device's IP address
SERVER_HOST = "0.0.0.0"
SERVER_PORT = 5001
# receive 40964 bytes each time which includes CRC checksum
BUFFER_SIZE = 40964


SEPARATOR = "<SEPARATOR>"
s = socket.socket()
s.bind((SERVER_HOST, SERVER_PORT))
s.listen(5)
print(f"[*] Listening as {SERVER_HOST}:{SERVER_PORT}")

client_socket, address = s.accept()
# if below code is executed, that means the sender is connected
print(f"[+] {address} is connected.")
# receive the file infos
# receive using client socket, not server socket
#received = client_socket.recv(BUFFER_SIZE).decode()
filename="testfile"
filesize=int("2367275")
#filename, filesize,crc = received.split(SEPARATOR)
# remove absolute path if there is
filename = os.path.basename(filename)
# convert to integer
filesize = int(filesize)
#print(f"[+] File CRC is {crc}")


crc32 = crcmod.mkCrcFun(0x104c11db7, 0, False, 0xFFFFFFFF)


# start receiving the file from the socket
# and writing to the file stream
progress = tqdm.tqdm(range(filesize), f"Receiving {filename}", unit="B", unit_scale=True, unit_divisor=1024)
#with open(filename, "wb") as f:
i=0
while True:
    # read bytes from the socket (receive)
    bytes_read = client_socket.recv(BUFFER_SIZE)
    if not bytes_read:
        # nothing is received
        # file transmitting is done
        break
    # write to the file the bytes we just received
    if (len(bytes_read) >= BUFFER_SIZE) :
        this_chunk_crc = crc32(bytes_read)
        print(this_chunk_crc)
        #partname=filename+"Part"+str(i)+" "+str(this_chunk_crc)
        with open(filename, "wb") as f:
            #bytes_read -= crc32(bytes_read).to_bytes(4, 'big')
            f.write(bytes_read)
    progress.update(len(bytes_read))


# close the client socket
client_socket.close()
# close the server socket
s.close()

正确的输出应该是类似的（该CRC定义的常数应为955982468，如您所见，并且与文件无关）


Receiving testfile:   0%|          | 0.00/2.26M [00:00<?, ?B/s]955982468
Receiving testfile:   2%|▏         | 40.0k/2.26M [00:21<20:26, 1.90kB/s]955982468
Receiving testfile:   3%|▎         | 80.0k/2.26M [00:22<08:57, 4.26kB/s]955982468
Receiving testfile:   5%|▌         | 120k/2.26M [00:23<04:58, 7.52kB/s] 955982468
Receiving testfile:   7%|▋         | 160k/2.26M [00:23<03:08, 11.7kB/s]955982468
Receiving testfile:   9%|▊         | 200k/2.26M [00:24<02:08, 16.9kB/s]955982468
Receiving testfile:  10%|█         | 240k/2.26M [00:24<01:31, 23.2kB/s]955982468
Receiving testfile:  12%|█▏        | 280k/2.26M [00:25<01:08, 30.2kB/s]955982468
Receiving testfile:  14%|█▍        | 320k/2.26M [00:26<01:02, 32.9kB/s]955982468
Receiving testfile:  16%|█▌        | 360k/2.26M [00:26<00:49, 40.4kB/s]955982468
Receiving testfile:  17%|█▋        | 400k/2.26M [00:27<00:40, 47.9kB/s]955982468
Receiving testfile:  19%|█▉        | 440k/2.26M [00:27<00:35, 53.9kB/s]955982468
Receiving testfile:  21%|██        | 480k/2.26M [00:28<00:30, 60.5kB/s]955982468
Receiving testfile:  22%|██▏       | 520k/2.26M [00:28<00:27, 66.2kB/s]955982468
Receiving testfile:  24%|██▍       | 560k/2.26M [00:29<00:25, 70.3kB/s]955982468
Receiving testfile:  26%|██▌       | 600k/2.26M [00:30<00:26, 67.2kB/s]955982468
Receiving testfile:  28%|██▊       | 640k/2.26M [01:01<06:52, 4.15kB/s]955982468
Receiving testfile:  29%|██▉       | 680k/2.26M [01:01<04:45, 5.85kB/s]955982468
Receiving testfile:  31%|███       | 720k/2.26M [01:02<03:19, 8.17kB/s]955982468
Receiving testfile:  33%|███▎      | 760k/2.26M [01:02<02:23, 11.1kB/s]955982468
Receiving testfile:  35%|███▍      | 800k/2.26M [01:03<01:43, 15.0kB/s]955982468
Receiving testfile:  36%|███▋      | 840k/2.26M [01:03<01:15, 19.8kB/s]955982468
Receiving testfile:  38%|███▊      | 880k/2.26M [01:04<01:01, 23.7kB/s]955982468
Receiving testfile:  40%|███▉      | 920k/2.26M [01:05<00:47, 29.9kB/s]955982468
Receiving testfile:  42%|████▏     | 960k/2.26M [01:05<00:37, 37.2kB/s]955982468
Receiving testfile:  43%|████▎     | 0.98M/2.26M [01:06<00:30, 44.5kB/s]955982468
Receiving testfile:  45%|████▍     | 1.02M/2.26M [01:06<00:26, 48.7kB/s]955982468
Receiving testfile:  47%|████▋     | 1.05M/2.26M [01:07<00:22, 55.8kB/s]955982468
Receiving testfile:  48%|████▊     | 1.09M/2.26M [01:07<00:18, 65.4kB/s]955982468
Receiving testfile:  50%|█████     | 1.13M/2.26M [01:08<00:17, 68.5kB/s]955982468
Receiving testfile:  52%|█████▏    | 1.17M/2.26M [01:08<00:15, 71.5kB/s]955982468
Receiving testfile:  54%|█████▎    | 1.21M/2.26M [01:09<00:15, 72.5kB/s]955982468
Receiving testfile:  55%|█████▌    | 1.25M/2.26M [01:09<00:14, 73.8kB/s]955982468
Receiving testfile:  57%|█████▋    | 1.29M/2.26M [01:10<00:13, 76.1kB/s]955982468
Receiving testfile:  59%|█████▉    | 1.33M/2.26M [01:10<00:12, 76.4kB/s]955982468
Receiving testfile:  61%|██████    | 1.37M/2.26M [01:11<00:12, 76.6kB/s]955982468
Receiving testfile:  62%|██████▏   | 1.41M/2.26M [01:11<00:11, 77.4kB/s]955982468
Receiving testfile:  64%|██████▍   | 1.45M/2.26M [01:12<00:11, 76.6kB/s]955982468
Receiving testfile:  66%|██████▌   | 1.48M/2.26M [01:12<00:09, 84.2kB/s]955982468
Receiving testfile:  67%|██████▋   | 1.52M/2.26M [01:13<00:09, 81.2kB/s]955982468
Receiving testfile:  69%|██████▉   | 1.56M/2.26M [01:13<00:09, 78.5kB/s]955982468
Receiving testfile:  71%|███████   | 1.60M/2.26M [01:14<00:08, 77.3kB/s]955982468
Receiving testfile:  73%|███████▎  | 1.64M/2.26M [01:15<00:08, 76.2kB/s]955982468
Receiving testfile:  74%|███████▍  | 1.68M/2.26M [01:15<00:07, 85.2kB/s]955982468
Receiving testfile:  76%|███████▌  | 1.72M/2.26M [01:15<00:06, 81.0kB/s]955982468
Receiving testfile:  78%|███████▊  | 1.76M/2.26M [01:16<00:05, 87.8kB/s]955982468
Receiving testfile:  80%|███████▉  | 1.80M/2.26M [01:16<00:05, 84.3kB/s]955982468
Receiving testfile:  81%|████████▏ | 1.84M/2.26M [01:17<00:05, 80.5kB/s]955982468
Receiving testfile:  83%|████████▎ | 1.88M/2.26M [01:17<00:05, 78.7kB/s]955982468
Receiving testfile:  85%|████████▍ | 1.91M/2.26M [01:18<00:04, 79.6kB/s]955982468
Receiving testfile:  87%|████████▋ | 1.95M/2.26M [01:18<00:03, 84.7kB/s]955982468
Receiving testfile:  88%|████████▊ | 1.99M/2.26M [01:19<00:03, 79.6kB/s]955982468
Receiving testfile:  90%|████████▉ | 2.03M/2.26M [01:20<00:03, 76.8kB/s]955982468
Receiving testfile:  92%|█████████▏| 2.07M/2.26M [01:20<00:02, 81.9kB/s]955982468
Receiving testfile:  93%|█████████▎| 2.11M/2.26M [01:21<00:02, 62.6kB/s]955982468
Receiving testfile:  95%|█████████▌| 2.15M/2.26M [01:22<00:01, 63.4kB/s]955982468
Receiving testfile:  97%|█████████▋| 2.19M/2.26M [01:22<00:01, 70.8kB/s]955982468
Receiving testfile:  99%|█████████▊| 2.23M/2.26M [01:23<00:00, 69.7kB/s]955982468
Receiving testfile: 2.26MB [02:00, 19.7kB/s]
原文

I'm working on a project that involves transferring a file with a CRC checksum calculation for each chunk of a file for example here 40960 bytes. My issue is that everything works fine when I debug the code line by line, but when I run the code completely, I get different CRC checksums at the receiver side. I am guessing the issue is from the buffer.

The problem is from sender's side.

Any help would be much appreciated and please let me know how to do this task.

Thank you.

Sender Side:

import socket
import tqdm
import os
import crcmod

SEPARATOR = "<SEPARATOR>"
BUFFER_SIZE = 40960 # bytes

host = "10.218.105.192"
port = 5001

crc32 = crcmod.mkCrcFun(0x104c11db7, 0, False, 0xFFFFFFFF)

s = socket.socket()
print(f"[+] Connecting to {host}:{port}")
s.connect((host, port))
print("[+] Connected.")

#filename = input('Enter file name --> ')
filename="testfile"
if not os.path.exists(filename):
    print("file-doesn't-exist")
    s.close()
filesize = os.path.getsize(filename)
crc_of_file = open(filename, 'rb')
crc_d = crc_of_file.read()
crc = crc32(crc_d)
# send the filename and filesize
#s.send(f"{filename}{SEPARATOR}{filesize}{SEPARATOR}{crc}".encode())
# start sending the file
chunk_crc=[]
progress = tqdm.tqdm(range(filesize), f"Sending {filename}", unit="B", unit_scale=True, unit_divisor=1024)
with open(filename, "rb") as f:
    while True:
        # read the bytes from the file
        bytes_read = f.read(BUFFER_SIZE)
        if not bytes_read:
            # file transmitting is done
            break
        bytes_read += crc32(bytes_read).to_bytes(4, 'big')# we use sendall to assure transimission in
        # busy networks
        #s.send(f"{this_chunk_crc}\n".encode())
        s.sendall(bytes_read)
        # update the progress bar
        progress.update(len(bytes_read))
# close the socket
s.close()

Receiver Side:

import socket
import tqdm
import os
import crcmod
# device's IP address
SERVER_HOST = "0.0.0.0"
SERVER_PORT = 5001
# receive 40964 bytes each time which includes CRC checksum
BUFFER_SIZE = 40964


SEPARATOR = "<SEPARATOR>"
s = socket.socket()
s.bind((SERVER_HOST, SERVER_PORT))
s.listen(5)
print(f"[*] Listening as {SERVER_HOST}:{SERVER_PORT}")

client_socket, address = s.accept()
# if below code is executed, that means the sender is connected
print(f"[+] {address} is connected.")
# receive the file infos
# receive using client socket, not server socket
#received = client_socket.recv(BUFFER_SIZE).decode()
filename="testfile"
filesize=int("2367275")
#filename, filesize,crc = received.split(SEPARATOR)
# remove absolute path if there is
filename = os.path.basename(filename)
# convert to integer
filesize = int(filesize)
#print(f"[+] File CRC is {crc}")


crc32 = crcmod.mkCrcFun(0x104c11db7, 0, False, 0xFFFFFFFF)


# start receiving the file from the socket
# and writing to the file stream
progress = tqdm.tqdm(range(filesize), f"Receiving {filename}", unit="B", unit_scale=True, unit_divisor=1024)
#with open(filename, "wb") as f:
i=0
while True:
    # read bytes from the socket (receive)
    bytes_read = client_socket.recv(BUFFER_SIZE)
    if not bytes_read:
        # nothing is received
        # file transmitting is done
        break
    # write to the file the bytes we just received
    if (len(bytes_read) >= BUFFER_SIZE) :
        this_chunk_crc = crc32(bytes_read)
        print(this_chunk_crc)
        #partname=filename+"Part"+str(i)+" "+str(this_chunk_crc)
        with open(filename, "wb") as f:
            #bytes_read -= crc32(bytes_read).to_bytes(4, 'big')
            f.write(bytes_read)
    progress.update(len(bytes_read))


# close the client socket
client_socket.close()
# close the server socket
s.close()

the correct output should be something like (constant for that CRC definition should be 955982468 at every step as you can see and it is not related to the file)


Receiving testfile:   0%|          | 0.00/2.26M [00:00<?, ?B/s]955982468
Receiving testfile:   2%|▏         | 40.0k/2.26M [00:21<20:26, 1.90kB/s]955982468
Receiving testfile:   3%|▎         | 80.0k/2.26M [00:22<08:57, 4.26kB/s]955982468
Receiving testfile:   5%|▌         | 120k/2.26M [00:23<04:58, 7.52kB/s] 955982468
Receiving testfile:   7%|▋         | 160k/2.26M [00:23<03:08, 11.7kB/s]955982468
Receiving testfile:   9%|▊         | 200k/2.26M [00:24<02:08, 16.9kB/s]955982468
Receiving testfile:  10%|█         | 240k/2.26M [00:24<01:31, 23.2kB/s]955982468
Receiving testfile:  12%|█▏        | 280k/2.26M [00:25<01:08, 30.2kB/s]955982468
Receiving testfile:  14%|█▍        | 320k/2.26M [00:26<01:02, 32.9kB/s]955982468
Receiving testfile:  16%|█▌        | 360k/2.26M [00:26<00:49, 40.4kB/s]955982468
Receiving testfile:  17%|█▋        | 400k/2.26M [00:27<00:40, 47.9kB/s]955982468
Receiving testfile:  19%|█▉        | 440k/2.26M [00:27<00:35, 53.9kB/s]955982468
Receiving testfile:  21%|██        | 480k/2.26M [00:28<00:30, 60.5kB/s]955982468
Receiving testfile:  22%|██▏       | 520k/2.26M [00:28<00:27, 66.2kB/s]955982468
Receiving testfile:  24%|██▍       | 560k/2.26M [00:29<00:25, 70.3kB/s]955982468
Receiving testfile:  26%|██▌       | 600k/2.26M [00:30<00:26, 67.2kB/s]955982468
Receiving testfile:  28%|██▊       | 640k/2.26M [01:01<06:52, 4.15kB/s]955982468
Receiving testfile:  29%|██▉       | 680k/2.26M [01:01<04:45, 5.85kB/s]955982468
Receiving testfile:  31%|███       | 720k/2.26M [01:02<03:19, 8.17kB/s]955982468
Receiving testfile:  33%|███▎      | 760k/2.26M [01:02<02:23, 11.1kB/s]955982468
Receiving testfile:  35%|███▍      | 800k/2.26M [01:03<01:43, 15.0kB/s]955982468
Receiving testfile:  36%|███▋      | 840k/2.26M [01:03<01:15, 19.8kB/s]955982468
Receiving testfile:  38%|███▊      | 880k/2.26M [01:04<01:01, 23.7kB/s]955982468
Receiving testfile:  40%|███▉      | 920k/2.26M [01:05<00:47, 29.9kB/s]955982468
Receiving testfile:  42%|████▏     | 960k/2.26M [01:05<00:37, 37.2kB/s]955982468
Receiving testfile:  43%|████▎     | 0.98M/2.26M [01:06<00:30, 44.5kB/s]955982468
Receiving testfile:  45%|████▍     | 1.02M/2.26M [01:06<00:26, 48.7kB/s]955982468
Receiving testfile:  47%|████▋     | 1.05M/2.26M [01:07<00:22, 55.8kB/s]955982468
Receiving testfile:  48%|████▊     | 1.09M/2.26M [01:07<00:18, 65.4kB/s]955982468
Receiving testfile:  50%|█████     | 1.13M/2.26M [01:08<00:17, 68.5kB/s]955982468
Receiving testfile:  52%|█████▏    | 1.17M/2.26M [01:08<00:15, 71.5kB/s]955982468
Receiving testfile:  54%|█████▎    | 1.21M/2.26M [01:09<00:15, 72.5kB/s]955982468
Receiving testfile:  55%|█████▌    | 1.25M/2.26M [01:09<00:14, 73.8kB/s]955982468
Receiving testfile:  57%|█████▋    | 1.29M/2.26M [01:10<00:13, 76.1kB/s]955982468
Receiving testfile:  59%|█████▉    | 1.33M/2.26M [01:10<00:12, 76.4kB/s]955982468
Receiving testfile:  61%|██████    | 1.37M/2.26M [01:11<00:12, 76.6kB/s]955982468
Receiving testfile:  62%|██████▏   | 1.41M/2.26M [01:11<00:11, 77.4kB/s]955982468
Receiving testfile:  64%|██████▍   | 1.45M/2.26M [01:12<00:11, 76.6kB/s]955982468
Receiving testfile:  66%|██████▌   | 1.48M/2.26M [01:12<00:09, 84.2kB/s]955982468
Receiving testfile:  67%|██████▋   | 1.52M/2.26M [01:13<00:09, 81.2kB/s]955982468
Receiving testfile:  69%|██████▉   | 1.56M/2.26M [01:13<00:09, 78.5kB/s]955982468
Receiving testfile:  71%|███████   | 1.60M/2.26M [01:14<00:08, 77.3kB/s]955982468
Receiving testfile:  73%|███████▎  | 1.64M/2.26M [01:15<00:08, 76.2kB/s]955982468
Receiving testfile:  74%|███████▍  | 1.68M/2.26M [01:15<00:07, 85.2kB/s]955982468
Receiving testfile:  76%|███████▌  | 1.72M/2.26M [01:15<00:06, 81.0kB/s]955982468
Receiving testfile:  78%|███████▊  | 1.76M/2.26M [01:16<00:05, 87.8kB/s]955982468
Receiving testfile:  80%|███████▉  | 1.80M/2.26M [01:16<00:05, 84.3kB/s]955982468
Receiving testfile:  81%|████████▏ | 1.84M/2.26M [01:17<00:05, 80.5kB/s]955982468
Receiving testfile:  83%|████████▎ | 1.88M/2.26M [01:17<00:05, 78.7kB/s]955982468
Receiving testfile:  85%|████████▍ | 1.91M/2.26M [01:18<00:04, 79.6kB/s]955982468
Receiving testfile:  87%|████████▋ | 1.95M/2.26M [01:18<00:03, 84.7kB/s]955982468
Receiving testfile:  88%|████████▊ | 1.99M/2.26M [01:19<00:03, 79.6kB/s]955982468
Receiving testfile:  90%|████████▉ | 2.03M/2.26M [01:20<00:03, 76.8kB/s]955982468
Receiving testfile:  92%|█████████▏| 2.07M/2.26M [01:20<00:02, 81.9kB/s]955982468
Receiving testfile:  93%|█████████▎| 2.11M/2.26M [01:21<00:02, 62.6kB/s]955982468
Receiving testfile:  95%|█████████▌| 2.15M/2.26M [01:22<00:01, 63.4kB/s]955982468
Receiving testfile:  97%|█████████▋| 2.19M/2.26M [01:22<00:01, 70.8kB/s]955982468
Receiving testfile:  99%|█████████▊| 2.23M/2.26M [01:23<00:00, 69.7kB/s]955982468
Receiving testfile: 2.26MB [02:00, 19.7kB/s]

如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论， 你可以免费 注册 一个本站的账号。
列表为空，暂无数据

关于作者

九厘米的零°

暂无简介

文章
评论
27 人气
发私信

相关话题

更多

热门标签

操作系统 程序设计 IT运维 Linux系统管理 JavaScript 服务器应用 solaris C/C++ PHP Shell BSD Vue.js aix Oracle Python HTML 系统管理 HTML5 CSS 前端
更多

推荐作者

琉璃梦幻

文章 0 评论 0

qq_4zWU6L

文章 0 评论 0

话少情深

文章 0 评论 0

西西弗的石头怪

文章 0 评论 0

彻夜缠绵

文章 0 评论 0

千寻…

文章 0 评论 0

更多

友情链接

文江博客
    我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站，即表示您同意使用 Cookies 和您的相关数据。
    原文