在Python中播放部分httpresponse

发布于 2025-02-12 14:14:09 字数 856 浏览 0 评论 0原文

我正在尝试从请求中接收部分数据。服务器不响应范围标题。

我尝试了以下内容：

def get_data(change_id):
    url = "https://my-api.com?id={}".format(change_id)
    r = requests.get(url, stream=True, headers=headers)

    for data in r.iter_content(chunk_size=512):
        return extract_change_id(data)

据我所知，这仍然完成了完整的请求。

该请求返回一个新的更改ID，该ID可以访问下一个河流的请求。这个想法是读取前几个字节并从身体中提取更改ID，因为更改ID首先出现在每个身体中，然后将其传递到要处理的新线程中。每个请求都超过5MB，需要同时处理以与河流保持最新状态。只需阅读整个请求，然后解析JSON太慢即可。

提取功能是一个简单的双重正则调查结果。

def extract_change_id(data):
    try:
        full_change_id = full_change_id_regex.search(data).group(0)
        change_id = change_id_regex.search(full_change_id).group(0)
        return change_id
    except Exception as e:
        return None

原文

I am trying to receive only partial data from a request. The server does not respond to the Range header.

I have attempted the following:

def get_data(change_id):
    url = "https://my-api.com?id={}".format(change_id)
    r = requests.get(url, stream=True, headers=headers)

    for data in r.iter_content(chunk_size=512):
        return extract_change_id(data)

This still completes the full request as far as I can see.

The request returns a new change id which gives access to the next request as a river.
The idea is to read the first few bytes and extract the change id from the body since the change id appears first in every body and then pass it off to a new thread to be processed. Each request is upwards of 5MBs and needs to be handled concurrently to stay up to date with the river. Simply reading the whole request and then parsing to json is too slow.

The extract function is a simple double regex find.

def extract_change_id(data):
    try:
        full_change_id = full_change_id_regex.search(data).group(0)
        change_id = change_id_regex.search(full_change_id).group(0)
        return change_id
    except Exception as e:
        return None

分享到QQ

分享到微博