如何使用用户代理列表随机对标头进行随机化？

Question

在我的脚本.py中，我有：

    headers = {'Connection': 'keep-alive',
              'Cache-Control': 'max-age=0',
              'Upgrade-Insecure-Requests': '1',
              'User-Agent': {agents},
              'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
              'Accept-Encoding': 'gzip, deflate',
              'Accept-Language': 'en-US,en;q=0.9,fr;q=0.8',
              'referer': 'bing.com'}

在.json文件上

    { 
      "user_agent_list": [
          "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36",
          "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
      ]
    }

，当我发送请求时：

find_file = (grequests.get(url_a, headers={headers}, timeout=None, stream=True, verify=False, allow_redirects=False) for url_a in [f'{site_link}{x}' for x in file_rootl])

如何随机化标题= {headers} for to remands ？当然，“ 对于用户_agent_list中的代理”

更好：

'user-agent'： {agents} ，我需要在头部内部随机化...

有什么想法吗？

Answer 1

您可以通过随机函数传递随机一个：

import random

user_agent_list = [
          "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36",
          "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
      ]

def get_headers():
    headers = {'Connection': 'keep-alive',
              'Cache-Control': 'max-age=0',
              'Upgrade-Insecure-Requests': '1',
              'User-Agent': random.choice(user_agent_list),
              'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
              'Accept-Encoding': 'gzip, deflate',
              'Accept-Language': 'en-US,en;q=0.9,fr;q=0.8',
              'referer': 'bing.com'}
    return headers

#then
find_file = (grequests.get(url_a, headers=get_headers(), timeout=None, stream=True, verify=False, allow_redirects=False) for url_a in [f'{site_link}{x}' for x in file_rootl])

其中user_agent_list具有从JSON导入的值。