如何使用 Python 向服务器发送 URL 请求而不实际打开浏览器（“不使用 webbrowser 模块”）？

Question

我想在登录时将此 URL 作为请求发送到服务器以更改网站上的某些内容。问题是，当我使用 mechanize 或 urllib2 打开该 URL 时，它不会更改网站上的任何内容。然而，当我使用网络浏览器模块时，它确实改变了网站上的内容。我想做 webbrowser 模块所做的事情，但不打开实际的浏览器。有办法做到这一点吗？为什么 mechanize 和 urllib2 不起作用？

编辑：我所说的“网站更改”是指我在网站上发布的信息得到了称为“共享”和“门票”的东西。我的程序找到准确的信息（如果是假的，他们会把你踢出去）并使用 URL 将其“插入”到网站中。

示例 URL（我的所有其他人都遵循此格式）：

http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=10206 34218&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%2Ffamily-dollar-store-1349194%3Flid%3D1349194

机械化代码：

import mechanize
br = mechanize.Browser()
url = http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1020634218&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%2Ffamily-dollar-store-1349194%3Flid%3D1349194
br.open(url)

urllib2 代码：

from urllib2 import urlopen
url = http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1020634218&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%2Ffamily-dollar-store-1349194%3Flid%3D1349194
page = urllib2.urlopen(url)
page.read()

webbrowser 代码：

import webbrowser
url = http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1020634218&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%2Ffamily-dollar-store-1349194%3Flid%3D1349194
webbrowser.open(url)

编辑#2 我刚才尝试了这段代码：

import urllib2
import urllib

def log_in():
    url = 'https://www.locationary.com/index.jsp?ACTION_TOKEN=tile_loginBar_jsp$JspView$LoginAction'
    values = {'inUserName' : '[email protected]',
              'inUserPass' : 'myPass'}
    data = urllib.urlencode(values)
    req = urllib2.Request(url, data)
    req.add_header('Host', 'www.locationary.com')
    req.add_header('User-Agent', 'Mozilla/5.0 (Windows NT 6.1; rv:8.0) Gecko/20100101 Firefox/8.0')
    req.add_header('Accept', 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8')
    req.add_header('Accept-Language', 'en-us,en;q=0.5')
    req.add_header('Accept-Encoding','gzip, deflate')
    req.add_header('Accept-Charset','ISO-8859-1,utf-8;q=0.7,*;q=0.7')
    req.add_header('Connection','keep-alive')
    req.add_header('Referer','http://www.locationary.com/')
    req.add_header('Cookie','site_version=REGULAR; __utma=47547066.1079503560.1321924193.1322707232.1324693472.36; __utmz=47547066.1321924193.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); nickname=jacob501; locaCountry=1033; locaState=1795; locaCity=Montreal; jforumUserId=1; PMS=1; TurnOFfTips=true; Locacookie=enable; __utma=47547066.1079503560.1321924193.1322707232.1324693472.36; __utmz=47547066.1321924193.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); nickname=jacob501; PMS=1; __utmb=47547066.15.10.1324693472; __utmc=47547066; JSESSIONID=DC7F5AB08264A51FBCDB836393CB16E7; PSESSIONID=28b334905ab6305f7a7fe051e83857bc280af1a9; __utmc=47547066; __utmb=47547066.15.10.1324693472; ACTION_RESULT_CODE=ACTION_RESULT_FAIL; ACTION_ERROR_TEXT=java.lang.NullPointerException')
    req.add_header('Content-Type','application/x-www-form-urlencoded')
    response = urllib2.urlopen(req)
    page = response.read()

url2 = 'http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1020634218&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%2Ffamily-dollar-store-1349194%3Flid%3D1349194'

log_in()
response2 = urllib2.urlopen(url2)
page2 = response2.read()

但它不起作用。

编辑3：托尼的代码对我不起作用。

import urllib2
import urllib
import cookielib

data = urllib.urlencode({"inUserName":"MYUSERNAMESHOULDBEHERE", "inUserPass":"MYPASSWORDSHOULDBEHERE"})
jar = cookielib.FileCookieJar("cookies")
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(jar))
request = urllib2.Request("https://www.locationary.com/index.jsp?ACTION_TOKEN=tile_loginBar_jsp$JspView$LoginAction", data)
opener.open(request) 
url = "http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1012432546&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%2Fdennys-13470813%3Flid%3D13470813"
anything = opener.open(url)
anything.read()

最终编辑！ 我终于按照托尼的建议让它工作了！

这是我最终有效的代码：

import urllib2
import urllib
import cookielib

data = urllib.urlencode({"inUserName":"[email protected]", "inUserPass":"mypassword"})
jar = cookielib.FileCookieJar("cookies")
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(jar))
opener.addheaders.append(('User-agent', 'Mozilla/4.0'))
opener.addheaders.append( ('Referer', 'http://www.hellboundhackers.org/index.php') )
opener.addheaders.append(('Cookie','site_version=REGULAR; __utma=47547066.912030359.1322003402.1324688192.1324930160.55; __utmz=47547066.1324655802.52.13.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=cache:dr23PN5fUj4J:www.locationary.com/%20locationary; nickname=jacob501; jforumUserId=1; PMS=1; locaCountry=1033; locaState=1786; locaCity=Vancouver; JSESSIONID=A8F241E1924CE7A25FAA8C5CA6597697; PSESSIONID=5c21c44245f978b917f17982c944a9ec2b5d2df5; Locacookie=enable; __utmb=47547066.5.10.1324930160; __utmc=47547066'))
request = urllib2.Request("https://www.locationary.com/index.jsp?ACTION_TOKEN=tile_loginBar_jsp$JspView$LoginAction", data)
response = opener.open(request) 
url = "http://www.locationary.com/"
anything = opener.open(url)
anything.read()

我所要做的就是添加行

opener.addheaders.append(('Cookie','site_version=REGULAR; __utma=47547066.912030359.1322003402.1324688192.1324930160.55; __utmz= 

等等（非常长的代码行，cookie），

我还添加了“Referer”和“User-Agent”标头以防万一。

谢谢托尼！！

Answer 1

首先，您应该用引号编写 url 变量：

url = "http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1020634218&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%2Ffamily-dollar-store-1349194%3Flid%3D1349194"

如果您想在不打开浏览器的情况下发送请求，您可以使用 urllib，就像您一样。

如果你需要身份验证（看起来像你这样做），你应该发送身份验证请求，获取cookie（使用cookielib.FileCookieJar）并将它们设置在opener中。然后您将能够打开页面并发送请求。

大约你需要这样的东西：

data=urllib.urlencode({"login":"your login or whatever, "pass":"password}) # be aware you need to change "login" and "pass" to names of fields in form you have.
jar = cookielib.FileCookieJar("cookies")
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(jar))
request = urllib2.Request("url for authentication", data)
opener.open(request) # now you should be authorized and able to send any request like logged in user, using opener

url = "http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=1020634218&xxx_c_1_f_987=http%3A%2F%2Fwww.yellowpages.com%2Fpittsburgh-pa%2Fmip%2Ffamily-dollar-store-1349194%3Flid%3D1349194"
anything = opener.open(url)
anything.read()

Answer 2

{"manifest":{"errorTimeout":0,"succeed":true,"errorCode":0,"serverVersion":"1.0","type":"locaaccess"},"saveResult":{"message":"You don't have permissions!","placeOpenedState":0,"isSucess":false}} 

我是通过将你的 urllib 放入我的浏览器中得到的。您需要首先在我相信的网站上验证自己的身份，然后执行此命令。我无法向您提供有关如何登录该网站的说明，但如果您转到登录页面，它可能有一个表单，您可以通过 urllib2 通过 url post 来模仿