在 C# 中使用 Selenium RemoteWebDriver

Question

我正在尝试在 C# 中使用 Selenium RemoteWebDriver，基本上我想做的就是在 javascript 完成操作 DOM 后以编程方式抓取网页的 html（不会弹出浏览器窗口）

。我像这样启动了selenium-server.jar：

C:\Program Files\selenium-server>java -jar "C:\Program Files\selenium-server\selenium-server.jar"

13:34:46.163 INFO - Java: Sun Microsystems Inc. 19.1-b02
13:34:46.166 INFO - OS: Windows 7 6.1 amd64
13:34:46.174 INFO - v2.0 [a2], with Core v2.0 [a2]
13:34:46.277 INFO - RemoteWebDriver instances should connect to: **http://127.0.0.1:4444/wd/hub**
13:34:46.278 INFO - Version Jetty/5.1.x
13:34:46.279 INFO - Started HttpContext[/selenium-server/driver,/selenium-server
/driver]
13:34:46.280 INFO - Started HttpContext[/selenium-server,/selenium-server]
13:34:46.280 INFO - Started HttpContext[/,/]
13:34:46.311 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler@6019d0
a1
13:34:46.312 INFO - Started HttpContext[/wd,/wd]
13:34:46.316 INFO - Started SocketListener on 0.0.0.0:4444
13:34:46.316 INFO - Started org.openqa.jetty.jetty.Server@199a0c7c

接下来我尝试从测试用例运行此行：

var driver = new RemoteWebDriver(new Uri("http://127.0.0.1:4444/wd/hub"), DesiredCapabilities.Chrome());

此行错误：

Test 'Housters.Test.ScrapeTest.TestSelenium' failed: OpenQA.Selenium.WebDriverException : Unexpected error. {"message":"java.lang.NullPointerException","localizedMessage":"java.lang.NullPointerException","cause":{"class":"java.lang.NullPointerException","stackTrace":[{"fileName":"DriverFactory.java","class":"java.lang.StackTraceElement","lineNumber":43,"className":"org.openqa.selenium.remote.server.DriverFactory","methodName":"getBestMatchFor","nativeMethod":false},{"fileName":"DriverFactory.java","class":"java.lang.StackTraceElement","lineNumber":76,"className":"org.openqa.selenium.remote.server.DriverFactory","methodName":"newInstance","nativeMethod":false},{"fileName":"Session.java","class":"java.lang.StackTraceElement","lineNumber":48,"className":"org.openqa.selenium.remote.server.Session$1","methodName":"call","nativeMethod":false},{"fileName":"Session.java","class":"java.lang.StackTraceElement","lineNumber":46,"className":"org.openqa.selenium.remote.server.Session$1","methodName":"call","nativeMethod":false},{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.util.concurrent.FutureTask$Sync","methodName":"innerRun","nativeMethod":false},{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.util.concurrent.FutureTask","methodName":"run","nativeMethod":false},{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.util.concurrent.ThreadPoolExecutor$Worker","methodName":"runTask","nativeMethod":false},{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.util.concurrent.ThreadPoolExecutor$Worker","methodName":"run","nativeMethod":false},{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.lang.Thread","methodName":"run","nativeMethod":false}]},"class":"java.util.concurrent.ExecutionException","stackTrace":[{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.util.concurrent.FutureTask$Sync","methodName":"innerGet","nativeMethod":false},{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.util.concurrent.FutureTask","methodName":"get","nativeMethod":false},{"fileName":"Session.java","class":"java.lang.StackTraceElement","lineNumber":68,"className":"org.openqa.selenium.remote.server.Session","methodName":"execute","nativeMethod":false},{"fileName":"Session.java","class":"java.lang.StackTraceElement","lineNumber":54,"className":"org.openqa.selenium.remote.server.Session","methodName":"<init>","nativeMethod":false},{"fileName":"DriverSessions.java","class":"java.lang.StackTraceElement","lineNumber":76,"className":"org.openqa.selenium.remote.server.DriverSessions","methodName":"newSession","nativeMethod":false},{"fileName":"NewSession.java","class":"java.lang.StackTraceElement","lineNumber":46,"className":"org.openqa.selenium.remote.server.handler.NewSession","methodName":"handle","nativeMethod":false},{"fileName":"ResultConfig.java","class":"java.lang.StackTraceElement","lineNumber":144,"className":"org.openqa.selenium.remote.server.rest.ResultConfig","methodName":"handle","nativeMethod":false},{"fileName":"DriverServlet.java","class":"java.lang.StackTraceElement","lineNumber":271,"className":"org.openqa.selenium.remote.server.DriverServlet","methodName":"handleRequest","nativeMethod":false},{"fileName":"DriverServlet.java","class":"java.lang.StackTraceElement","lineNumber":256,"className":"org.openqa.selenium.remote.server.DriverServlet","methodName":"doPost","nativeMethod":false},{"fileName":"HttpServlet.java","class":"java.lang.StackTraceElement","lineNumber":727,"className":"javax.servlet.http.HttpServlet","methodName":"service","nativeMethod":false},{"fileName":"HttpServlet.java","class":"java.lang.StackTraceElement","lineNumber":820,"className":"javax.servlet.http.HttpServlet","methodName":"service","nativeMethod":false},{"fileName":"ServletHolder.java","class":"java.lang.StackTraceElement","lineNumber":428,"className":"org.openqa.jetty.jetty.servlet.ServletHolder","methodName":"handle","nativeMethod":false},{"fileName":"ServletHandler.java","class":"java.lang.StackTraceElement","lineNumber":677,"className":"org.openqa.jetty.jetty.servlet.ServletHandler","methodName":"dispatch","nativeMethod":false},{"fileName":"ServletHandler.java","class":"java.lang.StackTraceElement","lineNumber":568,"className":"org.openqa.jetty.jetty.servlet.ServletHandler","methodName":"handle","nativeMethod":false},{"fileName":"HttpContext.java","class":"java.lang.StackTraceElement","lineNumber":1530,"className":"org.openqa.jetty.http.HttpContext","methodName":"handle","nativeMethod":false},{"fileName":"HttpContext.java","class":"java.lang.StackTraceElement","lineNumber":1482,"className":"org.openqa.jetty.http.HttpContext","methodName":"handle","nativeMethod":false},{"fileName":"HttpServer.java","class":"java.lang.StackTraceElement","lineNumber":909,"className":"org.openqa.jetty.http.HttpServer","methodName":"service","nativeMethod":false},{"fileName":"HttpConnection.java","class":"java.lang.StackTraceElement","lineNumber":820,"className":"org.openqa.jetty.http.HttpConnection","methodName":"service","nativeMethod":false},{"fileName":"HttpConnection.java","class":"java.lang.StackTraceElement","lineNumber":986,"className":"org.openqa.jetty.http.HttpConnection","methodName":"handleNext","nativeMethod":false},{"fileName":"HttpConnection.java","class":"java.lang.StackTraceElement","lineNumber":837,"className":"org.openqa.jetty.http.HttpConnection","methodName":"handle","nativeMethod":false},{"fileName":"SocketListener.java","class":"java.lang.StackTraceElement","lineNumber":245,"className":"org.openqa.jetty.http.SocketListener","methodName":"handleConnection","nativeMethod":false},{"fileName":"ThreadedServer.java","class":"java.lang.StackTraceElement","lineNumber":357,"className":"org.openqa.jetty.util.ThreadedServer","methodName":"handle","nativeMethod":false},{"fileName":"ThreadPool.java","class":"java.lang.StackTraceElement","lineNumber":534,"className":"org.openqa.jetty.util.ThreadPool$PoolThread","methodName":"run","nativeMethod":false}]}
    at OpenQA.Selenium.Remote.RemoteWebDriver.UnpackAndThrowOnError(Response errorResponse)
    at OpenQA.Selenium.Remote.RemoteWebDriver.Execute(DriverCommand driverCommandToExecute, Dictionary`2 parameters)
    at OpenQA.Selenium.Remote.RemoteWebDriver.StartSession(ICapabilities desiredCapabilities)
    at OpenQA.Selenium.Remote.RemoteWebDriver..ctor(Uri remoteAddress, ICapabilities desiredCapabilities)
    ScrapeTest.cs(36,0): at Housters.Test.ScrapeTest.TestSelenium()

在服务控制台窗口中，它显示此错误：

13:44:55.558 INFO - WebDriver remote server: Executing: [new session: null] at URL: /session)
13:44:55.560 INFO - WebDriver remote server: Exception: java.lang.NullPointerException

我正在尝试从 Windows 7 x64 执行此操作。我做错了什么？对于我想做的事情来说，这似乎是很多工作......

Answer 1

我看到你已经有了一个被接受的答案，但因为它回答了完全不同的问题，我会加上我的 5 美分。

因此，如果您想连接到删除网络驱动程序，而不是行：

var driver = new FirefoxDriver();

您必须执行以下操作：

var driver = new RemoteWebDriver(new Uri("http://localhost:4444/wd/hub"), DesiredCapabilities.Firefox()); // instead of this url you can put the url of your remote hub

不要忘记在上一行之前包含远程命名空间：

using OpenQA.Selenium.Remote;

Answer 2

如果您尝试在本地主机上运行 Selenium2，则不需要使用RemoveWebDriver() 和 Selenium Server - 您可以使用以下内容：

WebDriver driver = new ChromeDriver();

我发现以这种方式在本地运行的问题比使用RemoveWebDriver时要少，您应该会发现您如果出现问题，可以随时获取更多信息。

或者，您可以直接使用 HtmlUnit，如 http://blog.stevensanderson.com/2010/03/30/using-htmlunit-on-net-for-headless-browser-automation/

Answer 3

获取您感兴趣的页面的 HTML 很容易，特别是在 .NET 中（使用 HttpWebRequest）。将其解析为类似 DOM 的结构，并在页面上应用 JavaScript 指示的转换，但没有那么多。它需要一个浏览器，或者至少需要一个带有 DOM 构建引擎的 HTML 解析器，以及一个 JS 脚本引擎来操作生成的 DOM。这不是一个小问题。此时，您的选择是：（1）使用 HtmlUnit、env.js 或其他“无头”浏览器项目之一，这些项目都不需要 WebDriver 来完成您想做的事情；或 (2) 弹出浏览器窗口。对于 (2) 的情况，使用原始 InternetExplorerDriver 或 FirefoxDriver（或即将推出的 2.0b4 中的 ChromeDriver）是比使用远程服务器更简单的选择。

Answer 4

自 Chrome 59 推出以来，您可以在无头模式下运行浏览器。请参阅此帖子