对 Web 表单进行逆向工程
我有一个网站，我从中下载 2-3 MB 的原始数据，然后将其输入 ETL 流程，将其加载到我的数据集市中。 不幸的是，数据提供者是美国农业部 (USDA)，他们不允许通过 FTP 下载。 他们要求我使用网络表单来选择我想要的元素，点击 2-3 个屏幕，最后点击下载文件。 我想自动化这个下载过程。 我不是网络开发人员，但不知何故，我似乎应该能够使用某种工具来准确地告诉我从最终请求到服务器的 put/get/magic 的内容。 如果我有一个工具说，“将这些参数传递到这个 url 并等待响应”，那么我就可以在 Perl 中将一些东西组合在一起来自动化这个过程。
我意识到，如果我解构他们的所有 5 个页面并通读 JavaScript 包含内容并点击我的治疗 3 次，我就可以从我有权访问的内容中获取此信息。 但我想要一个更快、更直接的路径，不需要我手动解析他们所有的 JS。
重述最后一个问题：是否有一种工具或方法可以清楚地显示从 Web 表单发送的最终请求是什么以及它的结构如何？
I have a web site which I download 2-3 MB of raw data from that then feeds into an ETL process to load it into my data mart. Unfortunately the data provider is the US Dept. of Ag (USDA) and they do not allow downloading via FTP. They require that I use a web form to select the elements I want, click through 2-3 screens and eventually click to download the file. I'd like to automate this download process. I am not a web developer but somehow it seems that I should be able to use some tool to tell me exactly what put/get/magic goes from the final request to the server. If I had a tool that said, "pass these parameters to this url and wait for a response" I could then hack something together in Perl to automate this process.
I realize that if I deconstructed all 5 of their pages and read through the JavaScript includes and tapped my heals together 3 times I could get this info from what I have access to. But I want a faster and more direct path that does not require me to manually parse all their JS.
Restatement of the final question: Is there a tool or method that will show clearly what the final request request sent from a web form was and how it was structured?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（5）
篡改者最好的朋友（这些是 Firefox 扩展，您也可以使用 Wireshark 之类的东西）
HTTPFox
篡改数据
祝你好运
A tamperer's best friends (these are firefox extensions, you could also use something like Wireshark)
HTTPFox
Tamper Data
Best of luck
使用 Fiddler2 作为代理来查看来回传递的内容。 我在其他类似情况下成功做到了这一点
主页在这里： http://www.fiddler2.com /fiddler2/
Use Fiddler2 as a proxy to see what is being passed back and forth. I've done this with success in other similar circumstances
Home page is here: http://www.fiddler2.com/fiddler2/
与其他回复一样，除了我选择的工具是 Charles
As with the other responses, except my tool of choice is Charles
使用 Web 测试工具包怎么样，例如 Watir 和 Ruby ？
很容易填写表格..只需使用输出..
What about using a web testing toolkit, like Watir and Ruby ?
Easy to fill in the forms.. just use the output..
使用WatiN并将其与WatiN TestRecorder（Google for it）结合起来
它可以“模拟”坐在前面的用户浏览器输入您可以从自己的 C# 代码提供的值...
Use WatiN and combine it with WatiN TestRecorder (Google for it)
It can "simulate" a user sitting in front of the browser punching in values which you can supply from your own C# code...