如何让 RoR 图像抓取/解析工具与需要密码才能进入的网站一起工作?
我最近与一名开发人员签约,开发了一个图像抓取工具,类似于 Facebook 的工具,它对于任何不需要密码进入的网站都非常有效,但在不久的将来,我想扩展它的实用性,以跨网站工作,例如ideeli.com、fab.com 或其他需要密码才能进入的网站。
另外,我假设用户在尝试从中抓取任何图像之前已经登录到这些网站之一。
关于如何创建此功能有什么想法吗?
感谢您抽出时间来回答!
I recently contracted a dev to build an image scraping tool, similar to Facebook's, and it works really well for any sites that don't need a password for entry, but in the near future I want to expand its utility to work across sites like ideeli.com, fab.com, or other sites that require a password for entry.
Also, I would assume that a user would already be logged in to one of these sites before they attempt to scrape any images from it.
Any ideas for how to go about creating this functionality?
Thanks for taking the time to answer!
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
我会使用“mechanize”gem (https://github.com/tenderlove/mechanize),它是自动化与网站交互的绝佳工具。
然而,如果您被发现自动执行登录过程,许多网站将禁止您。我过去在使用谷歌时遇到过麻烦。
I'd use the "mechanize" gem (https://github.com/tenderlove/mechanize), it's a great tool for automating interactions with websites.
Many sites, however, will ban you if you're caught automating the login process. I've had trouble with Google in the past.