如何使用显示器的输出作为程序的输入？
一些背景：我希望制作一个可以基于视觉输入玩视频游戏的程序。虽然我可以在游戏显示在屏幕上时将网络摄像头对准显示器，但我更愿意通过某种方式将屏幕上的任何像素信息发送到我的程序。
具体来说，我希望以约 30fps 的速率对屏幕截图进行采样并对其进行计算。到目前为止，我唯一能想到的是，每当我的程序“按下”某个键时，使用 Fraps 来截取屏幕截图，但这些屏幕截图的最大速率只能为每秒一张，并且需要使用此外部程序。我希望有某种方法可以更直接地拦截此屏幕信息。
我目前计划使用 Java 和 Matlab 的组合，但我很乐意切换到任何一种能够快速抓取屏幕截图的语言。哦，我正在 Windows 7 上执行此操作，以防此屏幕抓取操作的级别足够低，因此很重要。
Some context: I am hoping to make a program that can play a videogame based on visual input. While I could just point a webcam at my monitor whilst the game is on the screen, I'd prefer to just have some way to send whatever pixel information is going to the screen to my program.
Specifically, I'm hoping to sample screenshots at a rate ~30fps and compute on them. So far the only thing I can think of is to use Fraps to take screenshots whenever my program 'presses' a certain key, but those can only be taken at a maximum rate of one per second and require using this outside program. I'm hoping there is some way to intercept this screen information more directly.
I'm currently planning on using a combination of Java and Matlab, but I'd be happy to switch to whichever language has a nice way to grab screenshots rapidly. Oh, and I'm doing this on Windows 7, in case this screen grabbing operation is low-level enough for that to matter.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（3）
我会尽力回答你的问题。基本答案是屏幕数据本身是内存映射的，因此它驻留在物理内存中的某个位置。但是，您的操作系统可能阻止您通过其虚拟内存系统直接访问该内存。因此，获得访问权限的唯一方法是：
a) 编写设备驱动程序来访问它
或者
b) 使用别人编写的设备驱动程序来访问它
在 Windows 上，您可以使用 DirectX 来访问屏幕数据。此网站说明了如何执行此操作：
http ://www.dgrigoriadis.net/post/2008/01/29/Taking-screenshots-with-DirectX-90.aspx
I'll take a stab at answering your question. The basic answer is that the screen data itself is memory mapped, so it resides somewhere in physical memory. However, your operating system has probably cut you off from accessing that memory directly through its virtual memory system. So, the only way to get access is to either:
a) Write a device driver to access it
or
b) Use a device driver written by someone else to access it
On Windows you could use DirectX to access the screen data. This website explains how to do that:
http://www.dgrigoriadis.net/post/2008/01/29/Taking-screenshots-with-DirectX-90.aspx
如果它是 2D Java 游戏，您只需将其发送到帧缓冲区以及视觉输入程序。如果它是 3D 且使用 Java，您可以使用编写游戏的库中的屏幕截图功能执行类似的操作。如果它是另一个窗口中的游戏，您可以尝试使用
java.awt.Robot(http://download.oracle.com/javase/1,5.0/docs/api/java/awt/Robot.html）并查看屏幕截图功能是否有效。您将获得一个 BufferedImage，您可以将其发送到视觉输入程序（就像您从 Java 2D 或 3D 应用程序发送游戏视图一样）。If it's a 2D Java game, you'd just send it to the framebuffer as well as your visual input program. If it's 3D and in Java, you could possibly do a similar thing with screenshotting functions in the library the game is written in. If it's a game in another window, you could try using
java.awt.Robot(http://download.oracle.com/javase/1,5.0/docs/api/java/awt/Robot.html) and see if the screenshot capability works. You get aBufferedImage, which you would send to the visual input program (like if you were sending the game view from a Java 2D or 3D app).你可以做到，但是需要复杂的图像处理、手势识别和机器学习算法。
如果你想赢得比赛，系统的响应应该是实时的，这自然会让你的研究变得复杂。
但如果原来的游戏是你自己开发的，那么你可能不需要任何图像处理、网络摄像头和FRAPS。在这种情况下，您所需要的只是用于玩游戏的机器学习。
You can do it , But with complex image processing, gesture recognition and machine learning algorithms.
And the response of the system should be real-time if you want to win that game which naturally makes your research complex.
But if the original game is developed by yourself , then you probably don't need any of image processing, ,webcam and FRAPS. In that case all you need is Machine Learning for game playing.