如何学习鼠标移动？
在过去的几周里，我一直在尝试开发一种在我的应用程序中合成类人鼠标运动的方法。一开始，我使用了多项式和样条插值等简单技术，但是即使有一点噪音，结果仍然无法显得足够像人类。
为了解决这个问题，我一直在研究将机器学习算法应用于真实人类鼠标运动生物识别的方法，以便通过学习记录的真实人类运动来合成鼠标运动。用户将编译记录的运动的配置文件，该配置文件将训练程序用于合成目的。
我已经搜索了几周并阅读了几篇关于逆向生物识别技术在生成小鼠动力学中的应用的文章，例如 小鼠动力学的逆向生物识别;然而，他们倾向于从随机生成动态中生成现实时间，而我希望生成一条从具体 A 到B. 另外，我实际上仍然需要想出一条路径，而不仅仅是从一条路径测量的一些动态。
有人有一些建议可以帮助菜鸟吗？
目前，测试是通过记录动作并让我和其他几位开发人员观看回放来完成的。理想情况下，该运动将能够欺骗自动生物识别分类器以及真实的、活生生的、会呼吸的智人。
I've been attempting to develop a means of synthesizing human-like mouse movement in an application of mine for the past few weeks. At the start I used simple techniques like polynomial and spline interpolation, however even with a little noise the result still failed to appear sufficiently human-like.
In an effort to remedy this issue, I've been researching into ways of applying machine learning algorithms on real human mouse movement biometrics in order to synthesize mouse movements by learning from recorded real human ones. Users would be compiling a profile of recorded movements that would trainh= the program for synthesis purposes.
I've been searching for a few weeks and read several articles on application of inverse biometrics in generating mouse dynamics, such as Inverse Biometrics for Mouse Dynamics; they tend to focus, however, on generating realistic time from randomly-generated dynamics, while I was hoping to generate a path from specifically A to B. Plus, I still need to actually need to come up with a path, not just a few dynamics measured from one.
Does anyone have a few pointers to help a noob?
Currently, testing is done by recording movements and having I and several other developers watch the playback. Ideally the movement will be able to trick both an automatic biometric classifier, as well as a real, live, breathing Homo sapien, too.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（7）
菲特定律很好地估计了定位鼠标指针所需的时间。在 推导 部分中有一个简单的解释，我认为您可以将其用作一个您的应用程序的基本构建块。从大的动作开始，在动作的方向和长度上增加一些不准确的地方，然后做一个较小的修正动作，依此类推......
Fitt's law gives a very good estimation of the time needed to position the mouse pointer. In the derivation section there is a simple explanation I think you could use this as one of the basic building blocks of your app. Start with big movements, put some inacurracy both in the direction and the length of the movement, then do a smaller correction movement and so on...
首先，我猜你记录了人类鼠标从 A 到 B 的运动。因为否则，尝试合成这种运动的模型对我来说似乎是不可能的。
其次，如何测量与“直接”路径的偏差，也许与时间有关。我实际上怀疑不同角度、路径长度等的运动看起来不同，但也许你可以先尝试标准化模型，你只需拉伸（在空间和时间上）并根据需要旋转即可。
第三，学习。最简单的事情就是只收集真实动作的集合（以我上面讨论的形式），并从该集合中取样。评估一下它看起来怎么样。如果你真的想要一个概率模型，那么你必须评估什么样的模型适合。使用从训练集中学习其参数的高斯噪声来模糊直接路径是否足够？或者一些（正弦）波浪偏差？或者“接近按钮”和“最终修正”的单独模型。费茨定律可能对评估有用。
First, i guess you record human mouse movements from A to B. Because otherwise, trying to synthesize a model for such movement does not seem possible to me.
Second, how about measuring the deviations from the "direct" path, maybe in relation to time. I actually suspect that movements look different for different angles, path lengths etc., but maybe you can try a normalized model first, that you just stretch (in space and time) and rotate like you need it.
Third, the learning. The easiest thing would be to just have a collection of real moves (in the form i discussed above), and sample from that collection. Evaluate how that looks like. If you really want a probabilistic model, then you have to evaluate what kind of models fit. is it enough to blurr the direct path with gaussian noise whose parameters you learn from your training set? Or some (sin-)wavy deviation? Or seperate models for "getting near the button" and "final corrections". Fitts law might be useful for evaluation.
这个问题让我想起了几年前知道的一个网站，所以我访问了它并发现
时间如此相似，让我觉得这个问题在某种程度上是相关的。事实上，该线程中的某人链接到了您所做的同一篇文章。如果不相关，那么，有一个链接指向很多人正在讨论您正在思考的内容。
This question reminded me of a website I knew about years ago, so I visited it and found this in-depth discussion on the topic.
The timing is so similar as to make me think this question is related in some way. In fact, someone in the thread linked to the same article you did. If it's not related, well, there's a link to a lot of people discussing exactly what you're thinking about.
我认为问题没有那么明确。有一个重要的概念到目前为止还没有提到，那就是上下文。 Chrome 获得焦点时屏幕上的鼠标移动与 Vim 获得焦点时的移动有很大不同。
I don't think the problem is all that well defined. There is a important notion not mentioned so far, which is context. The mouse movement on my screen when Chrome has focus is massively different that the motion when Vim has focus.
鼠标移动的方式根据设备类型、操作类型、涉及的 UI 元素、对 UI 的熟悉程度、用户尝试完成任务的速度、用户的技能、初始失败而有所不同用户的行为（例如误点击）、用户的情绪状态（以及许多其他因素）。您是否计划创建多种路径策略来对应不同的环境？另外，你对你试图欺骗的算法了解多少？我假设不会广泛使用，或者您只需直接针对该算法进行编程。
如果人类正在查看路径，他们可能能够识别与路径策略相关的状态，并且如果他们将其识别为人类状态（例如，用户匆忙、错过点击、快速关闭），则可能更容易被愚弄由此产生的弹出窗口，再次尝试时速度会更慢）。 UI 的作用不仅仅是大小和位置。我经常快速指向工具栏，然后滑动选项，直到到达目标。另一个例子是，当我扫描目标或将鼠标悬停在正在阅读的文本上时，我通常会在菜单项上暂停。您是否试图模仿人类行为或只是模仿他们的鼠标运动（因为我认为它们是在臀部连接的）？
The way a mouse moves varies based on the type of the device, the type of action, the UI elements involved, familiarity with the UI, the speed at which the user is attempting to complete their task, the skill of the user, initial failure of the user (eg miss-clicks), the user's emotional state (as well as many other factors). Do you plan on creating several pathing strategies to correspond to different contexts? Also how well do you know the algorithm you are trying to fool? I assume not extensively or you would simply program directly against that algorithm.
If a human is looking at the pathing, they might be able to identify the state associated with a pathing strategy and may be more inclined to be fooled if they identify it as a human state (eg user is rushing, miss-clicks, quickly closes a resulting popup, tries again slower). UI comes into play with not just size and position. I often quickly point to a toolbar, then slide across the options until I get to my target. Another example is that I typically pause on menu items while I am scanning for my target or hover over text I am reading. Are you attempting to emulate human behavior or just their mouse movements (because I think they are joined at the hip)?
您是否想模拟类似人类的鼠标移动，因为您正在为您的游戏进行实时在线训练？如果您的训练序列是静态的，只需记录您的鼠标移动，并在每次单击鼠标按钮时播放鼠标单击声音效果。没有任何鼠标移动比您自己的鼠标移动更让您感觉“足够真实”。
就我个人而言，我觉得软件专家在培训视频中移动鼠标的速度太快了。我更喜欢我见过的截屏视频软件所采用的方法，它总是从 A 点线性移动鼠标 --> A 点。 B. 诀窍是，视频中的每次鼠标移动都花费相同的时间，无论距离如何，例如 3/4 秒，然后是鼠标点击音效。
我相信他们以这种方式移动鼠标，因为这样观看者就可以通过鼠标开始移动的方向和速度来预测鼠标的着陆区域。在训练情况下，我认为像这样的常规动作对眼睛来说更温和，也许更容易记住/回忆。
Are you wanting to simulate human-like mouse movement because you are doing real-time online training for your game? If your training sequences are static, just record your mouse movements and play a mouse clicking sound effect whenever you click the mouse button. No mouse movement is going to feel "real enough" to you more than your own.
Personally, I feel experts in software move their mice too quickly in training videos. I prefer an approach taken by screencast video software I've seen that always moves the mouse linearly from point A --> B. The trick was, every mouse move made in the video always took the same amount of time regardless of distance, say 3/4 of a second and then followed by a mouse click sound effect.
I believe they moved the mouse in this way because then the viewer could anticipate the landing area of the mouse by the direction and velocity the mouse moved at the start. In a training situation, I suppose that regular movements like this are gentler on the eye and perhaps easier to retain/recall.
您是否考虑过在应用程序中添加鼠标跟踪，以便从本质上记录用户如何移动鼠标，然后分析记录？
我最近没有研究过这个问题，但我相信 Swing 应用程序中的 MouseListener 可以获取您需要的信息。
Have you considered adding mouse tracking to your application so you essentially record how the user moves the mouse and then analyze the recordings?
I have not looked into this recently but I believe that a MouseListener in a Swing application get the information you need.