使仿真具有确定性（基于 qemu 的仿真器）
我正在使用 Marss 周期精确模拟器，它使用 QEMU。它是一个完整的系统模拟器，提供用户和内核统计信息。 然而，即使我只获取用户统计信息，不同运行之间的统计信息也会有很大差异。我在marss网站上问过这个问题，但没有得到很好的答案。 我想知道这是否与qemu有关。或者任何可以使模拟具有确定性的 qemu 选项/变体。 我尝试使用 -icount auto ，但仍然存在一些变化。 对于简单标量 eio 文件，我从未观察到任何变化。 如果您能提供一些帮助，我将不胜感激。
I am using Marss cycle-accurate simulator, which uses QEMU. It is a full system simulator and gives both user and kernel stats.
However, even if I take only user-stats, the statistics vary a lot between different runs. I have asked this question on marss site, but could not get good answer.
I was wondering if it has something to do with qemu. Or any qemu option/variation, that can make simulation deterministic.
I tried using -icount auto and still some variation is there.
With simplescalar eio files, I have never observed any variation.
I would be grateful for some help.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
我没有使用过 MARSS/QEMU，但我熟悉这个问题、一般的模拟器和 SimpleScalar。
一般来说，让完整的系统模拟器具有确定性是非常困难的。例如，如果您的程序打印一天中的时间，并且一天中的时间发生变化，则将遵循不同的代码路径。更一般地说，如果您正在进行完整的系统模拟，并且时间 ls，那么操作系统和用户代码的确切交错将根据磁盘头的位置、磁盘缓存中有多少数据等而有所不同。更不用说什么了系统上正在运行其他进程。
SimpleScalar eio（以及工业中的模拟器）通过记录外部 I/O 事件的跟踪来提供确定性，以便以与之前运行完全相同的指令数应用于模拟。基本上，您只是在录制时进行完整的系统模拟
。您需要为 MARSS/QEMU 寻找类似的选项。不幸的是，我还没有找到这样的。
缺少这些......
但我可以听到你说“我的程序不读取一天中的时间。它不是应该每次都以完全相同的方式运行吗？”也许是这样...但是 (1) 许多 Linux 图书馆在道德上相当于阅读时间。 (2)还有其他进程在运行用户代码。
所以你需要控制跑步。不要一路启动你的虚拟机（不是硬件虚拟机，我的意思只是你在模拟器上运行的操作系统）。停止在单用户状态。编辑 /etc/rc 以停止尽可能多的进程运行。完整的系统模拟与真实机器上的基准测试没有什么不同。在真实的机器上，我们甚至可以在每次运行基准测试时重新启动机器。禁用（虚拟）网络。等等。
如果可以的话，不要在真正的操作系统上运行，而是在最小的监视器上运行。
然而，最好找到 I/O 跟踪重放工具。有时这些隐藏在调试器选项下。
这是 2008 年有关 qemu 中 I/O 重放补丁的参考文献。
http://wiki.qemu.org/Features/FaultTolerance - 进行 I/O 重放以确保可靠性/ 容错。
QEMU 用户空间模拟可能更具可重复性。我不知道它是否可以与 MARSS 一起使用
顺便说一下：再现性问题不仅仅与模拟器有关。它还与可靠性和调试相关。
您也许能够找到一个通用的 Linux 重放工具，您可以使用它来使 QEMU/MARSS 之上的系统更具可重现性。
在 MARSS 内部，您可以使用 ptlcall 检查点工具在基准测试（假设您正在运行 SPEC 等）完成大量初始 I/O 的点之后、下一个 IO 之前获取一个检查点基准测试执行大量初始 I/O，然后在用户代码中运行一段时间，然后执行一些最终 I/O。通过使用这些检查点，您可以避免随时间变化的系统调用。
类似的 Simpoints， http://www.marss86.org/~marss86/index.php/ Simpoints
顺便说一句，我刚刚想起程序可能具有非确定性的另一种方式（除了意外地依赖于时间或多处理器交互）：它们可能显式且本质上依赖于准时。
例如，Langer98 版本的 Livermore Loops 基准测试循环花费的时间，如果花费的时间不足，则会增加循环计数。您可以在此处记录外部交互，例如时间和系统调用，并且程序可能会变得可重现 - 但您不会准确地测量它将在更新、更快的计算机上执行的操作。
相反：假设您有一个编译器优化开关，其形式为“应用这个非常昂贵的优化不超过 2 小时，然后停止......”
I haven't used MARSS/QEMU, but I am familiar with the issue, with simulators in general, and with SimpleScalar.
It is very hard to make full system simulators in general deterministic. E.g. if your program prints the time of day, and the time of day changes, different code paths will be followed. More generally, if you are doing a full system simulation, and time ls, then the exact interleaving of OS and user code will differ based on the position of the disk head, how much data is in the disk cache, etc. Let alone what other processes are running on the system.
SimpleScalar eio (and simulators in industry) provide determinism by recording a trace of external I/O events to apply to the simulation at exactly the same instruction count as in previous runs. Basically, you are only doing full system simulation when you record
You need to look for similar options for MARSS/QEMU. Unfortunately, I have not found such.
Lacking these...
But I can hear you say "My program doesn't read the time of day. Shouldn't it run exactly the same way each time?" Maybe so... but (1) many Linux libraruies do the moral equivalent of reading the time. And (2) there are other processes running user code.
So you need to control the run. Don't boot your virtual machine (not a hardware virtual machine, i mean just the OS you are running on the simulator) all the way up. Stop in singleuser. Edit /etc/rc to stop as many processes as possible running. Full system simulation is not unlike benchmarking on a real machine. And on real machines, we may even reboot the machine afresh each time we run a benchmark. Disable (virtual) networking. Etc.
If you can, don't run on a real OS, but in a minimal monitor.
However, really best to find I/O trace replay facility. Sometimes these hide under debugger options.
Here is a ref that talks, in 2008, about patches for I/O replay in qemu.
http://wiki.qemu.org/Features/FaultTolerance - does I/O replay for reliability / fault tolerance.
QEMU user space emulation may be more reproducible. I do not know if it works in conjunction with MARSS
By the way: the issue of reproducibility is not just relevant to simulators. It is also relevant to reliability, and to debugging.
You may be able to find a generic Linux replay facility, that you can use to make your system on top of QEMU/MARSS more reproducible.
Inside MARSS, you may be able to use the ptlcall checkpoint facilities to take a checkpoint just after the point where your benchmark (assuming you are running SPEC or the like) has done a lot of initial I/O, and before the next I.O. Many benchmarks do a lot of initial I/O, then run in user code for a while, and then do some final I/O. By using these checkpoints, you may be able to avoid time varying system calls.
Similarly Simpoints, http://www.marss86.org/~marss86/index.php/Simpoints
By the way, I was just reminded of another way programs may be non-deterministic (apart from accidentally depending on time, or multiprocessor interactions): they may explicitly and intrinsically depend on time.
E.g. the langer98 version of the Livermore Loops benchmark times how long a loop took, and if insufficient time is spent, ups the loop count. You can record the external interactions, such as time and system calls here, and the program may become reproducible - but you won't be measuring exactly what it will do on your newer, faster, computer.
The converse: imagine that you have a compiler optimization switch of the form "apply this really expensive optimization for no more than 2 hours, then stop..."
QEMU 内置确定性记录和重放
QEMU 2.9.0 现在通过保存外部输入来实现确定性记录和重放，但暂时被破坏：如何使用 QEMU Linux 内核启动的确定性记录和重放功能？
主要文档是：
QEMU built-in deterministic record and replay
QEMU 2.9.0 now kind of has deterministic record and replay by saving external inputs, but it is broken temporarily: How to use QEMU's deterministic record and replay feature for a Linux kernel boot?
The main documentations are: