I/O 完成端口可以帮助数据库而不是文件写入吗？
我正在阅读 IOCP，据我所知，异步写入仅适用于写入文件的上下文。我所说的“文件”并不仅仅指磁盘文件，而是 Windows 上的“文件”类型输出设备。
我计划以某种方式使用 IOCP 来实现一个服务器，该服务器从客户端获取消息，然后将这些消息异步写入数据库（MySQL 或 SQLite）。但是，据我了解，IOCP 中的异步写入涉及将要写入的数据传递给设备驱动程序 - 并且“设备驱动程序”的提及似乎排除了在数据库上使用 IOCP 和异步写入的可能性，因为有从应用程序编写者的角度来看，写入数据库不涉及“设备驱动程序”。
那么，IOCP 真的可以帮助实现写入数据库的服务器吗？我有一种挥之不去的感觉，我误解了某些东西。
如果 IOCP 在这种情况下无法提供帮助，那么对于我应该研究什么来实现在 Windows 上异步写入数据库的服务器，是否有任何建议？
I am reading up on IOCP, and from what I understand so far, the asynchronous writes only apply in the context of writing to Files. By "Files", I don't mean just disk file, but "File" type output devices on Windows.
I am planning to somehow use IOCP in implementing a server that takes messages from clients and then writes those messages asynchronously to database (either MySQL or SQLite). But, from what I understand, async writes in IOCP involves passing the data to be written to a device driver - and the very mention of "device driver" seems to rule out the possibility of using IOCP and async writes on databases, because there is no "device driver" involved in writing to databases from the application writer's point of view.
So, can IOCP actually help in implementing servers that write to database? I have a nagging feeling that I am misunderstanding something.
If IOCP can't help in this case, are there any recommendations on what I should look into for implementing a server that does async writes to database on Windows?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（4）
通常，数据库会为您提供一个 API，您可以使用它来写入数据。这通常隐藏了它所做的所有复杂的事情。有时，该 API 可能是通用的，并且可以与许多数据库一起使用，例如 OLE-DB 和 ODBC，有时该 API 可能是特定于数据库的。
您使用的数据库不太可能公开一个级别足够低以使用 IOCP 的 API，尽管它可能在内部使用 IOCP。
您可能想问的是，我可以使用异步写入来写入我的数据库吗？也就是说，您可以在不阻塞正在执行“触发”的线程的情况下触发数据库写入吗？
虽然当服务器的其余部分使用 IOCP 与其自己的日志文件和套接字读/写进行通信时，IOCP“友好”数据库 API 会很好，但最好的方法是使用线程池来允许您发出阻止数据库 API 所需的调用，而不阻止服务器执行的其余工作。我倾向于将其称为“业务逻辑线程池”，它与我用于所有非阻塞 I/O 的 IOCP 线程池是分开的。
我写了一些关于构建服务器的文章，这些文章使用这种设计来写入数据库，代码和文章链接可以找到 此处。
Normally a database will give you an API that you use to write to it. This usually hides all the complex things it does. Sometimes that API might be generic and work with lots of databases, like OLE-DB and ODBC and sometimes the API might be database specific.
It's unlikely that the database you use will expose an API which is low level enough to use IOCP though it may use IOCP internally.
What you probably want to be asking is, can I use asynchronous writes to write to my database, that is, can you fire off a db write without blocking the thread that's doing the 'firing off'.
Whilst an IOCP 'friendly' database API would be nice when the rest of your server is using IOCP to communicate with its own log files and socket read/writes often the best you can get is to use a thread pool to allow you to issue the blocking calls that the database API requires without blocking the rest of the work that your server does. I tend to call this a 'business logic thread pool' and it's separate to the IOCP thread pool that I use for all my non blocking I/O.
I wrote some articles for building servers that used such a design for writing to databases and the code and links to the articles can be found here.
Io 完成端口是一种通用机制，可以通过多种方式使用来实现可扩展性。
在“最佳”情况下，Io 完成端口与与重叠 Io 一起使用的操作系统句柄相关联。但这实际上根本不是一个要求：Io 完成端口机制足够通用，可以提供可扩展性，即使使用的所有 API 都是阻塞的和/或不公开所需的句柄。
在一个非常简单的模型中，可以使用 CreateIoCompletionPort 调用 PostQueuedCompletionStatus 来创建用户定义的“作业”。创建一个工作线程池，所有工作线程都在
GetQueuedCompletionStatus上循环 - 并在处理排队作业时简单地调用工作线程内的阻塞 Io 例程。每当一个工作线程在任何内核函数上阻塞时，Io 完成端口机制就会发现并发计数较低，并释放另一个工作线程。显然，使用这种方式，活动工作线程的数量可能会超过并发计数，但是，假设作业是对称的，当线程返回到其“GetQueuedCompletionStatus”调用时，这个问题应该会很快自行解决。
Io Completion Ports are a versatile mechanism that can be used in a number of ways to achieve scalability.
In the "best" case the Io Completion Port is associated with the OS handles that are being used with overlapped Io. But this is actually not a requirement at all: the Io Completion Port mechanism is general enough to provide scalability even if all the APIs used are blocking and/or do not expose the handles required.
In a very simple model, one can use
CreateIoCompletionPort- the callPostQueuedCompletionStatusto create user defined "jobs". Create a pool of worker threads all looping onGetQueuedCompletionStatus- and simply call your blocking Io routines inside the worker threads when processing a queued job. Each time one of the worker threads blocks on any kernel function, the Io Completion Port mechanism will see that the concurrency count is low, and release another worker thread.Obviously, used this way, the number of active worker threads can exceed the concurrency count, but, assuming the jobs are symetrical, that should resolve itself quite quickly when the thread returns to its `GetQueuedCompletionStatus' call.
当您写入文件并且不想在任何情况发生时阻塞时，它们可以在任何地方提供帮助。很难想象数据库会需要这样的功能，除非编写完整性并不重要的文本日志文件。
They can help anywhere when you write to a file and don't want to block while any of it happens. It's hard to imagine a database wanting that, except maybe in writing text log files whose integrity doesn't really matter.
IOCP 在实现数据库时可以提供很大帮助。
IOCP 与 FILE_FLAG_NO_BUFFERING 和 FILE_FLAG_WRITE_THROUGH 以及正确对齐的块相结合，使数据库引擎可以控制缓存行为，避免不必要的重复缓存和块副本，以正确的顺序进行写入，控制哪些写入可以同时进行，等等
。当然，需要实现数据库才能使用这些功能，使用 SQLite 和 Mysql，您可以获得其他东西。
有关这如何帮助实现数据库的更多详细信息，Gray 和 Reuter 撰写的“事务处理：概念和技术”是一个很好的参考资料。
IOCP can help a great deal when implementing a database.
IOCP combined with FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH and correctly aligned blocks lets the database engine control cache behaviour, avoid unnecessary duplicate caching and block copies, get writes in the correct order, control which writes can be in-flight at any the same time, etc.
Of course the database needs to be implemented to use these features, with SQLite and Mysql you get something else.
For more details on how this would help with implementing a database, "Transaction Processing: Concepts and Techniques" by Gray and Reuter is an excellent reference.