You are correct, as a general concept, a low latency system will take a shorter amount of time to process a single operation and therefore could process more messages than the same system that exhibits a longer latency.
But in practice, especially in programming, latency of a system can be affected by the throughput. We may need to allow for resources to be cleaned up and to become ready again between cycles, some of these resources may be databases that enforce throttling limits or other processes that themselves have safe operating limits. At some point we will often hit limitations with a given processing model that will force us to change our process.
If we scale out our operator processors over more resources you may observe a significant rise in the cost of processing per message, even then you may still run into maximal throughput issues.
In these systems it is common to observe a pattern of latency increasing as the throughput requirements increases. In these systems low latency can only be affordably observed at low throughput rates.
IoT and realtime processing is a common domain where we may need to achieve a higher throughput than our low latency system can achieve, often this is realized by implementing batch processing.
Batch processing is generally a significantly higher latency than most per message flows, but overall it can allow for processing of a higher volume of messages using less resources.
In a batching system we can tune the throughput by altering the size of the batch, more messages in the batch will mean that those messages will have to wait longer to be processed, so this increases latency, but overall larger batch sizes may increase total throughput.
It's is this batch scenario that this dialog of low latency = low throughput generally comes from. It is alluded to in this clip: https://www.youtube.com/watch?v=PXHLZGp-XMc
It is not that low latency systems can only produce low throughput, but more specifically that low throughput systems can more easily achieve lower latencies.
发布评论
评论（1）
您是正确的，作为一般概念，低延迟系统将花费更短的时间来处理单个操作，因此可以比表现出更长延迟的同一系统处理更多的消息。
但在实践中，尤其是在编程中，系统的延迟可能会受到吞吐量的影响。我们可能需要允许资源在周期之间被清理并再次准备好，其中一些资源可能是强制限制的数据库或本身具有安全操作限制的其他进程。在某些时候，我们经常会遇到给定处理模型的限制，这将迫使我们改变我们的流程。
如果我们将操作员处理器扩展到更多资源，您可能会发现每条消息的处理成本显着上升，即使如此，您仍然可能遇到最大吞吐量问题。
在这些系统中，通常会观察到延迟随着吞吐量要求的增加而增加的模式。在这些系统中，只有在低吞吐率下才能经济地观察到低延迟。
物联网和实时处理是一个常见领域，我们可能需要实现比低延迟系统更高的吞吐量，这通常是通过实施批处理来实现的。
批处理通常比大多数消息流的延迟要高得多，但总的来说，它可以使用更少的资源处理更多的消息。
在批处理系统中，我们可以通过改变批处理的大小来调整吞吐量，批处理中的消息越多意味着这些消息将需要等待更长的时间才能处理，因此这会增加延迟，但总体而言较大的批处理大小可能会增加总吞吐量。
低延迟 = 低吞吐量的对话通常来自这种批处理场景。此剪辑中提到了这一点：https://www.youtube.com/watch? v=PXHLZGp-XMc
You are correct, as a general concept, a low latency system will take a shorter amount of time to process a single operation and therefore could process more messages than the same system that exhibits a longer latency.
But in practice, especially in programming, latency of a system can be affected by the throughput. We may need to allow for resources to be cleaned up and to become ready again between cycles, some of these resources may be databases that enforce throttling limits or other processes that themselves have safe operating limits. At some point we will often hit limitations with a given processing model that will force us to change our process.
If we scale out our operator processors over more resources you may observe a significant rise in the cost of processing per message, even then you may still run into maximal throughput issues.
In these systems it is common to observe a pattern of latency increasing as the throughput requirements increases. In these systems low latency can only be affordably observed at low throughput rates.
IoT and realtime processing is a common domain where we may need to achieve a higher throughput than our low latency system can achieve, often this is realized by implementing batch processing.
Batch processing is generally a significantly higher latency than most per message flows, but overall it can allow for processing of a higher volume of messages using less resources.
In a batching system we can tune the throughput by altering the size of the batch, more messages in the batch will mean that those messages will have to wait longer to be processed, so this increases latency, but overall larger batch sizes may increase total throughput.
It's is this batch scenario that this dialog of low latency = low throughput generally comes from. It is alluded to in this clip: https://www.youtube.com/watch?v=PXHLZGp-XMc