在屏幕上绘制许多对象
我正在开展一个项目，我们需要以热图的形式汇总大量数据。这些数据将尽可能长时间地保存在数据库中。在某些时候，我们需要将摘要存储在矩阵中（可能？），然后才能将热图的块绘制到屏幕上。我们正在使用 C# 创建一个 Windows 窗体应用程序。
我们假设热图将总结在线地图程序（例如谷歌地图）的日志文件。它将根据向特定地址或区域发出请求的次数为该区域/地址分配颜色。它可以总结不同详细程度的数据。也就是说，热图上的每个块都可以汇总特定地址的数据（最大细节，因此数十亿/数百万个块），或者它可以汇总对街道、城市或国家的请求（最小细节 - 很少的块，因为它们每个代表一个国家）。想象一下，有数百万个地址请求。我们考虑过用数据库来总结这一点。问题是我们需要在屏幕上绘制如此多的块（最多数十亿，但通常要少得多）。我们假设这些数据被汇总在一个数据库表中，该表存储了较大区域的命中数。我们是否可以将块绘制到窗口，而不为每个区域构造一个对象，甚至不引入数据库表中的所有信息？这是我最关心的问题，因为如果我们确实构建了一个矩阵，对于要求较高的请求来说，它可能需要 10 GB 左右。
我很想知道我们可以在屏幕上绘制多少块以及最好的方法是什么（即 direct3d、XNA）。从上图可以看出，范围会有很大差异，我们预计需要绘制数十亿个正方形。我们将有一个垂直滚动条，可以快速向下滚动以查看其他块。
总的来说，我想知道我们如何使用 C# 来实现这一目标？为苛刻的请求创建矩阵可能需要大约 10 GB。有没有一种方法可以在不需要大量内存的情况下绘制到屏幕上（即为每个块创建一个对象）。如果我们可以将 SQL 查询的结果直接转换为屏幕上的渲染块，那将是理想的（即不构造对象等）。我们需要的只是正方形，它们唯一的属性是颜色，我们可能需要为每个块维护一个数字。
笔记： 我们非常确定如何绘制热图（如何向用户显示缩放、滚动等）。澄清一下，我更关心我们如何实现我们的想法。是否有一个库或某种方法可以让我们绘制这么多对象，而无需构建十亿个对象并使用千兆字节的数据。每个块本质上是一组相同颜色的像素 (20x20)。我不认为这需要建造 10 亿个物体。
谢谢！
I'm working on a project in which we need to summarize a substantial amount of data in the form of a heat map. This data will be kept in a database for as long as possible. At some point, we will need to store a summary in a matrix (possibly?) before we can draw the blocks for the heat map to the screen. We are creating a windows form application with C#.
Let's assume the heat map is going to summarize a log file for an online mapping program such as google maps. It will assign a color to a particular address or region based on the number of times a request was made to that region/address. It can summarize the data at differing levels of detail. That is, each block on the heat map can summarize data for a particular address (max detail, therefore billions/millions of blocks) or it can summarize for requests to a street, city, or country (minimum detail -- few blocks as they each represent a country). Imagine that millions of requests were made for addresses. We have considered summarizing this with a database. The problem is that we need to draw so many blocks to the screen (up to billions, but usually much less). Let's assume this data is summarized in a database table that stores the number of hits to the larger regions. Can we draw the blocks to the window without constructing an object for each region or even bringing in all of the information from the db table? That's my primary concern, because if we did construct a matrix, it could be around 10 GB for a demanding request.
I'm curious to know how many blocks we can draw to the screen and what the best approach to this may be (i.e. direct3d, XNA). From above, you can see the range will vary substantially and we expect the potential for billions of squares that need to be drawn. We will have a vertical scroll bar to scroll down quickly to see other blocks.
Overall, I'm wondering how we might accomplish this with C#? Creating the matrix for the demanding request could require around 10 Gigabytes. Is there a way to draw to the screen that will not require a substantial amount of memory (i.e. creating an object for each block). If we could have the results of a SQL query be translated directly into rendered blocks on the screen, that would be ideal (i.e. not constructing objects, etc etc). All we need are squares and their only property is color and we might need to maintain a number for each block.
Note:
We are pretty sure about how we will draw the heat map (how zooming, scrolling, etc should appear to user). To clarify, I'm more concerned about how we will implement our idea. Is there a library or some method that allows us to draw this many objects without constructing a billion objects and using Gigabytes of data. Each block is essentially a group of pixels (20x20) that are the same color. I don't believe this should necessitate constructing 1 billion objects.
Thanks!
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
如果这确实是图形热图，那么我同意这样的评论：至少 780 笔记本电脑屏幕宽的图像是不切实际的。如果您在 SQL(?) 数据库中的某处拥有此信息，那么您可以执行一个奇特的查询，将结果划分为特定宽度的存储桶。数据库应该能够有效地将这些记录聚合到 1680（像素宽）的存储桶中。
此外，如果您的存储桶具有固定宽度（产生固定宽度的热图图像），您可以为数据库中的“地址”预先生成存储桶编号。如果索引正确，按此分组会非常快。
如果您确实需要查看 1:1 图像，您可能会考虑仅渲染滚动到的图像的一部分。这将显着减少存储当前视图所需的内存量。假设您实际上不需要 100% 查看所有 780 个屏幕的数据（特别是如果您将其与上面的“大图片视图”策略结合起来），那么您也将节省处理时间。
“大图视图”的聚合函数可能是 MAX、SUM、AVG。如果这些功能不合适，请详细解释您在热图中寻找的特定功能。
就绘图本身而言，您不需要每个框的“对象”，您只需在图形对象上绘制像素即可。
If this is really for a graphic heat map, then I agree with the comments that an image that's at least 780 laptop screens wide is impractical. If you have this information in a SQL(?) database somewhere, then you can do a fancy query that partitions your results into buckets of a certain widths. The database should be able to aggregate these records into 1680 (pixels wide) buckets efficiently.
Furthermore, if your buckets are of a fixed width (yielding a fixed width heat-map image) you could pre-generate the bucket numbers for the "addresses" in your database. Indexed properly, grouping by this would be very fast.
If you DO need to see a 1:1 image, you might consider only rendering a section of the image that you're scrolled to. This would significantly reduce the amount of memory necessary to store the current view. Assuming you don't need to actually view all 780 screens worth of data at 100% (especially if you couple this with the "big picture view" strategy above) then you'll save on processing too.
The aggregate function for the "big picture view" might be MAX, SUM, AVG. If these functions aren't appropriate, please explain more about the particular features you'd be looking for in the heat-map.
As far as the drawing itself, you don't need "objects" for each box, you just need to draw the pixels on a graphics object.
我认为您正在寻找的技术称为“虚拟化”。现在我指的不是硬件虚拟化，而是技术，您仅为可见的项目创建具体的视觉对象。许多网格和列表使用这种技术以正常速度和内存消耗显示数千个项目。您还可以在交换具体数据对象时重用这些视觉对象。
我还会质疑显示数十亿细节的必要性。您应该使其类似于数据的缩放或聚合，以仅显示少数项目，然后让用户选择特定的部分或数据片段。但我想你已经有了这样的想法。
I think technique you are looking for is called "virtualization". Now I don't mean hardware virtualization, but technique, where you create concrete visual object only for items, that are visible. Many grids and lists use this technique to show thousands of hundreds of items at normal speeds and memory consumption. You can also reuse those visual objects while swaping concrete data objects.
I would also question necesity of displaying bilions of details. You should make it similiar to zooming or agregation of data to show only few items and then let the user choose specific part or pice of data. But I guess you have that thought out.