通过延迟执行和延迟加载获取 Count()

发布于 2024-11-17 20:44:40 字数 1683 浏览 2 评论 0原文

我有一些使用延迟执行和延迟加载的代码:

    public static IEnumerable<XElement> GetStreamElementP(string fileId, ListProgressEventHandler progressHandler, int total)
    {
        var filePath = Utility.GetEContentFilePath(fileId);
        using (var reader = XmlReader.Create(filePath, new XmlReaderSettings { IgnoreWhitespace = true, }))
        {
            var cnt = 0;
            reader.MoveToContent();
            // Parse the file and display each of the p nodes.
            reader.Read();
            while (reader.NodeType == XmlNodeType.Element && reader.Name == "p")
            {
                cnt++;
                var returnedValue = XElement.ReadFrom(reader) as XElement;

                int rem = cnt % _streamElementCallBackSize;
                if (progressHandler != null && rem == 0)
                {
                    progressHandler(null, new ListProgressEventArgs { ItemsProcessed = cnt, TotalItemsToProcess = total, });
                }
                yield return returnedValue;
            }
            reader.Close();
        }

    }

我希望获得元素数量的简单计数。我们当前使用的代码是:

    public static int FileElementsCount(string fileId)
    {
        var cnt = 0;
        foreach (XElement e in GetStreamElementP(fileId))
        {
            cnt++;
        }
        return cnt;
    }

我可以将其改进为吗?

    public static int FileElementsCount(string fileId)
    {
        return GetStreamElementP(fileId).Count<XElement>();
    }

或者这会导致在获取计数时使用更多内存?在某些情况下,我们会处理非常大的文件,并尝试尽可能将内存使用量保持在最低限度。

我试图找到一个具体的例子来解释每种情况下如何使用内存,但没有成功。

预先感谢您的任何帮助。

I have some code that uses deferred execution and lazy loading:

    public static IEnumerable<XElement> GetStreamElementP(string fileId, ListProgressEventHandler progressHandler, int total)
    {
        var filePath = Utility.GetEContentFilePath(fileId);
        using (var reader = XmlReader.Create(filePath, new XmlReaderSettings { IgnoreWhitespace = true, }))
        {
            var cnt = 0;
            reader.MoveToContent();
            // Parse the file and display each of the p nodes.
            reader.Read();
            while (reader.NodeType == XmlNodeType.Element && reader.Name == "p")
            {
                cnt++;
                var returnedValue = XElement.ReadFrom(reader) as XElement;

                int rem = cnt % _streamElementCallBackSize;
                if (progressHandler != null && rem == 0)
                {
                    progressHandler(null, new ListProgressEventArgs { ItemsProcessed = cnt, TotalItemsToProcess = total, });
                }
                yield return returnedValue;
            }
            reader.Close();
        }

    }

I'm looking to get a simple count on the number of elements. The current code we are using is:

    public static int FileElementsCount(string fileId)
    {
        var cnt = 0;
        foreach (XElement e in GetStreamElementP(fileId))
        {
            cnt++;
        }
        return cnt;
    }

Can I improve this to?

    public static int FileElementsCount(string fileId)
    {
        return GetStreamElementP(fileId).Count<XElement>();
    }

Or will this cause more memory to be used when getting the count? We are dealing with very large files in some cases and attempting to keep memory usage to a minimum where possible.

I have tried to find a concrete example that explains how the memory is used in each case without any success.

Thanks in advance for any help.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

又爬满兰若 2024-11-24 20:44:40

这并不重要。您的方法和 count 方法都在内部对 GetStreamElementP 的结果执行直接循环(这里没有懒惰的东西)。不涉及缓存或任何内容。

如果您希望速度更快,则必须找到一种智能方法来缓存/预先计算 GetStreamElementP- 的结果,或者使用 GetStreamElementP 的变体,它可以直接对文件进行更智能的计数

It doesnt really matter. Both your method and the count method internally perform a direct loop (no lazy stuff here) over the result of GetStreamElementP. There is no caching or whatsoever involved.

If you want this to be faster, you either have to find a smart way of caching / pre-calculating the result of GetStreamElementP- or have a variant on GetStreamElementP which does a smarter count on the file directly

如果没结果 2024-11-24 20:44:40

在您的情况下,两种计算计数的方法都会执行相同的操作。

此函数的内存消耗应与

元素的大小成正比。因此,如果有很多小元素,它不应该消耗大量内存。如果您的大型元素相对较少,则可能会消耗大量内存,因为您要为每个元素创建一个 XElement。如果是这种情况,通过根本不创建它们可以使内存消耗小得多。

In your case, both ways of computing the count are will do the same.

The memory consumption of this function should be proportional the size of the <p> elements. So, if there is lots of small elements, it shouldn't consume large amounts of memory. If you have relatively few huge elements, this could consume quite a lot of memory, because you're creating an XElement out of each of them. If this was the case, the memory consumption could be made much smaller by not creating them at all.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文