用户的顺序。被观察到的清单不下降

发布于 2025-01-24 07:18:32 字数 3887 浏览 0 评论 0 原文

为了同步邮箱,我的应用程序关注同步建议用户邮箱中最新消息的历史ID。然后,我们将其用于部分同步。

最近,我们注意到行为暗示了这些同步问题。一个解释是,我们收到了一个较旧的消息和历史ID。我已经测试了我们的功能,并且似乎可以正常工作。尽管如此,为了排除潜在的根本原因,我添加了一些检查以检测 users.messages.list api返回结果的降序订单返回结果。这些检查最终被击中,这表明这是一个问题。

这是我在Go中找到最新历史ID的功能。这包括我添加的其他检查以验证订购的其他检查 - 本质上是而是使用 Message.get 对于列表中的第一个条目,它还在列表中获取最后一个条目,然后比较日期/历史记录IDS:列表中的第一个条目应具有最大的历史记录ID和日期。

func getLatestHistoryID(ctx context.Context, gmailService *gmail.Service) (uint64, time.Time, error) {
    messagesResponse, err := gmailService.Users.Messages.List("me").IncludeSpamTrash(true).Context(ctx).Do()
    if err != nil {
        return 0, time.Time{}, err
    }

    messagesList := messagesResponse.Messages
    if messagesList == nil || len(messagesList) == 0 {
        return 0, time.Time{}, nil
    }

    latestMessage, err := gmailService.Users.Messages.Get("me", messagesList[0].Id).Context(ctx).Do()
    if err != nil {
        return 0, time.Time{}, err
    } else if latestMessage == nil {
        return 0, time.Time{}, nil
    }

    earliestMessage, err := gmailService.Users.Messages.Get("me", messagesList[len(messagesList)-1].Id).Context(ctx).Do()
    if err != nil {
        log.Errorf("error doing optional check to validate ordering of message list. %v", err)
    } else if earliestMessage == nil {
        log.Errorf("unexpected earliest message not retrieved")
    } else {
        if latestMessage.HistoryId < earliestMessage.HistoryId {
            return 0, time.Time{}, fmt.Errorf("message list was not in the expected order by history id! first in list %d (%s), last %d (%s)",
                latestMessage.HistoryId, latestMessage.Id,
                earliestMessage.HistoryId, earliestMessage.Id)
        }

        // This could probably fail in rare clock skew cases, but right now we're observing this being a several hour difference between dates.
        if latestMessage.InternalDate < earliestMessage.InternalDate {
            return 0, time.Time{}, fmt.Errorf("message list was not in the expected order by date! first in list %s (%s), last %s (%s)",
                time.UnixMilli(latestMessage.InternalDate).String(), latestMessage.Id,
                time.UnixMilli(earliestMessage.InternalDate).String(), earliestMessage.Id)
        }
    }

    return latestMessage.HistoryId, time.UnixMilli(latestMessage.InternalDate), nil
}

我找到了几个资源,可以确认 users.messages.list 有望按日期/历史ID降下:

当我在上面测试上面的函数时,它可以按预期工作,并且最后一行的返回语句被击中。但是我观察到了数百次的订单检测错误。在失败中,我看到历史ID检查失败的9/10倍。我相信这在很大程度上是在一小部分邮箱上失败的,而且目前我不确定发生了什么比例用法(努力收集此内容)。

API是否有任何理由可能会退出结果?我的支票做出的假设有什么问题吗?

In order to sync mailboxes my application follows the sync recommendations by attempting to find the history ID of the latest message in the users mailbox. We then use this for partial syncs going forward.

Recently we noticed behavior that suggested an issue with these syncs. One explanation was that we were receiving a much older message and history ID. I've tested our functionality and it appears to work correctly. Still, in an attempt to rule out a potential root cause, I added some checks to detect if the users.messages.list API return results out of descending order. These checks ended up being hit suggesting that this is an issue.

Here is my function, in Go, for finding the latest history ID. This includes the additional checks I added to validate the ordering -- essentially instead of using messages.get for the first entry in the list, it also gets the last entry in the list and then compares dates/history IDs: the first entry in the list should have the greatest history ID and date.

func getLatestHistoryID(ctx context.Context, gmailService *gmail.Service) (uint64, time.Time, error) {
    messagesResponse, err := gmailService.Users.Messages.List("me").IncludeSpamTrash(true).Context(ctx).Do()
    if err != nil {
        return 0, time.Time{}, err
    }

    messagesList := messagesResponse.Messages
    if messagesList == nil || len(messagesList) == 0 {
        return 0, time.Time{}, nil
    }

    latestMessage, err := gmailService.Users.Messages.Get("me", messagesList[0].Id).Context(ctx).Do()
    if err != nil {
        return 0, time.Time{}, err
    } else if latestMessage == nil {
        return 0, time.Time{}, nil
    }

    earliestMessage, err := gmailService.Users.Messages.Get("me", messagesList[len(messagesList)-1].Id).Context(ctx).Do()
    if err != nil {
        log.Errorf("error doing optional check to validate ordering of message list. %v", err)
    } else if earliestMessage == nil {
        log.Errorf("unexpected earliest message not retrieved")
    } else {
        if latestMessage.HistoryId < earliestMessage.HistoryId {
            return 0, time.Time{}, fmt.Errorf("message list was not in the expected order by history id! first in list %d (%s), last %d (%s)",
                latestMessage.HistoryId, latestMessage.Id,
                earliestMessage.HistoryId, earliestMessage.Id)
        }

        // This could probably fail in rare clock skew cases, but right now we're observing this being a several hour difference between dates.
        if latestMessage.InternalDate < earliestMessage.InternalDate {
            return 0, time.Time{}, fmt.Errorf("message list was not in the expected order by date! first in list %s (%s), last %s (%s)",
                time.UnixMilli(latestMessage.InternalDate).String(), latestMessage.Id,
                time.UnixMilli(earliestMessage.InternalDate).String(), earliestMessage.Id)
        }
    }

    return latestMessage.HistoryId, time.UnixMilli(latestMessage.InternalDate), nil
}

I've found several resources that confirm that users.messages.list is expected to be descending by date/history ID:

When I test the function above locally it works as expected, and the return statement on the last line is hit. Yet I've observed the out of order detection errors hundred of times. Of the failures, ~9/10 times I'm seeing the HistoryId check fail. I believe this is largely failing on a small set of mailboxes, and I am currently not sure what proportion usages this occurs (working on gathering this).

Is there any reason the API may return results out of order? Is there anything wrong with the assumptions made by my checks?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

飘落散花 2025-01-31 07:18:32

API返回结果以降序订购。

如果您检查 users.messages.messages.list 您会发现没有按参数订单。这意味着您没有办法保证数据到达的顺序。

有时可能会以降序和其他时间降低顺序到达。如果有文档中的命令,则无法保证它。

##限制 对订单没有任何提及,它只会提及它可能会提及它可能会提到它可能会表明它可能会提及它可能只能或可能不可能。

历史记录通常至少有一个星期,而且通常更长。

您应该始终在本地对此进行分类。

API return results out of descending order.

If you check the documentation for users.messages.list you will find that there is no order by parameter. Which means that there is no way for you to guarantee the order the data arrives in.

It could arrive sometimes in descending order and other times not in descending order. There is no way to guarantee it if there was it would state the order in the docs.

#limitations does not mention anything about order it only mentions that it may or may not be alliable.

History records are typically available for at least one week and often longer.

you should always sort this locally.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文