应用程序引擎批量加载程序下载警告“__key__ 上没有降序索引,正在执行串行下载”

发布于 2024-10-03 08:30:29 字数 744 浏览 0 评论 0原文

我正在使用以下内容下载我的一种类型的所有实例:

appcfg.py download_data --config_file=bulkloader.yaml --kind=ModelName --filename=ModelName.csv --url=http://appid.appspot.com/remote_api

如果该类型的实例数量多于批量大小,那么我会收到此警告:

No descending index on __key__, performing serial download

我没有任何自定义索引,也没有任何禁用索引的属性。

我是否“需要”采取措施来解决此警告,或者这只是我可以安全忽略的警告?对下载速度有影响吗?

这篇关于bulkloader的文章在示例输出中包含警告消息,但没有提及。

关于应用引擎群组的这篇文章说我需要创建一个索引。然而,添加更多索引会减慢对实体的写入速度 - 我宁愿不这样做,因为我将比批量数据下载更频繁地写入实体。

谢谢。

I'm using the following to download all instances of one of my kinds:

appcfg.py download_data --config_file=bulkloader.yaml --kind=ModelName --filename=ModelName.csv --url=http://appid.appspot.com/remote_api

If the kind has more instances than the batch size, then I get this warning:

No descending index on __key__, performing serial download

I don't have any custom indexes, or any properties with indexes disabled.

Do I 'need' to do something to resolve this warning, or is it just a warning I can safely ignore? Does it effect the speed of the download?

This article on the bulkloader includes the warning message in the sample output, but makes no mention of it.

This post on the app engine group says that I need to create an index. However adding more indexes would slow down writes to my entities - which I'd rather not do as I am going to be writing entities more often than I will be doing bulk data downloads.

Thanks.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

玩心态 2024-10-10 08:30:29

正如错误所描述的,如果您正在下载的模型的 __key__ 上没有降序索引,则批量加载器必须串行下载。如果按照描述添加索引,它将能够并行下载。如果不这样做,它会正常工作,但下载速度会较慢,因为它是串行运行的。

请注意,附加索引对延迟的影响很小,因为索引行与实体写入并行写入,这意味着写入只需要最慢的更新时间。

As the error describes, without a descending index on __key__ for the model you're downloading, the bulkloader has to download serially. If you add the index as described, it will be able to download in parallel. If you don't, it will work fine, but will be slower to download, as it operates serially.

Note that an additional index has only a small impact on latency, as index rows are written in parallel to the entity write, meaning the write only takes as long as the slowest update.

秋千易 2024-10-10 08:30:29

我通过将此代码添加到index.yaml解决了这个问题

kind: books
- properties:
  name: __key__
    - direction: desc
kind: books
- properties:
  name: another_indexes_here

I've solved this problem by add this code to index.yaml

kind: books
- properties:
  name: __key__
    - direction: desc
kind: books
- properties:
  name: another_indexes_here
另类 2024-10-10 08:30:29

如果您使用 JAVA 和 datastore-indexes.xml 文件。

将其添加到 datastore-indexes.xml 文件(假设该种类的名称为“Books”):

<datastore-index kind="Books" ancestor="false" source="auto"> 
    <property name="__key__" direction="desc"/> 
</datastore-index>

然后重新部署您的应用程序。确保检查数据存储索引选项卡以查看 __key__ 正在提供服务。
然后您可以再次尝试下载。

If your using JAVA and the datastore-indexes.xml file.

Add this (Assuming the name of the kind is "Books") to the datastore-indexes.xml file:

<datastore-index kind="Books" ancestor="false" source="auto"> 
    <property name="__key__" direction="desc"/> 
</datastore-index>

Then redeploy your app. Ensure you check the datastore index tab to see that the __key__ is serving.
Then you can try your download again.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文