DatastoreMutationPool 是否在回调之前刷新到数据存储区？
我正在使用mapreduce，并且当某些实体不在数据存储中时，我需要保留它们。我将新实体添加到 DatastoreMutationPool 中，以便可以通过批量调用来保留这些实体。当mapreduce结束时，回调函数被调用。回调函数将使用其中一些实体。我的问题是，在调用回调函数之前，所有实体是否都会刷新到数据存储中，或者它们仍然可以位于 DatastoreMutationPool 中但不在数据存储中。
谢谢。
映射器示例：
public class MyMapper extends AppEngineMapper<Key, Entity, NullWritable, NullWritable> {
@Override
public void map(Key key, Entity value, Context context) {
...
DatastoreMutationPool mutationPool = this.getAppEngineContext(context).getMutationPool();
mutationPool.put(entity);
}
}
回调示例：
@RequestMapping(value="/callback/function",method=RequestMethod.POST)
public void callback(@RequestParam("job_id") String jobIdName){
JobID jobId = JobID.forName(jobIdName);
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
//search for some entities persisted in the mapper
...
}
I'm using mapreduce and I need to persist some entities when they are not in the datastore. I add the new entities to a DatastoreMutationPool so these entities could be persisted with batched calls. When the mapreduce ends a callback function is invoked. The callback function will use some of these entities. My question is, will all they entities be flushed to the datastore before the callback function is invoked or they can still be in the DatastoreMutationPool but not in the datastore.
Thanks.
Example of mapper:
public class MyMapper extends AppEngineMapper<Key, Entity, NullWritable, NullWritable> {
@Override
public void map(Key key, Entity value, Context context) {
...
DatastoreMutationPool mutationPool = this.getAppEngineContext(context).getMutationPool();
mutationPool.put(entity);
}
}
Example of callback:
@RequestMapping(value="/callback/function",method=RequestMethod.POST)
public void callback(@RequestParam("job_id") String jobIdName){
JobID jobId = JobID.forName(jobIdName);
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
//search for some entities persisted in the mapper
...
}
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
突变池是针对每个映射器的，而一旦映射缩减完成，回调就会在映射器外部运行。因此，您可以预期在回调运行时所有突变池都将被刷新。
The mutation pool is per-mapper, while the callback is run outside the mappers once the mapreduce completes. As a result, you can expect that all the mutation pools will have been flushed by the time your callback is run.