Apache Beam：平坦的PCollection＆lt; list＆lt; foo＆gt;＆gt;＆gt; pcollection＆lt; foo＆gt;

发布于 2025-02-03 19:06:42 字数 736 浏览 3 评论 0 原文

假设我们有一些嵌套列表：

["a", "b"]
["c", "d"]

我们可以像这样轻松地在流API中进行襟翼映射：

Stream
        .of(List.of("a", "b"), List.of("c", "d"))
        .flatMap(List::stream)
        .forEach(System.out::println);

但是使用“ Flatmapelements”进行，这是一团糟：

Pipeline pipeline = Pipeline.create();
pipeline.apply(Create.of(List.of(List.of("a", "b"), List.of("c", "d"))))
        .apply(FlatMapElements.into(TypeDescriptor.of(String.class)).via(list -> list))
        .apply(ParDo.of(new SomeOutputFunction()));

我们可以使用平面地图功能做得更好吗？

简单的flatmap工作不应该那么复杂，所以我认为我缺少一些东西。
由于类型推理问题，我什至无法替换 .via（list - ＆gt; list） 。

原文

Say we have some nested list:

["a", "b"]
["c", "d"]

And we can easily do the flap map in Stream API like this:

Stream
        .of(List.of("a", "b"), List.of("c", "d"))
        .flatMap(List::stream)
        .forEach(System.out::println);

But doing it with "FlatMapElements", it was quite a mess:

Pipeline pipeline = Pipeline.create();
pipeline.apply(Create.of(List.of(List.of("a", "b"), List.of("c", "d"))))
        .apply(FlatMapElements.into(TypeDescriptor.of(String.class)).via(list -> list))
        .apply(ParDo.of(new SomeOutputFunction()));

Can we do anything else better with the flat map function?
A simple flatmap job should not be that complicated so I think I am missing something.
I cannot even replace .via(list -> list) to .via(Function.identity()) due to the type inference problem.

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

迟月 2025-02-10 19:06:42

“ .apply（flatten.iterables（））”
它将转换“ pcollection＆lt; list＆gt;”到“ PCollection”

回复收藏 0 原文

烏雲後面有陽光 2025-02-10 19:06:42

请参阅《 apache beam编程指南：

PCollectionList<String> collections = PCollectionList.of(pc1).and(pc2).and(pc3);

PCollection<String> merged = collections.apply(Flatten.<String>pCollections());

Please refer to the Apache Beam programming guide https://beam.apache.org/documentation/programming-guide/#flatten:

PCollectionList<String> collections = PCollectionList.of(pc1).and(pc2).and(pc3);

PCollection<String> merged = collections.apply(Flatten.<String>pCollections());

回复收藏 0 原文

~没有更多了~