我可以在较旧版本中使用Hadoop纱线的Docker运行时功能(Hadoop 2.7.3)

发布于 2025-02-02 20:42:25 字数 608 浏览 4 评论 0原文

YARN提供了将Linux Executor容器与Docker Runtime一起使用的功能,较早的YARN提供了创建Docker Executor容器而不是Linux Executor容器的功能,因此我想知道Docker Runtime的新功能是否向后兼容。

在后来版本的Hadoop中介绍的Docker运行时间到了下面讨论的Docker Executor容器的缺点:

较大的架构问题是,在Yarn中,您可以使用每个NodeManager使用一个containerexecutor。所有任务都将使用节点配置中指定的contairexecutor。结果,一旦将群集配置为使用DockerContainerexecutor,用户将无法启动常规的MapReduce,tez或Spark作业。此外,实施新的ContainExecutor意味着现有LinuxContainereXecutor的所有好处(例如CGROUP和流量塑造)现在需要在新的ContaineRexecutor中重新完成。由于这些挑战,DockerContainerexecutor已被弃用,以支持新的抽象(集装箱运行时间),并且将在未来的Apache Hadoop版本中删除DockerContainereXecutor。

我可以在旧版本的Hadoop(2.7.3)中使用纱线Docker运行时。

Yarn provides a feature to use linux executor container with the docker runtime , earlier yarn provides a functionality to create docker executor container instead of linux executor container , so i want to know if the new feature of docker runtime is backward compatible .

docker runtime that was introduced in the later version of hadoop to over come the shortcoming of docker executor container discussed below :

the bigger architectural issue is that in YARN, you can use one ContainerExecutor per NodeManager. All tasks will use the ContainerExecutor specified in the node’s configuration. As a result, once the cluster was configured to use DockerContainerExecutor, users would be unable to launch regular MapReduce, Tez, or Spark jobs. Additionally, implementing a new ContainerExecutor means that all of the benefits of the existing LinuxContainerExecutor (such as cgroups and traffic shaping) now need to be reimplemented in the new ContainerExecutor. As a result of these challenges, DockerContainerExecutor has been deprecated in favor of a newer abstraction – container runtimes – and DockerContainerExecutor will be removed in a future Apache Hadoop release.

Can i use the yarn docker runtime in older versions of hadoop (2.7.3).

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文