我可以在较旧版本中使用Hadoop纱线的Docker运行时功能(Hadoop 2.7.3)
YARN提供了将Linux Executor容器与Docker Runtime一起使用的功能,较早的YARN提供了创建Docker Executor容器而不是Linux Executor容器的功能,因此我想知道Docker Runtime的新功能是否向后兼容。
在后来版本的Hadoop中介绍的Docker运行时间到了下面讨论的Docker Executor容器的缺点:
较大的架构问题是,在Yarn中,您可以使用每个NodeManager使用一个containerexecutor。所有任务都将使用节点配置中指定的contairexecutor。结果,一旦将群集配置为使用DockerContainerexecutor,用户将无法启动常规的MapReduce,tez或Spark作业。此外,实施新的ContainExecutor意味着现有LinuxContainereXecutor的所有好处(例如CGROUP和流量塑造)现在需要在新的ContaineRexecutor中重新完成。由于这些挑战,DockerContainerexecutor已被弃用,以支持新的抽象(集装箱运行时间),并且将在未来的Apache Hadoop版本中删除DockerContainereXecutor。
我可以在旧版本的Hadoop(2.7.3)中使用纱线Docker运行时。
Yarn provides a feature to use linux executor container with the docker runtime , earlier yarn provides a functionality to create docker executor container instead of linux executor container , so i want to know if the new feature of docker runtime is backward compatible .
docker runtime that was introduced in the later version of hadoop to over come the shortcoming of docker executor container discussed below :
the bigger architectural issue is that in YARN, you can use one ContainerExecutor per NodeManager. All tasks will use the ContainerExecutor specified in the node’s configuration. As a result, once the cluster was configured to use DockerContainerExecutor, users would be unable to launch regular MapReduce, Tez, or Spark jobs. Additionally, implementing a new ContainerExecutor means that all of the benefits of the existing LinuxContainerExecutor (such as cgroups and traffic shaping) now need to be reimplemented in the new ContainerExecutor. As a result of these challenges, DockerContainerExecutor has been deprecated in favor of a newer abstraction – container runtimes – and DockerContainerExecutor will be removed in a future Apache Hadoop release.
Can i use the yarn docker runtime in older versions of hadoop (2.7.3).
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论