HDFS中的数据码与纱线的Node Manager之间的连接/关系是什么?

发布于 2025-01-19 13:34:07 字数 306 浏览 1 评论 0原文

我正在阅读有关纱线和Hadoop文件系统的基础知识。一些博客在线告诉我,纱线只是资源管理系统,HDFS与存储有关。但是我在《 hadoop deveritive指南》一书中遇到了以下几行: “在此处输入图像说明” 在这一行中,我可以推断出数据码的位置与节点管理器节点之间应该存在一些联系。也许他们可以在同一地方。这与我从博客中获得的知识相矛盾。 谁能帮助解释这一点?

我通过“ DataNode和Node Manager之间的连接”进行了搜索,但找不到直接答案。

I am reading the basics about Yarn and Hadoop FileSystem. I was told by some blogs online that Yarn is just resource management system and HDFS is about storage. But I encountered the following lines in the book Hadoop Definitive Guide:
enter image description here
In this line, I can infer that there should be some connection between the location of Datanodes and Node Manager Node. Maybe they can be in the same place. That contradicts the knowledge I got from the blog.
Can anyone helps to explain this?

I googled a lot by"connection between Datanode and Node Manager" and I can not find direct answer to that.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

红墙和绿瓦 2025-01-26 13:34:07

纱线是操作系统,计算功率。
HDFS是磁盘。

如果有益将计算移至数据所在的节点。节点通常会具有管理计算(纱线)和数据节点(HDFS)的节点管理器。因此,可以在1个节点/服务器上将容器和纱线/Hadoop作业的文件进行共归因。也是如此,您可以在不是数据节点的节点上拥有一个节点管理器。而且您可以拥有一个数据节点,这不是一个nodemanager。两者是独立的,但要利用数据局部性是有意义的。毕竟,谁想要一个没有磁盘的操作系统? (它们实际上是一个用例,但不要进入“计算节点”)

Yarn is the OS, the compute power.
HDFS is the Disk.

If beneficial to move the compute to a node where the data is located. A node will often have a node manager that manages the compute(yarn) and a data node(HDFS). So both a container, and files for a yarn/hadoop job, can be colocated on 1 node/server. It's also the case you can just have a node manager on a node that isn't a data node. And you could have a data node, that wasn't a nodemanager. The two are independent, but frequently it makes sense to collocate them, to take advantage of data locality. After-all who wants a OS without a disk? (Their is actually a use case for this but lets not get into "compute nodes")

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文