HDFS默认将文件存储在本地哪里?
我正在使用单节点集群的默认配置运行 hadoop,并且希望找到 HDFS 在本地存储文件的位置。
有什么想法吗?
谢谢。
I am running hadoop with default configuration with one-node cluster, and would like to find where HDFS stores files locally.
Any ideas?
Thanks.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(6)
您需要在 hdfs-default.xml 配置文件中查找 dfs.data.dir 设置。默认设置为:${hadoop.tmp.dir}/dfs/data,注意${hadoop.tmp.dir}实际上是在core-default.xml中描述的此处。
配置选项在此处进行了描述。该设置的描述是:
You need to look in your hdfs-default.xml configuration file for the dfs.data.dir setting. The default setting is: ${hadoop.tmp.dir}/dfs/data and note that the ${hadoop.tmp.dir} is actually in core-default.xml described here.
The configuration options are described here. The description for this setting is:
似乎当前版本(2.7.1)的目录
基于 dfs.datanode.data.dir 、hadoop.tmp.dir 设置:
http://hadoop.apache .org/docs/r2.7.1/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml
http://hadoop.apache .org/docs/r2.7.1/hadoop-project-dist/hadoop-common/core-default.xml
Seems like for the current version(2.7.1) the dir is
Based on
dfs.datanode.data.dir
,hadoop.tmp.dir
setting from:http://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml
http://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-common/core-default.xml
作为“更新的答案”并澄清hadoop版本号:
如果您使用Hadoop 1.2.1(或类似的东西),@Binary Nerd的答案仍然是正确的。
但是如果您使用Hadoop 2.1.0-beta(或类似的版本),您应该阅读配置文档此处,您要设置的选项是:
dfs.datanode.data.dir
As "more recent answer" and to clarify hadoop version numbers:
If you use Hadoop 1.2.1 (or something similar), @Binary Nerd's answer is still true.
But if you use Hadoop 2.1.0-beta (or something similar), you should read the configuration documentation here and the option you want to set is:
dfs.datanode.data.dir
对于hadoop 3.0.0,hdfs根路径由属性
“dfs.datanode.data.dir”
给出For hadoop 3.0.0, the hdfs root path is as given by the property
"dfs.datanode.data.dir"
首先找到 /usr/lib 中的 Hadoop 目录。在那里您可以找到 etc/hadoop 目录,其中包含所有配置文件。
在该目录中,您可以找到 hdfs-site.xml 文件,其中包含有关 HDFS 的所有详细信息。在那里你可以找到 2 个属性:
dfs.namenode.name.dir – 告诉 namenode 在本地文件系统上存储元数据的位置。
dfs.datanode.data.dir – 告诉 datanode 在本地文件系统上存储数据的位置
First find the Hadoop directory present in /usr/lib. There you can find the etc/hadoop directory, where all the configuration files are present.
In that directory you can find the hdfs-site.xml file which contains all the details about HDFS. There you find 2 properties:
dfs.namenode.name.dir – tells where the namenode stores the metadata on the local filesystem.
dfs.datanode.data.dir – tells where the datanode stores the data on the local filesystem
在
cmd
提示符下运行此命令,您将获得 HDFS 位置:Run this in the
cmd
prompt, and you will get the HDFS location: