客户端机器上的hadoop api配置
超级菜鸟。我有一台带有 cdh3u1 伪发行版的服务器计算机,以及一台带有使用 cdh3u1 API 的 java 应用程序的客户端计算机。
如何配置客户端与服务器通信?我已经搜索了几个小时,但找不到“客户端配置”文件在哪里。对我来说,“hdfs-default”、“core-default”和“mapred-default”及其对应的“-site”都看起来像服务器(名称节点和数据节点)配置。
它只是“多用途客户端服务器”配置吗?我应该在这些文件中挑选适合客户端的属性吗?他们是谁?这里可能遗漏了一些大东西...
谢谢,Ido
ultra-noob. I have a server machine with cdh3u1 pseudo-distrib, and a client machine with a java application using the cdh3u1 API.
How do I configure the client to talk to the server? I've been googling for hours and couldn't find where is the "client configuration" file. The "hdfs-default", "core-default" and "mapred-default" and their "-site" counterparts all look like server (namenode and datanode) config to me.
Is it just "multipurpose client server" config and I should cherry-pick the attributes in these files that are appropriate to the client? which are they? probably missing something big here...
Thanks, Ido
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
确保客户端机器可以访问hadoop服务器机器ip。如果您为 hadoop 服务器 (cdh3 vm) 使用 virtualbox,则添加“仅主机”网络接口(请参阅此处的详细信息:使用 virtualbox 进行仅主机网络。我假设您的 hadoop 服务器的静态 IP 是
192.168.56.101
并且您可以从客户端 ping 它,并在服务器和客户端计算机中为您的 hadoop 服务器计算机配置主机名。大象”,将以下行添加到两台计算机的 /etc/hosts 中:
192.168.56.101 local-elephant
。在服务器计算机中转到/etc/hadoop/conf 将以下属性的值从“localhost”更改为“local-elephant”:在 core-site.xml 中,
fs.default.name
的值和在 mapred-site 中mapred.job.tracker
的值,.xml客户端计算机中的 在 java 应用程序的类路径中创建 core-site.xml 和 mapred-site.xml 在这些文件中仅放置 <。代码>fs.default.name和mapred.job.tracker属性。
make sure that the client machine can access the hadoop server machine ip. If you use a virtualbox for the hadoop server (cdh3 vm), then add a "host-only" network interface (see details here: host-only networking with virtualbox. I'm assuming that your static ip for the hadoop server is
192.168.56.101
and that you're able to ping it from your client.configure a hostname for your hadoop server machine in both the server and client machine. If you want to name your hadoop server "local-elephant", add the following line to /etc/hosts in both machines:
192.168.56.101 local-elephant
.in the server machine goto /etc/hadoop/conf change the values of the following properties from "localhost" to "local-elephant": in core-site.xml the value of
fs.default.name
and in mapred-site.xml the value ofmapred.job.tracker
.in the client machine, create core-site.xml and mapred-site.xml in the classpath of your java application. In those files put only the
fs.default.name
andmapred.job.tracker
properties.