在 udf 中加载外部属性文件

发布于 2024-11-08 03:39:57 字数 202 浏览 0 评论 0原文

当编写 UDF 时，比如说 EvalFunc，

properties = new Properties();
properties.load(new FileInputStream("conf/config.properties"));

在 Hadoop 模式下运行时是否可以传递配置文件？

最好的，将要

原文

When writing a UDF let's say a EvalFunc, is it possible to pass a configuration file with

properties = new Properties();
properties.load(new FileInputStream("conf/config.properties"));

when running in Hadoop Mode?

Best,
Will

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

早乙女 2024-11-15 03:39:57

这是从 Hadoop DFS 读取和写入文件的简单示例，来自 http://wiki .apache.org/hadoop/HadoopDfsReadWriteExample

也许您可以在其中找到一些有用的代码来完成您的工作。

以下是我的代码，它成功在 hadoop 中加载属性文件，我使用了 Apache Commons Configuration http://commons.apache.org/configuration/

public static void loadProperites(String path) throws ConfigurationException, IOException {
    Configuration conf = new Configuration();
    FileSystem fs = FileSystem.get(conf);
    Path inFile = new Path(path);
    FSDataInputStream in = fs.open(inFile);

    PropertiesConfiguration config = new PropertiesConfiguration();
    config.load(in);

    in.close();
}

Here is Simple Example to Read and Write files from Hadoop DFS from http://wiki.apache.org/hadoop/HadoopDfsReadWriteExample

maybe you can find some useful code in it to complete your job.

Following is my code, it successfully load a properties file in hadoop, I used the Apache Commons Configuration http://commons.apache.org/configuration/

public static void loadProperites(String path) throws ConfigurationException, IOException {
    Configuration conf = new Configuration();
    FileSystem fs = FileSystem.get(conf);
    Path inFile = new Path(path);
    FSDataInputStream in = fs.open(inFile);

    PropertiesConfiguration config = new PropertiesConfiguration();
    config.load(in);

    in.close();
}

回复收藏 0 原文

迷鸟归林 2024-11-15 03:39:57

使用 Apache Commons Configuration2 和 vfs2：


Parameters params = new Parameters();
FileBasedConfigurationBuilder<PropertiesConfiguration> builder =
                    new FileBasedConfigurationBuilder<>(PropertiesConfiguration.class)
                            .configure(params.fileBased().setFileSystem(new VFSFileSystem())
                                    .setLocationStrategy(new FileSystemLocationStrategy())
                                    .setEncoding("UTF-8").setFileName(propertyPath));
config = builder.getConfiguration();

Use the Apache Commons Configuration2 and vfs2:


Parameters params = new Parameters();
FileBasedConfigurationBuilder<PropertiesConfiguration> builder =
                    new FileBasedConfigurationBuilder<>(PropertiesConfiguration.class)
                            .configure(params.fileBased().setFileSystem(new VFSFileSystem())
                                    .setLocationStrategy(new FileSystemLocationStrategy())
                                    .setEncoding("UTF-8").setFileName(propertyPath));
config = builder.getConfiguration();

回复收藏 0 原文

~没有更多了~