如何将命名管道尾接到 Flume Windows 节点（Flume 节点服务）
不确定这是一个 SO、SuperUser 还是 ServerFault 问题，但我刚刚在 Windows 计算机上安装了 FlumeNode 服务，并且我想通过命名管道将日志信息发送到 FlumeNode。
Flume Cookbook 指出我们可以将命名管道尾随 Flume ，但这似乎需要 cygwin，并且我们不太可能在所有 Flume 节点上安装 cygwin。以下是食谱中的说明：
控制台源非常有用，因为我们可以将数据通过管道传输到 Flume 中 直接地。下一个示例将数据从程序传输到 Flume 中 然后交付。
$ <external process> | flume node_nowatch -1 -s -n foo -c
'foo:console|agentBESink("collector");'
理想情况下，您可以将数据写入命名管道，然后让 Flume 读取 使用文本或尾部来自命名管道的数据。不幸的是，这个版本 Flume 的文本和尾部目前与命名管道不兼容 在Linux环境中。但是，您可以通过管道将数据传输到 Flume 节点 在标准输入控制台上监听：
$ tail -f namedpipe | flume node_nowatch -1 -s -n foo -c
'foo:console|agentBESink;'
或者您可以使用 exec 源来获取其输出数据：
$ flume node_nowatch -1 -s -n bar -c 'bar:exec("cat
pipe")|agentBESink;'
我还发现了关于 谷歌小组讨论了这个想法，Claudera 的人似乎说这是可能的，但没有具体解释 Flume 如何连接到管道以及在这种情况下它充当服务器还是客户端。
理想情况下，我应该能够在我的应用程序中创建 NamedPipeClientSteram 或 NamedPipeServerStream 。客户端需要运行命名管道服务器，并且客户端查找连接到它的客户端。我假设 Flume 将充当服务器，但我无法确认这一点。例如
NamedPipeClientStream pipeClient = new NamedPipeClientStream(".", "FlumeNamedPipe", PipeDirection.Out, pIpeOptions.None);
pipeClient.Connect();
，在 Windows 中启动 Flume 服务时，是否可以将某些内容添加到命令行参数中，以便跟踪管道？
Not sure if this is a SO, SuperUser or ServerFault question, but I just installed the FlumeNode service on my windows machine and I wanted to send logging infromation via a named pipe to the FlumeNode.
The Flume Cookbook states that we can tail the named pipe to Flume, but it seems that this requires cygwin and it's unlikely that we're going to install cygwin on all the Flume nodes. Here are the instructions from the cookbook:
Console sources are useful because we can pipe data into Flume
directly. The next example pipes data from a program into Flume which
then delivers it.
$ <external process> | flume node_nowatch -1 -s -n foo -c
'foo:console|agentBESink("collector");'
Ideally, you could write data to a named pipe and just have Flume read
data from a named pipe using text or tail. Unfortunately, this version
of Flume’s text and tail are not currently compatible with named pipes
in a Linux environment. However, you could pipe data to a Flume node
listening on the stdin console:
$ tail -f namedpipe | flume node_nowatch -1 -s -n foo -c
'foo:console|agentBESink;'
Or you can use the exec source to get its output data:
$ flume node_nowatch -1 -s -n bar -c 'bar:exec("cat
pipe")|agentBESink;'
I also found a discussion on google groups regarding this idea and it seems that the Claudera people say it's possible, but there is nothing specific explaining how Flume would connect to a pipe and if it acts as the server or a client in that case.
Ideally I should be able to create a NamedPipeClientSteram or a NamedPipeServerStream in my application. The client requires a named pipe server to be running and the clientlooks for clients which connect to it. I assume that the Flume will be acting as a server, but I can't confirm this. E.g.
NamedPipeClientStream pipeClient = new NamedPipeClientStream(".", "FlumeNamedPipe", PipeDirection.Out, pIpeOptions.None);
pipeClient.Connect();
Is there something that can be added to the command line arguments, when starting the Flume service in Windows, in order to tail a pipe?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
我收到了 Claudera 的回复，他们指出，由于 Windows 没有内置
tail程序，因此需要使用 Cygwin 来实现此类功能。我们不想安装 cygwin，所以我正在寻找替代解决方案。根据 Claudera 的说法，另一种方法是使用 C# Thrift 绑定，它将数据发送到具有 Thrift 源的 Flume 代理同一个 Windows 盒子。我不太确定如何为 Flume 代理指定 Thrift 源，但据说可以做到。
I got a response from Claudera and they state that since windows does not have a built-in
tailprogram, it's required that one uses Cygwin for such functionality. We don't want to install cygwin, so I'm looking for an alternative solution.According to Claudera, the alternative is to use C# Thrift bindings which would send the data a Flume agent with a Thrift source on the same Windows box. I'm not exactly sure how you specify a Thrift source for a Flume agent, but supposedly it can be done.
作为尾随日志的替代方案，我为 NLog 和 log4net 创建了一个 .Net Flume 附加程序。它使用 Flume 公开的 thrift 绑定。这些现在称为 ThriftLegacySource。 Flume 基于 Avro 的新绑定目前没有 C# 客户端，因此这是我能想到的唯一明智的替代方案。
附加器使用 nuget 安装：
Install-Package DotNetFlumeNG.Client.log4net
Install-Package DotNetFlumeNG.Client.NLog
完整文档位于：https://github.com/marksl/DotNetFlumeNG.Clients
As an alternative to tailing logs, I created a .Net Flume appender for NLog and log4net. It uses the thrift bindings exposed by Flume. These are now referred to as the ThriftLegacySource. Flume's newer bindings based on Avro do not currently have C# clients, so this was the only sensible alternative I could think of.
The appenders are installed with nuget:
Install-Package DotNetFlumeNG.Client.log4net
Install-Package DotNetFlumeNG.Client.NLog
Full documentation is here: https://github.com/marksl/DotNetFlumeNG.Clients