使用 Quartz 工作线程分割 log4j 输出

发布于 2024-07-06 10:35:42 字数 852 浏览 7 评论 0原文

我正在开发一个应用程序,该应用程序由一个基于 Quartz 的整体调度程序和使用 CronTriggers 运行的“CycledJob”组成。 该应用程序的目的是根据来源国家/地区处理来自不同电子邮件收件箱的输入。

根据它来自的国家(即美国、英国、法国等),应用程序会触发一个作业线程来运行每个国家的处理周期,因此会有一个英国工作线程,一个用于美国、法国等。将输出格式化为 log4j 时,我使用了 thread 参数,因此它会发出 [ApplicationName_Worker-1]、[ApplicationName_Worker-2] 等。尽我所能,我找不到命名线程的方法,因为它们'重新从 Quartz 的线程池中退出。 虽然我可能会扩展 Quartz,但我想制定一个不同的解决方案,而不是搞乱标准库。

问题是:使用 log4j 时,我希望将美国线程中的所有日志项输出到仅限美国的文件,对于每个国家/地区线程也是如此。 我不在乎它们是否留在一个统一的 ConsoleAppender 中,FileAppender 拆分正是我在这里所追求的。 我已经知道如何指定多个文件附加程序等,我的问题是我无法根据国家/地区进行区分。 应用程序中有 20 多个类可以位于执行链上,其中很少有我想承担通过每个方法传递额外“上下文”参数的知识的负担……我考虑了一种扩展策略模式log4j 包装器类,但除非我能让链中的每个类都知道它在哪个线程上来参数化记录器调用,否则这似乎是不可能的。 无法命名线程也会带来挑战(否则这会很容易!)。

因此,问题是:建议采用什么方法来允许应用程序中的许多从属类(每个类用于每个不同的线程来处理输入)知道它们在记录日志时位于特定国家/地区线程的上下文中?

祝您理解顺利,请提出澄清问题! 我希望有人能够帮助我找到一个合适的方法来解决这个问题。 欢迎所有建议。

I'm working on an application that consists of an overall Quartz-based scheduler and "CycledJob" run using CronTriggers. The purpose of the application is to process inputs from different email inboxes based on the source country.

Based on the country that it comes in from (i.e. US, UK, FR, etc.) the application triggers one job thread to run each country's processing cycle, so there would be a UK Worker thread, one for US, France, etc. When formatting the output to log4j, I'm using the thread parameter, so it emits [ApplicationName_Worker-1], [ApplicationName_Worker-2] etc. Try as I might, I can't find a way to name the threads since they're pulled out of Quartz's Thread Pools. Although I could possibly go so far as to extend Quartz, I'd like to work out a different solution instead of messing with the standard library.

Here's the problem: When using log4j, I'd like to have all log items from the US thread output to a US only file, likewise for each of the country threads. I don't care if they stay in one unified ConsoleAppender, the FileAppender split is what I'm after here. I already know how to specify multiple file appenders and such, my issue is I can't differentiate based on country. There are 20+ classes within the application that can be on the execution chain, very few of which I want to burden with the knowledge of passing an extra "context" parameter through EVERY method... I've considered a Strategy pattern extending a log4j wrapper class, but unless I can let every class in the chain know which thread it's on to parameterize the logger call, that seems impossible. Without being able to name the thread also creates a challenge (or else this would be easy!).

So here's the question: What would be a suggested approach to allow many subordinate classes in an application that are each used for every different thread to process the input know that they are within the context of a particular country thread when they are logging?

Good luck understanding, and please ask clarifying questions! I hope someone is able to help me figure out a decent way to tackle this. All suggestions welcome.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(4

活雷疯 2024-07-13 10:35:42

在每个国家/地区处理线程的顶部,将国家/地区代码放入 Log4j 的映射诊断上下文 (MDC) 中。 这使用了 ThreadLocal 变量,因此您不必显式地在调用堆栈中上下传递国家/地区。 然后创建一个查看 MDC 的自定义过滤器,并过滤掉任何不包含当前附加程序国家/地区代码的事件。

在您的 Job 中:

...
public static final String MDC_COUNTRY = "com.y.foo.Country";
public void execute(JobExecutionContext context)
  /* Just guessing that you have the country in your JobContext. */
  MDC.put(MDC_COUNTRY, context.get(MDC_COUNTRY));
  try {
    /* Perform your job here. */
    ...
  } finally {
    MDC.remove(MDC_COUNTRY);
  }
}
...

编写自定义 过滤器

package com.y.log4j;

import org.apache.log4j.spi.LoggingEvent;

/**
 * This is a general purpose filter. If its "value" property is null, 
 * it requires only that the specified key be set in the MDC. If its 
 * value is not null, it further requires that the value in the MDC 
 * is equal.
 */
public final class ContextFilter extends org.apache.log4j.spi.Filter {

  public int decide(LoggingEvent event) {
    Object ctx = event.getMDC(key);
    if (value == null)
      return (ctx != null) ? NEUTRAL : DENY;
    else
      return value.equals(ctx) ? NEUTRAL : DENY;
  }

  private String key;
  private String value;

  public void setContextKey(String key) { this.key = key; }
  public String getContextKey() { return key; }
  public void setValue(String value) { this.value = value; }
  public String getValue() { return value; }

}

在您的 log4j.xml 中:

<appender name="fr" class="org.apache.log4j.FileAppender">
  <param name="file" value="france.log"/>
  ...
  <filter class="com.y.log4j.ContextFilter">
    <param name="key" value="com.y.foo.Country" />
    <param name="value" value="fr" />
  </filter>
</appender> 

At the top of each country's processing thread, put the country code into Log4j's mapped diagnostic context (MDC). This uses a ThreadLocal variable so that you don't have to pass the country up and down the call stack explicitly. Then create a custom filter that looks at the MDC, and filters out any events that don't contain the current appender's country code.

In your Job:

...
public static final String MDC_COUNTRY = "com.y.foo.Country";
public void execute(JobExecutionContext context)
  /* Just guessing that you have the country in your JobContext. */
  MDC.put(MDC_COUNTRY, context.get(MDC_COUNTRY));
  try {
    /* Perform your job here. */
    ...
  } finally {
    MDC.remove(MDC_COUNTRY);
  }
}
...

Write a custom Filter:

package com.y.log4j;

import org.apache.log4j.spi.LoggingEvent;

/**
 * This is a general purpose filter. If its "value" property is null, 
 * it requires only that the specified key be set in the MDC. If its 
 * value is not null, it further requires that the value in the MDC 
 * is equal.
 */
public final class ContextFilter extends org.apache.log4j.spi.Filter {

  public int decide(LoggingEvent event) {
    Object ctx = event.getMDC(key);
    if (value == null)
      return (ctx != null) ? NEUTRAL : DENY;
    else
      return value.equals(ctx) ? NEUTRAL : DENY;
  }

  private String key;
  private String value;

  public void setContextKey(String key) { this.key = key; }
  public String getContextKey() { return key; }
  public void setValue(String value) { this.value = value; }
  public String getValue() { return value; }

}

In your log4j.xml:

<appender name="fr" class="org.apache.log4j.FileAppender">
  <param name="file" value="france.log"/>
  ...
  <filter class="com.y.log4j.ContextFilter">
    <param name="key" value="com.y.foo.Country" />
    <param name="value" value="fr" />
  </filter>
</appender> 
神魇的王 2024-07-13 10:35:42

我希望我能比这更有帮助,但您可能想使用一些过滤器进行调查? 也许您的日志记录可以输出国家/地区代码,并且您可以根据该代码匹配您的过滤器?

StringMatchFilter 应该能够为您匹配它。

无法使以下地址作为链接正常工作,但如果您查看它,它包含一些使用过滤器进行单独文件日志记录的内容。

http://mail-archives.apache.org/mod_mbox /logging-log4j-user/200512.mbox/<1CC26C83B6E5AA49A9540FAC8D35158B01E2968E@pune.kaleconsultants.com> (只需删除>之前的空格)

http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/spi/Filter.html

I wish I could be a bit more helpful than this, but you may want to investigate using some filters? Perhaps your logging could output the country code and you could match your filter based on that?

A StringMatchFilter should probably be able to match it for you.

Couldn't get the below address to work properly as a link, but if you look at it, it has some stuff on separate file logging using filters.

http://mail-archives.apache.org/mod_mbox/logging-log4j-user/200512.mbox/<1CC26C83B6E5AA49A9540FAC8D35158B01E2968E@pune.kaleconsultants.com > (just remove the space before the >)

http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/spi/Filter.html

拥醉 2024-07-13 10:35:42

当您的作业开始设置线程名称时,为什么不直接调用 Thread.setName() 呢? 如果存在访问问题,请配置quartz以使用您自己的线程池。

Why not just call Thread.setName() when your job starts to set the name the Thread? If there is an access problem, configure quartz to use your own thread pool.

¢好甜 2024-07-13 10:35:42

我可能完全偏离了我对你想要完成的目标的理解,但我会尝试解决方案。 听起来您想要为您正在处理电子邮件的每个国家/地区建立一个单独的日志文件。 基于这种理解,这里是一个可能的解决方案:

  1. 在 log4j 配置中为您希望单独登录的每个国家/地区设置一个附加程序(提供了美国示例):

    log4j.appender.usfile=org.apache.log4j.FileAppender

    log4j.appender.usfile.File=us.log

    log4j.appender.usfile.layout=org.apache.log4j.PatternLayout

    log4j.appender.usfile.layout.ConversionPattern=%m%n

  2. 为每个国家/地区创建一个记录器,并将每个记录器定向到适当的附加程序(提供美国示例):

    log4j.logger.my-us-logger=debug,usfile

  3. 在您的代码中,根据处理电子邮件的国家/地区创建记录器:

    Logger logger = Logger.getLogger("my-us-logger");

  4. 确定如何完成后续方法调用的步骤 3。 您可以在每个类/方法中重复步骤 3; 或者您可以修改方法签名以接受记录器作为输入; 或者您可以使用 ThreadLocal在方法之间传递 Logger。

额外信息:如果您不希望日志语句发送到父记录器(例如 rootLogger),您可以将其可加性标志设置为 false(提供了美国示例):

log4j.additivity.my-us-logger=false

I may be completely off base on my understanding of what you are attempting to accomplish, but I will take a stab at the solution. It sounds like you want a separate log file for each country for which you are processing email. Based on that understanding, here is a possible solution:

  1. Set up an appender in your log4j configuration for each country for which you wish to log separately (US example provided):

    log4j.appender.usfile=org.apache.log4j.FileAppender

    log4j.appender.usfile.File=us.log

    log4j.appender.usfile.layout=org.apache.log4j.PatternLayout

    log4j.appender.usfile.layout.ConversionPattern=%m%n

  2. Create a logger for each country and direct each of them to the appropriate appender (US example provided):

    log4j.logger.my-us-logger=debug,usfile

  3. In your code, create your Logger based on the country for which the email is being processed:

    Logger logger = Logger.getLogger("my-us-logger");

  4. Determine how you will accomplish step 3 for the subsequent method calls. You could repeat step 3 in each class/method; or you could modify the method signatures to accept a Logger as input; or you could possibly use ThreadLocal to pass the Logger between methods.

Extra info: If you do not want the log statements going to parent loggers (e.g. the rootLogger), you can set their additivity flags to false (US example provided):

log4j.additivity.my-us-logger=false
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文