春季批处理 - 通过编程无需注释的读取器/作者台阶语

发布于 2025-01-21 20:58:56 字数 6691 浏览 1 评论 0原文

第一个版本


我会以编程方式定义多个弹簧批次读取器/作家,尤其是对于阶跃语言。

用注释@stepscope等定义读者是常见的,但这不是我的场景。

AS-IS:

@Bean
public Job exporterJobForLUS149A() {
    Job jobWithStep = buildJobWithStep(LUS149A.class, readerForLUS149A(), writerForLUS149A());
    return jobWithStep;
}

@StepScope
@Bean
public EntityMongoItemReader<LUS149A> readerForLUS149A() {
    final EntityMongoItemReader<LUS149A> reader = reader(LUS149A.class);
    return reader;
}

@StepScope
@Bean
public CommonEntityToFlatFileItemWriter<LUS149A> writerForLUS149A() {
    final CommonEntityToFlatFileItemWriter<LUS149A> writerForLUS149A = writer(LUS149A.class);
    return writerForLUS149A;
}

但是我有很多工作(读者和作家)来定义(〜20),并且可以进一步到达...总的来说,我不会为每个新工作/读者/作者修改代码提供。

因此,这个想法(伪代码):

for (String entityClass: entities) {
    // this provides the entityClass, in some way
    Class<? extends AccountSessionAware> entityClass = buildEntityClass(entitiesDefaultPackage, entityName, 
    output.getEntityPackage());
    
    // for readers
    EntityMongoItemReader<? extends AccountSessionAware> reader = buildReader(entityClass);
    GenericBeanDefinition readerBeanDefinition = new GenericBeanDefinition();
    readerBeanDefinition.setBeanClassName(EntityMongoItemReader.class.getName());
    readerBeanDefinition.setInstanceSupplier(() -> reader);
    // this setScope allows scope from BeanDefinition (application/singleton, infrastructure, prototype, etc...) but I would use StepScope
    readerBeanDefinition.setScope(STEP_SCOPE_I_HAVE_NOT);
    registry.registerBeanDefinition("readerFor" + entityName, readerBeanDefinition);
    log.info("registered: " + "readerFor" + entityName);

    // for writers
    CommonEntityToFlatFileItemWriter<? extends AccountSessionAware> writer = buildWriter(entityClass);
    GenericBeanDefinition writerBeanDefinition = new GenericBeanDefinition();
    writerBeanDefinition.setBeanClassName(EntityMongoItemReader.class.getName());
    writerBeanDefinition.setInstanceSupplier(() -> writer);
    // same problem as reader above
    writerBeanDefinition.setScope(STEP_SCOPE_I_HAVE_NOT);
    registry.registerBeanDefinition("writerFor" + entityName, writerBeanDefinition);
    log.info("registered: " + "writerFor" + entityName);


    Job jobWithStep = buildJobWithStep(entityClass, reader, writer);
    GenericBeanDefinition jobBeanDefinition = new GenericBeanDefinition();
    jobBeanDefinition.setBeanClassName(Job.class.getName());
    jobBeanDefinition.setInstanceSupplier(() -> jobWithStep);
    // for Job we have singleton scope 
    jobBeanDefinition.setScope(BeanDefinition.SCOPE_SINGLETON);
    registry.registerBeanDefinition("jobFor" + entityName, jobBeanDefinition);
    log.info("registered: " + "jobFor" + entityName);
}

下面,构建读者和作家的通用方法,常见的情况和情景:

private public <eASAx extends AccountSessionAware> EntityMongoItemReader<eASAx> reader(final Class<eASAx> entityToReadClass) {
    LinkedHashMap<String, SortType> commonSort = outputFromDomain.getSort().getCommon();
    final LinkedHashMap<String, SortType> furtherSort = csvExporterType.getOutputs().get(entityToReadClass.getSimpleName()).getAdditionalSort();
    final EntityMongoItemReader<eASAx> reader = new EntityMongoItemReader<>(entityToReadClass, readingPageSize, commonSort, furtherSort,            mongoTemplate);
    return reader;
}

private <eASAx extends AccountSessionAware> CommonEntityToFlatFileItemWriter<eASAx> writer(final Class<eASAx> eASAxxxClass) {
    final String[] headerColumnNames = csvExporterType.getOutputs().get(eASAxxxClass.getSimpleName()).getColumns().split(ExporterConstants.COMMA);
    final String outputFileName = eASAxxxClass.getSimpleName();

    final EntityToFlatFileItemWriter<eASAx> writer = new EntityToFlatFileItemWriter<>(eASAxxxClass, headerColumnNames, outputDir, outputFileName,   fieldsSeparator, writingChunkSize);

    // if required, create multifile writer instance
    if (resourceLimit > 0) {
        final EntityToMultiResourceFlatFileItemWriter<eASAx> entityToMultiResourceFlatFileItemWriter = new EntityToMultiResourceFlatFileItemWriter<>(writer, resourceLimit);
        return entityToMultiResourceFlatFileItemWriter;
    }

    return writer;
}

不幸的是,我无法找到一种方法来指定步骤的方式申请读者/作家通用定义。 另外,我找到了org.springframework.batch.core.scope.stepscope类,似乎有责任将该范围应用于@stepscope注释的bean,并在@configuration注释类中发表了@enableBatchProcessing Ontotation of Class的注释。本身,@configuration。 不幸的是,我也无法找到使用该步骤的方法。 我需要读者/作者的步骤镜,因为它们需要一些来自工作上下文的参数,例如:

@Value("#{jobParameters['session']}")
private String session;
@Value("#{jobParameters['sort']}")
private boolean toSort;

不提供读取器/作者的步骤范围,当然,这些注入将不起作用,我将具有无效的值在步骤之间传递将无法使用(使用stepexecutionContext)。

那么,有什么建议吗?当然,也使用其他方式……一种与我的方法不同的方法。


第二版(已更新)

现在,代码是:

@EnableBatchProcessing
@Configuration
public class ExporterBatchConfigDynamic extends DefaultBatchConfigurer {

  @Autowired
  private GenericApplicationContext genericApplicationContext;

[...]

  @PostConstruct
  public void init() {
    for (String entityClass: entities) {

      GenericEntityMongoItemReader reader = reader(entityClass);                 
      genericApplicationContext.registerBean("readerFor" + entityName, GenericEntityMongoItemReader.class, () -> reader, Customizers::stepScoped);

      
CommonEntityToFlatFileItemWriter<AccountSessionAware> writer = writer(entityClass);
      genericApplicationContext.registerBean("writerFor" + entityName, CommonEntityToFlatFileItemWriter.class, () -> writer, Customizers::stepScoped);

      Job jobWithStep = buildJobWithStep(entityClass, reader, writer, namesToJobsMapping());
      genericApplicationContext.registerBean("jobFor" + entityName, Job.class, () -> jobWithStep, Customizers::jobScoped);
      log.info(entityName + ":: registered: " + "readerFor" + entityName + ", writerFor" + entityName + ", jobFor" + entityName);       
    }
  }
  
  [...]

  public static class Customizers {
    public static void stepScoped(final BeanDefinition bd) {
      bd.setScope(SCOPE_STEP);
    }

    public static void jobScoped(final BeanDefinition bd) {
      bd.setScope(SCOPE_JOB);
    }
  }

  // after this, also the ScopeConfiguration is declared, including Step and Job from comment below
}

现在读者和作家(显然是)逐步分析。 。 在第二个通话中,读者读取0个条目,然后将0个块传递给当然是0的作家。 我知道,当读者/作家没有被逐步划分时,我知道这会发生:因此,我用于Bean注册的定制器无法正常工作,或者通常认为注册方式不正确。

还有进一步的建议吗?

谢谢

FIRST VERSION


I would define multiple Spring Batch readers/writers programmatically, especially for StepScope.

It's common to define a reader with annotation @StepScope and so on, but this is not my scenario.

AS-IS:

@Bean
public Job exporterJobForLUS149A() {
    Job jobWithStep = buildJobWithStep(LUS149A.class, readerForLUS149A(), writerForLUS149A());
    return jobWithStep;
}

@StepScope
@Bean
public EntityMongoItemReader<LUS149A> readerForLUS149A() {
    final EntityMongoItemReader<LUS149A> reader = reader(LUS149A.class);
    return reader;
}

@StepScope
@Bean
public CommonEntityToFlatFileItemWriter<LUS149A> writerForLUS149A() {
    final CommonEntityToFlatFileItemWriter<LUS149A> writerForLUS149A = writer(LUS149A.class);
    return writerForLUS149A;
}

But I have a lot of job (with its reader and writer) to define (~20) and further could arrive... and in general I would not modify code for each new job/reader/writer I have to provide.

So, the idea, TO-BE (pseudo-code):

for (String entityClass: entities) {
    // this provides the entityClass, in some way
    Class<? extends AccountSessionAware> entityClass = buildEntityClass(entitiesDefaultPackage, entityName, 
    output.getEntityPackage());
    
    // for readers
    EntityMongoItemReader<? extends AccountSessionAware> reader = buildReader(entityClass);
    GenericBeanDefinition readerBeanDefinition = new GenericBeanDefinition();
    readerBeanDefinition.setBeanClassName(EntityMongoItemReader.class.getName());
    readerBeanDefinition.setInstanceSupplier(() -> reader);
    // this setScope allows scope from BeanDefinition (application/singleton, infrastructure, prototype, etc...) but I would use StepScope
    readerBeanDefinition.setScope(STEP_SCOPE_I_HAVE_NOT);
    registry.registerBeanDefinition("readerFor" + entityName, readerBeanDefinition);
    log.info("registered: " + "readerFor" + entityName);

    // for writers
    CommonEntityToFlatFileItemWriter<? extends AccountSessionAware> writer = buildWriter(entityClass);
    GenericBeanDefinition writerBeanDefinition = new GenericBeanDefinition();
    writerBeanDefinition.setBeanClassName(EntityMongoItemReader.class.getName());
    writerBeanDefinition.setInstanceSupplier(() -> writer);
    // same problem as reader above
    writerBeanDefinition.setScope(STEP_SCOPE_I_HAVE_NOT);
    registry.registerBeanDefinition("writerFor" + entityName, writerBeanDefinition);
    log.info("registered: " + "writerFor" + entityName);


    Job jobWithStep = buildJobWithStep(entityClass, reader, writer);
    GenericBeanDefinition jobBeanDefinition = new GenericBeanDefinition();
    jobBeanDefinition.setBeanClassName(Job.class.getName());
    jobBeanDefinition.setInstanceSupplier(() -> jobWithStep);
    // for Job we have singleton scope 
    jobBeanDefinition.setScope(BeanDefinition.SCOPE_SINGLETON);
    registry.registerBeanDefinition("jobFor" + entityName, jobBeanDefinition);
    log.info("registered: " + "jobFor" + entityName);
}

Below, common methods to build reader and writer, common to AS-IS and TO-BE scenarios:

private public <eASAx extends AccountSessionAware> EntityMongoItemReader<eASAx> reader(final Class<eASAx> entityToReadClass) {
    LinkedHashMap<String, SortType> commonSort = outputFromDomain.getSort().getCommon();
    final LinkedHashMap<String, SortType> furtherSort = csvExporterType.getOutputs().get(entityToReadClass.getSimpleName()).getAdditionalSort();
    final EntityMongoItemReader<eASAx> reader = new EntityMongoItemReader<>(entityToReadClass, readingPageSize, commonSort, furtherSort,            mongoTemplate);
    return reader;
}

private <eASAx extends AccountSessionAware> CommonEntityToFlatFileItemWriter<eASAx> writer(final Class<eASAx> eASAxxxClass) {
    final String[] headerColumnNames = csvExporterType.getOutputs().get(eASAxxxClass.getSimpleName()).getColumns().split(ExporterConstants.COMMA);
    final String outputFileName = eASAxxxClass.getSimpleName();

    final EntityToFlatFileItemWriter<eASAx> writer = new EntityToFlatFileItemWriter<>(eASAxxxClass, headerColumnNames, outputDir, outputFileName,   fieldsSeparator, writingChunkSize);

    // if required, create multifile writer instance
    if (resourceLimit > 0) {
        final EntityToMultiResourceFlatFileItemWriter<eASAx> entityToMultiResourceFlatFileItemWriter = new EntityToMultiResourceFlatFileItemWriter<>(writer, resourceLimit);
        return entityToMultiResourceFlatFileItemWriter;
    }

    return writer;
}

Unfortunately, I am not able to find a way to specify the StepScope to apply to reader/writer GenericBeanDefinition.
Also, I found org.springframework.batch.core.scope.StepScope class, which seems to be responsible to apply that scope to @StepScope annotated bean, and declared in a @Configuration annotated class also having the @EnableBatchProcessing annotation on top of class itself, among the @Configuration.
Unfortunately, I was also not able to find a way to use that StepScope.
And I need StepScope for reader/writer because they need some parameters from job context, such as:

@Value("#{jobParameters['session']}")
private String session;
@Value("#{jobParameters['sort']}")
private boolean toSort;

Not providing the step scope for reader/writer, of course, those injection will not work and I will have null value, as for as the data passing between step will not work (using StepExecutionContext).

So, any suggestion? Also using other way, of course... a different approach than mine.


SECOND VERSION (UPDATED)

now the code is something as:

@EnableBatchProcessing
@Configuration
public class ExporterBatchConfigDynamic extends DefaultBatchConfigurer {

  @Autowired
  private GenericApplicationContext genericApplicationContext;

[...]

  @PostConstruct
  public void init() {
    for (String entityClass: entities) {

      GenericEntityMongoItemReader reader = reader(entityClass);                 
      genericApplicationContext.registerBean("readerFor" + entityName, GenericEntityMongoItemReader.class, () -> reader, Customizers::stepScoped);

      
CommonEntityToFlatFileItemWriter<AccountSessionAware> writer = writer(entityClass);
      genericApplicationContext.registerBean("writerFor" + entityName, CommonEntityToFlatFileItemWriter.class, () -> writer, Customizers::stepScoped);

      Job jobWithStep = buildJobWithStep(entityClass, reader, writer, namesToJobsMapping());
      genericApplicationContext.registerBean("jobFor" + entityName, Job.class, () -> jobWithStep, Customizers::jobScoped);
      log.info(entityName + ":: registered: " + "readerFor" + entityName + ", writerFor" + entityName + ", jobFor" + entityName);       
    }
  }
  
  [...]

  public static class Customizers {
    public static void stepScoped(final BeanDefinition bd) {
      bd.setScope(SCOPE_STEP);
    }

    public static void jobScoped(final BeanDefinition bd) {
      bd.setScope(SCOPE_JOB);
    }
  }

  // after this, also the ScopeConfiguration is declared, including Step and Job from comment below
}

Now the readers and writers are (apparently) step-scoped.. the application works, but only for the first call.
On second call, the reader reads 0 entries, and pass 0 chunks to writer, which wrotes 0, of course.
I know this happens when readers/writers are not step-scoped: so, my Customizer used for bean registration is not working properly, or in general that registration way is not right.

Any further suggestion?

Thanks

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

难如初 2025-01-28 20:58:56

您可以使用genericBeanDefinition#setScope(String)。因此,在您的情况下,可能是:

readerBeanDefinition.setScope("step");

请注意,范围“步骤”应在应用程序上下文中注册,因为默认情况下未注册(这是Spring Batch的自定义范围)。

You can use GenericBeanDefinition#setScope(String). So in your case it could be something like:

readerBeanDefinition.setScope("step");

Note that the scope "step" should be registered in the application context as it is not registered by default (it is a custom scope from Spring Batch).

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文