我正在开发一个Spring批处理作业,它使用MultiResourcePartitioner并行处理多个输入文件。在ItemProcessor中,我需要获取当前输入文件中的记录数。我从步骤上下文中获取当前文件名,并读取文件中的行数:
StepSynchronizationManager.register(stepExecution);
StepContext stepContext = StepSynchronizationManager.getContext();
StepSynchronizationManager.close();
log.trace("stepContext: " + stepContext.getStepExecution().getExecutionContext().entrySet().toString());
...
UrlResource currentFile = new UrlResource(stepContext.getStepExecution().getExecutionContext().getString("fileName"));
这一切似乎工作,但我得到例外时,从处理器线程访问步骤上下文:
2013-05-15 11:44:35,178 DEBUG [org.springframework.batch.core.step.tasklet.TaskletStep] <taskExecutor-3> - Rollback for RuntimeException: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'scopedTarget.processor': Scope 'step' is not active for the current thread; consider defining a scoped proxy for this bean if you intend to refer to it from a singleton; nested exception is java.lang.IllegalStateException: No context holder available for step scope
2013-05-15 11:44:35,194 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] <taskExecutor-3> - Returning JDBC Connection to DataSource
2013-05-15 11:44:35,194 DEBUG [org.springframework.batch.repeat.support.RepeatTemplate] <taskExecutor-3> - Handling exception: org.springframework.beans.factory.BeanCreationException, caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'scopedTarget.processor': Scope 'step' is not active for the current thread; consider defining a scoped proxy for this bean if you intend to refer to it from a singleton; nested exception is java.lang.IllegalStateException: No context holder available for step scope
2013-05-15 11:44:35,194 DEBUG [org.springframework.batch.repeat.support.RepeatTemplate] <taskExecutor-3> - Handling fatal exception explicitly (rethrowing first of 1): org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'scopedTarget.processor': Scope 'step' is not active for the current thread; consider defining a scoped proxy for this bean if you intend to refer to it from a singleton; nested exception is java.lang.IllegalStateException: No context holder available for step scope
2013-05-15 11:44:35,210 ERROR [org.springframework.batch.core.step.AbstractStep] <taskExecutor-3> - Encountered an error executing the step
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'scopedTarget.processor': Scope 'step' is not active for the current thread; consider defining a scoped proxy for this bean if you intend to refer to it from a singleton; nested exception is java.lang.IllegalStateException: No context holder available for step scope
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:341)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:192)
at org.springframework.aop.target.SimpleBeanTargetSource.getTarget(SimpleBeanTargetSource.java:33)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:182)
at $Proxy14.process(Unknown Source)
at org.springframework.batch.core.step.item.SimpleChunkProcessor.doProcess(SimpleChunkProcessor.java:125)
at org.springframework.batch.core.step.item.SimpleChunkProcessor.transform(SimpleChunkProcessor.java:291)
at org.springframework.batch.core.step.item.SimpleChunkProcessor.process(SimpleChunkProcessor.java:190)
at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:74)
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:386)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:130)
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:264)
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:76)
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:367)
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:214)
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:143)
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:250)
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:195)
at org.springframework.batch.core.partition.support.TaskExecutorPartitionHandler$1.call(TaskExecutorPartitionHandler.java:120)
at org.springframework.batch.core.partition.support.TaskExecutorPartitionHandler$1.call(TaskExecutorPartitionHandler.java:118)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.IllegalStateException: No context holder available for step scope
at org.springframework.batch.core.scope.StepScope.getContext(StepScope.java:197)
at org.springframework.batch.core.scope.StepScope.get(StepScope.java:139)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:327)
... 24 more
有没有办法从分区作业的处理器获取当前输入文件名?
下面是相关配置:
<batch:job id="acct">
<batch:step id="init" parent="abstractStep" next="processStep">
<batch:tasklet ref="fileDeleteTasklet" />
</batch:step>
<batch:step id="processStep">
<batch:partition step="processInput" partitioner="partitioner">
<batch:handler grid-size="2" task-executor="taskExecutor" />
</batch:partition>
</batch:step>
</batch:job>
<batch:step id="processInput">
<batch:tasklet>
<batch:chunk reader="reader" processor="processor" writer="writer" commit-interval="3">
<batch:streams>
<batch:stream ref="reader"/>
<batch:stream ref="flatFileItemWriter"/>
</batch:streams>
</batch:chunk>
</batch:tasklet>
</batch:step>
<bean id="partitioner" class="org.springframework.batch.core.partition.support.MultiResourcePartitioner" scope="step">
<property name="keyName" value="fileName"/>
<property name="resources" value="file:#{jobParameters['inputFilePattern']}"/>
</bean>
<bean id="taskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="5"/>
<property name="maxPoolSize" value="5"/>
</bean>
<bean id="reader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
<property name="resource" value="#{stepExecutionContext[fileName]}" />
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="lineTokenizer">
<bean class="org.springframework.batch.item.file.transform.FixedLengthTokenizer">
<property name="names" value="recordType,reserved,aCode,bCode,acctNumber,idType,cCode" />
<property name="columns" value="1,2,3-6,7-9,10-15,16,17-19" />
<property name="strict" value="false" />
</bean>
</property>
<property name="fieldSetMapper">
<bean class="com.example.batch.acct.AcctInputFieldSetMapper" />
</property>
</bean>
</property>
</bean>
<bean id="processor" class="com.example.batch.acct.AcctProcessor" scope="step">
<property name="soapClient" ref="soapClient" />
<property name="maxNumIds" value="${batch.soap.numberOfIds}" />
<aop:scoped-proxy />
</bean>
<bean id="itemProcessor" class="com.myApp.MyItemProcessor" scope="step">
<property name="stepContext" value="#{stepExecutionContext}"/>
</bean>
在这种情况下,当stepContex
是org的一种类型时。springframework。一批果心范围上下文StepContext
,如何显示此属性的名称,我遇到了异常:
组织。springframework。豆。工厂BeanCreationException:创建名为“scopedTarget”的bean时出错。在类路径资源[META-INF/jobs/Job.xml]中定义的itemProcessor':bean初始化失败;嵌套的异常是org。springframework。豆。ConversionNotSupportedException:未能转换“java”类型的属性值。util。集合$UnmodifiableMap“到所需类型”组织。springframework。一批果心范围上下文属性“StepContext”的“StepContext”;嵌套的异常是java。lang.IllegalStateException:无法转换“java”类型的值。util。集合$UnmodifiableMap“到所需类型”组织。springframework。一批果心范围上下文属性“StepContext”的“StepContext”:未找到匹配的编辑器或转换策略
我需要访问ItemProcessor
内部的jobExecutionId
,因此我已将此配置更改为:
<bean id="itemProcessor" class="com.myApp.MyItemProcessor" scope="step">
<property name="stepExecution" value="#{stepExecution}"/>
</bean>
属性“stepExecution”
是一种类型组织。springframework。一批果心步骤执行
在Spring Batch论坛上回答:
ItemProcessor
接口没有公开访问StepContext
的方法,因此您需要自己插入它:
<bean id="itemProcessor" class="com.myApp.MyItemProcessor" scope="step">
<property name="stepContext" value="#{stepExecutionContext}"/>
</bean>
假设您的处理器类有一个名为stepContext
的公开属性。
-Michael Minella,Spring Batch负责人
给定一个使用分区的Spring批处理作业,是否可能有多个分区步骤? 例如: 在上述示例中,是否可以将另一个添加到(最好不需要为每个分区步骤提供分区器)?如果没有,是否有其他方法来配置多个步骤,这些步骤将针对每个分区逐个执行?
我设置了以下测试用例: 我想使用Groovy脚本测试步骤获得SoapRequest测试步骤的状态。 它可以如下所示完成: 但是我不想通过脚本运行TestStep,而只是使用soapui testrunner。在datasink测试步骤中,我可以使用这个: 不幸的是,上述方法在GroovyScript TestStep中不起作用 有什么想法吗?
我试图配置我的第一个多线程作业。我们有大约200,000条记录的主目录,我们需要处理。我想将文件分解为10个文件并处理它们。拆分文件tasklet工作正常 主步骤在我的配置中运行,但从步骤不运行。下面是我的配置。 分割者: MultiResourceItemReader: FlatFileItemWriter: 作业配置: 从属步骤配置: 请告知我做错了什么。我没有看到处理器urlFileItem
我正在尝试为分区配置Spring批处理步骤。这里很好的示例显示了一个关于“ID范围”的分区,但我不知道如何从“数据页”范围开始。 在我的顺序步骤中,我有: null
使用Spring Batch 3.0.4.Release。 我将作业配置为使用分区步骤。从机步骤使用块大小1。任务执行器中有六个线程。我使用从六到数百的各种网格大小来运行这个测试。我的网格大小是从StepExecutions的数量,我希望==我的分区器创建的ExecutionContexts的数量。 下面是Java配置代码:
1. 预处理的步骤 现在我们全面了解一下C编译器做语法解析之前的预处理步骤: 1、把第 2 节 “常量”提到过的三连符替换成相应的单字符。 2、把用\字符续行的多行代码接成一行。例如: #define STR "hello, "\ "world" 经过这个预处理步骤之后接成一行: #define STR "hello, " "world" 这种续行的写法要求\后面紧跟换行,中间不能有其它空白